Testing Against Learned Reference Data

• Documentation
• PyPI
• Sources
• Issues

Installing it is pretty easy:

pip install test2ref

Testing Against Learned Reference Data.

Concept

A unit test creates files in a temporary folder tmp_path. :any:assert_refdata() is called at the end of the test.

There are two modes:

• Testing: Test result in tmp_path is compared against a known reference. Any deviation in the files, causes a fail.
• Learning: The test result in tmp_path is taken as reference and is copied to the reference folder, which should be committed to version control and kept as reference.

The file .test2ref in the project root directory selects the operation mode. If the file exists, Learning Mode is selected. If the files does not exists, the Testing Mode is selected.

Next to that, stdout, stderr and logging can be included in the reference automatically.

Minimal Example

Example

>>> def test_something(tmp_path, capsys):
...     (tmp_path / "file.txt").write_text("Hello Mars")
...     print("Hello World")
...     assert_refdata(test_something, tmp_path, capsys=capsys)

API

Functions:

Name	Description
configure	Configure.
assert_refdata	Compare Output of arg generated at path with reference.
assert_paths	Compare Output of ref_path with gen_path.

Attributes:

Name	Type	Description
Search	TypeAlias	Possible Search Pattern.
Replacements	TypeAlias	Replacements - Pairs of Search Pattern and Things to be Replaced.

Search module-attribute

Search = Path | str | Pattern

Possible Search Pattern.

File System Path, string or regular expression.

Replacements module-attribute

Replacements = Iterable[tuple[Search, str]]

Replacements - Pairs of Search Pattern and Things to be Replaced.

configure

configure(
    ref_path=None,
    ref_update=None,
    excludes=None,
    add_excludes=None,
    rm_excludes=None,
    ignore_spaces=False,
)

Configure.

Other Parameters:

Name	Type	Description
ref_path	Path | None	Path for reference files. "tests/refdata" by default
ref_update	bool | None	Update reference files. True by default if .test2ref file exists.
excludes	Excludes | None	Paths to be excluded in all runs.
add_excludes	Excludes | None	Additionally Excluded Files
rm_excludes	Excludes | None	Not Excluded Files
ignore_spaces	bool | None	Ignore Space Changes - False by default

assert_refdata

assert_refdata(
    arg,
    path,
    capsys=None,
    caplog=None,
    replacements=None,
    excludes=None,
    flavor="",
    known=None,
)

Compare Output of arg generated at path with reference.

Use replacements to mention things which vary from test to test. path and the project location are already replaced by default.

Parameters:

Name	Type	Description	Default
arg	Callable | Path	Test Function or Path to reference data	required
path	Path	Path with generated files to be compared.	required

Other Parameters:

Name	Type	Description
capsys	Any	pytest capsys fixture. Include stdout/stderr too.
caplog	Any	pytest caplog fixture. Include logging output too.
replacements	Replacements | None	pairs of things to be replaced.
excludes	Iterable[str] | None	Files and directories to be excluded.
flavor	str	Flavor for different variants.
known	Path | None	Path with directories and files which are known and excluded as soon as they are identical.

Minimal Example

def test_example(tmp_path):
    (tmp_path / "file.txt").write_text("Content")
    assert_refdata(test_example, tmp_path)

Full Example

import logging

def test_example(tmp_path, capsys, caplog):
    (tmp_path / "file.txt").write_text("Content")

    # print on standard-output - captured by capsys
    print("Hello World")

    # logging - captured by caplog
    logging.getLogger().warning("test")

    assert_refdata(test_example, tmp_path, capsys=capsys, caplog=caplog)

The following replacements are included automatically:

• Python Installation Directories: $SITE
• Current Working Directory: $PRJ
• Argument path: $GEN
• Home Directory: $HOME

assert_paths

assert_paths(ref_path, gen_path, excludes=None)

Compare Output of ref_path with gen_path.

Parameters:

Name	Type	Description	Default
ref_path	Path	Path with reference files to be compared.	required
gen_path	Path	Path with generated files to be compared.	required

Other Parameters:

Name	Type	Description
excludes	Iterable[str] | None	Files and directories to be excluded.