Sybil Documentation Testing

Sybil validates code examples embedded in documentation and docstrings by parsing and executing them as part of normal test runs.

Official Documentation: https://sybil.readthedocs.io/en/latest/

Installation

bash
1pip install sybil[pytest]

Pytest Integration

Configure in conftest.py. See pytest integration docs.

python
1from sybil import Sybil
2from sybil.parsers.markdown.codeblock import PythonCodeBlockParser
3from sybil.parsers.markdown.skip import SkipParser
4
5pytest_collect_file = Sybil(
6    parsers=[
7        SkipParser(),
8        PythonCodeBlockParser(),
9    ],
10    patterns=["*.md", "**/*.py"],
11).pytest()

Sybil Parameters

See API reference.

Document Types

See API reference.

• Default: Parse entire file
• PythonDocument: Import .py file as module, names available in namespace
• PythonDocStringDocument: Parse only docstrings from .py files

python
1from sybil.document import PythonDocStringDocument
2
3pytest_collect_file = Sybil(
4    parsers=[...],
5    patterns=["src/**/*.py"],
6    document_types={".py": PythonDocStringDocument},
7).pytest()

Fixtures

python
1import pytest
2from sybil import Sybil
3
4
5@pytest.fixture
6def my_fixture():
7    return {"key": "value"}
8
9
10pytest_collect_file = Sybil(
11    parsers=[...],
12    patterns=["*.md"],
13    fixtures=["my_fixture"],  # Available in document namespace
14).pytest()

Setup/Teardown

python
1def sybil_setup(namespace):
2    namespace["helper"] = lambda x: x * 2
3
4
5def sybil_teardown(namespace):
6    pass  # Cleanup if needed
7
8
9pytest_collect_file = Sybil(
10    parsers=[...],
11    setup=sybil_setup,
12    teardown=sybil_teardown,
13).pytest()

Disable pytest's Built-in Doctest

Add to pyproject.toml to prevent conflicts:

toml
1[tool.pytest.ini_options]
2addopts = "-p no:doctest"

Markdown Parsers

See Markdown parsers docs.

python
1from sybil.parsers.markdown.codeblock import PythonCodeBlockParser, CodeBlockParser
2from sybil.parsers.markdown.skip import SkipParser
3from sybil.parsers.markdown.clear import ClearNamespaceParser

Skip Directives

See skip directive docs.

SkipParser must come before other parsers to handle skip directives.

markdown
1<!-- skip: next -->
2
3```python
4# This example is skipped
5```

python
1# Skipped and reported as skipped test with reason

python
1# Multiple examples

python
1# All skipped

python
1# Conditionally skipped based on namespace variable

## Invisible Code Blocks

Setup code that doesn't render in documentation. See [invisible code blocks docs](https://sybil.readthedocs.io/en/latest/markdown.html#invisible-code-blocks).

```markdown
<!-- invisible-code-block: python
setup_var = "hidden setup"
-->

Clear Namespace

Reset the document namespace for isolation. See clear namespace docs.

markdown
1<!-- clear-namespace -->

Custom Evaluators

See evaluators API.

python
1from sybil.parsers.markdown.codeblock import CodeBlockParser
2from sybil.evaluators.python import PythonEvaluator
3
4# Custom evaluator with future imports
5evaluator = PythonEvaluator(future_imports=["annotations"])
6
7parser = CodeBlockParser(language="python", evaluator=evaluator)

Running Sybil Tests

Sybil tests are collected like regular pytest tests. To run only Sybil tests:

bash
1# Run tests from specific directory containing documented code
2pytest src/mypackage/ -v
3
4# Exclude regular tests, only run documentation examples
5pytest docs/ -v

To exclude Sybil tests from regular test runs, use pytest's --ignore flag or configure addopts in pyproject.toml.

Example: Complete conftest.py

python
1"""Sybil configuration for docstring testing."""
2
3from sybil import Sybil
4from sybil.document import PythonDocStringDocument
5from sybil.evaluators.python import PythonEvaluator
6from sybil.parsers.markdown.codeblock import CodeBlockParser
7from sybil.parsers.markdown.skip import SkipParser
8
9import mypackage
10
11
12def sybil_setup(namespace):
13    """Pre-populate namespace for all examples."""
14    namespace["pkg"] = mypackage
15
16
17def sybil_teardown(namespace):
18    """Cleanup after document."""
19    pass
20
21
22pytest_collect_file = Sybil(
23    parsers=[
24        SkipParser(),
25        CodeBlockParser(language="python", evaluator=PythonEvaluator()),
26        CodeBlockParser(language="py", evaluator=PythonEvaluator()),
27    ],
28    patterns=["src/mypackage/**/*.py"],
29    document_types={".py": PythonDocStringDocument},
30    setup=sybil_setup,
31    teardown=sybil_teardown,
32    excludes=[
33        "**/tests/**",
34        "**/_internal/**",
35    ],
36).pytest()