{"id":5338,"library":"nbval","title":"nbval","description":"nbval is a pytest plugin designed to validate Jupyter notebooks. It allows users to execute notebooks as part of their test suite and compare cell outputs against previously saved outputs, ensuring reproducibility and correctness. The current version is 0.11.0, and it has a sporadic but active release cadence, typically with minor releases every 6-12 months.","status":"active","version":"0.11.0","language":"en","source_language":"en","source_url":"https://github.com/computationalmodelling/nbval","tags":["pytest","testing","jupyter","notebooks","ci"],"install":[{"cmd":"pip install nbval pytest","lang":"bash","label":"Install nbval and pytest"}],"dependencies":[{"reason":"nbval is a plugin for pytest and requires it to function.","package":"pytest"}],"imports":[{"note":"As a pytest plugin, direct imports from `nbval` are generally not needed for basic usage. Configuration is primarily done via pytest command-line options or `pytest.ini`.","symbol":"nbval functionality","correct":"nbval is primarily used via the pytest CLI (e.g., `pytest --nbval`) and typically does not require direct Python imports for its core validation features. pytest automatically discovers and loads the plugin upon installation."}],"quickstart":{"code":"import pytest\nimport json\nimport os\n\n# 1. Create a dummy Jupyter notebook file for testing\nnotebook_content = {\n    \"cells\": [\n        {\n            \"cell_type\": \"code\",\n            \"execution_count\": 1,\n            \"metadata\": {},\n            \"outputs\": [\n                {\"name\": \"stdout\", \"output_type\": \"stream\", \"text\": [\"Hello from nbval!\\n\"]}\n            ],\n            \"source\": [\"print('Hello from nbval!')\"]\n        }\n    ],\n    \"metadata\": {\n        \"kernelspec\": {\"display_name\": \"Python 3\", \"language\": \"python\", \"name\": \"python3\"},\n        \"language_info\": {\"codemirror_mode\": {\"name\": \"ipython\", \"version\": 3}, \"file_extension\": \".py\", \"mimetype\": \"text/x-python\", \"name\": \"python\", \"nbconvert_exporter\": \"python\", \"pygments_lexer\": \"ipython3\", \"version\": \"3.9.7\"}\n    },\n    \"nbformat\": 4,\n    \"nbformat_minor\": 5\n}\n\nnotebook_filename = \"example.ipynb\"\nwith open(notebook_filename, \"w\") as f:\n    json.dump(notebook_content, f, indent=4)\n\nprint(f\"Created {notebook_filename}. Running nbval tests...\")\n\n# 2. Run pytest programmatically with nbval\n#    The --nbval flag enables the plugin, and the notebook filename specifies the target.\nexit_code = pytest.main([\"--nbval\", notebook_filename])\n\n# 3. Report results and clean up\n# os.remove(notebook_filename) # Uncomment to clean up the notebook file after run\n\nif exit_code == 0:\n    print(f\"\\nnbval tests passed for {notebook_filename}!\")\nelse:\n    print(f\"\\nnbval tests failed for {notebook_filename} with exit code {exit_code}.\")\n","lang":"python","description":"This quickstart script demonstrates how to programmatically create a simple Jupyter notebook and then run `pytest` with the `nbval` plugin to validate its output. It shows the minimal setup required to get started with notebook testing using `nbval`."},"warnings":[{"fix":"Remove `--nbval-sanitize` from your pytest command-line arguments or `pytest.ini` configuration. If sanitation is still required, consider pre-processing notebooks or using `nbval`'s cell metadata options for more granular control over output comparison (e.g., `nbval-ignore_data_priority`).","message":"The `--nbval-sanitize` command-line option was removed in version 0.10.0. This option was previously used to sanitize notebook outputs before comparison but was deprecated in earlier versions.","severity":"breaking","affected_versions":">=0.10.0"},{"fix":"Upgrade your Python environment to Python 3.7 or newer. The current minimum requirement is Python 3.7.","message":"Support for Python 3.6 was dropped in version 0.9.0. Users on Python 3.6 will not be able to install or run `nbval` versions 0.9.0 or newer.","severity":"breaking","affected_versions":">=0.9.0"},{"fix":"To get a simpler pass/fail indication without detailed diffs, you can set `nbval_diff=False` in your `pytest.ini` or use `--no-nbval-diff` on the command line. For more controlled output validation, explore options like `--nbval-lax` or specific cell metadata for ignoring certain outputs.","message":"By default, `nbval` performs a strict diff on notebook outputs. This can lead to very verbose output in case of failures, especially with slight differences in timestamps, floating-point precision, or minor formatting changes. This behavior is controlled by the `nbval_diff` option.","severity":"gotcha","affected_versions":"All"},{"fix":"Ensure you understand how `nbval` handles outputs. For cells that are expected to fail, add `\"raises-exception\": true` to their cell metadata. For cells with no relevant output or to skip execution, use `\"skip-execution\": true` or `\"skip-compare\": true`. Use `--nbval-lax` for more permissive output comparison, or `--nbval-errors` to fail on any executed code error regardless of output comparison.","message":"nbval's primary function is *output validation*, not just execution and failure on any runtime error. It compares the newly generated outputs against the stored outputs in the notebook. If the notebook is expected to raise an exception or has no outputs for a cell, this needs to be explicitly indicated using cell metadata (e.g., `raises-exception`, `skip-execution`).","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-13T00:00:00.000Z","next_check":"2026-07-12T00:00:00.000Z"}