LangChain Standard Tests

1.1.6 · active · verified Sat Apr 11

langchain-tests provides a collection of standard unit and integration tests designed for verifying LangChain implementations and integrations. It ensures consistent behavior across various components (e.g., LLMs, ChatModels, Embeddings, Retrievers). The library is part of the broader LangChain ecosystem and sees frequent updates, typically alongside `langchain-core` releases. The current version is 1.1.6.

Warnings

gotcha The `langchain-tests` package provides the actual test *implementations* and fixtures. However, the `StandardTestSuite` base class and general testing utilities (e.g., for creating mock integrations) that users extend to implement their own tests are typically found in `langchain-core` (e.g., `langchain_core.utils.standard_tests`).
Fix: When building a new integration, derive your test classes from `langchain_core.utils.standard_tests.StandardTestSuite` and implement its abstract methods. Then, use `pytest` to discover and run your implementation alongside the standard `langchain-tests`.
gotcha Many tests within `langchain-tests` (especially integration tests) require specific environment variables (e.g., `OPENAI_API_KEY`, `GOOGLE_API_KEY`, `ANTHROPIC_API_KEY`) and/or external service configurations to run successfully. Tests that cannot connect to required services or find necessary credentials will often be skipped.
Fix: Consult the LangChain documentation for the specific integration you are testing to understand its environmental requirements. Ensure all necessary API keys and service endpoints are correctly configured as environment variables or passed during test setup.
gotcha The primary way to 'use' `langchain-tests` is by running `pytest` from the command line, allowing it to discover the contained test functions and modules. Direct programmatic imports of classes or functions from `langchain_tests` for use in application logic are generally not common or intended, as the package's purpose is testing.
Fix: To execute the tests, use `pytest --pyargs langchain_tests` in your terminal. If you need to write custom tests for your integration, you'll typically interact with base classes and utilities from `langchain-core`.
breaking `langchain-tests` is part of the `langchain` monorepo and has tight dependency coupling with `langchain-core` and other `langchain` packages. Incompatibility issues can arise if `langchain-tests` is used with significantly older or newer versions of `langchain-core`.
Fix: Always install `langchain-tests` with a version that aligns with your `langchain-core` installation. Check the `pyproject.toml` of `langchain-tests` on GitHub for its exact `langchain-core` dependency range. It is often safest to upgrade both packages concurrently.

Install

pip install langchain-tests pytest Install library and test runner

Quickstart

The primary way to use `langchain-tests` is by running `pytest` to discover and execute its test modules. This example shows how to programmatically invoke `pytest` to run all tests within the `langchain_tests` package.

import pytest
import sys

# This script demonstrates how to programmatically run tests from the langchain-tests package.
# In practice, you would typically run 'pytest' directly from your terminal.

# Ensure langchain-tests and pytest are installed:
# pip install langchain-tests pytest

# To run all tests within the 'langchain_tests' package:
# Use '--pyargs' to treat the target as a Python package, allowing pytest to discover tests.
test_target = "langchain_tests" 

print(f"Running tests for target: {test_target}")

# pytest.main() returns an exit code. sys.exit() uses this code.
# -v for verbose output.
exit_code = pytest.main(["-v", "--pyargs", test_target])

if exit_code != 0:
    print(f"Tests failed with exit code {exit_code}")
else:
    print("All specified tests passed!")

# You can also run specific test modules:
# exit_code_specific = pytest.main(["-v", "--pyargs", "langchain_tests.unit_tests.test_retrievers"])
# if exit_code_specific != 0: print("Specific retriever tests failed.")

view raw JSON →