Semantic Router

0.1.12 · active · verified Wed Apr 15

Semantic Router is a superfast decision-making layer for LLMs and agents. It routes requests based on semantic meaning using vector space, rather than relying on slow LLM generations for tool-use or safety decisions. It supports various embedding models (e.g., OpenAI, Cohere, Hugging Face) and integrates with vector databases like Pinecone and Qdrant. The library is actively maintained with frequent updates and a focus on speed, safety, and scalability.

Warnings

breaking The `RouteLayer` class has been renamed to `SemanticRouter` and moved from the top-level `semantic_router` module to `semantic_router.routers`.
Fix: Update imports from `from semantic_router import RouteLayer` to `from semantic_router.routers import SemanticRouter`.
deprecated The `retrieve_multiple_routes` method on the router has been removed.
Fix: For similar functionality, you might use `router.__call__(limit=None)` or `router.acall(limit=None)`, or the deprecated `_semantic_classify_multiple_routes` in versions 0.1.0-0.1.2. Consult documentation for the specific version you are using.
gotcha Many encoders (e.g., OpenAIEncoder, CohereEncoder) require API keys set as environment variables (e.g., `OPENAI_API_KEY`). The library will not function correctly without these being properly configured.
Fix: Set the required API keys as environment variables before initializing the corresponding encoder, e.g., `os.environ["OPENAI_API_KEY"] = "your_key_here"`.
gotcha Specific features like using local models (`HuggingFaceEncoder`, `LlamaCppLLM`) or hybrid routing (`HybridRouter`) require extra installation commands (e.g., `pip install "semantic-router[local]"`).
Fix: Install the necessary optional dependencies using the `pip install "semantic-router[extra]"` syntax for the features you intend to use.
gotcha To prevent incorrect routing decisions, especially for queries that don't strongly match any defined route, it is crucial to set an appropriate similarity threshold.
Fix: Configure the `threshold` parameter during router initialization. For dynamic thresholding, ensure `OPENAI_API_KEY` is set if using an `OpenAIEncoder`.

Install

pip install semantic-router Standard Installation
pip install "semantic-router[local]" Local Models (HuggingFaceEncoder, LlamaCppLLM)
pip install "semantic-router[hybrid]" Hybrid Routing
pip install "semantic-router[cohere]" Cohere Encoder
pip install "semantic-router[openai]" OpenAI Encoder

Imports

Route
```
from semantic_router import Route
```
SemanticRouter
wrong:
```
from semantic_router import RouteLayer
```
correct:
```
from semantic_router.routers import SemanticRouter
```
`RouteLayer` was renamed to `SemanticRouter` and moved to `semantic_router.routers` in versions 0.1.3+.

OpenAIEncoder

from semantic_router.encoders import OpenAIEncoder

CohereEncoder

from semantic_router.encoders import CohereEncoder

LocalIndex

from semantic_router.index import LocalIndex

HybridRouter

from semantic_router.routers import HybridRouter

Quickstart

This quickstart demonstrates how to define semantic routes, initialize an encoder (OpenAI in this case), create a router with a local index, and then use it to categorize incoming queries based on their semantic meaning. Ensure your OpenAI API key is set as an environment variable.

import os
from semantic_router import Route
from semantic_router.encoders import OpenAIEncoder
from semantic_router.routers import SemanticRouter
from semantic_router.index import LocalIndex

# Ensure API key is set or prompt for it
if not os.getenv("OPENAI_API_KEY"): #
    print("Please set the OPENAI_API_KEY environment variable.")
    # In a real application, you might raise an error or use getpass
    os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-YOUR_OPENAI_KEY_HERE")

# 1. Define routes
politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
        "they're going to destroy this country!"
    ]
)
chitchat = Route(
    name="chitchat",
    utterances=[
        "how's the weather today?",
        "how are things going?",
        "lovely weather today"
    ]
)
routes = [politics, chitchat]

# 2. Initialize an encoder (e.g., OpenAIEncoder)
encoder = OpenAIEncoder()

# 3. Create a SemanticRouter instance with an index (e.g., LocalIndex)
# For persistent storage, consider PineconeIndex, QdrantIndex, etc.
router = SemanticRouter(
    encoder=encoder,
    routes=routes,
    index=LocalIndex()
)

# 4. Route a query
query1 = "What do you think about the government?"
route_result1 = router(query1)
print(f"Query: '{query1}' -> Routed to: {route_result1.name} (Score: {route_result1.score:.2f})")

query2 = "How's life treating you?"
route_result2 = router(query2)
print(f"Query: '{query2}' -> Routed to: {route_result2.name} (Score: {route_result2.score:.2f})")

query3 = "Tell me a story."
route_result3 = router(query3) # Should return None if no match above threshold
print(f"Query: '{query3}' -> Routed to: {route_result3.name if route_result3 else 'None'} (Score: {route_result3.score:.2f} if route_result3 else 'N/A')")

view raw JSON →