GPTCache

0.1.44 · active · verified Tue Apr 14

GPTCache is a powerful caching library designed to speed up and lower the cost of chat applications that rely on Large Language Model (LLM) services. It functions as a semantic cache, storing and retrieving responses for similar (not just exact) queries using embedding algorithms and vector stores. The library is actively maintained with frequent minor releases.

Warnings

gotcha When integrating with LangChain, particularly with Pydantic v2, older versions of GPTCache might have caused 'metaclass conflict errors' or 'LangChain chat pydantic bugs'.
Fix: Upgrade GPTCache to version 0.1.43 or newer to resolve known compatibility issues with Pydantic v2 and LangChain.
gotcha Using certain features like remote Redis cache stores or distributed caching might require explicit installation of `redis` and can encounter connection issues in older versions.
Fix: Ensure `pip install gptcache[redis]` if you plan to use Redis. Upgrade to at least 0.1.36 for critical Redis connection fixes, and 0.1.43 to benefit from `redis` being an optional dependency, avoiding unnecessary installs.
gotcha Changes in external LLM APIs (e.g., OpenAI's API base for embeddings) can cause unexpected behavior or errors if GPTCache is not updated to reflect these changes.
Fix: Regularly update GPTCache to its latest version to ensure compatibility with evolving LLM APIs. Version 0.1.38 addressed specific OpenAI API base changes.

Install

pip install gptcache Install core library
pip install gptcache[openai] Install with OpenAI support
pip install gptcache[langchain] Install with LangChain support
pip install gptcache[redis] Install with Redis support

Imports

cache
```
from gptcache import cache
```
Commonly used pre-configured global cache instance for quick setup.
GPTCache
```
from gptcache import GPTCache
```
The main class for creating a GPTCache instance, allowing custom configuration.
openai
```
from gptcache.adapter import openai
```
Adapter to integrate GPTCache with the OpenAI API calls.

Quickstart

This quickstart demonstrates how to integrate GPTCache with the OpenAI API. After initializing the cache, subsequent OpenAI calls will automatically leverage the semantic caching capabilities. The first query will likely go to the LLM, while identical or semantically similar subsequent queries will be served from the cache.

import os
from gptcache import cache
from gptcache.adapter import openai

# Set your OpenAI API key from an environment variable
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-...")

# Initialize GPTCache
cache.init()

# The gptcache.adapter.openai module automatically wraps the openai library
# Subsequent OpenAI API calls will use the cache
response1 = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, what is the capital of France?"}
    ]
)
print(f"First response (likely from LLM): {response1.choices[0].message.content}")

# A second identical request will hit the cache for faster response and cost savings
response2 = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, what is the capital of France?"}
    ]
)
print(f"Second response (likely from cache): {response2.choices[0].message.content}")

view raw JSON →