lmcache
JSON →lmcache is a Python library that provides an LLM serving engine extension. It aims to reduce Time To First Token (TTFT) and increase throughput, particularly in scenarios involving long contexts. The current version is 0.4.3, and it appears to have an active development cadence.
Traffic · last 30 days stale · no recent hits
total hits 13
actors 4 distinct systems
last hit 17d ago AhrefsBot
top countries 🇺🇸 United States · 🇨🇦 Canada · 🇩🇪 Germany
API endpoints
full doc /v1/registry/lmcache
install /v1/registry/lmcache/install
compatibility /v1/registry/lmcache/compatibility