lmcache
JSON →lmcache is a Python library that provides an LLM serving engine extension. It aims to reduce Time To First Token (TTFT) and increase throughput, particularly in scenarios involving long contexts. The current version is 0.4.3, and it appears to have an active development cadence.
Traffic · last 30 days ↑350% vs prev 7d
total hits 17
actors 7 distinct systems
last hit 22h ago human
top countries 🇺🇸 United States · 🇨🇦 Canada · 🇫🇷 France · 🇩🇪 Germany · VN
API endpoints
full doc /v1/registry/lmcache
install /v1/registry/lmcache/install
compatibility /v1/registry/lmcache/compatibility