SGLang Router

SGLang Router (PyPI: `sglang-router`, current version 0.3.2) is a high-performance, Rust-based load balancer designed for SGLang instances, facilitating data parallelism and advanced request routing. It supports multiple load balancing algorithms, including cache-aware, power of two, random, and round robin, and is specialized for prefill-decode disaggregated serving architectures. The project has been evolving into the 'SGLang Model Gateway', aiming to become a full OpenAI API server with features like native tool calling and session management. It maintains an active development pace with frequent updates and bug fixes.