{"library":"sglang-router","title":"SGLang Router","description":"SGLang Router (PyPI: `sglang-router`, current version 0.3.2) is a high-performance, Rust-based load balancer designed for SGLang instances, facilitating data parallelism and advanced request routing. It supports multiple load balancing algorithms, including cache-aware, power of two, random, and round robin, and is specialized for prefill-decode disaggregated serving architectures. The project has been evolving into the 'SGLang Model Gateway', aiming to become a full OpenAI API server with features like native tool calling and session management. It maintains an active development pace with frequent updates and bug fixes.","language":"python","status":"active","last_verified":"Sun May 17","install":{"commands":["pip install sglang-router"],"cli":null},"imports":["from sglang_router import Router"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import os\nfrom sglang_router import Router\n\n# NOTE: This quickstart assumes SGLang worker instances are running\n# at the specified URLs (e.g., http://localhost:8000).\n# Replace with actual worker URLs if available.\nworker_urls = [\n    os.environ.get('SGLANG_WORKER_URL_1', 'http://localhost:8000'),\n    os.environ.get('SGLANG_WORKER_URL_2', 'http://localhost:8001')\n]\n\ntry:\n    # Initialize the SGLang Router\n    # By default, it runs in regular HTTP routing mode.\n    router = Router(worker_urls=worker_urls)\n    print(f\"SGLang Router initialized with workers: {worker_urls}\")\n\n    # In a real application, you would typically start the router\n    # (e.g., in a separate thread or process) and then send requests to it.\n    # For demonstration, we just show initialization.\n\n    # Example of running the router process (requires a running event loop or main function)\n    # This part is conceptual as `Router` doesn't expose a simple `run()` method directly\n    # in this programmatic interface; it's often launched via `python -m`.\n    print(\"Router instance created. To run, typically use 'python -m sglang_router.launch_router'\\n\"+\n          \"or integrate into an ASGI app. Refer to SGLang documentation for full deployment.\")\n\nexcept Exception as e:\n    print(f\"Error initializing SGLang Router: {e}\")\n    print(\"Ensure SGLang worker instances are running and accessible at the provided URLs.\")\n","lang":"python","description":"This quickstart demonstrates how to programmatically initialize the SGLang Router by providing a list of SGLang worker URLs. The router acts as a load balancer for these workers. For actual deployment and to handle incoming requests, the router is often launched as a separate process using the `python -m sglang_router.launch_router` command, or integrated into an ASGI application. Ensure that SGLang worker instances are running and accessible at the specified URLs for the router to function correctly.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-17","installed_version":"0.3.2","pypi_latest":"0.3.2","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":7.7,"avg_import_s":null,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"145.3M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":8.9,"import_time_s":null,"mem_mb":null,"disk_size":"132M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"149.5M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":7.6,"import_time_s":null,"mem_mb":null,"disk_size":"136M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"141.0M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":5.4,"import_time_s":null,"mem_mb":null,"disk_size":"128M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"140.7M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":5.9,"import_time_s":null,"mem_mb":null,"disk_size":"128M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":"145.3M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"sglang-router","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":10.5,"import_time_s":null,"mem_mb":null,"disk_size":"132M"}]}}