{"library":"sglang","code":"import os\nfrom openai import OpenAI\nimport time\n\n# --- Step 1: Launch SGLang Server (Run this in a separate terminal) ---\n# Command: python -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --port 30000\n# Note: Replace 'meta-llama/Llama-3.1-8B-Instruct' with a model you have access to\n# and ensure you have logged into Hugging Face CLI if it's a gated model.\n# Server output will indicate when it's ready, e.g., 'Uvicorn running on http://0.0.0.0:30000'\n\n# --- Step 2: Interact with the SGLang server using OpenAI-compatible client ---\n# Wait a moment for the server to start, or adjust the sleep duration\ntime.sleep(5) \n\nclient = OpenAI(\n    base_url=os.environ.get('SGLANG_SERVER_URL', 'http://localhost:30000/v1'),\n    api_key=os.environ.get('SGLANG_API_KEY', 'EMPTY') # 'EMPTY' is common for local SGLang instances\n)\n\ntry:\n    response = client.chat.completions.create(\n        model=\"meta-llama/Llama-3.1-8B-Instruct\", # Model name must match server's loaded model\n        messages=[\n            {\"role\": \"user\", \"content\": \"What is the capital of France?\"}\n        ],\n        max_tokens=50,\n        stream=False\n    )\n    print(\"Response from SGLang server:\", response.choices[0].message.content)\nexcept Exception as e:\n    print(f\"Error connecting to SGLang server or making request: {e}\")\n    print(\"Please ensure the SGLang server is running in a separate terminal.\")","lang":"python","description":"This quickstart demonstrates how to interact with an SGLang server using the OpenAI Python client. First, launch the SGLang server in a separate terminal, specifying the model to serve. Then, use the provided Python script to connect to this server and send a chat completion request. Remember to replace the example model path with a valid one and handle Hugging Face authentication if using gated models.","tag":null,"tag_description":null,"last_tested":"2026-04-24","results":[{"runtime":"python:3.10-alpine","exit_code":-1},{"runtime":"python:3.10-slim","exit_code":-1},{"runtime":"python:3.11-alpine","exit_code":-1},{"runtime":"python:3.11-slim","exit_code":-1},{"runtime":"python:3.12-alpine","exit_code":1},{"runtime":"python:3.12-slim","exit_code":-1},{"runtime":"python:3.13-alpine","exit_code":1},{"runtime":"python:3.13-slim","exit_code":-1},{"runtime":"python:3.9-alpine","exit_code":1},{"runtime":"python:3.9-slim","exit_code":1}]}