{"id":6700,"library":"leptonai","title":"Lepton AI Platform","description":"The LeptonAI Python library is a framework designed to simplify AI service building, enabling developers to convert research and modeling code into production-ready services with minimal Python. It provides abstractions for launching models from platforms like HuggingFace, includes AI-tailored features such as autobatching and background jobs, and offers a Python client for interacting with deployed services as native functions. The library also comes with a command-line interface (`lep`) for local and cloud service management. Currently at version 0.27.0, it maintains an active development and release cadence with frequent updates.","status":"active","version":"0.27.0","language":"en","source_language":"en","source_url":"https://github.com/leptonai/leptonai","tags":["AI","MLOps","serverless","HuggingFace","model deployment","inference","photon","client"],"install":[{"cmd":"pip install -U leptonai","lang":"bash","label":"Install core library and CLI"}],"dependencies":[],"imports":[{"symbol":"Client, local","correct":"from leptonai.client import Client, local"},{"note":"Used for defining custom AI services (Photons).","symbol":"Photon","correct":"from leptonai.photon import Photon"}],"quickstart":{"code":"import os\nfrom leptonai.client import Client, local\n\n# NOTE: This example assumes you have run 'lep photon runlocal --name gpt2 --model hf:gpt2' in your terminal.\n# It also requires the Lepton AI CLI to be installed and potentially HuggingFace credentials\n# if you're using models that require them (set HUGGING_FACE_HUB_TOKEN env var).\n\n# Initialize client for local service running on port 8080\nc = Client(local(port=8080))\n\n# Check available paths (endpoints) of the deployed service\nprint(f\"Available paths: {c.paths()}\")\n\n# Call the 'run' method of the gpt2 model\nresponse = c.run(inputs=\"I enjoy walking with my cute dog, and\")\nprint(f\"Model response: {response}\")\n\n# Example of using a remote client (replace with your actual workspace/token/deployment)\n# from leptonai.client import Client\n# LEPTIN_API_TOKEN = os.environ.get('LEPTON_API_TOKEN', 'YOUR_LEPTON_API_TOKEN')\n# # If you have a specific workspace_id and deployment_id\n# # remote_client = Client(f\"workspace_id\", \"deployment_id\", token=LEPTIN_API_TOKEN)\n# # Or if using a direct URL:\n# # remote_client = Client(\"https://your-deployment-url.lepton.run/api\", token=LEPTIN_API_TOKEN)\n# # response = remote_client.run(inputs=\"What is Lepton AI?\")\n# # print(f\"Remote model response: {response}\")\n","lang":"python","description":"This quickstart demonstrates how to interact with a locally running Lepton AI service (a \"Photon\") using the Python client. First, you typically launch a model (e.g., a HuggingFace GPT-2 model) using the `lep` CLI tool (`lep photon runlocal --name gpt2 --model hf:gpt2`). Then, the Python client can connect to this local service to send inputs and receive outputs. For remote deployments, an API token and deployment details would be required."},"warnings":[{"fix":"Check LeptonAI's official documentation for supported HuggingFace pipelines. For unsupported models, you may need to define a custom Photon.","message":"Not all HuggingFace models are supported out-of-the-box. Models containing custom code or non-standard pipelines may not function directly with LeptonAI's abstractions.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use the `-hf` suffix in the model name when deploying Llama2 models via HuggingFace on LeptonAI.","message":"When attempting to use Llama2 models, it is crucial to specify the HuggingFace-compatible version (e.g., `hf:meta-llama/Llama-2-7b-chat-hf`) to ensure proper integration with LeptonAI's pipelines.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always refer to the specific example's documentation or `requirements.txt` for additional dependencies. It is highly recommended to use Python virtual environments to manage these dependencies.","message":"While `pip install leptonai` installs the core library, many advanced examples or specific AI models (e.g., for image generation, specific LLMs) require additional Python dependencies. These are often not automatically installed with the base package.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure you are logged in via `lep login` for CLI operations or provide necessary API tokens as environment variables or directly to the client for programmatic access.","message":"Accessing remote Lepton AI services or certain gated HuggingFace models often requires authentication. This typically involves using the `lep login` command via the CLI or setting API tokens (e.g., `LEPTON_API_TOKEN`, `HUGGING_FACE_HUB_TOKEN`) as environment variables.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Monitor official announcements from Lepton AI and Nvidia for information on migration paths, new terms, or changes to the platform's independent availability.","message":"Reports indicate a potential acquisition of Lepton AI by Nvidia and rebranding to DGX Cloud Lepton. This may lead to changes in platform access, features, and pricing models for standalone users in future versions.","severity":"breaking","affected_versions":"Future versions beyond 0.27.0"}],"env_vars":null,"last_verified":"2026-04-15T00:00:00.000Z","next_check":"2026-07-14T00:00:00.000Z","problems":[]}