Lepton AI Platform

Warnings

• gotcha Not all HuggingFace models are supported out-of-the-box. Models containing custom code or non-standard pipelines may not function directly with LeptonAI's abstractions.
  Fix: Check LeptonAI's official documentation for supported HuggingFace pipelines. For unsupported models, you may need to define a custom Photon.
• gotcha When attempting to use Llama2 models, it is crucial to specify the HuggingFace-compatible version (e.g., `hf:meta-llama/Llama-2-7b-chat-hf`) to ensure proper integration with LeptonAI's pipelines.
  Fix: Always use the `-hf` suffix in the model name when deploying Llama2 models via HuggingFace on LeptonAI.
• gotcha While `pip install leptonai` installs the core library, many advanced examples or specific AI models (e.g., for image generation, specific LLMs) require additional Python dependencies. These are often not automatically installed with the base package.
  Fix: Always refer to the specific example's documentation or `requirements.txt` for additional dependencies. It is highly recommended to use Python virtual environments to manage these dependencies.
• gotcha Accessing remote Lepton AI services or certain gated HuggingFace models often requires authentication. This typically involves using the `lep login` command via the CLI or setting API tokens (e.g., `LEPTON_API_TOKEN`, `HUGGING_FACE_HUB_TOKEN`) as environment variables.
  Fix: Ensure you are logged in via `lep login` for CLI operations or provide necessary API tokens as environment variables or directly to the client for programmatic access.
• breaking Reports indicate a potential acquisition of Lepton AI by Nvidia and rebranding to DGX Cloud Lepton. This may lead to changes in platform access, features, and pricing models for standalone users in future versions.
  Fix: Monitor official announcements from Lepton AI and Nvidia for information on migration paths, new terms, or changes to the platform's independent availability.

Install

• pip install -U leptonai Install core library and CLI

Imports

• Client, local

  from leptonai.client import Client, local

• Photon

  from leptonai.photon import Photon

  Used for defining custom AI services (Photons).

Quickstart

This quickstart demonstrates how to interact with a locally running Lepton AI service (a "Photon") using the Python client. First, you typically launch a model (e.g., a HuggingFace GPT-2 model) using the `lep` CLI tool (`lep photon runlocal --name gpt2 --model hf:gpt2`). Then, the Python client can connect to this local service to send inputs and receive outputs. For remote deployments, an API token and deployment details would be required.

import os
from leptonai.client import Client, local

# NOTE: This example assumes you have run 'lep photon runlocal --name gpt2 --model hf:gpt2' in your terminal.
# It also requires the Lepton AI CLI to be installed and potentially HuggingFace credentials
# if you're using models that require them (set HUGGING_FACE_HUB_TOKEN env var).

# Initialize client for local service running on port 8080
c = Client(local(port=8080))

# Check available paths (endpoints) of the deployed service
print(f"Available paths: {c.paths()}")

# Call the 'run' method of the gpt2 model
response = c.run(inputs="I enjoy walking with my cute dog, and")
print(f"Model response: {response}")

# Example of using a remote client (replace with your actual workspace/token/deployment)
# from leptonai.client import Client
# LEPTIN_API_TOKEN = os.environ.get('LEPTON_API_TOKEN', 'YOUR_LEPTON_API_TOKEN')
# # If you have a specific workspace_id and deployment_id
# # remote_client = Client(f"workspace_id", "deployment_id", token=LEPTIN_API_TOKEN)
# # Or if using a direct URL:
# # remote_client = Client("https://your-deployment-url.lepton.run/api", token=LEPTIN_API_TOKEN)
# # response = remote_client.run(inputs="What is Lepton AI?")
# # print(f"Remote model response: {response}")