{"id":8181,"library":"foundry-local-sdk","title":"Foundry Local SDK","description":"Foundry Local SDK for Python provides a control-plane interface to Foundry Local, a unified on-device AI runtime that enables local generative AI inference. It allows developers to interact with locally run models, perform chat completions, and manage local AI resources without network latency or per-token costs. The current version is 1.0.0, and it has moved to General Availability, suggesting a stable, but potentially evolving, release cadence.","status":"active","version":"1.0.0","language":"en","source_language":"en","source_url":"https://github.com/microsoft/Foundry-Local","tags":["AI","LLM","local","on-device","inference","Microsoft","gen-ai","offline"],"install":[{"cmd":"pip install foundry-local-sdk","lang":"bash","label":"Install stable version"}],"dependencies":[],"imports":[{"note":"The package installs as 'foundry-local-sdk', but the primary module is `foundry`, from which `FoundryLocal` is imported.","wrong":"from foundry_local_sdk import FoundryLocal","symbol":"FoundryLocal","correct":"from foundry import FoundryLocal"}],"quickstart":{"code":"from foundry import FoundryLocal\nimport os\n\n# Ensure the Foundry Local runtime is installed and running.\n# For example, download the 'phi3-mini-4k-instruct' model via the Foundry Local CLI:\n# foundry download phi3-mini-4k-instruct\n\ntry:\n    client = FoundryLocal()\n    # You might need to explicitly download a model if not already present.\n    # client.download_model(\"phi3-mini-4k-instruct\") # Uncomment and run once if needed\n\n    print(\"FoundryLocal client initialized. Attempting chat completion...\")\n    response = client.chat.completions.create(\n        model=\"phi3-mini-4k-instruct\", # Replace with a model you have downloaded\n        messages=[\n            {\"role\": \"user\", \"content\": \"What is the capital of France?\"}\n        ],\n        max_tokens=50\n    )\n    print(\"Response:\", response.choices[0].message.content)\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\n    print(\"Please ensure the Foundry Local runtime is installed, running, and the specified model is downloaded.\")\n","lang":"python","description":"Initializes the Foundry Local client and performs a simple chat completion using a locally available model. Before running, ensure the Foundry Local runtime is installed and running, and a model (e.g., `phi3-mini-4k-instruct`) has been downloaded via the `foundry` CLI."},"warnings":[{"fix":"Install the Foundry Local runtime from official Microsoft sources and ensure it's running before using the Python SDK. Check its status via the `foundry` CLI.","message":"The `foundry-local-sdk` Python library is a client to the Foundry Local runtime. It requires the Foundry Local application/service to be installed and running on your system for any API calls to function.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Use the `foundry download <model_name>` CLI command or `client.download_model(<model_name>)` in the SDK to fetch the desired model before attempting inference.","message":"Models used with the SDK must be downloaded to your local Foundry Local instance. Attempting to use a model that hasn't been downloaded will result in a `ModelNotFoundError`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Refer to the latest Foundry Local documentation for current tool-calling capabilities and limitations. For older versions, simplify tool_choice to ensure only one tool is invoked.","message":"Early versions (pre-1.0.0, e.g., v0.8.113) had a limitation of supporting only one tool call per request. While this might be improved in newer versions, be aware of potential limitations with complex tool-calling scenarios.","severity":"gotcha","affected_versions":"<1.0.0"},{"fix":"Ensure your Foundry Local runtime and SDK are updated to the latest available version, as many such issues are resolved in subsequent releases. Check official release notes for hardware-specific fixes.","message":"Some users experienced issues with tool calling on NVIDIA GPUs with older Foundry Local runtime versions (e.g., v0.8.117). This indicates potential hardware-specific runtime bugs.","severity":"gotcha","affected_versions":"<1.0.0"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Start the Foundry Local runtime application/service. Verify its status using the `foundry status` CLI command. Ensure no firewall is blocking access to its local port.","cause":"The Foundry Local runtime service is not running or is not accessible at its default port (usually 8008 or similar). The Python SDK cannot connect to the local AI engine.","error":"ConnectionRefusedError: [Errno 111] Connection refused"},{"fix":"Download the required model using the Foundry Local CLI (`foundry download <model_name>`) or `client.download_model(<model_name>)` via the SDK. Verify the exact model name from `foundry list`.","cause":"You are trying to use a model name that has not been downloaded to your local Foundry Local instance, or the model name is misspelled.","error":"foundry.exceptions.ModelNotFoundError: Model 'my-non-existent-model' not found"},{"fix":"Change your import statement to `from foundry import FoundryLocal`.","cause":"Incorrect import statement. The top-level package for `foundry-local-sdk` exposes the main class via `foundry` not `foundry_local_sdk`.","error":"ImportError: cannot import name 'FoundryLocal' from 'foundry_local_sdk'"},{"fix":"Ensure `max_tokens` is set to a positive integer value greater than 0, corresponding to the maximum number of tokens you want the model to generate.","cause":"The `max_tokens` parameter for `chat.completions.create` was provided with a non-positive value (e.g., 0 or a negative number).","error":"ValueError: max_tokens must be positive"}]}