{"id":7791,"library":"tinker","title":"Tinker Python SDK","description":"Tinker is the official Python SDK for the Tinker API, designed for fine-tuning large language models (LLMs). It abstracts away the complexities of distributed GPU training, allowing developers to focus on data and algorithms. The current version is 0.18.0, and it is actively maintained with ongoing development and documentation updates.","status":"active","version":"0.18.0","language":"en","source_language":"en","source_url":"https://github.com/thinking-machines-lab/tinker","tags":["LLM","fine-tuning","AI","SDK","API","machine learning"],"install":[{"cmd":"pip install tinker","lang":"bash","label":"Install stable release"},{"cmd":"uv pip install tinker","lang":"bash","label":"Install with uv (recommended by Tinker docs)"}],"dependencies":[{"reason":"Required for training functionalities, but is an optional dependency.","package":"torch","optional":true}],"imports":[{"note":"The primary entry point is typically accessed via the top-level 'tinker' module after import.","wrong":"from tinker import ServiceClient","symbol":"ServiceClient","correct":"import tinker\nclient = tinker.ServiceClient()"}],"quickstart":{"code":"import os\nimport tinker\nfrom tinker import types\n\nos.environ['TINKER_API_KEY'] = os.environ.get('TINKER_API_KEY', 'your_tinker_api_key_here')\n\n# Initialize the Tinker ServiceClient\nservice_client = tinker.ServiceClient()\n\n# Create a LoRA training client (example for fine-tuning)\ntraining_client = service_client.create_lora_training_client(\n    base_model=\"meta-llama/Llama-3.2-1B\",\n    rank=32,\n)\n\n# Example of an asynchronous operation (replace with actual data and loss_fn)\nasync def run_optim_step():\n    # In a real scenario, you would have actual data and define a loss function\n    # For this quickstart, we'll simulate a minimal Datum and OptimStepRequest\n    dummy_model_input = types.ModelInput(\n        text=\"This is a dummy prompt.\", \n        tokens=[1, 2, 3]\n    )\n    dummy_loss_fn_inputs = {\"labels\": types.TensorData(data_float=[1.0, 2.0])}\n    datum = types.Datum(model_input=dummy_model_input, loss_fn_inputs=dummy_loss_fn_inputs)\n    \n    optim_request = types.OptimStepRequest(\n        datums=[datum],\n        # Other required fields like loss_fn, optim_params, etc. would go here\n        # This is a simplified example; refer to full docs for actual usage\n        loss_fn=types.LossFunction.CROSS_ENTROPY,\n        optim_params=types.AdamParams(learning_rate=1e-5)\n    )\n    optim_future = await training_client.optim_step_async(optim_request)\n    # await optim_future.get_result_async()\n    print(\"Optim step initiated.\")\n\nimport asyncio\nasyncio.run(run_optim_step())","lang":"python","description":"This quickstart demonstrates how to initialize the Tinker SDK, set up a service client, and create a LoRA training client. It also includes a basic asynchronous example for initiating an optimizer step, highlighting the typical pattern for interacting with the Tinker API. A Tinker API key must be set as an environment variable (TINKER_API_KEY)."},"warnings":[{"fix":"Update your code to use the new field names (`header`, `output`, `stop_overlap`) when working with `RenderedMessage` objects. Adapt `Renderer` implementations to inherit from `ABC`.","message":"The `RenderedMessage` fields `prefix`, `content`, and `suffix` were renamed to `header`, `output`, and `stop_overlap` respectively. The `Renderer` interface also changed from `Protocol` to `ABC`.","severity":"breaking","affected_versions":"0.x.x (exact version for breaking change not specified, but occurred in tinker-cookbook changelog prior to current)"},{"fix":"Always use the `_async` variants of API calls and use `asyncio.gather` for concurrent requests. For example, use `sampling_client.sample_async()` instead of `sampling_client.sample()` in loops.","message":"Making sequential API calls instead of utilizing asynchronous patterns is a major performance bottleneck, especially for operations like `sample` or `optim_step`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always create a new sampling client after saving model weights (`training_client.save_weights_and_get_sampling_client()`) to ensure it reflects the latest model state.","message":"A sampling client created before saving new weights will silently sample from old, stale weights, leading to unexpected model behavior.","severity":"gotcha","affected_versions":"All versions"},{"fix":"When using LoRA, consult `hyperparam_utils.get_lr(model_name)` or experiment with learning rates approximately 10 times higher than you would for full fine-tuning.","message":"LoRA fine-tuning typically requires a significantly higher learning rate compared to full fine-tuning, often around 10x higher.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Set the `TINKER_API_KEY` environment variable before running your application. For example: `export TINKER_API_KEY=\"your_api_key_here\"` in your shell, or `os.environ['TINKER_API_KEY'] = 'your_api_key_here'` in your Python script (for testing, not recommended for production).","cause":"The `TINKER_API_KEY` environment variable is not set, which is required for authenticating with the Tinker API.","error":"KeyError: 'TINKER_API_KEY'"},{"fix":"Ensure all calls to asynchronous Tinker methods are preceded by `await`. For example, `await client.some_async_method(...)`.","cause":"An asynchronous Tinker API call (ending with `_async`) was made but not `await`ed, leading to a coroutine object not being executed.","error":"TypeError: 'coroutine' object is not awaited"},{"fix":"Check your internet connection and verify the Tinker API endpoint configuration if specified. If the issue persists, check the Tinker status page or contact support, as it might be a server-side problem. For timeouts, consider increasing the client timeout if it's a known long-running operation, though very long timeouts might indicate server-side hangs.","cause":"These errors indicate network issues or problems reaching the Tinker API server, potentially due to connectivity problems, incorrect endpoint configuration, or server-side issues.","error":"APIConnectionError or APITimeoutError"}]}