Cog (Replicate)

0.17.2 · active · verified Fri Apr 10

Cog is an open-source tool for packaging machine learning models into standard Docker containers. It allows you to define a Python `Predictor` class with `setup` and `predict` methods, which Cog then uses to build a Docker image for local testing or deployment to platforms like Replicate. The current version is 0.17.2, and it receives frequent minor updates with occasional major releases introducing significant architectural changes.

Warnings

breaking In v0.17.0+, `setup()` and `predict()` methods are synchronous by default. If your model's operations are truly asynchronous (e.g., awaiting external I/O), you *must* explicitly declare these methods as `async def`.
Fix: Prefix `def` with `async` (e.g., `async def setup(self):`) for asynchronous operations. Ensure your code properly `await`s async calls within these methods.
breaking In v0.17.0+, `Input()` arguments without a default value now default to `None` if not provided by the client, instead of Pydantic raising an error. This can lead to `None` type errors in your `predict` method if you expect a non-null value.
Fix: Either provide a default value to `Input()` (e.g., `Input(default=...)`) or explicitly handle `None` checks in your `predict` method for any input that might be optional.
gotcha Cog relies on Docker to build and run model containers. Docker Desktop (or equivalent Docker engine) must be installed and running on your system for `cog build`, `cog push`, and local `cog predict` commands to function.
Fix: Install Docker Desktop (for Windows/macOS) or Docker Engine (for Linux) and ensure the Docker daemon is running before using Cog commands.
gotcha The `cog.yaml` configuration file is crucial for defining your model's environment (e.g., Python version, system packages, `requirements.txt` path). Incorrect or missing configuration often leads to build failures or runtime errors within the container.
Fix: Always initialize `cog.yaml` using `cog init` and carefully specify `python_version`, `build.python_packages`, `build.system_packages`, and `build.python_version` as needed for your model. Double-check paths to `requirements.txt` and other build files.
deprecated The `extra_model_headers` field in `cog.yaml` has been removed in v0.17.0+ and will cause an error if present.
Fix: Remove `extra_model_headers` from your `cog.yaml` file. If you relied on this for custom headers, you may need to implement a custom HTTP handler or find an alternative approach.

Install

pip install cog Install Cog Python SDK
curl -o /usr/local/bin/cog -L 'https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m)' && chmod +x /usr/local/bin/cog Install Cog CLI (Linux/macOS)

Imports

BasePredictor
```
from cog import BasePredictor
```
Input
```
from cog import Input
```
Path
```
from cog import Path
```
Used for file inputs/outputs, typically `pathlib.Path` or `str` representations of paths.
replicate
```
from cog import replicate
```
Used for programmatic interaction with the Replicate API, not for defining models.

Quickstart

Define a `Predictor` class inheriting from `BasePredictor`. Implement `setup()` to load your model and `predict()` to handle inference. `Input` specifies prediction inputs with types, descriptions, and defaults. `Path` is used for file-based inputs/outputs.

from cog import BasePredictor, Input, Path
import torch

class Predictor(BasePredictor):
    def setup(self):
        """Load the model into memory to make running multiple predictions efficient"""
        # Example: self.model = torch.load("./weights.pth")
        self.model = "a dummy model"

    def predict(
        self,
        text_input: str = Input(description="A text input"),
        scale: float = Input(description="Factor to scale by", default=1.5)
    ) -> str:
        """Run a single prediction on the model"""
        # Example: processed_input = self.preprocess(text_input)
        # output = self.model(processed_input, scale)
        output = f"Processed '{text_input}' with scale {scale}"
        return output

# To run locally with Cog CLI:
# 1. Create a cog.yaml file: `cog init`
# 2. Add dependencies (e.g., torch) to requirements.txt
# 3. Run prediction: `cog predict -i text_input="hello" -i scale=2.0`

view raw JSON →