Outerbounds Metaflow Extensions

1.6.17 · active · verified Thu Apr 16

The `ob-metaflow-extensions` library provides custom plugins, decorators, and features to seamlessly integrate Metaflow workflows with the Outerbounds Platform. This includes functionalities like authentication, deployment tools, and specialized data store configurations for an Outerbounds-managed environment. It's actively maintained with frequent updates, typically aligning with Metaflow and Outerbounds Platform releases. The current version is 1.6.17.

Common errors

```
ModuleNotFoundError: No module named 'metaflow'
```
cause `ob-metaflow-extensions` requires `metaflow` as a peer dependency, but it's not explicitly listed as a direct dependency during `pip install ob-metaflow-extensions`.

fix Install `metaflow` separately: `pip install metaflow`.
```
ob_metaflow_extensions.auth.client.OuterboundsAuthException: Authentication failed.
```
cause The Outerbounds Platform could not authenticate your request, likely due to a missing or invalid authentication token/credentials.

fix Set the `OUTERBOUNDS_TOKEN` environment variable with a valid token, or use the Outerbounds CLI to log in (`ob login`) before running your flow.
```
An error occurred (403) when calling the PutObject operation: Forbidden (for S3-related operations)
```
cause Your Outerbounds workspace or the underlying AWS credentials do not have sufficient permissions to write to the specified S3 bucket, often for storing Metaflow artifacts.

fix Verify that your Outerbounds workspace and associated IAM roles have `s3:PutObject` and `s3:GetObject` permissions for the S3 bucket used for Metaflow artifacts. Check `AWS_REGION` and other AWS-related environment variables for correctness.

Warnings

gotcha Version compatibility with `metaflow` is crucial. `ob-metaflow-extensions` is built against specific Metaflow versions. Mismatches can lead to runtime errors or unexpected behavior.
Fix: Always check the `ob-metaflow-extensions` documentation or release notes for compatible `metaflow` versions. Upgrade both packages in tandem if necessary.
gotcha Deploying or running flows on the Outerbounds Platform requires proper authentication. This typically involves setting the `OUTERBOUNDS_TOKEN` environment variable or logging in via the Outerbounds CLI (`ob login`).
Fix: Ensure `OUTERBOUNDS_TOKEN` is correctly set in your environment, or run `ob login` to authenticate your CLI session before deploying or interacting with platform features.
gotcha Platform-specific features (e.g., S3 artifact storage, Kubernetes orchestration) often require additional environment variables or cluster-side configurations. Errors related to permissions or resource access are common.
Fix: Consult Outerbounds Platform documentation for required environment variables (e.g., `AWS_REGION`, `OUTERBOUNDS_NAMESPACE`) and ensure your Outerbounds workspace has the necessary permissions configured for the actions your flow performs.

Install

pip install ob-metaflow-extensions Install library
pip install metaflow Install core Metaflow (recommended)

Imports

outerbounds

from ob_metaflow_extensions.decorators import outerbounds

OuterboundsClient

from ob_metaflow_extensions.auth.client import OuterboundsClient

S3Client

from ob_metaflow_extensions.s3.s3_client import S3Client

Quickstart

This quickstart demonstrates a basic Metaflow flow using the `@outerbounds` decorator, which enables deployment to the Outerbounds Platform. It requires Metaflow to be installed and Outerbounds authentication configured (e.g., via `OUTERBOUNDS_TOKEN` environment variable or Outerbounds CLI login).

from metaflow import FlowSpec, step
from ob_metaflow_extensions.decorators import outerbounds

# Ensure OUTERBOUNDS_TOKEN environment variable is set for deployment
# or ensure you are logged in via Outerbounds CLI

@outerbounds
class MyOuterboundsFlow(FlowSpec):
    """
    A simple Metaflow flow deployable to the Outerbounds Platform.
    """
    @step
    def start(self):
        print("Starting MyOuterboundsFlow")
        self.message = "Hello from Outerbounds!"
        self.next(self.end)

    @step
    def end(self):
        print(f"MyOuterboundsFlow finished! Message: {self.message}")

if __name__ == '__main__':
    # To run locally:
    # python your_flow_file.py run
    # To deploy to Outerbounds (requires CLI login or token):
    # python your_flow_file.py deploy
    MyOuterboundsFlow()

view raw JSON →