OpenXLab Python SDK

0.1.3 · active · verified Sun Apr 05

OpenXLab is a Python library providing tools to interact with the OpenXLab platform, focusing on managing datasets and models. It includes a command-line interface (CLI) and a Python SDK to programmatically access platform features such as dataset information, upload, and download. The library is currently at version 0.1.3 and appears to have a regular release cadence, with updates often reflecting platform-level changes within the broader OpenXLab ecosystem.

Warnings

gotcha Older versions of `openxlab` (e.g., 0.0.37) have been reported to have `setuptools` dependency conflicts (requiring `~=60.2.0` while other packages might need newer versions).
Fix: While potentially resolved in newer versions, if you encounter `setuptools` conflicts, try isolating `openxlab` in a virtual environment or explicitly pinning `setuptools` to a compatible version as a temporary workaround. Check release notes for explicit dependency updates.
gotcha Authentication (Access Key and Secret Key) is crucial. If not configured via `openxlab config`, `~/.openxlab/config.json`, or environment variables (`OPENXLAB_AK`, `OPENXLAB_SK`), API calls will fail. Direct `openxlab.login` or SDK calls can also fail with 503 Server Error if the backend service is unavailable.
Fix: Always ensure your `OPENXLAB_AK` and `OPENXLAB_SK` environment variables are correctly set and valid, or that `openxlab config` has been run. Verify network connectivity to the OpenXLab API endpoints. For programmatic access, using environment variables is the most recommended and secure approach.
gotcha The `openxlab` PyPI package and its GitHub presence can be confusing due to similar names with other projects (e.g., `OpenXLA`, `openclaw`) and its integration within a broader OpenXLab/OpenDataLab ecosystem. The primary source for the Python SDK functionality for datasets and models resides within modules imported from `openxlab`.
Fix: Refer to the official OpenXLab documentation (浦源-文档中心) for the most accurate and up-to-date SDK usage patterns and import paths. Confirm you are installing the correct `openxlab` package (`pip install openxlab`) if interacting with the OpenXLab platform.

Install

pip install openxlab Install latest version

Imports

get
```
from openxlab.dataset import get
```

upload_folder

from openxlab.dataset import upload_folder

get_jwt
```
from openxlab.xlab.handler.user_token import get_jwt
```
The token utility is nested within `xlab.handler.user_token`.

Quickstart

This quickstart demonstrates how to authenticate with OpenXLab using environment variables for Access Key (AK) and Secret Key (SK), and then how to use the `openxlab.dataset.get` function to download a public dataset. Replace 'YOUR_AK_HERE' and 'YOUR_SK_HERE' with your actual credentials, or ideally, set them as environment variables (OPENXLAB_AK, OPENXLAB_SK). The example attempts to download the 'OpenDataLab/MNIST' dataset, which is a common public dataset.

import os
from openxlab.xlab.handler.user_token import get_jwt
from openxlab.dataset import get

# Set your Access Key and Secret Key as environment variables
# These can be obtained from your OpenXLab user center.
# For example: export OPENXLAB_AK="YOUR_AK_HERE"
# For example: export OPENXLAB_SK="YOUR_SK_HERE"
ak = os.environ.get('OPENXLAB_AK', 'YOUR_AK_HERE')
sk = os.environ.get('OPENXLAB_SK', 'YOUR_SK_HERE')

if ak == 'YOUR_AK_HERE' or sk == 'YOUR_SK_HERE':
    print("Please set OPENXLAB_AK and OPENXLAB_SK environment variables or replace placeholders.")
    # Attempt to use CLI config if env vars are not set for demonstration purposes
    # In a real scenario, user would run 'openxlab config' or set env vars.
    # For programmatic access, env vars are preferred.
else:
    try:
        # Authenticate and get a JWT token
        token = get_jwt(ak=ak, sk=sk)
        print(f"Successfully obtained JWT token (first 10 chars): {token[:10]}...")

        # Example: Get (download) a public dataset
        # Replace 'OpenDataLab/MNIST' with a valid dataset repository and adjust target_path
        # Ensure the target_path exists or can be created.
        dataset_repo_name = 'OpenDataLab/MNIST'
        target_download_path = './MNIST_dataset'
        print(f"\nAttempting to download dataset: {dataset_repo_name} to {target_download_path}")
        
        # Create a dummy directory for download if it doesn't exist
        os.makedirs(target_download_path, exist_ok=True)

        # Note: The 'get' function might automatically use configured AK/SK,
        # but explicit authentication often precedes other SDK calls.
        get(dataset_repo=dataset_repo_name, target_path=target_download_path)
        print(f"Dataset '{dataset_repo_name}' downloaded to '{target_download_path}' successfully.")

    except Exception as e:
        print(f"An error occurred: {e}")
        print("Ensure your AK/SK are correct and you have network access to OpenXLab.")

view raw JSON →