{"library":"runai-model-streamer","title":"Run:ai Model Streamer","description":"The Run:ai Model Streamer is an open-source Python SDK designed to accelerate the loading of large AI models onto accelerators, such as GPUs or TPUs. It achieves this by streaming tensors directly from various storage locations (local, S3, GCS, Azure Blob Storage) to GPU memory, bypassing local disk buffering, and optimizing for the SafeTensors file format. The current version is 0.15.8, with releases occurring somewhat regularly, indicating active development.","language":"python","status":"active","last_verified":"Sat May 16","install":{"commands":["pip install runai-model-streamer","pip install runai-model-streamer-gcs","pip install runai-model-streamer-s3","pip install runai-model-streamer-azure","pip install vllm[runai]"],"cli":null},"imports":["from runai_model_streamer import SafetensorsStreamer"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import os\nfrom runai_model_streamer import SafetensorsStreamer\n\n# This is a placeholder for a safetensors file. In a real scenario,\n# 'model.safetensors' would be a path to your model file.\n# For a runnable example, you might create a dummy file or adapt to a real path.\n# If streaming from cloud storage, ensure appropriate backend package is installed\n# and environment variables for authentication are set (e.g., GOOGLE_APPLICATION_CREDENTIALS for GCS).\n# For local testing, ensure 'model.safetensors' exists or is mocked.\n\n# Example of creating a dummy safetensors file for local quickstart demonstration\ntry:\n    from safetensors.torch import save_file\n    import torch\n    dummy_tensor = {'tensor_key': torch.randn(10, 10)}\n    save_file(dummy_tensor, 'model.safetensors')\n    file_path = \"model.safetensors\"\n\n    print(f\"Attempting to stream from: {file_path}\")\n\n    with SafetensorsStreamer() as streamer:\n        streamer.stream_file(file_path)\n        print(\"Successfully started streaming.\")\n        # In a real scenario, you would then iterate and process tensors:\n        # for name, tensor in streamer.get_tensors():\n        #    gpu_tensor = tensor.to('cuda:0') # Or another accelerator\n        #    print(f\"Streamed tensor: {name}, shape: {gpu_tensor.shape}\")\n\n    print(\"Streamer context closed.\")\nexcept ImportError:\n    print(\"To run this quickstart with a dummy file, install 'safetensors' and 'torch':\")\n    print(\"pip install safetensors torch\")\n    print(\"Alternatively, replace 'model.safetensors' with an actual path to your model file.\")\nexcept Exception as e:\n    print(f\"An error occurred during quickstart: {e}\")\n    print(\"Ensure the file_path is correct and necessary system libraries (libcurl4, libssl1.1_1) are installed.\")\n    print(\"If streaming from cloud storage, verify environment variables for authentication are set.\")\n","lang":"python","description":"This quickstart demonstrates how to use `SafetensorsStreamer` to initiate streaming of a model. It creates a dummy `safetensors` file for a runnable example. For actual use, `file_path` should point to your model. When working with cloud storage (S3, GCS, Azure), ensure the respective `runai-model-streamer-*` package is installed and authentication environment variables (e.g., `GOOGLE_APPLICATION_CREDENTIALS` for GCS, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` for S3, `AZURE_CLIENT_ID` for Azure) are correctly configured. The `stream_file` method starts the streaming process, and `get_tensors()` can then be used to retrieve tensors.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-16","installed_version":"0.15.9","pypi_latest":"0.15.9","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":38,"avg_install_s":17.5,"avg_import_s":5.75,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"runai-model-streamer-azure","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"runai-model-streamer-gcs","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"runai-model-streamer-s3","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"runai-model-streamer","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":75.8,"import_time_s":3.87,"mem_mb":64,"disk_size":"4.7G"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"runai-model-streamer-azure","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.2,"import_time_s":null,"mem_mb":null,"disk_size":"197M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"runai-model-streamer-gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":5.7,"import_time_s":null,"mem_mb":null,"disk_size":"299M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"runai-model-streamer-s3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.2,"import_time_s":null,"mem_mb":null,"disk_size":"208M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"runai-model-streamer-azure","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"runai-model-streamer-gcs","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"runai-model-streamer-s3","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"runai-model-streamer","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":68.1,"import_time_s":6.69,"mem_mb":69.1,"disk_size":"4.8G"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"runai-model-streamer-azure","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3.9,"import_time_s":null,"mem_mb":null,"disk_size":"206M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"runai-model-streamer-gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":5.2,"import_time_s":null,"mem_mb":null,"disk_size":"308M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"runai-model-streamer-s3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3.7,"import_time_s":null,"mem_mb":null,"disk_size":"216M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"runai-model-streamer-azure","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"runai-model-streamer-gcs","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"runai-model-streamer-s3","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"runai-model-streamer","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":65.1,"import_time_s":6.86,"mem_mb":68,"disk_size":"4.7G"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"runai-model-streamer-azure","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3.5,"import_time_s":null,"mem_mb":null,"disk_size":"197M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"runai-model-streamer-gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.6,"import_time_s":null,"mem_mb":null,"disk_size":"298M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"runai-model-streamer-s3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3.4,"import_time_s":null,"mem_mb":null,"disk_size":"206M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"runai-model-streamer-azure","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"runai-model-streamer-gcs","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"runai-model-streamer-s3","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"runai-model-streamer","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":57.8,"import_time_s":5.59,"mem_mb":68.5,"disk_size":"4.7G"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"runai-model-streamer-azure","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3.6,"import_time_s":null,"mem_mb":null,"disk_size":"195M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"runai-model-streamer-gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.5,"import_time_s":null,"mem_mb":null,"disk_size":"297M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"runai-model-streamer-s3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":3,"import_time_s":null,"mem_mb":null,"disk_size":"205M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"runai-model-streamer-azure","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"runai-model-streamer-gcs","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"runai-model-streamer-s3","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"runai-model-streamer","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"runai-model-streamer-azure","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.8,"import_time_s":null,"mem_mb":null,"disk_size":"197M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"runai-model-streamer-gcs","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":6.8,"import_time_s":null,"mem_mb":null,"disk_size":"299M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"runai-model-streamer-s3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"broken","install_time_s":4.6,"import_time_s":null,"mem_mb":null,"disk_size":"208M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"runai","exit_code":1,"wheel_type":null,"failure_reason":"timeout","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null}]}}