{"id":9740,"library":"fastsafetensors","title":"High-Performance Safetensors Model Loader","description":"fastsafetensors is a Python library designed for high-performance loading of safetensors models, particularly optimized for GPU environments (CUDA, ROCm). It aims to offer faster loading times compared to the standard `safetensors` library for large models. The current version is `0.2.2`, and it maintains an active release cadence with frequent bug fixes and performance improvements.","status":"active","version":"0.2.2","language":"en","source_language":"en","source_url":"https://github.com/foundation-model-stack/fastsafetensors","tags":["safetensors","pytorch","tensorflow","paddlepaddle","gpu","ml","high-performance","model-loading","rocm"],"install":[{"cmd":"pip install fastsafetensors","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core library for safetensors format specification and basic operations.","package":"safetensors","optional":false},{"reason":"Required to load tensors as PyTorch `torch.Tensor` objects.","package":"torch","optional":true},{"reason":"Required to load tensors as TensorFlow `tf.Tensor` objects.","package":"tensorflow","optional":true},{"reason":"Required to load tensors as PaddlePaddle `paddle.Tensor` objects.","package":"paddlepaddle","optional":true}],"imports":[{"symbol":"FastSafetensorsFile","correct":"from fastsafetensors import FastSafetensorsFile"},{"note":"Used for advanced scenarios involving lazy tensor loading and custom tensor creation.","symbol":"LazyTensorFactory","correct":"from fastsafetensors.lazy_tensor_factory import LazyTensorFactory"}],"quickstart":{"code":"import torch\nfrom safetensors.torch import save_file\nfrom fastsafetensors import FastSafetensorsFile\nimport os\n\n# 1. Create a dummy safetensors file for demonstration\ndummy_data = {\n    \"layer1.weight\": torch.randn(128, 64),\n    \"layer1.bias\": torch.zeros(128),\n    \"layer2.weight\": torch.ones(64, 32)\n}\ndummy_file_path = \"dummy_model.safetensors\"\nsave_file(dummy_data, dummy_file_path)\n\nprint(f\"Created dummy safetensors file: {dummy_file_path}\\n\")\n\n# 2. Load the safetensors file using FastSafetensorsFile\ntry:\n    fsf = FastSafetensorsFile(dummy_file_path)\n\n    # 3. Inspect tensor metadata (does not load data into memory)\n    print(\"Tensors available in the file (metadata only):\")\n    for name, metadata in fsf.get_tensors().items():\n        print(f\"  - {name}: {metadata}\")\n\n    # 4. Access a specific tensor (this triggers loading for that tensor)\n    tensor_name = \"layer1.weight\"\n    loaded_tensor = fsf[tensor_name]\n    print(f\"\\nSuccessfully loaded '{tensor_name}':\")\n    print(f\"  Type: {type(loaded_tensor)}\")\n    print(f\"  Shape: {loaded_tensor.shape}\")\n    print(f\"  First 5 elements:\\n{loaded_tensor.flatten()[:5]}\\n\")\n\n    # Access another tensor\n    print(f\"Accessing 'layer2.weight' (shape: {fsf['layer2.weight'].shape})\\n\")\n\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\nfinally:\n    # 5. Clean up the dummy file\n    if os.path.exists(dummy_file_path):\n        os.remove(dummy_file_path)\n        print(f\"Cleaned up dummy file: {dummy_file_path}\")\n","lang":"python","description":"This quickstart demonstrates how to create a dummy safetensors file using the standard `safetensors` library, then load it with `fastsafetensors.FastSafetensorsFile`. It shows how to inspect the file's metadata and how to lazily load individual tensors by accessing them like dictionary items. Note that `torch` is used here for tensor creation and loading, implying it should be installed for this specific example."},"warnings":[{"fix":"Use `safetensors` for saving models: `from safetensors.torch import save_file; save_file(model_state, 'model.safetensors')`.","message":"`fastsafetensors` is designed for *loading* safetensors files efficiently, particularly on GPU. It does not provide functionality to *save* safetensors files. For saving, you should use the core `safetensors` library (e.g., `safetensors.torch.save_file`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be aware that `fsf.get_tensors()` returns metadata (name, shape, dtype, data_offsets), not the actual tensor data. Access specific tensors by key (e.g., `my_tensor = fsf['my_key']`) to trigger their loading.","message":"`FastSafetensorsFile` implements lazy loading. Tensors are not fully loaded into memory when the file is opened or when `get_tensors()` is called. They are loaded only when accessed (e.g., `fsf['tensor_name']`). This design optimizes memory usage and startup time but might surprise users expecting eager loading.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Install the necessary framework (e.g., `pip install torch`, `pip install tensorflow`, `pip install paddlepaddle`) if you intend to work with framework-specific tensor objects.","message":"To leverage `fastsafetensors` for specific deep learning frameworks (PyTorch, TensorFlow, PaddlePaddle), those frameworks must be installed separately. `fastsafetensors` does not include them as direct dependencies but will convert loaded data into their respective tensor types if available.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Upgrade to `fastsafetensors` version `0.2.1` or newer to benefit from critical bug fixes related to CUDA stream synchronization and device initialization.","message":"Older versions (0.2.0 and prior) had known issues with CUDA device initialization and stream synchronization, potentially leading to incorrect behavior or suboptimal performance in multi-GPU or complex asynchronous operations.","severity":"deprecated","affected_versions":"<=0.2.0"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Verify that the file path is correct and the safetensors file exists at that location.","cause":"The path provided to `FastSafetensorsFile` does not point to an existing safetensors file.","error":"FileNotFoundError: [Errno 2] No such file or directory: 'non_existent_model.safetensors'"},{"fix":"Use `fsf.get_tensors()` to list all available tensor names and their metadata in the file, then use a correct name.","cause":"Attempting to access a tensor by a name that is not present in the loaded safetensors file.","error":"KeyError: 'tensor_name_does_not_exist'"},{"fix":"Install PyTorch: `pip install torch` (or `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121` for CUDA-enabled versions, adjusting for your specific CUDA version).","cause":"You are trying to load a tensor into a PyTorch `torch.Tensor` object, but PyTorch is not installed in your environment.","error":"ModuleNotFoundError: No module named 'torch'"},{"fix":"Ensure your system has a CUDA-enabled GPU, that the necessary NVIDIA drivers and CUDA toolkit are installed, and that PyTorch (or relevant framework) is installed with CUDA support. For ROCm, ensure the AMD ROCm platform is correctly set up.","cause":"Attempting to load or process tensors on a CUDA device when no CUDA-enabled GPU is detected or properly configured, or when a CUDA-specific operation is attempted without a GPU.","error":"RuntimeError: No CUDA GPUs are available"}]}