{"id":8707,"library":"tensorizer","title":"Tensorizer","description":"Tensorizer is a Python library developed by CoreWeave for fast serialization and deserialization of PyTorch modules, models, and tensors. It aims to reduce model load times and CPU memory usage by efficiently streaming tensor data, supporting local filesystems, HTTP/HTTPS, and S3 endpoints. The current version is 2.12.0, with ongoing development and updates primarily driven by CoreWeave's needs for serving large AI models.","status":"active","version":"2.12.0","language":"en","source_language":"en","source_url":"https://github.com/coreweave/tensorizer","tags":["pytorch","serialization","deserialization","deep-learning","mlops","performance"],"install":[{"cmd":"pip install tensorizer","lang":"bash","label":"Install from PyPI"},{"cmd":"pip install git+https://github.com/coreweave/tensorizer","lang":"bash","label":"Install directly from GitHub"}],"dependencies":[{"reason":"Fundamental dependency for PyTorch model and tensor operations.","package":"torch"},{"reason":"Required for tensor encryption/decryption functionality.","package":"libsodium","optional":true}],"imports":[{"symbol":"TensorSerializer","correct":"from tensorizer import TensorSerializer"},{"symbol":"TensorDeserializer","correct":"from tensorizer import TensorDeserializer"},{"symbol":"tensorizer_saving","correct":"from tensorizer.torch_compat import tensorizer_saving"},{"symbol":"tensorizer_loading","correct":"from tensorizer.torch_compat import tensorizer_loading"}],"quickstart":{"code":"import torch\nfrom torch import nn\nfrom tensorizer import TensorSerializer, TensorDeserializer\nimport os\n\n# 1. Define a simple PyTorch model\nclass SimpleModel(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.linear1 = nn.Linear(10, 50)\n        self.relu = nn.ReLU()\n        self.linear2 = nn.Linear(50, 2)\n        self.dummy_attribute = 'some_string_data' # Non-tensor attribute\n\n    def forward(self, x):\n        return self.linear2(self.relu(self.linear1(x)))\n\nmodel = SimpleModel()\ndummy_input = torch.randn(1, 10)\noriginal_output = model(dummy_input)\n\nfile_path = \"./simple_model.tensors\"\n\n# 2. Serialize the model's state_dict\nprint(f\"Serializing model to {file_path}...\")\nserializer = TensorSerializer(file_path)\nserializer.write_state_dict(model.state_dict())\nserializer.close()\nprint(\"Serialization complete.\")\n\n# 3. Create a new model instance for deserialization\nloaded_model = SimpleModel()\n\n# 4. Deserialise the model's state_dict into the new instance\nprint(f\"Deserializing model from {file_path}...\")\nwith TensorDeserializer(file_path, device='cpu') as loaded_state_dict:\n    loaded_model.load_state_dict(loaded_state_dict)\nprint(\"Deserialization complete.\")\n\n# Verify deserialized model output\nloaded_output = loaded_model(dummy_input)\nassert torch.allclose(original_output, loaded_output), \"Outputs do not match after serialization/deserialization!\"\nprint(\"Model successfully serialized and deserialized with matching outputs.\")\n\n# Clean up the created file\nos.remove(file_path)\n","lang":"python","description":"This quickstart demonstrates how to serialize and deserialize a PyTorch model's state dictionary using `TensorSerializer` and `TensorDeserializer`. It creates a simple neural network, saves its state, loads it into a new instance, and verifies that the outputs match. The `TensorDeserializer` is used as a context manager for proper resource handling."},"warnings":[{"fix":"For models with non-tensor attributes, you must manually serialize and deserialize these components separately or use `tensorizer.torch_compat` which can save arbitrary objects (via pickle) alongside tensors. Be mindful of the security implications of pickling untrusted data.","message":"Tensorizer explicitly serializes only tensors. Unlike `torch.save`, it does NOT use Python's `pickle` module for arbitrary Python objects. If your `torch.nn.Module` contains non-tensor attributes critical to its functionality (e.g., custom configuration objects, tokenizers), these will NOT be saved by `TensorSerializer.write_module` or `write_state_dict` and will be missing upon deserialization.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Only load models from trusted sources when using `tensorizer.torch_compat` or `torch.load`. For maximum security, use `TensorSerializer` and `TensorDeserializer` directly for tensors and manage non-tensor data serialization with a safer format (e.g., JSON, YAML, Protocol Buffers).","message":"When using the `tensorizer.torch_compat` module as a drop-in replacement for `torch.save` and `torch.load`, `torch.load` still uses the `pickle` module internally for any non-tensor data. Loading untrusted pickled files can lead to arbitrary code execution, posing a security risk. This warning applies even though `tensorizer` itself only handles data.","severity":"security","affected_versions":"All versions"},{"fix":"Install `libsodium` on your operating system. For Ubuntu/Debian, use `sudo apt-get install libsodium23`. Refer to `libsodium`'s official documentation for other platforms.","message":"Tensor encryption/decryption functionality requires the external `libsodium` library to be installed on your system. Without it, attempts to use encryption features will fail.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use `TensorDeserializer` within a `with` statement: `with TensorDeserializer(...) as state_dict: ...`","message":"`TensorDeserializer` is designed as a context manager, especially important for lazy loading. While it can be instantiated without `with`, using it as a context manager ensures proper resource cleanup and file closing, preventing potential file handle leaks or unexpected behavior, especially when streaming from remote sources.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Monitor CPU and VRAM usage during serialization/deserialization. Scale your compute resources (CPU RAM, GPU VRAM) or consider using smaller models if resource constraints are an issue.","message":"Loading and serializing very large models (e.g., EleutherAI/gpt-j-6B) with `tensorizer` requires substantial CPU RAM (up to ~20GB) and GPU VRAM (~16GB), even with `tensorizer`'s efficiency. Ensure your environment has sufficient resources.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Modify your serialization logic to separately save and load non-tensor attributes using standard Python serialization (e.g., `json`, `pickle` if secure, or other custom methods). Ensure your model re-initialization logic correctly reconstructs these attributes, or use `tensorizer.torch_compat` with awareness of its `pickle` usage.","cause":"Attempting to access a non-tensor attribute (e.g., a custom configuration object, tokenizer) that was part of the original PyTorch model but was not explicitly saved or serialized by `Tensorizer`, because `Tensorizer` only handles tensors by default.","error":"AttributeError: 'SimpleModel' object has no attribute 'my_config_object'"},{"fix":"Install the `libsodium` development package on your system. For Debian/Ubuntu, use `sudo apt-get install libsodium-dev` or `libsodium23`. For other operating systems, consult the `libsodium` official documentation for installation instructions.","cause":"The `libsodium` shared library, required for tensor encryption/decryption, is not installed or not discoverable in the system's library paths.","error":"libsodium.so.23: cannot open shared object file: No such file or directory"},{"fix":"Ensure `TensorDeserializer` is used as a context manager when loading state dictionaries or accessing its contents that behave like a dictionary: `with TensorDeserializer(uri) as state_dict: model.load_state_dict(state_dict)`.","cause":"Attempting to iterate over a `TensorDeserializer` object directly, or treating it as a dictionary outside of its context manager usage, particularly when trying to load a state dictionary.","error":"TypeError: 'TensorDeserializer' object is not iterable"}]}