{"id":6574,"library":"compress-pickle","title":"compress-pickle","description":"compress-pickle is a Python library (version 2.1.0) that thinly wraps the standard `pickle` package with various standard compression libraries (gzip, bz2, lzma, zipfile, and optionally lz4). It provides an interface similar to `pickle.dump`, `pickle.load`, `pickle.dumps`, and `pickle.loads` to seamlessly serialize and deserialize Python objects to disk or file-like objects in a compressed manner. The library has an infrequent release cadence, with its last major update in September 2021.","status":"active","version":"2.1.0","language":"en","source_language":"en","source_url":"https://github.com/lucianopaz/compress_pickle","tags":["pickle","compression","serialization","data persistence","object storage"],"install":[{"cmd":"pip install compress-pickle","lang":"bash","label":"Basic installation"},{"cmd":"pip install compress-pickle[lz4]","lang":"bash","label":"With optional lz4 support"}],"dependencies":[{"reason":"Provides additional LZ4 compression support.","package":"lz4","optional":true}],"imports":[{"note":"Used for serializing and compressing an object to a file-like object.","symbol":"dump","correct":"from compress_pickle import dump"},{"note":"Used for decompressing and deserializing an object from a file-like object.","symbol":"load","correct":"from compress_pickle import load"},{"note":"Used for serializing and compressing an object to a bytes object.","symbol":"dumps","correct":"from compress_pickle import dumps"},{"note":"Used for decompressing and deserializing an object from a bytes object.","symbol":"loads","correct":"from compress_pickle import loads"}],"quickstart":{"code":"from compress_pickle import dump, load\nimport os\n\ndata = {'key': 'value', 'numbers': [1, 2, 3]}\nfilename = 'my_compressed_data.pkl.gz'\n\n# Dump (serialize and compress) the data\ndump(data, filename)\nprint(f\"Data saved to {filename} with inferred gzip compression.\")\n\n# Load (decompress and deserialize) the data\nloaded_data = load(filename)\nprint(f\"Data loaded successfully: {loaded_data}\")\n\n# Clean up the created file\nos.remove(filename)","lang":"python","description":"This quickstart demonstrates how to use `compress_pickle.dump` to serialize and compress a Python dictionary to a file, and `compress_pickle.load` to decompress and deserialize it back. The compression method (gzip in this case) is automatically inferred from the file extension '.pkl.gz'."},"warnings":[{"fix":"Upgrade Python to 3.6+ or pin `compress-pickle` to version `1.1.1` for Python 3.5 environments.","message":"Python 3.5 support was dropped starting from version 2.0.0. If you need to support Python 3.5, you must install `compress-pickle==1.1.1`.","severity":"breaking","affected_versions":">=2.0.0"},{"fix":"Only unpickle data from trusted sources. Consider alternative serialization formats like JSON for untrusted data.","message":"Like the underlying `pickle` module, `compress-pickle` is not secure against maliciously constructed data. Deserializing data from untrusted sources can lead to arbitrary code execution.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Evaluate file sizes for your specific use case. Compression is generally beneficial for larger objects.","message":"For very small Python objects, the overhead introduced by compression might cause the compressed file size to be larger than the uncompressed pickled data.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use appropriate file extensions or explicitly pass the `compression` argument to `dump` and `load`.","message":"By default, `dump` and `load` infer the compression protocol from the file extension (e.g., '.pkl.gz' for gzip). If the file extension does not match the desired compression or is missing, explicitly set the `compression` argument (e.g., `compression='gzip'`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be mindful of the Python versions used for pickling and unpickling. Specify a compatible `pickler_kwargs={'protocol': N}` if cross-version compatibility is critical.","message":"The `pickle` protocol used can affect compatibility. Objects pickled with newer Python versions or higher protocols may not be readable by older Python versions or lower protocols. `compress-pickle` inherits this behavior.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-15T00:00:00.000Z","next_check":"2026-07-14T00:00:00.000Z","problems":[]}