{"id":5298,"library":"lilcom","title":"Lilcom","description":"Lilcom is a Python library that provides lossy compression for sequence data stored in NumPy arrays. It efficiently compresses floating-point or 16-bit integer NumPy arrays into byte strings, typically used in machine learning applications for storing training data and models. The current version is 1.8.2, and it has a fairly active release cadence, with updates addressing compatibility and functionality.","status":"active","version":"1.8.2","language":"en","source_language":"en","source_url":"https://github.com/danpovey/lilcom","tags":["compression","numpy","machine-learning","audio-processing","data-storage"],"install":[{"cmd":"pip install lilcom","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core functionality relies on NumPy arrays for data handling.","package":"numpy","optional":false},{"reason":"Required for building from source if pre-compiled wheels are not available for your system/Python version. Version 1.6 and later typically provide pre-compiled libraries.","package":"cmake","optional":true}],"imports":[{"symbol":"lilcom","correct":"import lilcom"},{"symbol":"compress","correct":"lilcom.compress(...)"},{"symbol":"decompress","correct":"lilcom.decompress(...)"}],"quickstart":{"code":"import numpy as np\nimport lilcom\n\n# Create a sample NumPy array\na = np.random.randn(300, 500).astype(np.float32)\n\n# Compress the array (default tick_power=-8, controls accuracy)\na_compressed = lilcom.compress(a)\n\n# Decompress the array, specifying original dtype\na_decompressed = lilcom.decompress(a_compressed, dtype=a.dtype)\n\nprint(f\"Original array shape: {a.shape}, dtype: {a.dtype}\")\nprint(f\"Compressed data size: {len(a_compressed)} bytes\")\nprint(f\"Decompressed array shape: {a_decompressed.shape}, dtype: {a_decompressed.dtype}\")\nprint(f\"Max absolute error: {np.max(np.abs(a - a_decompressed)):.2e}\")\n","lang":"python","description":"This quickstart demonstrates how to compress a NumPy array using `lilcom.compress()` and then decompress it using `lilcom.decompress()`. Note that `lilcom` is a lossy compression utility, so the decompressed array will not be exactly identical to the original. The `tick_power` argument controls the precision of the compression."},"warnings":[{"fix":"Be aware of the lossy nature and choose an appropriate `tick_power` for your application's accuracy requirements. `2**(tick_power-1)` gives the maximum error per element.","message":"Lilcom provides *lossy* compression. The decompressed data will not be bit-for-bit identical to the original array. The amount of error is controlled by the `tick_power` argument (default -8), which determines the step size for discretized values.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If migrating from older code, replace `bits_per_sample` with `tick_power`. The `tick_power` argument (default: -8) is the power of 2 used for the step size between discretized values.","message":"The method for controlling compression accuracy changed. Older versions of Lilcom might have used `bits_per_sample`. Current versions (e.g., 1.8.2 and recent GitHub README) use `tick_power` to specify the quantization step size.","severity":"breaking","affected_versions":"Prior to ~v1.7 (exact transition not specified, but observed in documentation updates)"},{"fix":"Ensure your environment uses Python 3.6 or higher. Upgrade Python if necessary.","message":"Lilcom requires Python 3.6 or newer. It is not compatible with Python 2.x. Attempts to install or run on unsupported Python versions will fail.","severity":"breaking","affected_versions":"All versions"},{"fix":"Implement robust error handling, retransmission, or checksums if storing or transmitting compressed data in environments prone to bit errors and where data recovery is critical.","message":"The underlying algorithm is highly vulnerable to transmission errors. Even a single bit error in the compressed byte string can make the entire file or sequence unreadable during decompression. This is acceptable for its target machine learning applications where data integrity is often handled at a higher level or re-generated.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure a C++ compiler is installed and accessible in your system's PATH before attempting a source installation (e.g., `apt-get install build-essential` on Debian/Ubuntu or Xcode Command Line Tools on macOS).","message":"If installing `lilcom` from source (e.g., if pre-compiled wheels are not available for your specific platform/Python version), a C++ compiler (like g++ or clang) is required on your system.","severity":"gotcha","affected_versions":"All versions (for source installs)"}],"env_vars":null,"last_verified":"2026-04-13T00:00:00.000Z","next_check":"2026-07-12T00:00:00.000Z"}