{"id":3547,"library":"lzfse","title":"LZFSE Python Bindings","description":"The `lzfse` library provides Python bindings to Apple's LZFSE (Lempel–Ziv Finite State Entropy) compression algorithm. LZFSE is an open-source lossless data compression algorithm developed by Apple, designed for faster decompression and higher energy efficiency compared to zlib, particularly optimized for modern micro-architectures like arm64. The Python bindings are currently at version 0.4.2 (as of April 19, 2024), with infrequent releases often tied to updates in the underlying C reference implementation or improvements to the Python packaging.","status":"active","version":"0.4.2","language":"en","source_language":"en","source_url":"https://github.com/m1stadev/lzfse","tags":["compression","apple","lzfse","data-compression"],"install":[{"cmd":"pip install lzfse","lang":"bash","label":"Install from PyPI"}],"dependencies":[{"reason":"The library is a wrapper around a C implementation; compiling from source (e.g., when wheels aren't available for your platform/Python version, especially on Windows) requires a C/C++ compiler and associated SDKs (e.g., Windows 10/11 SDK for Visual Studio Build Tools).","package":"C/C++ Build Tools","optional":true}],"imports":[{"note":"The original project was named 'pyliblzfse' and imported as `liblzfse`. The current `lzfse` package on PyPI is a fork and rename, which should be imported directly as `lzfse`.","wrong":"import liblzfse","symbol":"lzfse","correct":"import lzfse"}],"quickstart":{"code":"import lzfse\n\noriginal_data = b\"This is some data to be compressed using LZFSE! \" * 100\n\n# Compress data\ncompressed_data = lzfse.compress(original_data)\nprint(f\"Original size: {len(original_data)} bytes\")\nprint(f\"Compressed size: {len(compressed_data)} bytes\")\n\n# Decompress data\ndecompressed_data = lzfse.decompress(compressed_data)\nprint(f\"Decompressed size: {len(decompressed_data)} bytes\")\n\n# Verify data integrity\nassert original_data == decompressed_data\nprint(\"Data compressed and decompressed successfully!\")\n\n# Handle decompression errors\ncorrupted_data = b'\\xde\\xad\\xbe\\xef' + compressed_data[4:] # Corrupt header\ntry:\n    lzfse.decompress(corrupted_data)\nexcept lzfse.error as e:\n    print(f\"Caught expected error during decompression: {e}\")","lang":"python","description":"This example demonstrates basic LZFSE compression and decompression of a byte string, including error handling for corrupted data."},"warnings":[{"fix":"Update import statements from `import liblzfse` to `import lzfse`.","message":"The current `lzfse` package is a fork and rename of an earlier project, `pyliblzfse`. Codebases written for `pyliblzfse` might use `import liblzfse`, which will break with the `lzfse` package. The correct import is now `import lzfse`.","severity":"breaking","affected_versions":"<0.4.0 (for pyliblzfse users), all (for lzfse users migrating from pyliblzfse)"},{"fix":"Ensure a compatible C/C++ build environment is installed on your system. For Windows, install the 'Desktop Development with C++' workload and the Windows 10/11 SDK using the Visual Studio Build Tools installer. For Linux, ensure `build-essential` or similar developer tools are installed.","message":"On platforms where pre-built wheels are not available (e.g., some Linux distributions, or when specific Python versions are not covered), installing `lzfse` might attempt to build from source. This requires a C/C++ compiler and potentially system-specific development headers. On Windows, this often means installing 'Desktop Development with C++' and the Windows 10/11 SDK via Visual Studio Build Tools, failing with errors like 'stddef.h: No such file or directory'.","severity":"gotcha","affected_versions":"All versions when installing from source."},{"fix":"Be aware of these limitations when processing very large files or unusual inputs. Consider pre-processing large data into smaller chunks if 2GiB+ performance degradation is a concern. Refer to the upstream LZFSE C library's GitHub issues for more details and potential workarounds.","message":"The underlying LZFSE C reference implementation has known limitations and potential issues, including degraded compression performance for inputs larger than 2GiB and documented memory-related crashes (e.g., double free when compressing empty files, malloc errors). As the Python library is a binding, these issues can manifest when using `lzfse`.","severity":"gotcha","affected_versions":"All versions."}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}