{"id":8800,"library":"zodbpickle","title":"Zodbpickle","description":"Zodbpickle is a fork of Python's built-in `pickle` module, primarily designed to provide a uniform pickling interface for ZODB (Zope Object Database). It extends Python 2.7's `pickle` and `cPickle` to support protocol 3 opcodes and introduces `zodbpickle.binary` for consistent binary value handling across Python 2 and 3. For Python 3, it forks the `pickle` module to re-add support for the `noload()` operation, which ZODB utilizes. The library is currently active, with version 4.3 as of the last check, and aims to ensure seamless data serialization and deserialization in ZODB environments spanning different Python versions.","status":"active","version":"4.3","language":"en","source_language":"en","source_url":"https://github.com/zopefoundation/zodbpickle","tags":["pickle","serialization","zodb","compatibility","python2","python3","data-persistence"],"install":[{"cmd":"pip install zodbpickle","lang":"bash","label":"Install with pip"}],"dependencies":[{"reason":"Requires Python 3.10 or newer.","package":"python","optional":false},{"reason":"ZODB depends on zodbpickle for its object persistence.","package":"ZODB","optional":true}],"imports":[{"note":"Use this to replace the standard `pickle` module for ZODB compatibility across Python 2 and 3 environments, leveraging `zodbpickle`'s enhancements.","wrong":"import pickle","symbol":"pickle","correct":"from zodbpickle import pickle"},{"note":"Imports the C implementation of `pickle` provided by `zodbpickle` for potentially faster performance.","symbol":"fastpickle","correct":"from zodbpickle import fastpickle"},{"note":"Imports the pure Python implementation of `pickle` from `zodbpickle`.","symbol":"slowpickle","correct":"from zodbpickle import slowpickle"}],"quickstart":{"code":"from zodbpickle import pickle\n\nclass MyObject:\n    def __init__(self, name, value):\n        self.name = name\n        self.value = value\n\n    def __eq__(self, other):\n        if not isinstance(other, MyObject):\n            return NotImplemented\n        return self.name == other.name and self.value == other.value\n\n# Pickle an object\nobj = MyObject('example', 123)\npickled_obj = pickle.dumps(obj, protocol=pickle.HIGHEST_PROTOCOL)\nprint(f\"Pickled object (bytes): {pickled_obj}\")\n\n# Unpickle the object\nunpickled_obj = pickle.loads(pickled_obj)\nprint(f\"Unpickled object: {unpickled_obj.name}, {unpickled_obj.value}\")\n\nassert obj == unpickled_obj","lang":"python","description":"This quickstart demonstrates basic serialization (pickling) and deserialization (unpickling) of a custom Python object using `zodbpickle.pickle`. It shows how to replace the standard `pickle` import to leverage `zodbpickle`'s specialized features, particularly useful for ZODB environments."},"warnings":[{"fix":"Only unpickle data from trusted sources. Implement additional security layers if data origin cannot be guaranteed.","message":"The `pickle` module, and by extension `zodbpickle`, is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source, as it could lead to arbitrary code execution.","severity":"breaking","affected_versions":"All versions"},{"fix":"In Python 2, use `zodbpickle.binary` for storing explicit binary data. When migrating existing Python 2 ZODB databases to Python 3, use migration tools like `zodbupdate` with `--convert-py3` and specify appropriate encoding to convert `str` attributes correctly.","message":"When migrating ZODB databases from Python 2 to Python 3, `Python 2 str` instances are by default loaded as `Python 3 str` (Unicode strings). If these `str` instances contained binary data, this can lead to `UnicodeDecodeError` or incorrect data interpretation. `zodbpickle.binary` was introduced to handle binary strings from Python 2 correctly as `bytes` in Python 3.","severity":"gotcha","affected_versions":"Python 2.x and 3.x interoperability"},{"fix":"For ZODB applications, use `from zodbpickle import pickle`. If performance is critical and not tied to ZODB's specific `noload()` requirement, consider profiling with both `zodbpickle.pickle` and the standard `pickle` module to determine the optimal choice for non-ZODB-specific serialization tasks in Python 3.","message":"While `zodbpickle` re-adds the `noload()` method (removed from standard Python 3 `pickle`) for ZODB compatibility, applications might encounter performance differences depending on the `pickle` protocol used. Python 3.4+ introduced `protocol 4` with significant performance impacts (e.g., framing), which standard `pickle` can leverage more directly than `zodbpickle`'s forks of earlier Python 3 `pickle` versions.","severity":"gotcha","affected_versions":"Python 3.x (especially 3.4+)"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure that Python 2 binary strings were stored using `zodbpickle.binary`. For existing Python 2 ZODB databases, use `zodbupdate --convert-py3 --encoding <your_encoding>` during migration to properly decode and convert string data.","cause":"Attempting to unpickle Python 2 `str` data (which could contain arbitrary bytes) in Python 3, where `str` objects are expected to be Unicode. This often occurs during ZODB database migrations.","error":"UnicodeDecodeError: 'utf-8' codec can't decode byte 0x... in position ...: invalid start byte"},{"fix":"Explicitly encode `str` to `bytes` (e.g., `s.encode('utf-8')`) or decode `bytes` to `str` (e.g., `b.decode('utf-8')`) as appropriate before concatenation. Review data handling logic, especially for values originally created in Python 2 or involving `zodbpickle.binary` to ensure type consistency.","cause":"This error typically arises in Python 3 code when attempting to concatenate `str` (Unicode) and `bytes` objects directly. It can be a symptom of incorrect handling of legacy Python 2 data in a ZODB migrated to Python 3, where `zodbpickle.binary` or `bytes` were not used consistently for binary data.","error":"TypeError: can't concat str to bytes"}]}