Base64 Streaming I/O

1.0.3 · active · verified Thu Apr 16

`base64io` is a Python library that provides a file-like object interface for streaming Base64 encoding and decoding. It allows processing large files without loading their entire content into memory by wrapping an existing byte stream. The current version is 1.0.3, and due to its stable and specialized utility, releases are infrequent.

Common errors

```
TypeError: a bytes-like object is required, not '_io.TextIOWrapper'
```
cause Attempting to initialize `base64io.Base64IO` with a file opened in text mode (`'r'` or `'w'`), but the library expects a binary stream.

fix Ensure the underlying file is opened in binary mode, e.g., `open('my_file.txt', 'rb')` for reading or `open('output.b64', 'wb')` for writing.
```
AttributeError: 'str' object has no attribute 'read'
```
cause Passing a raw `str` or `bytes` object directly to the `base64io.Base64IO` constructor instead of a file-like object.

fix Wrap the `str` (after encoding to bytes) or `bytes` object in an `io.BytesIO` object before passing it to `Base64IO`, or use the standard `base64` module for direct string/bytes operations.
```
ValueError: Non-base64 character found
```
cause When `Base64IO` is in decode mode, the underlying stream contains characters that are not part of the standard Base64 alphabet (A-Z, a-z, 0-9, +, /, =).

fix Verify that the input stream consists solely of valid Base64 characters. Pre-process the stream to remove any unexpected characters or padding if necessary.

Warnings

gotcha The `Base64IO` class expects a binary file-like object (opened in `rb` or `wb` mode) as its input. Passing a text-mode file handle (e.g., `open('file.txt', 'r')`) will lead to a `TypeError`.
Fix: Always open the underlying file with a binary mode, such as `rb` for reading or `wb` for writing.
gotcha Unlike the standard library's `base64.b64decode` or `base64.b64encode` functions, `base64io.Base64IO` does not accept a raw `str` or `bytes` object directly as its first argument. It strictly requires a file-like object that implements `read()` (or `write()` for encoding).
Fix: For non-streaming operations with raw strings or bytes, use `base64.b64decode()` or `base64.b64encode()` directly. If you need to treat a `bytes` object as a stream for `Base64IO`, wrap it in `io.BytesIO` (e.g., `base64io.Base64IO(io.BytesIO(my_bytes_data), 'decode')`).

Install

pip install base64io Install stable version

Imports

Base64IO
```
from base64io import Base64IO
```

Quickstart

This quickstart demonstrates how to use `base64io.Base64IO` to stream encode and decode content. It creates dummy binary and Base64 files, then uses `Base64IO` to transform them, verifying the output. Note that `Base64IO` always expects and produces byte streams.

import base64io
import tempfile
import os

# Create dummy input files in a temporary directory
dummy_input_bin_content = b"Hello, base64io streaming!"
dummy_input_b64_content = b"SGVsbG8sIGJhc2U2NGlvIHN0cmVhbWluZyE=" # Base64 of the above

with tempfile.TemporaryDirectory() as tmpdir:
    input_bin_path = os.path.join(tmpdir, 'input.bin')
    input_b64_path = os.path.join(tmpdir, 'input.b64')
    output_b64_path = os.path.join(tmpdir, 'output.b64')
    output_bin_path = os.path.join(tmpdir, 'output.bin')

    # Write dummy content to temporary files
    with open(input_bin_path, 'wb') as f:
        f.write(dummy_input_bin_content)
    with open(input_b64_path, 'wb') as f:
        f.write(dummy_input_b64_content)

    print(f"Temporary directory: {tmpdir}")

    # --- Encoding example ---
    # Open original binary input, wrap with Base64IO for encoding
    with open(input_bin_path, 'rb') as fin, base64io.Base64IO(fin, 'encode') as fenc:
        # Write encoded content to output file
        with open(output_b64_path, 'wb') as fout:
            fout.write(fenc.read())
    
    # Verify encoded output
    with open(output_b64_path, 'rb') as f:
        encoded_output = f.read()
    print(f"Encoded content written to {output_b64_path}: {encoded_output}")
    assert encoded_output == dummy_input_b64_content

    # --- Decoding example ---
    # Open Base64 encoded input, wrap with Base64IO for decoding
    with open(input_b64_path, 'rb') as fin, base64io.Base64IO(fin, 'decode') as fdec:
        # Write decoded content to output file
        with open(output_bin_path, 'wb') as fout:
            fout.write(fdec.read())
            
    # Verify decoded output
    with open(output_bin_path, 'rb') as f:
        decoded_output = f.read()
    print(f"Decoded content written to {output_bin_path}: {decoded_output}")
    assert decoded_output == dummy_input_bin_content

print("Quickstart successful!")

view raw JSON →