filesplit Python Library

4.1.0 · active · verified Sat Apr 11

filesplit is a Python module designed for splitting large files into smaller, manageable chunks and subsequently merging them back together. It supports splitting by size, number of chunks, or number of lines, and works with both text and binary files. The current version is 4.1.0, and it maintains an active, somewhat regular release cadence for minor improvements and bug fixes, with major versions introducing breaking changes.

Warnings

breaking Version 4.0.0 introduced significant breaking changes by renaming primary splitting methods. For instance, the generic `split()` method and specific `split_by_encoding()` were replaced with more explicit names like `split_by_size()`, `split_by_chunks()`, and `split_by_lines()`.
Fix: Update method calls from generic `split()` or `split_by_encoding()` to `fs.split_by_size()`, `fs.split_by_chunks()`, or `fs.split_by_lines()` depending on your splitting criteria. Refer to the official documentation for the updated method signatures.
breaking Version 3.0.0 changed the primary class name from `FileSplit` to `Filesplit` (case change) and removed the `splitbyencoding()` method, integrating its functionality into the `split()` method (which itself was later refactored in v4.0.0).
Fix: Ensure you are importing and instantiating `Filesplit` (capital F) instead of `FileSplit`. If using `splitbyencoding()`, refactor to use the `split()` method (for v3.x) or the more specific `split_by_...` methods (for v4.x and above).
gotcha The library explicitly requires Python 3. Versions prior to 2.0.0 supported Python 2, but all subsequent versions dropped Python 2 compatibility.
Fix: Ensure your project is running on Python 3.x. The library supports Python versions 3.0 to 3.11.
gotcha When splitting files, ensure the `output_dir` provided to `split_by_size`, `split_by_chunks`, or `split_by_lines` either exists or is created by your script. The library does not automatically create the output directory.
Fix: Before calling a split method, use `os.makedirs(output_dir, exist_ok=True)` to ensure the target directory for split files exists.

Install

pip install filesplit Install stable version

Imports

Filesplit
```
from filesplit.split import Filesplit
```
The primary class 'Filesplit' is located within the 'filesplit.split' submodule, not directly under 'filesplit'.
merge_files
```
from filesplit.merge import merge_files
```
The 'merge_files' function is found in the 'filesplit.merge' submodule, distinct from the 'split' submodule.

Quickstart

This quickstart demonstrates how to create a dummy file, split it into smaller chunks using `Filesplit.split_by_size`, and then merge those chunks back into a single file using `merge_files`. It includes cleanup of all generated files and directories.

import os
from filesplit.split import Filesplit
from filesplit.merge import merge_files

# Create a dummy file for splitting
dummy_content = "This is a test file for filesplit.\n" * 50
input_file = "my_large_file.txt"
output_dir = "output_chunks"
merged_file = "my_merged_file.txt"

with open(input_file, "w") as f:
    f.write(dummy_content)

print(f"Created dummy file: {input_file}")

# Initialize Filesplit
fs = Filesplit()

# --- Split the file by size (e.g., 100 bytes per chunk) ---
os.makedirs(output_dir, exist_ok=True)
print(f"Splitting '{input_file}' into '{output_dir}' by size...")
fs.split_by_size(
    file=input_file,
    size=100,
    output_dir=output_dir
)
print("Splitting complete.")

# --- Merge the files back ---
print(f"Merging files from '{output_dir}' back into '{merged_file}'...")
merge_files(
    input_dir=output_dir,
    output_file=merged_file
)
print("Merging complete.")

# --- Clean up ---
import shutil
if os.path.exists(input_file): os.remove(input_file)
if os.path.exists(merged_file): os.remove(merged_file)
if os.path.exists(output_dir): shutil.rmtree(output_dir)

print("Cleanup complete.")

view raw JSON →