semchunk

semchunk is a Python library for splitting text into smaller chunks while preserving as much local semantic context as possible. It supports advanced features like AI-powered hierarchical chunking, chunk overlapping, and processing Isaacus Legal Graph Schema (ILGS) Documents, working seamlessly with various tokenizers. Actively developed by Isaacus, the library has frequent releases, with version 4.0.0 notably introducing AI chunking and ILGS Document support.