TinySegmenter

TinySegmenter in Python is a Python port of the original JavaScript-based TinySegmenter, an extremely compact (23KB) Japanese tokenizer. It offers character-based segmentation with approximately 95% precision for Japanese news articles, compatible with MeCab + IPADic segmentation units, without relying on external dictionaries. The latest version, 0.4, was released on September 16, 2018, and its development is not actively maintained, though contributions are welcome.