{"library":"segtok","title":"segtok: Sentence Segmentation and Word Tokenization","description":"Segtok is a fast, rule-based Python library for sentence segmentation and word tokenization. It is designed for well-orthographed texts, particularly in English, German, and Romance languages, offering high precision and Unicode support. The current version is 1.5.11. While functional, it is largely superseded by 'syntok' (segtok v2) which offers improved performance and handles more edge cases. It is in a maintenance phase with no active development.","language":"python","status":"maintenance","last_verified":"Fri May 15","install":{"commands":["pip install segtok"],"cli":null},"imports":["from segtok.segmenter import split_multi","from segtok.tokenizer import web_tokenizer","from segtok.tokenizer import split_contractions","from segtok.tokenizer import word_tokenizer"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"from segtok.segmenter import split_multi\nfrom segtok.tokenizer import web_tokenizer, split_contractions\n\ntext = \"Hello, Mr. Man. He smiled!! This, i.e. that, is it. Don't worry.\"\nsentences = split_multi(text)\n\nall_tokens = []\nfor sentence in sentences:\n    tokens = list(split_contractions(web_tokenizer(sentence)))\n    all_tokens.append(tokens)\n\nprint(\"Original Text:\", text)\nprint(\"\\nSentences:\")\nfor s in sentences:\n    print(f\"- {s}\")\n\nprint(\"\\nTokens per sentence:\")\nfor i, tokens in enumerate(all_tokens):\n    print(f\"Sentence {i+1}: {tokens}\")","lang":"python","description":"This quickstart demonstrates basic sentence segmentation using `split_multi` and then tokenizes each sentence using `web_tokenizer` followed by `split_contractions` for English-specific handling.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-15","installed_version":"1.5.11","pypi_latest":"1.5.11","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":2.2,"avg_import_s":0.12,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.07,"mem_mb":1.7,"disk_size":"20.6M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.4,"import_time_s":0.04,"mem_mb":1.7,"disk_size":"22M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.29,"mem_mb":1.7,"disk_size":"22.8M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.1,"import_time_s":0.23,"mem_mb":1.7,"disk_size":"24M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.16,"mem_mb":1.7,"disk_size":"14.6M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":1.9,"import_time_s":0.15,"mem_mb":1.7,"disk_size":"16M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.06,"mem_mb":1.7,"disk_size":"14.4M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":1.9,"import_time_s":0.06,"mem_mb":1.5,"disk_size":"15M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.06,"mem_mb":1.7,"disk_size":"20.1M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"segtok","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.8,"import_time_s":0.06,"mem_mb":1.7,"disk_size":"21M"}]}}