{"id":2269,"library":"sacrebleu","title":"sacreBLEU","description":"sacreBLEU is a Python library providing hassle-free computation of shareable, comparable, and reproducible BLEU, chrF, and TER scores for machine translation evaluation. It is actively maintained, with the current version being 2.6.0, and releases typically occur several times a year to add new test sets, tokenizers, or minor features.","status":"active","version":"2.6.0","language":"en","source_language":"en","source_url":"https://github.com/mjpost/sacrebleu","tags":["NLP","BLEU","metrics","machine translation","evaluation","NMT"],"install":[{"cmd":"pip install sacrebleu","lang":"bash","label":"Install sacreBLEU"}],"dependencies":[{"reason":"Required for Korean tokenization (e.g., '--tok ko-mecab').","package":"mecab-ko","optional":true},{"reason":"Extra for Japanese tokenization, installing `mecab3-python`.","package":"sacrebleu[ja]","optional":true}],"imports":[{"symbol":"corpus_bleu","correct":"import sacrebleu\nscore = sacrebleu.corpus_bleu(hyp, [ref])"},{"note":"Newer object-oriented API for metrics introduced after v1.x.","symbol":"BLEU, CHRF, TER","correct":"from sacrebleu.metrics import BLEU, CHRF, TER"}],"quickstart":{"code":"import sacrebleu\n\n# Example hypothesis and reference sentences\nhypothesis = \"The cat sat on the mat.\"\nreferences = [\n    \"The cat is on the mat.\",\n    \"A cat sat on the mat.\"\n]\n\n# Calculate corpus BLEU score\n# Note: sacrebleu expects lists of sentences, even for a single hypothesis/reference\nbleu_score = sacrebleu.corpus_bleu([hypothesis], [references])\n\nprint(f\"BLEU score: {bleu_score.score:.2f}\")\nprint(f\"BLEU string: {bleu_score.format()}\")","lang":"python","description":"Calculates the corpus BLEU score for a given hypothesis against one or more reference translations. Outputs the raw score and the formatted string."},"warnings":[{"fix":"Update scripts to parse JSON output. Use the `--format text` option to revert to the old single-line format for compatibility, e.g., `sacrebleu -t wmt14 -l en-de --format text`.","message":"The default output format for the CLI utility changed from single-line to JSON. Tools or scripts parsing sacrebleu's standard output will need to adapt.","severity":"breaking","affected_versions":">=2.0.0"},{"fix":"Upgrade your Python environment to 3.9 or higher. For older Python versions, use an earlier sacrebleu version (e.g., sacrebleu<2.0 for Python <3.6, sacrebleu<2.6 for Python 3.8).","message":"Python 3.5, 3.6, and 3.8 support have been dropped in recent major versions. The library now requires Python >=3.9.","severity":"breaking","affected_versions":">=2.0.0, >=2.6.0"},{"fix":"Review your code to ensure the desired smoothing method (`floor`, `add-k`, `exp`) and `smooth_value` are explicitly passed to `corpus_bleu()` or `sentence_bleu()` to ensure consistent scoring behavior.","message":"The default smoothing method and floor value for `corpus_bleu()` and `sentence_bleu()` changed, potentially yielding different scores compared to earlier versions.","severity":"breaking","affected_versions":">=1.5.0"},{"fix":"Install with optional extras, e.g., `pip install sacrebleu[ja]` for Japanese. For Korean tokenizers like `ko-mecab`, you might need to manually install `mecab-ko`.","message":"Some language-specific tokenizers require extra dependencies that are not installed by default. For example, Japanese (`-tok ja`) and Korean (`-tok ko-mecab`) tokenizers need additional packages.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}