Python ROUGE Score Implementation

1.0.1 · active · verified Sun Apr 12

The 'rouge' library provides a full, native Python implementation of the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metric, used for evaluating automatic text summarization and machine translation. Unlike some other ROUGE packages, it is not a wrapper around the original Perl script. The current stable version is 1.0.1, with releases occurring periodically to introduce features and fixes.

Warnings

gotcha This 'rouge' library (pltrdy/rouge) is a native Python implementation and explicitly states its results may be slightly different from the 'official' ROUGE-155 Perl script. If exact replication of Perl ROUGE-155 is critical, or if using a different Python ROUGE implementation (like 'rouge-score' by Google) results in discrepancies, this is the expected behavior for this package.
Fix: Be aware of potential minor score differences. If strict adherence to ROUGE-155 results is required, consider using packages that wrap the Perl script or `rouge-score` by Google, which aims for Perl script replication.
gotcha There are multiple Python libraries for ROUGE, notably `rouge` (this package, from pltrdy) and `rouge-score` (from Google Research). They have different import paths and API interfaces. Confusing them is a common footgun.
Fix: Ensure you are importing `from rouge import Rouge` for this library. If you intend to use Google's implementation, install `rouge-score` and import `from rouge_score import rouge_scorer.RougeScorer`.
gotcha The library is language-agnostic and expects tokenized input. For optimal results, especially with non-English texts or specific NLP tasks, users should pre-process and tokenize the hypothesis and reference texts before passing them to the `get_scores` method.
Fix: Pre-tokenize your input strings into space-separated words or n-grams before passing them to the `Rouge().get_scores()` method. For example, `hypothesis = 'word1 word2 word3'`.
deprecated Prior to version 0.3, there was an error in ROUGE-L calculation when handling sequences with multiple sentences. This was fixed in version 0.3.
Fix: Upgrade to version 0.3 or higher to ensure correct ROUGE-L calculations, especially for multi-sentence inputs.

Install

pip install rouge Install latest version

Imports

Rouge
```
from rouge import Rouge
```

Quickstart

Calculate ROUGE scores (ROUGE-1, ROUGE-2, ROUGE-L) for a single hypothesis-reference pair. The `get_scores` method returns a list of dictionaries, each containing 'f' (F1-score), 'p' (precision), and 'r' (recall) for each ROUGE type.

from rouge import Rouge

hypothesis = "the cat sat on the mat"
reference = "the cat is on the mat"

rouge = Rouge()
scores = rouge.get_scores(hypothesis, reference)
print(scores)

view raw JSON →