HDR Histogram (Numpy-based)

0.3.3 · active · verified Thu Apr 16

hdrpy is a Python library that provides a NumPy-based implementation of High Dynamic Range (HDR) histograms. It was initially forked from HDRHistogram_py and replaced its C code dependency with NumPy. This library is designed for efficient recording and analyzing of sampled data value counts across a configurable integer range with specified value precision, making it particularly useful in latency and performance-sensitive applications.

Common errors

```
AttributeError: module 'hdrpy' has no attribute 'HdrHistogram'
```
cause The HdrHistogram class might not be directly exposed at the top level of the `hdrpy` package if an incorrect import path is used, or the package structure differs from expectation.

fix Ensure you are importing `HdrHistogram` directly from the `hdrpy` package: `from hdrpy import HdrHistogram`.
```
TypeError: HdrHistogram() takes no arguments
```
cause Attempting to instantiate `HdrHistogram` without the required initialization arguments: `lowest_discernible_value`, `highest_trackable_value`, and `number_of_significant_value_digits`.

fix Always initialize `HdrHistogram` with its mandatory parameters, for example: `histogram = HdrHistogram(1, 3600000000, 3)` where the arguments specify the range and precision.
```
ValueError: value out of range
```
cause Attempting to record a value using `record_value()` or `record_corrected_value()` that is outside the `lowest_discernible_value` and `highest_trackable_value` range defined during histogram initialization.

fix Ensure that all values you intend to record fall within the `lowest_discernible_value` and `highest_trackable_value` specified when creating the `HdrHistogram` instance. Adjust the histogram's range if necessary to accommodate your data.

Warnings

gotcha The library's last release was in 2018, indicating a slow development cadence. While stable for its core functionality, new features or rapid bug fixes are unlikely.
Fix: Be aware of potential lack of active maintenance; consider contributions if specific new features or bug fixes are required.
gotcha When measuring latency, 'coordinated omission' can lead to inaccurate statistics. Use `record_corrected_value()` to account for dropped or delayed samples that might otherwise skew results.
Fix: For latency measurements where samples might be omitted due to system overload, use `histogram.record_corrected_value(value, expected_interval)` to accurately reflect the true distribution. The `expected_interval` is the expected sampling interval.
gotcha The precision of the histogram is determined by `number_of_significant_value_digits` during initialization. Choosing too few digits may lead to coarser granularity and loss of detail, especially for values at the lower end of the range.
Fix: Carefully consider the `number_of_significant_value_digits` parameter when initializing `HdrHistogram` to ensure the required resolution for your data, particularly for small values. Higher precision requires more memory.

Install

pip install hdrpy Install with pip

Imports

HdrHistogram
```
from hdrpy import HdrHistogram
```

Quickstart

Initialize an HdrHistogram instance, record values, and retrieve various statistics like mean, standard deviation, and percentiles.

import random
from hdrpy import HdrHistogram

# Create a histogram that can track values from 1 to 3,600,000,000
# with 3 significant digits of precision (e.g., 1 microsecond to 1 hour)
histogram = HdrHistogram(1, 3600 * 1000 * 1000, 3)

# Record some simulated latency values (in microseconds)
for _ in range(100000):
    latency = random.randint(100, 5000000) # values between 0.1ms and 5s
    histogram.record_value(latency)

# Record a value with correction for coordinated omission (e.g., expected interval of 10ms)
histogram.record_corrected_value(random.randint(100, 5000000), 10000)

print(f"Total count: {histogram.get_total_count()}")
print(f"Mean: {histogram.get_mean()} µs")
print(f"Standard Deviation: {histogram.get_stddev()} µs")
print(f"99th Percentile: {histogram.get_value_at_percentile(99.0)} µs")
print(f"Max Value: {histogram.get_max_value()} µs")

view raw JSON →