DDSketch (Distributed Quantile Sketches)

3.0.1 · active · verified Thu Apr 09

DDSketch is a Python library for distributed quantile sketches, an algorithm for estimating quantiles with guaranteed relative accuracy. It allows for merging sketches from different sources while maintaining accuracy. The current version is 3.0.1. Releases are made as needed for bug fixes, Python version compatibility, or feature enhancements.

Warnings

breaking Python 2.x and Python 3.6 support has been dropped. The library now requires Python >= 3.7.
Fix: Upgrade your Python environment to 3.7 or newer. If you need older Python support, use `ddsketch<3.0.0`.
breaking The `numpy` dependency was removed. If your project explicitly relied on `ddsketch` pulling in `numpy`, you will need to add it as a direct dependency to your project.
Fix: No action needed for `ddsketch` functionality, but if you require `numpy`, add `pip install numpy` to your project's dependencies.
breaking Many `DDSketch` attributes such as `mapping`, `store`, `negative_store`, `zero_count`, `relative_accuracy`, `min`, and `max` were removed from direct access.
Fix: Access these attributes via new properties or methods (e.g., `s.min_value`, `s.max_value`) or use the newly exposed top-level `stores` and `mappings` for custom implementations. Review the v2.0.0 release notes for specific attribute replacements.
deprecated Protobuf serialization is now an optional dependency. If you use `DDSketch.to_proto()` or `DDSketch.from_proto()`, you must install the `serialization` extra.
Fix: Install with `pip install ddsketch[serialization]` to ensure Protobuf functionality is available.
gotcha The import path for `DDSketch` changed from `ddsketch.ddsketch.DDSketch` to `ddsketch.DDSketch` to simplify top-level access.
Fix: Update import statements from `from ddsketch.ddsketch import DDSketch` to `from ddsketch import DDSketch`.

Install

pip install ddsketch Base installation
pip install ddsketch[serialization] With Protobuf serialization support

Imports

DDSketch
```
from ddsketch import DDSketch
```
The DDSketch class was moved to the top-level package namespace in v2.0.0 for easier access.

Quickstart

This example demonstrates initializing a DDSketch, adding data points, retrieving quantiles, and merging with another sketch.

from ddsketch import DDSketch

# Initialize a DDSketch with a desired relative accuracy
s = DDSketch(relative_accuracy=0.01) # 1% relative accuracy

# Add data points
s.add(1.0)
s.add(2.0)
s.add(3.0)
s.add(10.0, count=5) # Add 10.0 five times

# Get quantiles
print(f"Value at 0.5 quantile: {s.get_value_at_quantile(0.5)}")
print(f"Value at 0.9 quantile: {s.get_value_at_quantile(0.9)}")

# Get min/max values
print(f"Min value: {s.min_value}")
print(f"Max value: {s.max_value}")

# Merge another sketch
s2 = DDSketch(relative_accuracy=0.01)
s2.add(20.0)
s.merge(s2)
print(f"Value at 0.95 quantile after merge: {s.get_value_at_quantile(0.95)}")

view raw JSON →