Sparse n-dimensional arrays

0.18.0 · active · verified Sat Apr 11

Sparse is a Python library that provides n-dimensional arrays for the PyData ecosystem, optimized for data with a large number of zero or 'fill' values. It aims to offer a drop-in replacement for NumPy arrays with support for N-dimensional operations, following the Array API standard. The current version is 0.18.0, and it maintains an active development cycle with frequent releases.

Warnings

breaking Automatic densification of sparse arrays into NumPy functions now raises a RuntimeError.
Fix: Explicitly convert the sparse array to a dense NumPy array using `.todense()` or `.toarray()` before passing it to NumPy functions that do not have sparse-aware implementations. This prevents unintended memory exhaustion.
gotcha The `*` operator performs element-wise multiplication, while `@` performs matrix multiplication (dot product).
Fix: Always use `@` for matrix multiplication with `sparse` arrays, consistent with NumPy's `ndarray` behavior. The `*` operator is reserved for element-wise operations.
gotcha Operations on `sparse` arrays, especially element-wise functions like `np.where`, propagate the `fill_value` (defaulting to 0). Unexpected results can occur if your data or mask implicitly relies on a non-zero `fill_value` behavior.
Fix: Be mindful of the `fill_value` of sparse arrays (which is usually 0). If your logic depends on a different implicit value or specific handling of `fill_value` during operations, ensure it's explicitly managed or converted to dense where necessary.
gotcha Directly applying general NumPy functions to `sparse` arrays without explicit conversion or a sparse-aware implementation can lead to inefficient computation or incorrect results.
Fix: Before applying a NumPy function, check if `sparse` (or `scipy.sparse` for 2D cases) offers a specialized sparse implementation. Otherwise, convert the sparse array to a dense NumPy array using `.todense()` or `.toarray()` first.
gotcha Migration from `scipy.sparse` matrix API (e.g., `csr_matrix`) to the array API (e.g., `csr_array`) involves changes in constructor names, operator behavior (`*` vs `@`), and potential changes in return dimensions.
Fix: Adopt the `_array` constructors (e.g., `csr_array`) for new code and when refactoring. Update matrix multiplication from `*` to `@`. Be aware that array-like operations might return 1D arrays where matrices used to return 2D.

Install

pip install sparse Install with pip

Imports

sparse
```
import sparse
```
The library is typically imported directly as 'sparse'.
COO
```
from sparse import COO
```
Specific sparse array formats can be imported from the top-level package.

Quickstart

This quickstart demonstrates how to create sparse arrays using the `COO` format from a dictionary of coordinates and values, or by converting a dense NumPy array. It also shows a basic arithmetic operation and how to convert a sparse array back to a dense NumPy array.

import sparse
import numpy as np

# Create a sparse array from a dictionary of coordinates and values
x = sparse.COO({(0, 0): 1, (1, 2): 2}, shape=(3, 3))
print("Sparse array x:\n", x)
print("Dense representation of x:\n", x.todense())

# Create a sparse array from a NumPy array
y = np.arange(9).reshape((3, 3))
z = sparse.COO.from_numpy(y)
print("Sparse array z from NumPy:\n", z)

# Perform an operation (addition) and convert to dense
print("Dense representation of (x + z):\n", (x + z).todense())

view raw JSON →