Linear Operator

0.6.1 · active · verified Sun Apr 12

LinearOperator is a PyTorch package for abstracting away linear algebra routines needed for structured matrices or operators, primarily designed for finite-dimensional positive definite operators (i.e., kernel matrices). It is actively developed, with its current version being 0.6.1, and typically releases maintenance updates and minor versions frequently.

Warnings

breaking Version 0.6 introduced stricter Python and PyTorch version requirements. Python 3.10+ and PyTorch 2.0+ are now mandatory.
Fix: Upgrade Python to 3.10 or higher and PyTorch to 2.0 or higher. For PyTorch, ensure you install the correct version for your CUDA toolkit.
gotcha The internal methods for sparse tensor construction were updated in v0.6.1. Users who have custom `LinearOperator` implementations or interact directly with sparse tensor construction might encounter changes or deprecation warnings.
Fix: Review your sparse tensor construction logic and update it to align with PyTorch's current best practices, as the library now uses deprecated sparse tensor construction methods internally.
breaking The initialization arguments for `_dtype_value_context` changed in version 0.4.0. If you have custom `LinearOperator` subclasses that override or interact with this internal context, your code may break.
Fix: Adjust the argument passing to `_dtype_value_context` in your custom `LinearOperator` implementations according to the new signature in v0.4.0.
gotcha In version 0.6.1, `BlockDiagLinearOperator` now converts its `base_linear_op` to a dense linear operator. This might lead to increased memory usage or altered performance if you were relying on lazy evaluation of the base operator within `BlockDiagLinearOperator` in previous versions.
Fix: Be aware of potential memory and performance impacts when using `BlockDiagLinearOperator` with complex `base_linear_op` instances. Evaluate if this change affects your specific use case.

Install

pip install linear_operator PyPI
conda install linear_operator -c gpytorch Conda

Imports

LinearOperator
```
from linear_operator.operators import LinearOperator
```
This is the base class for all linear operators. Specific operators like DiagLinearOperator, KroneckerProductLinearOperator, etc., are also imported from `linear_operator.operators`.

DiagLinearOperator

from linear_operator.operators import DiagLinearOperator

LowRankRootLinearOperator

from linear_operator.operators import LowRankRootLinearOperator

Quickstart

This example demonstrates how to construct a large-scale structured matrix (a sum of a low-rank matrix and a diagonal matrix) as a `LinearOperator` object. The library implicitly handles the algebraic structure, allowing efficient operations like linear solves using `torch.linalg.solve` without ever forming the full, dense matrix, which is crucial for large-scale problems.

import torch
from linear_operator.operators import LowRankRootLinearOperator, DiagLinearOperator

# Example: Represent a 10000 x 10000 matrix A = C C^T + D
# A is never explicitly instantiated as a dense matrix, saving memory
C = torch.randn(10000, 20) # A "skinny" matrix (e.g., 10000x20)
d = torch.randn(10000).abs() + 1e-6 # Diagonal elements (ensure positive)
b = torch.randn(10000)

A = LowRankRootLinearOperator(C) + DiagLinearOperator(d)

# Perform a linear solve efficiently without instantiating the full dense matrix
# This uses structure-exploiting algorithms like Woodbury formula under the hood
x = torch.linalg.solve(A, b)

print(f"Shape of A: {A.shape}")
print(f"Shape of b: {b.shape}")
print(f"Shape of x (solution): {x.shape}")

view raw JSON →