Bitsandbytes

Bitsandbytes is a Python library that provides k-bit optimizers and matrix multiplication routines, primarily designed for making large language models (LLMs) more accessible through quantization in PyTorch. It focuses on dramatically reducing memory consumption for both inference and training via 8-bit and 4-bit quantization, including techniques like LLM.int8() and QLoRA. The library is actively maintained, currently at version 0.49.2, and frequently updated.