faster-fifo

1.5.2 verified Sat May 09 auth: no python

A drop-in replacement for Python's multiprocessing.Queue that uses a shared-memory ring buffer and a POSIX semaphore for lower latency and higher throughput. Version 1.5.2 requires Python >=3.9. Actively maintained.

pip install faster-fifo

Common errors

error ModuleNotFoundError: No module named 'faster_fifo' ↓

cause The library is not installed, or installed in a different environment.

fix

Run pip install faster-fifo --upgrade in the correct environment.

error OSError: [Errno 28] No space left on device ↓

cause Shared memory filesystem (e.g., /dev/shm) is full. Faster-fifo uses POSIX shared memory.

fix

Increase /dev/shm size (e.g., mount -o remount,size=2G /dev/shm) or reduce queue/message size.

error AttributeError: module 'faster_fifo' has no attribute 'Queue' ↓

cause Importing the module instead of the class: `import faster_fifo` then using `faster_fifo.Queue` is correct, but some users mistakenly do `from faster_fifo import faster_fifo_queue`.

fix

Use from faster_fifo import Queue or from faster_fifo import Queue as FQueue.

Warnings

gotcha The underlying implementation uses shared memory; do not pass large objects (e.g., >1GB) without testing, as they may cause memory pressure. ↓

fix For very large data, consider serializing to disk and sending a path.

gotcha faster-fifo Queue is not a drop-in replacement for multiprocessing.Queue in all cases; e.g., it does not support the `timeout` parameter in `get()` (it blocks indefinitely by default). ↓

fix Use `q.get(timeout=0.5)` is not supported; implement your own timeout with a separate thread/process.

gotcha Objects must be picklable (default serializer is pickle). If you use custom objects that are not picklable, the queue will fail. ↓

fix Ensure all queued objects are picklable or switch to a different serializer (unsupported).

Imports

Queue

wrong

from faster_fifo import faster_fifo_queue

correct

from faster_fifo import Queue

The main class is named `Queue`, not `faster_fifo_queue`.

Queue
wrong
```
from multiprocessing import Queue
```
correct
```
from faster_fifo import Queue
```
Use the faster_fifo Queue to get speed improvements; multiprocessing.Queue is the fallback.

Quickstart

Create a Queue, put a message, get it back. Works like multiprocessing.Queue but faster.

from faster_fifo import Queue
import os

q = Queue()
q.put('hello')
msg = q.get()
print(msg)  # hello