persist-queue

1.1.0 verified Fri May 01 auth: no python

A thread-safe, disk-based persistent queue for Python with support for file, SQLite, MySQL, and auto-recovery. Current version 1.1.0, updated with priority queues and sync API. Release cadence: occasional.

pip install persist-queue

Common errors

error ImportError: cannot import name 'Queue' from 'persistqueue' ↓

cause Old import path `from persistqueue import Queue` not available in versions before 1.0.0; or using wrong package name.

fix

Upgrade to persist-queue >=1.0.0: pip install -U persist-queue.

error AttributeError: 'Queue' object has no attribute 'full' ↓

cause Method `full()` was added in v0.8.1. Older versions do not have it.

fix

Upgrade to v0.8.1 or later: pip install -U persist-queue.

error TypeError: put() got an unexpected keyword argument 'raw' ↓

cause The `raw` argument is only supported for SQLite-based queues, not file-based Queue.

fix

Use q.put(item, raw=True) only with SQLiteQueue or MySQLQueue.

Warnings

breaking In v1.0.0, Python 2 support was removed and import paths changed. Top-level Queue moved from `persistqueue.file` to `persistqueue`. ↓

fix Update imports: `from persistqueue import Queue` instead of `from persistqueue.file import Queue`.

gotcha File-based queue stores data in pickled format by default. Using different Python versions or serializers (e.g., msgpack, cbor) may cause compatibility issues. ↓

fix Always use the same serializer and Python version when reading old queues. Consider using SQLite-based queues for cross-version compatibility.

deprecated Python 2 and Python 3.4 are deprecated since v0.8.0 and removed in v1.0.0. These versions are no longer supported. ↓

fix Upgrade to Python 3.5+ and persist-queue v1.0.0+.

gotcha SQLiteQueue does not support multiple writers across separate processes. Only use within a single process or use MySQLQueue for concurrent access. ↓

fix Use MySQLQueue if cross-process concurrency is needed.

Imports

Queue

wrong

from persistqueue.file import Queue

correct

from persistqueue import Queue

Queue is now at top-level since v1.0.0

SQLiteQueue

from persistqueue.sqlite import SQLiteQueue

MySQLQueue

from persistqueue.mysql import MySQLQueue

Quickstart

Basic usage of file and SQLite persistent queues.

import os
from persistqueue import Queue

# File-based queue
q = Queue('myqueue')
q.put('foo')
q.put('bar')
print(q.get())  # foo
print(q.get())  # bar

# SQLite-based queue
from persistqueue.sqlite import SQLiteQueue
sqlq = SQLiteQueue('myqueue.db', auto_commit=True)
sqlq.put({'task': 'example'})
print(sqlq.get())  # {'task': 'example'}

# To ensure auth check (not needed for persist-queue):
db_path = os.environ.get('PERSIST_QUEUE_PATH', 'default.db')
print('Queue path:', db_path)