POSIX IPC for Python

Common errors

• posix_ipc.ExistentialError: semaphore doesn't exist

  cause Attempted to acquire or release a semaphore that has been closed (via `sem.close()`) or unlinked, or to open a semaphore with default flags when it doesn't exist.
  fix Ensure the semaphore object is valid and has not been closed or unlinked. If opening, use `posix_ipc.O_CREAT` to create it if it doesn't exist, or `posix_ipc.O_CREAT | posix_ipc.O_EXCL` for exclusive creation.

• posix_ipc.PermissionsError: Operation not permitted

  cause The current process lacks the necessary permissions to create, open, or operate on the IPC object (e.g., trying to write to a read-only shared memory segment, or opening an object with incompatible permissions).
  fix Check the `mode` parameter when creating IPC objects (e.g., `0o600` for owner read/write). Ensure the process has sufficient user/group permissions for the IPC object. Verify that `umask` settings are not overly restrictive.

• posix_ipc.ExistentialError: Message queue doesn't exist

  cause Attempting to interact with a message queue that was either never created, has been unlinked, or the process does not have permission to access it, and `O_CREAT` was not used. Also common on unsupported platforms like macOS.
  fix Ensure the message queue is created with `posix_ipc.O_CREAT`. If on macOS, remember that message queues are not supported by `posix-ipc`. Verify that the queue name is consistent across processes and that permissions allow access.

Warnings

• deprecated The module constants `PAGE_SIZE` and `SEMAPHORE_VALUE_MAX` have been deprecated as of version 1.3.0 and will be removed in a future release. Avoid using them in new code.
  Fix: Migrate away from using `posix_ipc.PAGE_SIZE` and `posix_ipc.SEMAPHORE_VALUE_MAX`. Consult system `os.sysconf` or `mmap` documentation for alternatives if needed.
• gotcha POSIX IPC objects (semaphores, shared memory, message queues) are persistent and survive program termination. Failing to explicitly unlink them can lead to resource leaks or unexpected behavior from stale objects.
  Fix: Always call `obj.unlink()` on IPC objects when they are no longer needed. Use `try...finally` blocks to ensure unlinking, even if errors occur during operation. Consider using `O_EXCL` with `O_CREAT` for exclusive creation to avoid conflicts with existing objects.
• gotcha The `posix_ipc.close()` method sets the internal C pointer to NULL but does not unlink the IPC object from the system. Attempting to acquire or release a semaphore after `close()` but before `unlink()` will raise an `ExistentialError`.
  Fix: Understand that `close()` is for releasing a process's reference, while `unlink()` removes the system-wide object. Call `unlink()` when the object is globally no longer needed by any process. Avoid using a `posix_ipc` object after calling `close()` on it, unless for `unlink()` or `name` access.
• gotcha Message queues and certain semaphore functions (`sem_getvalue()`, `sem_timedwait()`) are not supported on macOS by the underlying OS, and thus `posix-ipc` cannot provide them. Windows requires Cygwin or WSL for POSIX IPC support.
  Fix: Be aware of platform limitations. Test IPC functionality on the target OS. For macOS, avoid message queues entirely when using `posix-ipc`. For cross-platform Python IPC, consider `multiprocessing` or `multiprocessing.shared_memory`.
• gotcha For inter-process communication *between Python programs*, it is generally recommended to use Python's built-in `multiprocessing` module (or `multiprocessing.shared_memory` for shared memory) instead of `posix-ipc`. `posix-ipc` is designed primarily for IPC between Python and *non-Python* applications.
  Fix: If all communicating processes are Python, evaluate `multiprocessing` first for simpler and more Pythonic IPC patterns.

Install

• pip install posix-ipc Install from PyPI

Imports

• Semaphore

  from posix_ipc import Semaphore

• SharedMemory

  from posix_ipc import SharedMemory

• MessageQueue

  from posix_ipc import MessageQueue

• O_CREAT

  from posix_ipc import O_CREAT

• O_EXCL

  from posix_ipc import O_EXCL

Quickstart

This quickstart demonstrates the basic usage of POSIX semaphores, shared memory, and message queues. It shows how to create, use, and explicitly clean up these IPC objects. Note the platform-specific behavior for message queues. For shared memory, only creation and unlinking are shown; actual data exchange requires `mmap`.

import posix_ipc
import os

# Semaphore Example
SEM_NAME = '/my_semaphore'
try:
    sem = posix_ipc.Semaphore(SEM_NAME, posix_ipc.O_CREAT, initial_value=1)
    print(f"Semaphore '{SEM_NAME}' created/opened. Value: {sem.value}")
    sem.acquire()
    print(f"Semaphore acquired. Value: {sem.value}")
    sem.release()
    print(f"Semaphore released. Value: {sem.value}")
finally:
    if 'sem' in locals() and sem.name == SEM_NAME: # Ensure it's our semaphore and still valid
        sem.close()
        sem.unlink()
        print(f"Semaphore '{SEM_NAME}' closed and unlinked.")

# Shared Memory Example
SHM_NAME = '/my_shared_memory'
SIZE = 128
try:
    shm = posix_ipc.SharedMemory(SHM_NAME, posix_ipc.O_CREAT, size=SIZE)
    print(f"Shared Memory '{SHM_NAME}' created/opened with size {SIZE}.")
    # In a real app, you'd mmap this and write/read bytes
    # e.g., memory_map = mmap.mmap(shm.fd, shm.size)
    # memory_map.write(b'Hello from shared memory!')
finally:
    if 'shm' in locals() and shm.name == SHM_NAME:
        shm.close_fd() # Close the file descriptor
        shm.unlink()
        print(f"Shared Memory '{SHM_NAME}' unlinked.")

# Message Queue Example (Note: Message queues are not supported on macOS by posix-ipc)
# This example will likely fail on macOS due to OS limitations.
# For a runnable example, execute on Linux/WSL.
MQ_NAME = '/my_message_queue'
if os.uname().sysname != 'Darwin': # Check if not macOS
    try:
        mq = posix_ipc.MessageQueue(MQ_NAME, posix_ipc.O_CREAT, max_messages=10, max_message_size=64)
        print(f"Message Queue '{MQ_NAME}' created/opened.")
        message = b"Hello from message queue!"
        mq.send(message)
        print(f"Sent: {message.decode()}")
        received_message, _ = mq.receive()
        print(f"Received: {received_message.decode()}")
    except posix_ipc.PermissionsError as e:
        print(f"Caught PermissionsError for MessageQueue: {e}. This might happen if queue exists with different permissions.")
    except posix_ipc.ExistentialError as e:
        print(f"Caught ExistentialError for MessageQueue: {e}. This might happen if queue exists with incompatible settings.")
    except Exception as e:
        print(f"An unexpected error occurred with MessageQueue: {e}")
    finally:
        if 'mq' in locals() and mq.name == MQ_NAME:
            try:
                mq.close()
                mq.unlink()
                print(f"Message Queue '{MQ_NAME}' closed and unlinked.")
            except Exception as e:
                print(f"Error during MessageQueue cleanup: {e}")
else:
    print("MessageQueue example skipped: Not supported on macOS by posix-ipc.")