Atomic File Writes

1.4.1 · deprecated · verified Thu Apr 09

Atomicwrites is a Python library that provides a simple, high-level API for performing atomic file writes. It achieves atomicity by writing to a temporary file in the same directory and then atomically moving it to the target location. This prevents file corruption in case of crashes or interruptions during the write process. The library is currently at version 1.4.1. While previously actively maintained, its status has shifted due to the maintainer's decision regarding PyPI 2FA requirements, leading to its deprecation. Users are encouraged to consider alternatives like `safer` for new projects.

Warnings

deprecated The `atomicwrites` library is officially deprecated by its maintainer as of July 2022. The maintainer indicated that Python 3's built-in `os.replace` and `os.rename` functions might suffice for many use cases, and cited PyPI's 2FA requirements as a reason for deprecation.
Fix: For new projects, consider actively maintained alternatives like `safer`. For simple atomic renames, Python's `os.replace` (for overwriting) or a combination of `tempfile` and `os.rename` can be used.
gotcha On Windows, the atomicity of the `MoveFileEx` system call (used by `atomicwrites`) is not universally guaranteed. It can, under certain conditions, silently fall back to a non-atomic copy-and-delete operation, potentially leading to data inconsistencies if the system crashes during this fallback.
Fix: Be aware of this platform-specific limitation. For critical applications on Windows requiring absolute atomicity, additional verification or platform-specific mechanisms might be necessary. Thorough testing on the target Windows environment is recommended.
gotcha Atomic file operations are only guaranteed if the temporary file created by `atomicwrites` and the final target file reside on the *same filesystem*. If they are on different filesystems (e.g., a network drive vs. local disk), the operation might fall back to a non-atomic copy, which is susceptible to interruption failures.
Fix: Ensure that the target file path is on the same filesystem as the directory where temporary files will be created (by default, `atomicwrites` creates temporary files in the same directory as the target path to mitigate this).
gotcha When `overwrite=False` is used in `atomic_write` (which employs `os.link` and `os.unlink` on POSIX systems), there is a brief time window where the file might be accessible under both its temporary name and the target name. Additionally, this method may lead to changes in the target file's permissions if the link operation doesn't preserve them as expected.
Fix: If `overwrite=False` is critical, carefully test and verify permission preservation on your target platform. Design applications to be robust against a brief dual-presence of the file.

Install

pip install atomicwrites PyPI

Imports

atomic_write
```
from atomicwrites import atomic_write
```
The primary high-level context manager for atomic file operations.
AtomicWriter
```
from atomicwrites import AtomicWriter
```
A lower-level, class-based API for more granular control over atomic writes.

Quickstart

This example demonstrates how to use `atomic_write` to safely write JSON data to a file. The `overwrite=True` parameter allows replacing an existing file. If an error occurs during the write, the original file remains untouched. The example also includes cleanup for re-runnability and demonstrates reading the content back.

import os
from atomicwrites import atomic_write
import json

file_path = "my_config.json"
config_data = {"api_key": os.environ.get('MY_API_KEY', 'default_secret'), "timeout": 30}

try:
    # Write configuration atomically
    with atomic_write(file_path, overwrite=True, encoding='utf-8') as f:
        json.dump(config_data, f, indent=2)
    print(f"Configuration successfully written to {file_path}")

    # Verify content by reading back
    with open(file_path, 'r', encoding='utf-8') as f:
        read_config = json.load(f)
    print(f"Read back configuration: {read_config}")

except Exception as e:
    print(f"An error occurred during atomic write: {e}")
    # In case of an error, the original file (if any) should be intact
    # or no partial file should exist if it was a new creation.
    if not os.path.exists(file_path):
        print(f"Target file {file_path} does not exist after error (expected behavior for new file).")
    else:
        print(f"Target file {file_path} exists after error (expected behavior if original was preserved).")

finally:
    # Clean up the created file for a runnable example
    if os.path.exists(file_path):
        os.remove(file_path)
        print(f"Cleaned up {file_path}")

view raw JSON →