{"id":4515,"library":"django-postgres-copy","title":"Django PostgreSQL Copy","description":"django-postgres-copy is a Django package that facilitates fast and efficient bulk import and export of delimited data (like CSV files) to and from PostgreSQL databases, leveraging PostgreSQL's native `COPY` command. It is significantly faster than using Django's ORM for large datasets. The library is actively maintained, currently at version 2.8.0, and regularly updates to support recent Django and Python versions.","status":"active","version":"2.8.0","language":"en","source_language":"en","source_url":"https://github.com/palewire/django-postgres-copy","tags":["Django","PostgreSQL","COPY","CSV","data import","data export","bulk operations","performance"],"install":[{"cmd":"pip install django-postgres-copy","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core framework dependency. Compatible with Django 4.2, 5.1, 5.2.","package":"Django"},{"reason":"PostgreSQL adapter. Version 2.8.0 introduced support for `psycopg` (v3), alongside continued compatibility with `psycopg2`. `psycopg[binary]` or `psycopg[c]` are common installation choices.","package":"psycopg"},{"reason":"PostgreSQL adapter (older version). Still supported but `psycopg` (v3) is recommended for new projects and newer Django versions.","package":"psycopg2"}],"imports":[{"note":"Attach this manager to your Django model to enable COPY operations.","symbol":"CopyManager","correct":"from postgres_copy import CopyManager"}],"quickstart":{"code":"import os\nfrom django.db import models\nfrom postgres_copy import CopyManager\n\n# Define a simple Django model with CopyManager\nclass Person(models.Model):\n    name = models.CharField(max_length=500)\n    number = models.IntegerField(null=True)\n    date = models.DateField(null=True)\n\n    objects = CopyManager()\n\n    class Meta:\n        app_label = 'myapp' # Replace with your app's name\n\n# Example CSV content for 'data.csv'\n# name,number,date\n# ben,1,2024-01-01\n# joe,2,2024-01-02\n# jane,3,2024-01-03\n\n# Create a dummy CSV file for the example (in a real scenario, this would be an existing file)\ncsv_content = \"name,number,date\\nben,1,2024-01-01\\njoe,2,2024-01-02\\njane,3,2024-01-03\"\nwith open(\"data.csv\", \"w\") as f:\n    f.write(csv_content)\n\n# --- Import data from CSV ---\n# In a real Django setup, ensure your database connection is configured and migrations are run.\n# Example: Person.objects.from_csv(\n#    \"./data.csv\", \n#    dict(name=\"name\", number=\"number\", date=\"date\") # Mapping model fields to CSV headers\n# )\n\nprint(\"To import from CSV:\")\nprint(\"Person.objects.from_csv('./data.csv', dict(name='name', number='number', date='date'))\")\n\n# --- Export data to CSV (assuming some data exists in the model) ---\n# Example: Person.objects.to_csv(\"./export.csv\")\n\nprint(\"\\nTo export to CSV:\")\nprint(\"Person.objects.to_csv('./export.csv')\")\n\n# Clean up dummy file\nos.remove(\"data.csv\")\n# if os.path.exists(\"export.csv\"): os.remove(\"export.csv\") # Uncomment if testing export\n","lang":"python","description":"To use `django-postgres-copy`, first attach `CopyManager` to your Django model. You can then use the `from_csv()` method on your model's manager to import data from a CSV file, providing a path to the file and a dictionary mapping model fields to CSV headers. Similarly, `to_csv()` exports data. The library handles creating temporary tables and inserting data efficiently."},"warnings":[{"fix":"Review `psycopg3` documentation for breaking changes if migrating from `psycopg2`. Ensure your project's database configuration is compatible with the `psycopg` version installed.","message":"The library added support for `psycopg` (v3) in version 2.8.0, while maintaining `psycopg2` compatibility. However, `psycopg3` introduces several breaking changes compared to `psycopg2` in its underlying API. Although `django-postgres-copy` has a compatibility layer, users with existing direct `psycopg2` database interactions or complex connection pooling setups may need to adjust their project configuration or code during migration to `psycopg3`.","severity":"gotcha","affected_versions":"2.8.0+"},{"fix":"Upgrade `django-postgres-copy` to the latest version compatible with your Django version. Consult the project's `pyproject.toml` or release notes for specific compatibility information.","message":"`django-postgres-copy` is regularly updated for Django version compatibility. Older versions of `django-postgres-copy` might not be fully compatible with newer Django versions, or vice-versa. For example, version 2.7.3 included a fix for 'Django 4.2+ compatible `setup_query`'. Always check the PyPI classifiers or GitHub releases for specific Django version support when upgrading either library.","severity":"breaking","affected_versions":"<2.7.3 with Django 4.2+"},{"fix":"Always sanitize or strictly validate any user-provided input used to construct dynamic SQL statements within `copy_field_template` or `setup_query` methods. Prefer fixed, predefined transformations where possible.","message":"The library allows for powerful SQL-based data transformations during import using methods like `copy_field_template` or `setup_query`. While flexible, if these SQL statements are constructed using unsanitized user-provided input, they can introduce SQL injection vulnerabilities.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be aware of this behavior for critical data. Consider performing imports within a robust transaction (though the library itself may operate outside one for performance) or backing up your database before large-scale imports. The `drop_constraints` and `drop_indexes` parameters can be set to `False` if strict integrity is needed throughout, though with a performance penalty.","message":"By default, for performance optimization during large imports, the `from_csv` method (via `CopyMapping`) temporarily drops database constraints and indexes, then recreates them. While this speeds up the process, it means data integrity checks are momentarily suspended, which could potentially expose data to inconsistencies if an unexpected error occurs during the import process.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If importing custom primary keys, consider manually updating the PostgreSQL sequence for the table after the `from_csv` operation, or set the `columns` parameter in `from_csv` to exclude the primary key column to let PostgreSQL assign it.","message":"When importing data that includes auto-incrementing primary keys (like Django's default `id` field) using `COPY FROM`, PostgreSQL's sequence generator for that table is not automatically updated. If you subsequently insert new records via Django's ORM, `IntegrityError` or `UniqueViolation` exceptions may occur if the ORM tries to insert an ID that conflicts with a manually `COPY`-inserted ID.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}