{"id":2616,"library":"ndjson","title":"NDJSON Decoder for Python","description":"The `ndjson` library for Python, currently at version `0.3.1`, provides a `JsonDecoder` and `JsonEncoder` for newline-delimited JSON (NDJSON), also known as JSON Lines. It offers a familiar interface similar to Python's built-in `json` module, enabling efficient reading and writing of NDJSON data to and from file-like objects and strings. This lightweight library has no external dependencies and is particularly useful for processing large datasets or streaming applications where each line represents a complete, independent JSON object. Although its last release was in 2020 and its PyPI status is 'Pre-Alpha', it is considered stable and functional for its stated purpose.","status":"active","version":"0.3.1","language":"en","source_language":"en","source_url":"https://github.com/rhgrant10/ndjson","tags":["json","ndjson","jsonlines","serialization","deserialization","stream","newline-delimited"],"install":[{"cmd":"pip install ndjson","lang":"bash","label":"Install stable version"}],"dependencies":[],"imports":[{"symbol":"ndjson","correct":"import ndjson"},{"symbol":"load","correct":"ndjson.load(file_object)"},{"symbol":"dump","correct":"ndjson.dump(data, file_object)"},{"symbol":"loads","correct":"ndjson.loads(string_data)"},{"symbol":"dumps","correct":"ndjson.dumps(data)"},{"symbol":"reader","correct":"ndjson.reader(file_object)"},{"symbol":"writer","correct":"ndjson.writer(file_object)"}],"quickstart":{"code":"import ndjson\nimport os\n\n# Example data\ndata_to_write = [\n    {\"name\": \"Alice\", \"age\": 30, \"city\": \"New York\"},\n    {\"name\": \"Bob\", \"age\": 24, \"city\": \"San Francisco\"},\n    {\"name\": \"Charlie\", \"age\": 35, \"city\": \"London\"}\n]\n\nfile_path = \"example.ndjson\"\n\n# --- Writing NDJSON to a file ---\nwith open(file_path, 'w', encoding='utf-8') as f:\n    # Using ndjson.dump for a list of objects\n    ndjson.dump(data_to_write, f)\nprint(f\"Data written to {file_path} using ndjson.dump\")\n\n# Alternatively, using ndjson.writer for streaming individual rows\nfile_path_writer = \"example_writer.ndjson\"\nwith open(file_path_writer, 'w', encoding='utf-8') as f:\n    writer = ndjson.writer(f)\n    for record in data_to_write:\n        writer.writerow(record)\nprint(f\"Data written to {file_path_writer} using ndjson.writer\")\n\n# --- Reading NDJSON from a file ---\nread_data_dump = []\nwith open(file_path, 'r', encoding='utf-8') as f:\n    # Using ndjson.load for reading all objects from a file\n    read_data_dump = ndjson.load(f)\nprint(f\"\\nData read from {file_path} (ndjson.load):\\n{read_data_dump}\")\n\n# Alternatively, using ndjson.reader for streaming individual rows\nread_data_reader = []\nwith open(file_path_writer, 'r', encoding='utf-8') as f:\n    reader = ndjson.reader(f)\n    for row in reader:\n        read_data_reader.append(row)\nprint(f\"\\nData read from {file_path_writer} (ndjson.reader):\\n{read_data_reader}\")\n\n# Clean up created files\nos.remove(file_path)\nos.remove(file_path_writer)\n","lang":"python","description":"This quickstart demonstrates how to write and read NDJSON data using the `ndjson` library's `dump`/`load` functions for bulk operations and `writer`/`reader` classes for streaming line-by-line processing, similar to Python's `csv` module. This is particularly efficient for large files, avoiding the need to load the entire dataset into memory."},"warnings":[{"fix":"Be aware of the PyPI status, but understand that the library is stable for its intended use cases. The version 0.3.1 has been consistent since 2020.","message":"The `ndjson` library's official PyPI status is \"2 - Pre-Alpha\", which might suggest instability or an experimental nature. However, the library has been stable since its 0.3.1 release in February 2020 and \"works as advertised\", making this status misleading for its current functional state.","severity":"gotcha","affected_versions":"0.1.0 - 0.3.1"},{"fix":"Use `ndjson.load(file_object)` or iterate with `ndjson.reader(file_object)` for stream-based parsing, which correctly handles newline-delimited JSON objects.","message":"Do not attempt to parse an entire NDJSON file using Python's built-in `json.load()` (e.g., `json.load(open('data.ndjson'))`). This will typically result in a `json.JSONDecodeError` because NDJSON files contain multiple top-level JSON objects, not a single one, or a `MemoryError` for very large files.","severity":"gotcha","affected_versions":"All versions (general Python usage)"},{"fix":"The `ndjson` library handles this correctly with `ndjson.dump()` and `ndjson.writer()`. When manually constructing NDJSON, always ensure one valid JSON object per line.","message":"When writing NDJSON, ensure each record is a valid, self-contained JSON object on a single line, terminated by a newline character (`\\n`). Do not wrap the entire set of objects in a JSON array (`[]`) or add commas between objects, as this violates the NDJSON format and will cause parsing issues.","severity":"gotcha","affected_versions":"All versions (general NDJSON format adherence)"},{"fix":"Evaluate if the current feature set meets your needs. For very large-scale or high-performance NDJSON processing, consider alternatives like `ijson` for incremental parsing or `orjson` for faster JSON operations, or libraries integrated with dataframes like `polars.read_ndjson`.","message":"The library's last release (`0.3.1`) was in February 2020. While its core functionality is stable and complete for handling NDJSON, users seeking active development, bug fixes beyond the existing scope, or new features might find the project inactive.","severity":"deprecated","affected_versions":"0.3.1 and earlier"},{"fix":"Always specify `encoding='utf-8'` when opening NDJSON files. If a BOM is present and causing issues, use `encoding='utf-8-sig'` to automatically strip it.","message":"NDJSON files are expected to be UTF-8 encoded. Parsing issues can occur with files saved with a Byte Order Mark (BOM) or mixed encodings. This is a common issue for many text-based file formats.","severity":"gotcha","affected_versions":"All versions (general file handling)"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}