Partial JSON Parser

0.2.1.1.post7 · active · verified Thu Apr 09

Partial JSON Parser is a lightweight and customizable Python library designed to parse incomplete or partially received JSON strings, commonly generated by Large Language Models (LLMs). It enables incremental processing of JSON data as it streams in, preventing errors that traditional `json.loads` would throw on incomplete input. The library is pure Python, has no external dependencies, and supports Python 3.6+.

Common errors

```
MalformedJSON: Malformed node or string on line 1.
```
cause The input string provided to `partial_json_parser.loads` is fundamentally malformed and cannot be parsed even with partial allowances enabled. This typically means the JSON structure is broken in an unrecoverable way, not just incomplete.

fix Ensure the input string, even if partial, adheres to a basic JSON structure that allows for completion. Review the raw string for severe syntax errors beyond simple truncation. For example, `loads("wrong")` will raise this error because it's not JSON at all.
```
ModuleNotFoundError: No module named 'partial_json_parser'
```
cause The `partial-json-parser` library is either not installed, or there's an attempt to import it using an incorrect module name. Python packages with hyphens in their name (`partial-json-parser`) are typically imported using underscores (`partial_json_parser`).

fix First, ensure the library is installed: `pip install partial-json-parser`. Then, correct the import statement to `from partial_json_parser import loads, Allow` (or other desired functions).
```
json.decoder.JSONDecodeError: Expecting value: line 1 column X (char Y)
```
cause This error occurs when an incomplete or malformed JSON string is passed to Python's standard `json.loads` function, which expects a fully valid JSON string. The `partial-json-parser` library is specifically designed to handle these scenarios.

fix Instead of `json.loads`, use `partial_json_parser.loads` and provide appropriate `Allow` flags to specify which types of partial structures are acceptable. For example: `from partial_json_parser import loads, Allow; result = loads('{"key": "v', Allow.STR | Allow.OBJ)`.
```
Bad escaped character in JSON at position X (line 1 column Y)
```
cause This error arises when the JSON string contains invalid escape sequences, such as unescaped backslashes (`\H` instead of `\\H`) that are not part of the standard JSON escape character set (e.g., `\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`). This is common with LLM outputs that might generate text with unescaped special characters.

fix Pre-process the LLM-generated JSON string to sanitize invalid escape sequences before passing it to the parser. A common fix is to replace single backslashes followed by an invalid character with double backslashes: `sanitized_json = json_string.replace(/\([^"\\/bfnrtu])/g, '\\$1')`. Also, wrap the parsing call in a `try-except` block to gracefully handle such failures.

Warnings

gotcha The `loads` function returns a Python object representing the parsed (potentially incomplete) JSON. If you need a *syntactically complete JSON string*, use `ensure_json` instead.
Fix: Use `from partial_json_parser import ensure_json` and call `ensure_json(partial_json_string)` to get a completed string, or use `json.dumps(loads(partial_json_string))` if you need to re-serialize the parsed object.
gotcha The `allow_partial` argument (using the `Allow` enum) is crucial for controlling what types of partialness the parser should tolerate. If not specified, it defaults to `Allow.ALL`, which might not always be the desired behavior for strict parsing.
Fix: Explicitly pass `allow_partial` using bitwise OR (`|`) with `Allow` members (e.g., `allow_partial=Allow.STR | Allow.OBJ`) to define precisely what incomplete structures are acceptable.
gotcha This library is designed for *partial* JSON (e.g., a string cut off mid-value or before an object is closed), not for *arbitrarily malformed* JSON with fundamental syntax errors like unquoted keys or missing commas where they should be. It assumes the underlying structure is mostly valid but incomplete.
Fix: Pre-validate or pre-process highly malformed JSON if it contains more than just incompleteness. This library excels at handling streaming JSON that is syntactically correct up to its truncation point, rather than repairing deep structural errors.

Install

pip install partial-json-parser Install stable version

Imports

loads
```
from partial_json_parser import loads
```
The primary function to parse partial JSON into a Python object.
Allow
```
from partial_json_parser import Allow
```
An Enum to specify what kind of partialness is permitted during parsing.
ensure_json
```
from partial_json_parser import ensure_json
```
Function to get a completed (fixed) JSON string from a partial one.

Quickstart

Demonstrates parsing partial JSON strings using `loads` with and without the `Allow` enum, and how to use `ensure_json` to get a syntactically complete JSON string.

from partial_json_parser import loads, Allow, ensure_json

# Example 1: Basic partial JSON parsing
partial_string_1 = '{"name": "Alice", "age": 3'
parsed_data_1 = loads(partial_string_1)
print(f"Parsed data (basic): {parsed_data_1}")

# Example 2: Parsing with specific allowed partialness (partial string and object)
partial_string_2 = '{"user": "John D'
parsed_data_2 = loads(partial_string_2, allow_partial=Allow.STR | Allow.OBJ)
print(f"Parsed data (with Allow): {parsed_data_2}")

# Example 3: Ensuring a complete JSON string
partial_string_3 = '{"city": "New York', "temp": 75'
completed_json_str = ensure_json(partial_string_3)
print(f"Completed JSON string: {completed_json_str}")

# Example 4: Edge case - empty string
empty_string_parsed = loads('')
print(f"Parsed empty string: {empty_string_parsed}")

view raw JSON →