Python Pickle Parser for JavaScript

Common errors

• TypeError: Do not know how to serialize a BigInt

  cause Attempting to `JSON.stringify()` an object parsed from a pickle file that contains `BigInt` values without a custom `replacer` function.
  fix Modify your `JSON.stringify()` call to include a `replacer` function that converts `BigInt` values to strings or another serializable type: `JSON.stringify(obj, (key, value) => typeof value === 'bigint' ? value.toString() : value)`.

• Error: Input must be a Buffer or Uint8Array

  cause The `parser.parse()` method was invoked with an incorrect data type (e.g., a plain string or an object) instead of a `Buffer` (Node.js) or `Uint8Array` (browser) containing the raw binary pickle data.
  fix Ensure the input data is read in binary format and converted to the correct type. For Node.js, use `fs.readFileSync(filePath, null)` to get a `Buffer`. For browsers, use a `FileReader` to read as `ArrayBuffer` and convert to `Uint8Array`.

• ReferenceError: pickleparser is not defined

  cause This error typically occurs when trying to use `new pickleparser.Parser()` in a browser environment where the `pickleparser` global script was not correctly loaded, or in a Node.js environment where this global access pattern is invalid.
  fix In browser applications, verify the library's `<script>` tag is correctly placed and loaded. In Node.js or module-based environments, use explicit ESM `import { Parser } from 'pickleparser';` or CJS `const { Parser } = require('pickleparser');`.

Warnings

• breaking The public API underwent a significant refactor for improved extensibility in version `0.1.0-beta.0`. Users upgrading from pre-`0.1.0-beta.0` versions must adjust their import statements and parsing logic.
  Fix: Refer to the latest documentation and examples for `Parser` class instantiation and method calls. The module now primarily uses named exports like `Parser`.
• gotcha Python pickle files can contain large integers that, when parsed, are represented as JavaScript `BigInt` primitives. Standard `JSON.stringify()` cannot directly serialize `BigInt` values and will throw a `TypeError` if not handled.
  Fix: Provide a custom `replacer` function to `JSON.stringify()` to convert `BigInt`s to strings or another serializable format. For example: `JSON.stringify(obj, (key, value) => typeof value === 'bigint' ? value.toString() + 'n' : value)`.
• breaking Earlier versions (prior to `0.0.2-alpha.2`) contained an issue with insecure string handling in pickle protocol 0. This security vulnerability was addressed in `0.0.2-alpha.2`.
  Fix: Upgrade to `pickleparser@0.0.2-alpha.2` or newer immediately to ensure all known security fixes are applied and to benefit from the most stable and secure parsing logic.
• gotcha Parsing untrusted Python pickle data is a significant security risk. In Python, unpickling arbitrary data can lead to arbitrary code execution due to the nature of the pickle protocol. While `pickleparser` is a JavaScript implementation and does not execute Python code, the reconstructed JavaScript objects could still contain malicious data structures (e.g., specific string values, function names) that, if processed without sanitization in the consuming JavaScript application, could lead to security vulnerabilities.
  Fix: **Never parse pickle files from untrusted or unverified sources.** Always treat parsed data as potentially hostile and implement robust validation and sanitization before using it within your application logic.

Install

• npm install pickleparser npm
• yarn add pickleparser yarn
• pnpm add pickleparser pnpm

Imports

• Parser
  wrong:

  const { Parser } = require('pickleparser');

  correct:

  import { Parser } from 'pickleparser';

  The `Parser` class is the primary interface for unpickling. While CommonJS (CJS) syntax might work in some setups, ESM is the recommended and best-supported import style for modern Node.js and bundlers.
• ParserOptions
  wrong:

  import { ParserOptions } from 'pickleparser';

  correct:

  import type { ParserOptions } from 'pickleparser';

  Used for configuring parser behavior (e.g., custom object handling). As `ParserOptions` is typically an interface or type, prefer `import type` to ensure it's treated as a type-only import, which avoids potential runtime issues or bundle size increases.
• pickleparser (global)
  wrong:

  import { pickleparser } from 'pickleparser';

  correct:

  const parser = new pickleparser.Parser();

  In browser environments where the library is loaded via a `<script>` tag, `pickleparser` becomes a global object. Do not attempt to use this global access pattern in Node.js or when using module bundlers, where explicit imports are required.

Quickstart

Demonstrates how to read a Python pickle file from disk using Node.js, parse it with `pickleparser`, and log the resulting JavaScript object to the console, including a `BigInt` replacer for `JSON.stringify()`.

import fs from 'node:fs/promises';
import path from 'node:path';
import { Parser } from 'pickleparser';

async function unpickleFile(filePath: string) {
    try {
        // Read the pickle file as raw binary data (Buffer in Node.js)
        const pklData = await fs.readFile(filePath, null); 
        const buffer = Buffer.from(pklData);

        // Initialize the parser and parse the buffer
        const parser = new Parser();
        const obj = parser.parse(buffer);
        
        // Log the parsed object. Use a replacer for JSON.stringify to handle BigInts.
        console.log(JSON.stringify(obj, (key, value) =>
            typeof value === 'bigint' ? value.toString() + 'n' : value
        , 2));
    } catch (error) {
        console.error(`Error unpickling file ${filePath}:`, error);
    }
}

// Example usage: Ensure a pickle file exists at the specified path.
// For demonstration, we use an environment variable or a default.
unpickleFile(process.env.PICKLE_FILE_PATH ?? './example.pkl');

// To create a dummy pickle file for testing (run this in Python):
// import pickle
// data = {'message': 'Hello from Python!', 'value': 12345678901234567890, 'items': [1, 2, 3]}
// with open('example.pkl', 'wb') as f:
//     pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)