Node.js LMDB Binding

Common errors

• MDB_MAP_FULL: Environment mapsize limit reached

  cause The configured mapSize for the LMDB environment is too small to accommodate new data or existing data growth.
  fix Increase the `mapSize` property in `env.open()` to a sufficiently large value. You may need to close and reopen the environment for the change to take effect.

• Error: MDB_BAD_TXN: Transaction has been aborted or is inactive

  cause Attempting to use a transaction object after it has been committed or aborted, or after the environment/database has been closed.
  fix Ensure all database operations are performed within the scope of an active transaction. Do not reuse `Txn` objects after `commit()` or `abort()`. Create a new transaction for each logical unit of work.

• Error: MDB_NOTFOUND: No matching key/data pair found

  cause An attempt was made to retrieve a key that does not exist in the specified database.
  fix This is often expected behavior. Check the return value of `get*()` methods, which will typically be `null` or `undefined` if the key is not found. Handle these cases gracefully in your application logic.

• node-gyp rebuild failed

  cause During `npm install`, the native C/C++ binding compilation failed due to missing build tools or incorrect environment setup.
  fix Install necessary build tools for your operating system. For Windows, install Visual Studio Build Tools. For Linux, install `build-essential` (Debian/Ubuntu) or `Development Tools` (Fedora/RHEL). Ensure Python 3 is installed and configured correctly for `node-gyp`.

Warnings

• gotcha Always explicitly close transactions using `commit()` or `abort()`. Failure to do so can lead to resource leaks, database corruption, or deadlocks, especially in environments with many concurrent operations.
  Fix: Ensure every call to `env.beginTxn()` is paired with a `txn.commit()` or `txn.abort()` in a `try...finally` block to guarantee transaction closure.
• breaking Versions prior to 0.7.0 might have different API signatures or less stable behavior. While specific breaking changes are not fully documented in the README, it's generally recommended to upgrade to the latest 0.10.x series for stability and features.
  Fix: Review the changelog or GitHub releases for specific breaking changes if migrating from older versions. Update to `node-lmdb@^0.10.0` and adapt code as necessary.
• gotcha The `mapSize` option in `env.open()` sets the maximum size the database file can ever grow to. If your data exceeds this, subsequent writes will fail with an `MDB_MAP_FULL` error.
  Fix: Carefully estimate your maximum database size requirements and set `mapSize` generously. It is advisable to set it larger than strictly necessary, as resizing often requires closing and reopening the environment.
• gotcha Node-lmdb is a native binding, meaning it compiles C/C++ code during installation. This can cause issues on systems without proper build tools (e.g., Python, C++ compiler).
  Fix: Ensure your development and deployment environments have `node-gyp` prerequisites installed (Python 3, build tools like GCC/Clang or Visual C++). Consult the `node-gyp` documentation for specific OS requirements.

Install

• npm install node-lmdb npm
• yarn add node-lmdb yarn
• pnpm add node-lmdb pnpm

Imports

• Env
  wrong:

  const Env = require('node-lmdb').Env;

  correct:

  import { Env } from 'node-lmdb';

  For ESM, use named imports. For CommonJS, destructure from the require call or access properties.
• Txn
  wrong:

  const Txn = require('node-lmdb');

  correct:

  import { Txn } from 'node-lmdb';

  Txn is a named export, not the default module export. Direct require() without destructuring will not work for specific classes.
• * as lmdb
  wrong:

  const lmdb = require('node-lmdb').default;

  correct:

  import * as lmdb from 'node-lmdb';

  This provides a namespace import for all exports. For CommonJS, `const lmdb = require('node-lmdb');` is the correct pattern to get the module object.

Quickstart

This quickstart demonstrates how to initialize an LMDB environment, open a database, perform read and write operations within a transaction, and then properly close resources. It covers storing strings and binary data, and includes cleanup.

import { Env } from 'node-lmdb';
import * as fs from 'fs';
import * as path from 'path';

const dbPath = path.join(__dirname, 'mydata');
if (!fs.existsSync(dbPath)) {
    fs.mkdirSync(dbPath);
}

const env = new Env();
env.open({
    path: dbPath,
    mapSize: 2 * 1024 * 1024 * 1024, // 2GB maximum database size
    maxDbs: 5 // Allow up to 5 named databases
});

const dbi = env.openDbi({
    name: 'myPrettyDatabase',
    create: true // Create the database if it doesn't exist
});

try {
    let txn = env.beginTxn();
    const key1 = 1;
    let value1 = txn.getString(dbi, key1);
    
    console.log(`Initial value for key ${key1}: ${value1}`);

    if (value1 === null) {
        txn.putString(dbi, key1, "Hello from node-lmdb!");
        console.log(`Set value for key ${key1} to 'Hello from node-lmdb!'`);
    } else {
        txn.del(dbi, key1);
        console.log(`Deleted value for key ${key1}`);
    }
    
    const key2 = 'my_string_key';
    txn.putBinary(dbi, key2, Buffer.from('Binary data example'));
    console.log(`Put binary data for key '${key2}'`);

    txn.commit(); // Commit the transaction

    // Read the binary data back in a new transaction
    txn = env.beginTxn();
    const binaryValue = txn.getBinary(dbi, key2);
    console.log(`Retrieved binary data for key '${key2}': ${binaryValue?.toString()}`);
    txn.abort(); // Abort a read-only transaction

} catch (e) {
    console.error('LMDB operation failed:', e);
} finally {
    dbi.close();
    env.close();
    fs.rmSync(dbPath, { recursive: true, force: true }); // Clean up
}