Buffer Layout

Common errors

• TypeError: require is not a function

  cause Attempting to `require('buffer-layout')` in an ES Module context, or `import`ing `buffer-layout` directly in a CommonJS context.
  fix Ensure you are using CommonJS `const lo = require('buffer-layout');` in a CJS file, or configure your bundler to handle CJS modules in an ESM project. If using native ESM, you might need to use `import * as lo from 'buffer-layout';` or ensure your `package.json` specifies `"type": "commonjs"` for the file.

• AssertionError: Buffers not equal

  cause Often occurs when `encode` or `decode` operations produce unexpected results, frequently due to incorrect layout definitions, especially missing padding for C-style structs.
  fix Review your layout definition against the target binary format or C struct definition. Pay close attention to data types, endianness, and explicit padding needed for memory alignment. Use `Buffer.compare` or `Buffer.equals` for byte-by-byte comparison during debugging.

• RangeError: offset is out of bounds

  cause Trying to encode or decode data beyond the allocated size of the `Buffer`, or specifying an incorrect offset during read/write operations within `buffer-layout` methods.
  fix Verify that the `Buffer.alloc()` size is sufficient for the defined layout (`layout.span`). Double-check any manual offset parameters passed to `encode` or `decode` methods.

Warnings

• breaking The `buffer-layout` package is currently abandoned. There will be no further updates, bug fixes, or security patches, which could lead to compatibility issues with newer Node.js versions or unaddressed vulnerabilities.
  Fix: Consider migrating to a maintained alternative like `@solana/buffer-layout` which is a TypeScript-enabled fork with active maintenance and similar API.
• gotcha When defining structures that mimic C `struct`s, developers must manually account for memory alignment and add explicit padding layouts. Failing to do so will result in incorrect buffer sizes, offsets, and ultimately corrupted data during encoding and decoding.
  Fix: Use `lo.seq(lo.u8(), N)` or `lo.blob(N)` with no property name to insert `N` bytes of padding where required by alignment rules of the target C structure.
• gotcha 64-bit integral values (e.g., `lo.u64`, `lo.s64`) are decoded into standard JavaScript `Number` types. Due to JavaScript's floating-point number representation (IEEE 754 double-precision), integers larger than 2^53 - 1 (or smaller than -2^53 + 1) cannot be precisely represented, leading to potential data loss or approximation.
  Fix: For precise handling of 64-bit integers, use layouts that return `BigInt` (if available in a fork/alternative) or implement manual conversion using `BigInt` after decoding the raw bytes. The original `buffer-layout` library does not natively support `BigInt`.
• gotcha The package does not ship with its own TypeScript declaration files. While community-maintained `@types/buffer-layout` might exist, their reliability and up-to-dateness can vary. This leads to a suboptimal developer experience in TypeScript projects.
  Fix: Install `@types/buffer-layout` if available, or consider migrating to `@solana/buffer-layout`, which is a TypeScript-first fork and provides native types.

Install

• npm install buffer-layout npm
• yarn add buffer-layout yarn
• pnpm add buffer-layout pnpm

Imports

• buffer-layout
  wrong:

  import lo from 'buffer-layout';

  correct:

  const lo = require('buffer-layout');

  This package is primarily CommonJS. Direct ESM import (`import lo from 'buffer-layout';`) will fail in a native ESM environment without a CommonJS wrapper or bundler. Named imports like `import { struct } from 'buffer-layout'` are not supported.
• lo.struct
  wrong:

  import { struct } from 'buffer-layout';

  correct:

  const { struct } = require('buffer-layout'); // or lo.struct

  Layout constructors like `struct` are properties of the default export. Destructuring from `require` is common for convenience, or they are accessed via the `lo` alias (e.g., `lo.struct`).
• lo.u8
  wrong:

  import { u8 } from 'buffer-layout';

  correct:

  const { u8 } = require('buffer-layout'); // or lo.u8

  Similar to `lo.struct`, `u8` (unsigned 8-bit integer) is a method available on the module's default export. Named ESM imports for these methods are incorrect.

Quickstart

Demonstrates defining both packed and aligned C-style structs using `buffer-layout`, showing how to encode JavaScript objects into Buffers and decode them back, including explicit padding for alignment.

const assert = require('assert');
const lo = require('buffer-layout');

// Define a C-like packed struct without padding
// C: struct { uint8_t a; uint16_t b; } __attribute__((__packed__)) MyPackedStruct;
const packedStruct = lo.struct([
  lo.u8('a'),
  lo.u16le('b') // u16le for unsigned 16-bit little-endian
]);

const packedBuffer = Buffer.alloc(3); // 1 byte for 'a', 2 bytes for 'b'
const data1 = { a: 0x12, b: 0x3456 };
packedStruct.encode(data1, packedBuffer);

console.log('Packed Buffer:', packedBuffer.toString('hex'));
assert.equal(packedBuffer.toString('hex'), '125634');
assert.deepStrictEqual(packedStruct.decode(packedBuffer), data1);

// Define a C-like struct WITH padding, simulating a 32-bit aligned machine
// C: struct { uint8_t v; uint32_t u32; } MyAlignedStruct;
// In C, u32 would typically be 4-byte aligned, causing 3 bytes of padding after u8.
const alignedStruct = lo.struct([
  lo.u8('v'),
  lo.seq(lo.u8(), 3, 'padding'), // explicit padding for 4-byte alignment
  lo.u32le('u32')
]);

const alignedBuffer = Buffer.alloc(8); // 1 byte (v) + 3 bytes (padding) + 4 bytes (u32)
alignedBuffer.fill(0xBD); // Fill with a recognizable pattern for padding
const data2 = { v: 0x01, u32: 0x12345678 };
alignedStruct.encode(data2, alignedBuffer);

console.log('Aligned Buffer (with padding):', alignedBuffer.toString('hex'));
assert.equal(alignedBuffer.toString('hex'), '01bdbdbd78563412');
assert.deepStrictEqual(alignedStruct.decode(alignedBuffer), { v: 1, u32: 0x12345678 });

console.log('Quickstart examples completed successfully.');