Avsc - Avro for JavaScript

Common errors

• RangeError: Attempt to access memory outside buffer bounds

  cause Attempting to decode a corrupted or malformed Avro buffer, or using a reader schema that is fundamentally incompatible with the writer's schema.
  fix Verify the integrity of the Avro binary data. Ensure the schema used for decoding (`reader's schema`) is compatible with the schema used for encoding (`writer's schema`). If reading from a file, check file corruption or incorrect file type.

• AvroTypeException: missing required field: 'fieldName'

  cause A JavaScript object being encoded is missing a field that is defined as 'required' (non-nullable) in the Avro schema.
  fix Ensure that all non-nullable fields defined in your Avro schema are present in the JavaScript object you are attempting to serialize, or make the field nullable in the schema definition (e.g., `['null', 'string']`).

• TypeError: "type" must be a valid Avro type.

  cause The schema provided to `Type.forSchema` is syntactically incorrect, incomplete, or contains invalid Avro type definitions.
  fix Review your Avro schema definition carefully against the Avro specification. Check for typos, missing `type` properties, incorrect capitalization of primitive types, or invalid complex type structures.

• Error: No matching type for schema 'avro.FullName'

  cause Occurs during schema resolution for evolution, indicating a named type (e.g., a record, enum, or fixed type) referenced in the writer's or reader's schema cannot be found or resolved.
  fix Ensure all named types are fully defined or accessible within the context of the combined schemas when performing schema evolution. Check for correct namespaces and names of referenced types.

Warnings

• breaking Version 4.0.0 introduced a significant breaking change in how Avro unions are represented. Unions are now unwrapped by default, meaning they no longer use an encapsulating object for their value. This can affect code expecting the old wrapped structure.
  Fix: Review and update code that handles Avro union types. If you relied on a wrapper object around union values, you may need to adjust your parsing logic or explicitly configure `avsc` to use the legacy wrapped union representation if available (though it's recommended to migrate to the unwrapped format).
• gotcha JavaScript's native `number` type is a 64-bit floating-point number, which can lead to precision loss for Avro's 64-bit integer (`long`) type if the values exceed `Number.MAX_SAFE_INTEGER` (2^53 - 1).
  Fix: For applications requiring exact 64-bit integer precision, configure `avsc` to use custom 'long' types such as `BigInt` (Node.js >= 10) or libraries like `long.js`. This is done via options when creating `Type` instances, e.g., `Type.forSchema(schema, { logicalTypes: { 'long': MyBigIntLongType } })`.
• gotcha The `avro.assemble` function for Avro IDL (AVDL) and JSON protocol (AVPR) import support, introduced in v3.3.0, does not fully support nested or external `import` statements within IDL files for schema parsing, unlike some other Avro implementations. This can complicate large, modular schema definitions.
  Fix: For complex IDL definitions with multiple imports, consider flattening your schema into a single file or using pre-processing steps to resolve imports before passing them to `avsc.readProtocol` or `avro.assemble`. `parseTypeSchema` specifically does not support imports.
• gotcha While `avsc` supports schema evolution, providing incompatible reader and writer schemas (e.g., removing required fields or changing fundamental types without a compatible evolution rule) will result in runtime errors during decoding.
  Fix: Strictly adhere to Avro schema evolution rules (e.g., only add nullable fields, do not remove existing fields, ensure compatible type changes). Thoroughly test schema compatibility between different versions of your data producers and consumers.

Install

• npm install avsc npm
• yarn add avsc yarn
• pnpm add avsc pnpm

Imports

• avro
  wrong:

  const avro = require('avsc');

  correct:

  import * as avro from 'avsc';

  While CommonJS `require` is common in older Node.js examples, ESM `import * as avro` is the correct pattern for modern TypeScript/ESM projects. TypeScript users will typically prefer named imports when possible.
• Type
  wrong:

  import avro from 'avsc'; const type = avro.Type; // 'avro' is not the default export

  correct:

  import { Type } from 'avsc';

  The primary `Type` class for schema and value operations is a named export. Ensure you import it directly or access it via the `avro` namespace.
• Service
  wrong:

  import avro from 'avsc'; const service = avro.Service; // 'avro' is not the default export

  correct:

  import { Service } from 'avsc';

  For Avro RPC implementations, the `Service` class is a named export. It is used to define and create servers based on Avro protocols.
• createFileDecoder
  wrong:

  import avro from 'avsc'; const decoder = avro.createFileDecoder;

  correct:

  import { createFileDecoder } from 'avsc';

  Utility functions like `createFileDecoder` are provided as named exports for direct import, simplifying usage and enabling tree-shaking in ESM environments.

Quickstart

This quickstart demonstrates how to define an Avro schema, create an Avro `Type`, and then use it to encode and decode JavaScript objects into binary buffers. It also shows basic type inference.

import { Type } from 'avsc';

const petSchema = {
  type: 'record',
  name: 'Pet',
  fields: [
    {
      name: 'kind',
      type: { type: 'enum', name: 'PetKind', symbols: ['CAT', 'DOG', 'FISH'] }
    },
    { name: 'name', type: 'string' }
  ]
};

// Create an Avro Type from a schema definition
const petType = Type.forSchema(petSchema);

// Encode a JavaScript object into an Avro binary buffer
const myPet = { kind: 'CAT', name: 'Albert' };
const buf = petType.toBuffer(myPet);

console.log('Original object:', myPet);
console.log('Encoded buffer:', buf.toString('hex'));

// Decode the Avro binary buffer back into a JavaScript object
const decodedPet = petType.fromBuffer(buf);

console.log('Decoded object:', decodedPet);

// Example of schema inference for similar structures
const addressType = Type.forValue({
  city: 'Cambridge',
  zipCodes: ['02138', '02139'],
  visits: 2
});

const otherAddress = { city: 'Seattle', zipCodes: ['98101'], visits: 3 };
const otherBuf = addressType.toBuffer(otherAddress);
console.log('\nInferred schema for:', otherAddress);
console.log('Encoded (inferred) buffer:', otherBuf.toString('hex'));