Avro-js JavaScript Implementation

Common errors

• Error: Invalid value for union type: expected 'null' or 'LongList', got object

  cause Attempting to serialize an object that uses an untagged record within a union type where the record type name is expected.
  fix When dealing with recursive or optional record types within a union, explicitly wrap the nested object with its Avro record name, e.g., `{ next: { LongList: { value: 2, next: null } } }` instead of `{ next: { value: 2, next: null } }`.

• ReferenceError: require is not defined

  cause Using CommonJS `require()` syntax in a JavaScript module that is treated as an ES Module (e.g., in a Node.js project with `"type": "module"` or in a browser via modern bundling without CommonJS transpilation).
  fix For modern JavaScript environments, use `import * as avro from 'avro-js';`. If running in Node.js ES Modules, ensure your `package.json` specifies `"type": "module"` and adjust imports accordingly. If `avro-js` is primarily CJS, use `import avro from 'avro-js';` with caution, as it implies a default export which may not exist, or `import * as avro from 'avro-js';`.

• ReferenceError: fs is not defined

  cause Attempting to use `avro.parse('./path.avsc')` or `avro.createFileDecoder()` in a browser environment, which lack Node.js file system (`fs`) modules.
  fix Ensure that `avro-js` file-based operations are only executed in Node.js. For browser applications, load schemas as pre-fetched JSON strings or JavaScript objects and use `avro.parse(jsonString)` or `avro.parse(jsonObject)`.

• Error: Missing required field: 'fieldName'

  cause An object being serialized is missing a field that is marked as required (i.e., not a union with 'null' and no default value) in the Avro schema.
  fix Ensure that the JavaScript object being passed for serialization contains all fields that are defined as required in the corresponding Avro schema. Provide a value for the missing field or update the schema to make the field optional or provide a default.

Warnings

• gotcha When defining recursive schemas, `avro-js` does not use duck-typing for nested records within unions. You must explicitly specify the record type name for recursive instances, e.g., `{ "RecordName": { ... } }` rather than just `{ ... }`.
  Fix: For a union type like `["null", "MyRecord"]` where `MyRecord` is recursive, always structure nested data as `{ MyRecord: { field1: value1, ... } }` instead of directly `{ field1: value1, ... }`.
• gotcha File system operations, such as `avro.parse('./path/to/schema.avsc')` for loading schema files or `avro.createFileDecoder()` for reading Avro container files, are strictly for Node.js environments and are not supported in web browsers.
  Fix: In browser environments, load schemas as JSON strings or JavaScript objects using `avro.parse(jsonString)` or `avro.parse(jsonObject)`. Avoid all file I/O methods.
• gotcha Avro `long` types correspond to 64-bit signed integers. JavaScript's native `Number` type uses 64-bit floating-point format and can only safely represent integers up to `2^53 - 1`. Handling `long` values outside this range can lead to precision loss without explicit configuration.
  Fix: To preserve precision for `long` types, configure `avro-js` to use `BigInt` or a library like `long.js` for `long` schema types. This often involves customizing logical types or type hooks during schema parsing. Refer to the `avro-js` documentation for specific configuration options related to 64-bit integers.
• gotcha While Avro schemas are JSON-based, strict adherence to the Avro specification is required for schema parsing and data validation. Common JSON syntax errors (e.g., trailing commas, unquoted property names) or Avro-specific schema definition errors (e.g., invalid field types, missing required attributes) can lead to parsing failures.
  Fix: Always validate Avro schemas carefully using online tools or a schema linter. Pay close attention to JSON syntax rules and Avro's complex type definitions, especially for `record`, `enum`, `array`, `map`, and `union` types. Ensure all required fields like `name` and `type` are correctly defined.

Install

• npm install avro-js npm
• yarn add avro-js yarn
• pnpm add avro-js pnpm

Imports

• avro
  wrong:

  import avro from 'avro-js';

  correct:

  import * as avro from 'avro-js';

  The avro-js library is a CommonJS module. For ESM environments, use `import * as avro` to import the entire module object, as there is no default export.
• avro.parse
  wrong:

  import { parse } from 'avro-js';
  const type = parse(schema);

  correct:

  import * as avro from 'avro-js';
  const type = avro.parse(schema);

  Specific methods like `parse` are properties of the main `avro` module object, not direct named exports. Access them via the imported `avro` namespace object.
• avro.createFileDecoder
  wrong:

  import { createFileDecoder } from 'avro-js';

  correct:

  import * as avro from 'avro-js';
  avro.createFileDecoder('./records.avro')

  This function is a property of the main `avro` module object and is Node.js-specific for file I/O operations. It is not available in browser environments.

Quickstart

Demonstrates basic Avro schema parsing, object serialization to a buffer, deserialization back to an object, random instance generation, and schema validation.

import * as avro from 'avro-js';

// 1. Define an Avro schema for a 'Pet' record
const petType = avro.parse({
  name: 'Pet',
  type: 'record',
  fields: [
    {name: 'kind', type: {name: 'Kind', type: 'enum', symbols: ['CAT', 'DOG', 'FISH']}},
    {name: 'name', type: 'string'},
    {name: 'age', type: 'int', default: 0}
  ]
});

// 2. Create a JavaScript object conforming to the schema
const myPet = {kind: 'CAT', name: 'Albert', age: 5};

// 3. Serialize the object to an Avro binary buffer
const buffer = petType.toBuffer(myPet);
console.log('Serialized Buffer:', buffer.toString('hex'));

// 4. Deserialize the buffer back into a JavaScript object
const deserializedPet = petType.fromBuffer(buffer);
console.log('Deserialized Object:', deserializedPet);

// 5. Generate a random instance of the schema
const randomIdType = avro.parse('{"type": "fixed", "name": "Id", "size": 4}');
const randomId = randomIdType.random();
console.log('Random ID (Buffer):', randomId.toString('hex'));

// 6. Check if an object is valid against a schema
const isValid = petType.isValid({kind: 'DOG', name: 'Buddy'});
console.log('Is valid pet object:', isValid);

const isInvalid = petType.isValid({kind: 'FISH', unknownField: 'extra'});
console.log('Is invalid pet object (extra field):', isInvalid);