Farrow Pipeline

raw JSON →
2.3.0 verified Thu Apr 23 auth: no javascript

Farrow Pipeline (`farrow-pipeline`) is a core module within the Farrow.js ecosystem, designed as a type-friendly middleware composition library for TypeScript and Node.js environments. It enables developers to construct highly extensible and declarative data processing flows by defining pipelines and middleware with strict input and output typing. Currently stable at version 2.3.0, the package follows a moderate release cadence as part of the broader Farrow.js framework development. A key differentiator is its emphasis on strong type safety, guiding developers to build predictable and composable logic without the common pitfalls of implicit context mutation found in some other middleware paradigms. It provides primitives like `createPipeline` and `defineMiddleware` to facilitate a functional approach to handling sequential operations, forming the foundation for request handling in web applications and other data transformation tasks.

error Argument of type '(...args: any) => any' is not assignable to parameter of type 'Middleware<any, any>'.
cause Attempting to pass a plain JavaScript function directly into a pipeline's `.use()` method without wrapping it in `defineMiddleware`, or without correctly specifying generic types for `defineMiddleware`.
fix
Always use defineMiddleware<InputType, OutputType>((input, next) => { ... }) to correctly type and construct your middleware functions, ensuring type compatibility with the pipeline.
error TypeError: Cannot read properties of undefined (reading 'property')
cause A middleware expected a certain property on the `input` object that was not provided by the previous middleware or the initial pipeline run, often due to an incorrect type definition or a middleware short-circuiting.
fix
Carefully review the type flow (InputType and OutputType) between your chained middlewares. Ensure that each middleware explicitly passes the required properties to the next in the chain via its OutputType or by modifying the input object before calling next().
error Module not found: Can't resolve 'farrow-pipeline' in '...' OR ReferenceError: require is not defined
cause This typically occurs when trying to use `farrow-pipeline` (which is designed for ES Modules) within a CommonJS environment without proper configuration, or due to incorrect import paths.
fix
Ensure your Node.js project is configured for ES Modules by adding "type": "module" to your package.json and using import statements. Verify the package is installed and the import path farrow-pipeline is correct.
breaking The Farrow.js ecosystem, including `farrow-pipeline`, underwent significant architectural and API changes between version 1.x and 2.x, primarily to enhance type safety and introduce new patterns. Direct upgrades from v1 to v2 often require substantial code refactoring.
fix Consult the official Farrow.js documentation and examples for Farrow 2.x to update your pipeline and middleware definitions. There are no direct migration scripts for v1 to v2 due to the fundamental shifts.
gotcha Using `farrow-pipeline` with TypeScript versions older than 4.3 can lead to complex and hard-to-debug type inference errors. The library heavily relies on advanced TypeScript features, particularly generics and conditional types, which were significantly improved in TS 4.3 and later.
fix Ensure your project's `tsconfig.json` specifies a `target` compatible with modern Node.js (e.g., `es2020`) and that your `devDependencies` include `"typescript": "^4.3.0"` or higher. Always use the latest stable TypeScript version for optimal type checking.
gotcha When chaining many middlewares, especially with intricate generic types or conditional type transformations, TypeScript's inference can sometimes struggle. This may result in overly broad `any` types being inferred for pipeline inputs/outputs, or incorrect type assertions, diminishing type safety.
fix Explicitly declare input and output types for `defineMiddleware` where inference becomes ambiguous. Consider breaking down complex pipelines into smaller, explicitly typed sub-pipelines or leveraging utility types provided by Farrow to guide inference.
gotcha Misunderstanding the `next` function's role in middleware (e.g., failing to call it, not `await`ing it in async contexts, or not returning its result) can lead to middlewares not executing, pipelines short-circuiting unexpectedly, or incorrect final return values.
fix Always call `next(input)` or `await next(input)` to pass control to the subsequent middleware. Ensure you explicitly `return` the result of `next()` if you intend for subsequent middleware's output to propagate, or return a final value if the middleware is meant to terminate the pipeline.
npm install farrow-pipeline
yarn add farrow-pipeline
pnpm add farrow-pipeline

This example demonstrates creating and running `farrow-pipeline` instances. It showcases defining synchronous and asynchronous type-safe middlewares with `defineMiddleware`, composing them using `createPipeline().use()` and `composePipeline()`, and handling input/output transformation and errors within the pipeline.

import { createPipeline, defineMiddleware, composePipeline } from 'farrow-pipeline';

// Define a simple logging middleware that enriches the output
const loggerMiddleware = defineMiddleware<{ name: string }, { name: string, loggedAt: number }>((input, next) => {
  console.log(`[Logger] Processing input for: ${input.name}`);
  const result = next(input);
  return { ...result, loggedAt: Date.now() };
});

// Define a validation middleware for the name property
const validatorMiddleware = defineMiddleware<{ name: string }, { name: string }>((input, next) => {
  if (!input.name || input.name.length < 3) {
    throw new Error('Validation Error: Name must be at least 3 characters long.');
  }
  console.log(`[Validator] Input name "${input.name}" is valid.`);
  return next(input);
});

// Define a data processing middleware that transforms the data
const dataProcessorMiddleware = defineMiddleware<{ name: string, data?: any }, { processedData: string }>((input, next) => {
  console.log(`[Processor] Handling data for: ${input.name}`);
  const processed = `Transformed data for ${input.name} at ${new Date().toISOString()}`;
  const nextResult = next(input);
  return { ...nextResult, processedData: processed };
});

// Create a pipeline composed of these synchronous middlewares
const myPipeline = createPipeline()
  .use(loggerMiddleware)
  .use(validatorMiddleware)
  .use(dataProcessorMiddleware);

// Define an asynchronous middleware for demonstration purposes
const asyncMiddleware = defineMiddleware<{ id: number }, { id: number, fetchedData: string }>(async (input, next) => {
  console.log(`[Async] Starting async operation for ID: ${input.id}`);
  await new Promise(resolve => setTimeout(resolve, 100)); // Simulate async work
  const fetchedData = `Data for item ${input.id} from external API`;
  console.log(`[Async] Finished async operation for ID: ${input.id}`);
  const result = await next(input); // Await the result of the next middleware
  return { ...result, fetchedData };
});

// Compose multiple middlewares (including async ones) into a single pipeline
const complexPipeline = composePipeline(
  asyncMiddleware,
  defineMiddleware<{ id: number, fetchedData?: string }, { finalResult: string }>(async (input) => {
    console.log(`[Final] Processing final stage for ID: ${input.id} with fetched data: ${input.fetchedData}`);
    return { finalResult: `Final output for ID ${input.id} based on: ${input.fetchedData}` };
  })
);

// Run the pipelines
async function runExamples() {
  try {
    console.log('\n--- Running Simple Pipeline (Valid Input) ---');
    const result1 = await myPipeline.run({ name: 'Alice' });
    console.log('Pipeline Result 1:', result1);
    // Expected: { name: 'Alice', loggedAt: <timestamp>, processedData: 'Transformed data for Alice...' }

    console.log('\n--- Running Simple Pipeline (Invalid Input) ---');
    try {
      await myPipeline.run({ name: 'Bo' }); // This should throw due to validation
    } catch (error: any) {
      console.error('Pipeline Error:', error.message);
      // Expected: 'Validation Error: Name must be at least 3 characters long.'
    }

    console.log('\n--- Running Complex Pipeline ---');
    const result2 = await complexPipeline.run({ id: 123 });
    console.log('Complex Pipeline Result 2:', result2);
    // Expected: { id: 123, fetchedData: 'Data for item 123 from external API', finalResult: 'Final output for ID 123...' }

  } catch (e: any) {
    console.error("An unexpected error occurred during example execution:", e.message);
  }
}

runExamples();