{"id":18066,"library":"stream-concat","title":"Node.js Stream Concatenation","description":"The `stream-concat` library provides a simple and efficient mechanism for concatenating multiple Node.js readable streams into a single output readable stream. Currently at version `2.0.0`, it maintains an active development status, though a specific release cadence isn't explicitly defined. A core differentiator is its ability to handle streams in two ways: either by accepting an array of pre-existing streams or, more efficiently, by using a user-supplied function that returns the next stream on demand. This 'on-the-fly' stream creation significantly reduces memory consumption and improves performance, particularly when dealing with large datasets or numerous files, by preventing all streams from being buffered simultaneously. It builds upon Node.js's `Transform` stream, leveraging its underlying options like `highWaterMark` and `objectMode` for fine-grained control over stream behavior.","status":"active","version":"2.0.0","language":"javascript","source_language":"en","source_url":"https://github.com/sedenardi/node-stream-concat","tags":["javascript","node","stream","concat","streams","concatenation"],"install":[{"cmd":"npm install stream-concat","lang":"bash","label":"npm"},{"cmd":"yarn add stream-concat","lang":"bash","label":"yarn"},{"cmd":"pnpm add stream-concat","lang":"bash","label":"pnpm"}],"dependencies":[],"imports":[{"note":"This package is CommonJS only. Using ES Modules `import` syntax directly will likely result in a TypeError or undefined import without a transpilation step or specific Node.js loader configuration.","wrong":"import StreamConcat from 'stream-concat';","symbol":"StreamConcat","correct":"const StreamConcat = require('stream-concat');"}],"quickstart":{"code":"const fs = require('fs');\nconst StreamConcat = require('stream-concat');\nconst path = require('path');\n\n// Create some dummy files for demonstration\nfs.writeFileSync('file1.csv', 'header1,data1a\\nheader2,data1b\\n');\nfs.writeFileSync('file2.csv', 'header3,data2a\\nheader4,data2b\\n');\nfs.writeFileSync('file3.csv', 'header5,data3a\\nheader6,data3b\\n');\n\nconst fileNames = ['file1.csv', 'file2.csv', 'file3.csv'];\nlet fileIndex = 0;\n\n// Use a function to defer stream creation, reducing memory footprint\nconst nextStream = () => {\n  if (fileIndex === fileNames.length) {\n    return null;\n  }\n  console.log(`Reading: ${fileNames[fileIndex]}`);\n  return fs.createReadStream(fileNames[fileIndex++]);\n};\n\nconst output = fs.createWriteStream('combined.csv');\n\nconst combinedStream = new StreamConcat(nextStream);\n\ncombinedStream.pipe(output)\n  .on('finish', () => {\n    console.log('All files combined into combined.csv');\n    // Clean up dummy files\n    fileNames.forEach(file => fs.unlinkSync(file));\n    fs.unlinkSync('combined.csv');\n  })\n  .on('error', (err) => {\n    console.error('Error combining streams:', err);\n  });\n","lang":"javascript","description":"This example demonstrates how to concatenate multiple files efficiently using a `nextStream` function. This approach defers opening new streams until they are needed, optimizing memory usage compared to providing an array of all streams upfront."},"warnings":[{"fix":"Upgrade your Node.js environment to version 12 or newer, or use an older `stream-concat` version if you are constrained to an older Node.js runtime.","message":"Version `2.0.0` and newer requires Node.js `v12` or higher. Older versions of Node.js (`v0.12` to `v10`) are only supported by `stream-concat` versions prior to `1.0.0`. Ensure your Node.js runtime meets the `engines` requirement.","severity":"breaking","affected_versions":">=1.0.0"},{"fix":"Instead of providing an array of streams, pass a function to the `StreamConcat` constructor that returns the next stream (or a Promise resolving to a stream) only when it's needed. This allows the library to manage buffering sequentially.","message":"When concatenating many or very large streams, providing an array of all streams directly to the `StreamConcat` constructor can lead to high memory consumption and reduced performance. All streams' read queues will be buffered simultaneously.","severity":"gotcha","affected_versions":">=0.12"}],"env_vars":null,"last_verified":"2026-04-25T00:00:00.000Z","next_check":"2026-07-24T00:00:00.000Z","problems":[{"fix":"Since `stream-concat` is a CommonJS package, you must use `const StreamConcat = require('stream-concat');` to import it correctly in both CJS and ESM environments (where ESM allows CJS `require`).","cause":"Attempting to use `import StreamConcat from 'stream-concat'` in an ES Modules (ESM) context without proper CJS-ESM interop.","error":"TypeError: StreamConcat is not a constructor"}],"ecosystem":"npm","meta_description":null,"install_score":null,"install_tag":null,"quickstart_score":null,"quickstart_tag":null}