{"id":17229,"library":"flumedb","title":"Flumedb: Modular Stream-Based Database","description":"Flumedb is a modular database system built around an append-only log and streaming views. It acts as an orchestrator, connecting a single `flumelog-*` module (the durable primary data store) with zero or more `flumeview-*` modules (which provide flexible, rebuildable indexes and query interfaces). The core innovation is its star-shaped pipeline architecture, where views stream data from the central log and build their own optimized models. Unlike older designs like `level-sublevel`, `flumedb` views can be easily updated or added; they simply rebuild from the main log if their version changes. This current stable version is 2.1.8. The project does not explicitly state a release cadence, but its components are actively maintained within the `flumedb` GitHub organization. Key differentiators include its robust view rebuilding mechanism, asynchronous views that automatically synchronize upon read, and the ability for views to optimize for queries rather than durability, as they can always regenerate from the canonical log.","status":"active","version":"2.1.8","language":"javascript","source_language":"en","source_url":"git://github.com/dominictarr/flumedb","tags":["javascript"],"install":[{"cmd":"npm install flumedb","lang":"bash","label":"npm"},{"cmd":"yarn add flumedb","lang":"bash","label":"yarn"},{"cmd":"pnpm add flumedb","lang":"bash","label":"pnpm"}],"dependencies":[],"imports":[{"note":"Flumedb v2 is primarily a CommonJS module. Direct ES Module imports are not supported without a transpiler or dynamic import.","wrong":"import Flume from 'flumedb'","symbol":"Flume","correct":"const Flume = require('flumedb')"}],"quickstart":{"code":"var MemLog = require('flumelog-memory') // In-memory log for simple testing\nvar Reduce = require('flumeview-reduce') // A view that aggregates data\nvar Flume = require('flumedb')\n\n// Initialize FlumeDB with an in-memory log\nvar db = Flume(MemLog())\n  // Attach a 'sum' view using flumeview-reduce\n  .use(\n    'sum',\n    Reduce(1, function (acc, item) {\n      // This reduce function sums the 'foo' property of appended items\n      return (acc || 0) + item.foo\n    })\n  )\n\n// Append some data to the log\ndb.append({ foo: 1 }, function (err, seq) {\n  if (err) throw err\n  console.log('Appended item with sequence:', seq)\n  \n  // Get the current sum from the 'sum' view\n  db.sum.get(function (err, value) {\n    if (err) throw err\n    console.log('Current sum from view:', value) // Expected output: 1\n  })\n})\n\ndb.append({ foo: 5 }, function (err, seq) {\n  if (err) throw err\n  console.log('Appended another item with sequence:', seq)\n\n  db.sum.get(function (err, value) {\n    if (err) throw err\n    console.log('Current sum after second append:', value) // Expected output: 6\n  })\n})\n","lang":"javascript","description":"This example demonstrates how to initialize FlumeDB with an in-memory log, attach a reducing view, and then append data and query the view."},"warnings":[{"fix":"Implement UI feedback (e.g., loading indicators) for operations that rely on potentially unsynced views, especially on application startup or after view definition changes. Use `flumedb.rebuild()` for explicit view reconstruction, and handle its callback.","message":"Views in Flumedb are asynchronous. While the main log's `append` callback indicates data has been written, views might not be immediately up-to-date. A read from an unsynced view will block until the view building is complete. Applications interacting with large datasets or complex views should consider displaying progress bars during initial view indexing or reindexing.","severity":"gotcha","affected_versions":">=2.0.0"},{"fix":"Optimize view logic for efficiency. For critical startup paths, consider strategies to defer less critical view synchronization or pre-build views in deployment. Monitor and benchmark view rebuild times to set user expectations.","message":"View rebuilding on startup can be a time-consuming process for large databases. If a view's version number changes, or a new view is added, it will rebuild from the entire log. This process, while robust, can take several minutes depending on data size and view complexity, impacting initial application responsiveness.","severity":"gotcha","affected_versions":">=2.0.0"},{"fix":"Always `npm install` the necessary `flumelog-*` (e.g., `flumelog-memory`, `flumelog-offset`) and `flumeview-*` (e.g., `flumeview-reduce`, `flumeview-level`) packages that your application uses.","message":"Flumedb itself is an orchestrator; it requires separate `flumelog-*` and `flumeview-*` modules to be installed and passed to its constructor and `.use()` method respectively. Forgetting to install these companion modules will lead to runtime errors.","severity":"gotcha","affected_versions":">=2.0.0"}],"env_vars":null,"last_verified":"2026-04-22T00:00:00.000Z","next_check":"2026-07-21T00:00:00.000Z","problems":[{"fix":"Ensure you have installed the specific log module: `npm install flumelog-memory` (or `flumelog-offset`, etc.)","cause":"A required log implementation module (like `flumelog-memory` or `flumelog-offset`) was not installed.","error":"Error: Cannot find module 'flumelog-memory'"},{"fix":"Verify that `db.use('myViewName', MyViewConstructor(version, ...))` is called correctly during initialization and that `MyViewConstructor` returns an object with the expected API methods. Also, ensure the view module is installed: `npm install flumeview-myviewname`.","cause":"The `flumeview-*` module was either not correctly passed to `db.use()`, or its public methods (like `get`) are not correctly exposed or named by the view module itself.","error":"TypeError: db.myViewName.get is not a function"}],"ecosystem":"npm","meta_description":null}