{"id":16481,"library":"pelias-dbclient","title":"Pelias Database Client","description":"`pelias-dbclient` is a core Node.js module within the Pelias geocoder ecosystem, providing a stream-based interface for efficiently bulk-inserting documents into Elasticsearch. Its primary function is to act as a crucial pipeline stage for Pelias import processes, transforming Pelias `Document` objects into Elasticsearch-compatible bulk operations. The current stable version is v3.3.0, with releases occurring on a feature-driven, rather than strict time-based, cadence, though a major version (v3.0.0) was released in March 2024. Key differentiators include its tight integration with the Pelias data model, its focus on streaming for large-scale data ingestion, and its commitment to open-source principles as part of the broader Pelias open-data geocoding project. It leverages the official `elasticsearch` client under the hood and is designed specifically for Node.js environments.","status":"active","version":"3.3.0","language":"javascript","source_language":"en","source_url":"https://github.com/pelias/dbclient","tags":["javascript","pelias","elasticsearch","client","stream"],"install":[{"cmd":"npm install pelias-dbclient","lang":"bash","label":"npm"},{"cmd":"yarn add pelias-dbclient","lang":"bash","label":"yarn"},{"cmd":"pnpm add pelias-dbclient","lang":"bash","label":"pnpm"}],"dependencies":[{"reason":"Provides core Pelias Document objects and stream mappers for data transformation.","package":"pelias-model","optional":false},{"reason":"Used to retrieve Elasticsearch client configuration and schema details.","package":"pelias-config","optional":false},{"reason":"The official Elasticsearch client used for direct interaction and operations like deleteByQuery.","package":"elasticsearch","optional":false},{"reason":"Utility for performing delete-by-query operations on Elasticsearch.","package":"elastic-deletebyquery","optional":false},{"reason":"Common utility for creating readable streams from arrays, often used in conjunction with dbclient in examples.","package":"stream-array","optional":true},{"reason":"Common utility for creating transform streams, often used in conjunction with dbclient in examples.","package":"through2","optional":true}],"imports":[{"note":"The module exports a function directly, not an object with named exports. ESM usage would be `import dbclient from 'pelias-dbclient';`.","wrong":"const { dbclient } = require('pelias-dbclient');","symbol":"dbclient","correct":"const dbclient = require('pelias-dbclient');"},{"note":"The primary export is a factory function that returns a stream instance. Call it to create a new stream.","symbol":"streamFactory","correct":"const streamFactory = require('pelias-dbclient');\nconst stream = streamFactory();"},{"note":"While not directly from `pelias-dbclient`, the Document object from `pelias-model` is fundamental for `dbclient`'s input stream.","symbol":"Document (from pelias-model)","correct":"const Document = require('pelias-model').Document;"}],"quickstart":{"code":"'use strict';\n\nconst streamify = require('stream-array');\nconst through = require('through2');\nconst Document = require('pelias-model').Document;\nconst dbMapper = require('pelias-model').createDocumentMapperStream;\nconst dbclient = require('pelias-dbclient');\n\nconst elasticsearch = require('elasticsearch');\nconst config = require('pelias-config').generate(); // Ensure pelias-config is properly set up\nconst elasticDeleteQuery = require('elastic-deletebyquery');\n\nconst timestamp = Date.now();\n\n// Simulate an upstream data source\nconst stream = streamify([1, 2, 3, 4, 5])\n  .pipe(through.obj((item, enc, next) => {\n    // Create a Pelias Document for each item\n    const uniqueId = [ 'docType', item ].join(':');\n    const doc = new Document( 'sourceType', 'venue', uniqueId );\n    doc.timestamp = timestamp;\n    doc.setName('default', `Test Venue ${item}`);\n    doc.setCentroid(item * 0.1, item * 0.2);\n    next(null, doc);\n  }))\n  .pipe(dbMapper()) // Map Pelias Document to Elasticsearch format\n  .pipe(dbclient()); // Bulk-insert documents into Elasticsearch\n\nstream.on('finish', () => {\n  console.log('All documents processed and sent to Elasticsearch.');\n  const client = new elasticsearch.Client(config.esclient);\n  // Example of a post-import operation: clean up old documents\n  const options = {\n    index: config.schema.indexName,\n    body: {\n      query: {\n        \"bool\": {\n          \"must\": [\n            {\"term\": { \"source\":  \"sourceType\" }}\n          ],\n          \"must_not\": [\n            {\"term\": { \"timestamp\":  timestamp }}\n          ]\n        }\n      }\n    }\n  };\n\n  client.deleteByQuery(options, (err, response) => {\n    if (err) {\n      console.error('Error during cleanup:', err);\n    } else {\n      console.log(`Cleaned up ${response.elements || response.deleted} old elements.`);\n    }\n    client.close();\n  });\n});\n\nstream.on('error', (err) => {\n  console.error('Stream encountered an error:', err);\n});","lang":"javascript","description":"This quickstart demonstrates how to use `pelias-dbclient` as a transform stream to bulk-insert Pelias Document objects into Elasticsearch, followed by a cleanup operation upon stream completion. It shows the typical integration within a Pelias import pipeline."},"warnings":[{"fix":"Ensure your Elasticsearch cluster is running v7 or v8. Update any Pelias configuration or custom data models that explicitly rely on or set the `_type` field, as it is deprecated in newer Elasticsearch versions.","message":"Version 3.0.0 of `pelias-dbclient` dropped support for Elasticsearch v6 and removed the internal handling of the `_type` field. This change was necessary to support Elasticsearch v8, with v7 remaining the recommended version.","severity":"breaking","affected_versions":">=3.0.0"},{"fix":"Upgrade your Node.js environment to version 10.0.0 or higher to ensure compatibility and leverage modern JavaScript features and security updates.","message":"Version 2.14.0 introduced a breaking change by dropping official support for Node.js 8. The minimum required Node.js version is now 10.0.0.","severity":"breaking","affected_versions":">=2.14.0"},{"fix":"When chaining operations or performing post-import tasks, always listen for the 'finish' event on the `dbclient` stream to ensure all data has been written to Elasticsearch.","message":"The `dbclient` stream is designed to emit its 'finish' event only after all documents have been successfully stored in Elasticsearch. This behavior is intentional to guarantee data persistence before subsequent operations (e.g., cleanup or indexing updates) are triggered.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Ensure that your `pelias-config` module is properly set up and configured for your environment, specifically the `esclient` and `schema.indexName` properties, before initializing `pelias-dbclient`.","message":"The Elasticsearch client configuration (`config.esclient`) and index name (`config.schema.indexName`) are sourced from `pelias-config`. Incorrect or missing Pelias configuration can lead to connection errors or documents being written to unintended indices.","severity":"gotcha","affected_versions":">=1.0.0"}],"env_vars":null,"last_verified":"2026-04-22T00:00:00.000Z","next_check":"2026-07-21T00:00:00.000Z","problems":[{"fix":"Verify your Pelias configuration files (`pelias.json` or environment variables) are correctly set up and accessible to the application. Ensure `pelias-config` can locate and parse them.","cause":"The `pelias-config` module was not able to generate a valid configuration object, specifically missing `esclient` or `schema.indexName`.","error":"TypeError: Cannot read properties of undefined (reading 'client') or Cannot read property 'indexName' of undefined"},{"fix":"Update your Pelias data models, import scripts, or any custom code to remove references to the `_type` field. Ensure `pelias-dbclient` is version 3.x or higher and your Elasticsearch is v7+.","cause":"Attempting to send requests or interact with Elasticsearch using the deprecated `_type` field, which was removed in Elasticsearch v7 and v8, and subsequently dropped by `pelias-dbclient` v3.x.","error":"ElasticsearchClientError: [es/search] 'type' is no longer supported"},{"fix":"Use `const dbclient = require('pelias-dbclient');` for CommonJS or `import dbclient from 'pelias-dbclient';` for ESM. Do not use `{ dbclient }`.","cause":"The `pelias-dbclient` module is imported incorrectly, often by attempting to destructure it as a named export when it is a default export function.","error":"TypeError: dbclient is not a function"},{"fix":"Upgrade your Node.js environment to version 10.0.0 or higher. The recommended version for current `pelias-dbclient` releases is Node.js 16 or newer.","cause":"Running `pelias-dbclient` on an unsupported Node.js version.","error":"Error: 'pelias-dbclient' requires Node.js version >=10.0.0. You are running Node.js 8.x.x."}],"ecosystem":"npm"}