chDB-core: In-process OLAP SQL Engine

Common errors

• Program received signal SIGSEGV, Segmentation fault.

  cause Often caused by out-of-memory conditions when processing large datasets within the application's process, or memory management conflicts, especially with `jemalloc`.
  fix Reduce the data volume processed by the query, utilize streaming APIs, or increase available RAM. If using `jemalloc`, try disabling it (`-DENABLE_JEMALLOC=0` during build) or using `LD_PRELOAD` to ensure consistent memory allocation across C++ and Python boundaries.

• AttributeError: module 'chdb' has no attribute 'DataStore'

  cause You installed `chdb-core` but are trying to use the higher-level, Pandas-compatible DataStore API which is now part of the separate `chdb` package.
  fix If you intend to use the DataStore API, install the `chdb` package: `pip install chdb`. If you only need the core SQL engine, refactor your code to use `chdb.query()` or the `chdb.dbapi` module directly.

• Error in Python UDF: Function 'my_udf' failed to execute.

  cause Typically arises from incorrect UDF implementation, such as attempting to maintain state across calls, incorrect argument types (expecting non-strings), or missing imports within the UDF scope.
  fix Ensure the UDF is stateless. Explicitly convert string inputs to desired types (e.g., `int()`, `float()`) at the start of the function. Move any necessary `import` statements (e.g., `import json`) *inside* the UDF function body. Review UDF best practices in the documentation.

Warnings

• breaking The `chdb` project has been split into two packages: `chdb-core` (this package) and `chdb`. `chdb-core` provides the low-level C++ engine and SQL interfaces (`query`, `dbapi`). If you were previously using `chdb` for its higher-level Pandas-compatible DataStore API, you must now explicitly install `chdb` (i.e., `pip install chdb`) instead of, or in addition to, `chdb-core`.
  Fix: Ensure you install the correct package: `pip install chdb-core` for core SQL engine, or `pip install chdb` for DataStore API built on `chdb-core`.
• gotcha As an in-process OLAP engine, chDB-core shares the application's memory space. Running queries that process or aggregate very large datasets (e.g., 10GB on an 8GB RAM machine) can lead to out-of-memory errors and application crashes, unlike a standalone ClickHouse server which might spill to disk. Similarly, long-running queries can block the entire application process.
  Fix: Optimize queries to reduce memory footprint, process data in chunks (streaming), or consider using a separate ClickHouse server for extremely large or long-running tasks. Monitor memory usage carefully for in-process deployments.
• gotcha User-Defined Functions (UDFs) must be stateless and pure Python functions. All required modules for a UDF must be imported *inside* the UDF function itself, not at the module level. Input arguments are always treated as strings, and the default return type is also a string, requiring explicit type conversions within the UDF for numerical operations.
  Fix: Design UDFs as pure functions without side effects. For example, `def my_udf(arg1, arg2): import json; return str(json.loads(arg1)['key'])`. Explicitly cast string inputs to appropriate types (e.g., `int(lhs)`, `float(rhs)`) and cast the result back to string if not specifying a return type.

Install

• pip install chdb-core Install chdb-core

Imports

• query

  import chdb
  chdb.query(...)

  The primary query function is directly available under the 'chdb' namespace after installing 'chdb-core'.
• dbapi

  import chdb.dbapi as dbapi
  conn = dbapi.connect(...)

  For Python DB-API 2.0 compliant connections and cursors.
• chdb_udf
  wrong:

  from chdb import chdb_udf

  correct:

  from chdb.udf import chdb_udf

  User-Defined Functions (UDFs) are located in the 'chdb.udf' submodule.

Quickstart

This quickstart demonstrates how to perform a basic SQL query using the `chdb.query` function and how to get results in different formats, including a Pandas DataFrame. The `chdb-core` package exposes its primary functionalities via the `chdb` module.

import chdb

# Execute a simple SQL query
result = chdb.query("SELECT 'Hello, chDB-core!' as message, version() as chdb_version", "Pretty")
print(result)

# Example with DataFrame output (requires pandas to be installed)
try:
    import pandas as pd
    df = chdb.query("SELECT number, number * 2 AS double FROM numbers(5)", "DataFrame")
    print(df)
    print(type(df))
except ImportError:
    print("Pandas not installed. Skipping DataFrame example.")