{"id":7000,"library":"arcticdb","title":"ArcticDB DataFrame Database","description":"ArcticDB is a high-performance, serverless DataFrame database engine designed for the Python Data Science ecosystem. It enables storing, retrieving, and processing Pandas DataFrames at scale, backed by commodity object storage like S3-compatible services, Azure Blob Storage, and local LMDB. It provides a Python API backed by a fast C++ data-processing and compression engine, supporting flexible schemas and bitemporal versioning. The library is actively maintained with frequent patch and minor releases.","status":"active","version":"6.12.1","language":"en","source_language":"en","source_url":"https://github.com/man-group/arcticdb","tags":["database","dataframe","time-series","data storage","pandas","serverless","object-storage"],"install":[{"cmd":"pip install arcticdb","lang":"bash","label":"Install ArcticDB"}],"dependencies":[{"reason":"Core functionality revolves around storing and retrieving Pandas DataFrames.","package":"pandas","optional":false},{"reason":"Frequently used with Pandas DataFrames for data generation and manipulation.","package":"numpy","optional":false}],"imports":[{"symbol":"Arctic","correct":"from arcticdb import Arctic"},{"symbol":"adb","correct":"import arcticdb as adb"}],"quickstart":{"code":"import arcticdb as adb\nimport pandas as pd\nimport numpy as np\nimport os\n\n# Use a temporary LMDB directory for local storage\nuri = f\"lmdb://{os.environ.get('ARCTICDB_PATH', '/tmp/arcticdb_quickstart')}\"\nac = adb.Arctic(uri)\n\n# Create a library\nlibrary_name = 'my_test_library'\nif not ac.library_exists(library_name):\n    ac.create_library(library_name)\n\nlib = ac.get_library(library_name)\n\n# Create a sample DataFrame\ndf = pd.DataFrame(np.random.randint(0, 100, size=(10, 3)), columns=list('ABC'))\ndf.index = pd.date_range('2023-01-01', periods=10, freq='D')\n\n# Write the DataFrame\nsymbol_name = 'my_test_symbol'\nlib.write(symbol_name, df)\nprint(f\"DataFrame written to {library_name}/{symbol_name}\")\n\n# Read the DataFrame back\nread_df = lib.read(symbol_name).data\nprint(\"Read DataFrame head:\")\nprint(read_df.head())\n\n# Clean up (optional for temporary storage)\n# ac.delete_library(library_name)","lang":"python","description":"This quickstart demonstrates how to initialize ArcticDB with local LMDB storage, create a library, write a Pandas DataFrame as a 'symbol', and then read it back. The example uses a temporary directory for the LMDB database."},"warnings":[{"fix":"Create the library using `ac.create_library(library_name, library_options={'dynamic_schema': True})` if schema evolution is required.","message":"When using `update` or `append` operations, ArcticDB enforces a static schema by default. If you intend to modify the schema (add/remove/change column types), you must create the library with `dynamic_schema=True`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Limit to a single `Arctic('lmdb://...')` instance per process for a given path. Use object storage (S3, Azure) for distributed access.","message":"For LMDB storage, ensure only one Arctic instance is open per LMDB database within a single Python process. LMDB also does not support remote filesystems.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Contact info@arcticdb.io for licensing details for production deployments.","message":"Production use of ArcticDB (including business or commercial environments) or its use as a Database Service requires a paid license from ArcticDB Limited.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Verify the `CA_cert_path` is correct and the certificate exists. For debugging, set the environment variable `export AZURE_LOG_LEVEL=1` to enable SDK debug logging.","message":"When connecting to Azure Blob Storage, if the `CA_cert_path` is incorrect or the certificate cannot be found, Azure exceptions may lack meaningful error codes, making debugging difficult.","severity":"gotcha","affected_versions":"All versions supporting Azure"},{"fix":"Migrate code using `QueryBuilder` to use the `LazyDataFrame` API for improved readability and maintainability.","message":"The `QueryBuilder` API is officially deprecated in favor of the more intuitive and recommended `LazyDataFrame` API for complex querying, filtering, group-bys, and aggregations.","severity":"deprecated","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Read the latest version of the symbol and ensure your update/append operation uses data of the corresponding type. If you need to change the schema, the library must be created with `dynamic_schema=True`.","cause":"You are trying to update or append data to an existing symbol, but the new data (NumPy array or Pandas DataFrame) has a different type than the original data stored in the symbol.","error":"Normalization Error 2000: Attempting to update or append an existing type with an incompatible object type"},{"fix":"Increase the map size when initializing `Arctic` for LMDB by passing a larger `map_size` in the URI or options. Example: `adb.Arctic('lmdb:///path/to/db?map_size=100GB')`.","cause":"The LMDB map size limit has been reached, meaning your local database file is full and cannot store more data.","error":"LMDB error code -30792"},{"fix":"Ensure your Pandas DataFrame uses a supported index type, such as `DatetimeIndex` for time-series data. Non-DatetimeIndex will be converted to a `RowCount` integer index.","cause":"ArcticDB only supports specific Pandas index types for its on-disk index (e.g., `DatetimeIndex` for time-series, or a `RowCount` integer index). You attempted to write data with an unsupported index type.","error":"Normalization Error 2003: A write of an incompatible index type has been attempted."},{"fix":"Verify the symbol name is correct. Use `library.list_symbols()` to check existing symbols. Ensure the symbol has been written at least once before attempting read/modify operations.","cause":"You are trying to read, update, or append to a symbol that does not exist in the specified library.","error":"ArcticException: A missing key has been requested. [...] The symbol being worked on does not exist."}]}