{"id":3937,"library":"connectorx","title":"ConnectorX","description":"ConnectorX is a high-performance Python library for loading data from databases into dataframes (Pandas, Polars, Apache Arrow). Written in Rust, it bypasses Python's GIL, offering significant speedups for data-intensive operations. The current version is 0.4.5, and it has an active development cycle with frequent minor releases.","status":"active","version":"0.4.5","language":"en","source_language":"en","source_url":"https://github.com/sfu-db/connector-x","tags":["database","data-connector","etl","sql","pandas","polars","apache-arrow","high-performance","rust"],"install":[{"cmd":"pip install connectorx","lang":"bash","label":"Install base package"},{"cmd":"pip install \"connectorx[pandas]\" # For pandas return_type\npip install \"connectorx[polars]\" # For polars return_type\npip install \"connectorx[arrow]\" # For arrow return_type","lang":"bash","label":"Install with optional dependencies"}],"dependencies":[{"reason":"Required for `return_type=\"pandas\"`.","package":"pandas","optional":true},{"reason":"Required for `return_type=\"polars\"`.","package":"polars","optional":true},{"reason":"Required for `return_type=\"arrow\"`.","package":"pyarrow","optional":true}],"imports":[{"symbol":"read_sql","correct":"from connectorx import read_sql"}],"quickstart":{"code":"import connectorx as cx\nimport pandas as pd\nimport os\n\n# Example PostgreSQL connection string\n# Replace with your actual database connection string\nDB_CONNECTION_STRING = os.environ.get('CX_DB_CONNECTION_STRING', 'postgresql://user:password@host:5432/database')\n\nif DB_CONNECTION_STRING == 'postgresql://user:password@host:5432/database':\n    print(\"Warning: Please set CX_DB_CONNECTION_STRING environment variable for a real database connection.\")\n    print(\"Using a dummy connection string, this example might not run without a database.\")\n    # For a runnable dummy, you might use SQLite in-memory, but ConnectorX doesn't directly support that\n    # The primary use case is external DBs.\n    exit(1) # Exit if not configured, as it won't connect without a real string\n\nquery = \"SELECT id, name FROM my_table WHERE id < 10\"\n\ntry:\n    df = cx.read_sql(DB_CONNECTION_STRING, query, return_type=\"pandas\")\n    print(\"Successfully read data:\")\n    print(df.head())\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\n    print(\"Ensure your database connection string and query are correct, and the database is accessible.\")","lang":"python","description":"This example demonstrates how to use `connectorx.read_sql` to fetch data from a database into a Pandas DataFrame. Remember to replace the placeholder connection string with your actual database credentials and ensure `pandas` is installed. For a runnable example, set the `CX_DB_CONNECTION_STRING` environment variable to a valid database connection string."},"warnings":[{"fix":"Upgrade your Python environment to 3.10 or newer.","message":"ConnectorX now requires Python 3.10 or higher. Users on older Python versions (3.9 and below) will not be able to install or run the latest versions.","severity":"breaking","affected_versions":">=0.4.5 (and earlier 0.4.x releases)"},{"fix":"Update `except` blocks to catch `ModuleNotFoundError` for cases where optional dependencies (like pandas, polars, pyarrow) might be missing for the specified `return_type`.","message":"Error handling for missing module dependencies has changed. Previously, a `ValueError` might have been raised; it is now `ModuleNotFoundError`.","severity":"breaking","affected_versions":">=0.4.0"},{"fix":"Install the necessary optional dependencies using `pip install \"connectorx[pandas]\"`, `\"connectorx[polars]\"`, or `\"connectorx[arrow]\"` depending on your required return type.","message":"ConnectorX's core functionality relies on optional dependencies (pandas, polars, pyarrow) based on the `return_type` parameter. If you request `return_type=\"pandas\"` without `pandas` installed, it will fail.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Review your `pyarrow` version dependencies. Ensure compatibility with the `pyarrow` version bundled or expected by ConnectorX, especially if you manually manage Arrow data within your application.","message":"Internal changes related to Arrow libraries (e.g., removal of `arrow2` in v0.4.2, and subsequent `pyarrow` bumps) may affect advanced users who rely on specific Arrow version compatibility or internal structures. This can lead to unexpected type conversions or performance regressions if not carefully managed with existing Arrow-based workflows.","severity":"gotcha","affected_versions":">=0.4.2"},{"fix":"Always double-check your connection string format against the ConnectorX documentation for your specific database (e.g., PostgreSQL, MySQL, MSSQL) to ensure correctness. Test connectivity with simpler tools if issues persist.","message":"ConnectorX's connection strings are strict and database-specific. Minor syntax errors (e.g., missing slashes, incorrect port, invalid parameters) will prevent connection without clear specific error messages, often resulting in generic connection failures.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}