{"id":3514,"library":"ibis-framework","title":"Ibis: The Portable Python Dataframe Library","description":"Ibis is a portable Python dataframe library that provides a Pythonic way to build and execute operations on data in various backends, including SQL databases, data warehouses, and data lakes. It offers a familiar dataframe API that compiles into the backend's native language, enabling local iteration and remote deployment by changing a single line of code. It is currently at version 12.0.0 and maintains an active release cadence with frequent updates.","status":"active","version":"12.0.0","language":"en","source_language":"en","source_url":"https://github.com/ibis-project/ibis","tags":["dataframe","sql","data-warehousing","etl","duckdb","polars","bigquery"],"install":[{"cmd":"pip install 'ibis-framework[duckdb,examples]'","lang":"bash","label":"Recommended for local development with example data"},{"cmd":"pip install ibis-framework","lang":"bash","label":"Core library only"},{"cmd":"pip install 'ibis-framework[all]'","lang":"bash","label":"All backends (large install)"}],"dependencies":[{"reason":"Default and recommended local backend for high performance.","package":"duckdb","optional":true},{"reason":"Commonly used for converting Ibis expressions to in-memory DataFrames with `.to_pandas()`.","package":"pandas","optional":true},{"reason":"Used for efficient data handling and interoperability between Ibis and other data libraries.","package":"pyarrow","optional":true},{"reason":"Backend for Apache Spark integration.","package":"pyspark","optional":true},{"reason":"Underpins many SQL database connections.","package":"sqlalchemy","optional":true}],"imports":[{"note":"Installing 'ibis' (a web templating framework) instead of 'ibis-framework' (the dataframe library) is a common mistake. Always install 'ibis-framework' from PyPI.","wrong":"import ibis","symbol":"ibis","correct":"import ibis"},{"note":"Enables eager evaluation for interactive exploration, though it can be slower for production workloads and is not strictly necessary.","symbol":"ibis.options.interactive","correct":"ibis.options.interactive = True"}],"quickstart":{"code":"import ibis\n\n# Connect to an in-memory DuckDB database (default backend)\ncon = ibis.duckdb.connect(':memory:')\n\n# Load example data (e.g., the 'penguins' dataset)\nt = ibis.examples.penguins.fetch()\n\n# Create a table in the connected database\ncon.create_table('penguins', t.to_pyarrow(), overwrite=True)\n\n# Get a table expression from the connection\ntable = con.table('penguins')\n\n# Perform a lazy computation: group by species and calculate mean bill length\nresult_expr = table.group_by('species').agg(avg_bill_length=table.bill_length_mm.mean())\n\n# Execute the expression and fetch results into a pandas DataFrame\ndf = result_expr.to_pandas()\n\nprint(df)\n","lang":"python","description":"This quickstart demonstrates how to connect to an in-memory DuckDB backend, load an example dataset, define a lazy dataframe expression to calculate the average bill length per penguin species, and then execute it to retrieve the results as a pandas DataFrame. It showcases Ibis's core lazy evaluation pattern."},"warnings":[{"fix":"Upgrade your Python environment to 3.10 or a later version.","message":"Python 3.9 is no longer supported. Users must upgrade to Python 3.10 or newer.","severity":"breaking","affected_versions":">=12.0.0"},{"fix":"If using the PySpark backend, ensure PySpark is version 3.5 or newer.","message":"PySpark versions older than 3.5 are no longer supported when using the PySpark backend.","severity":"breaking","affected_versions":">=12.0.0"},{"fix":"Replace `dt.name` with `dt.__class__.__name__` where `dt` is an Ibis DataType object.","message":"The `DataType.name` attribute has been removed. Use `DataType.__class__.__name__` instead to get the string name of a data type.","severity":"breaking","affected_versions":">=11.0.0"},{"fix":"Instead of explicitly naming ephemeral in-memory tables, use `connection.create_table()` or `connection.create_view()` for persistent or named views.","message":"Explicit naming of `memtable`s is no longer supported. Use `create_table` or `create_view` for named objects.","severity":"breaking","affected_versions":">=11.0.0"},{"fix":"Always append `.execute()` or `.to_pandas()` (or similar materialization methods) to your Ibis expression chain to trigger computation and retrieve results.","message":"Ibis uses lazy evaluation. Operations define an expression graph but do not execute immediately. To get results, you must explicitly call methods like `.execute()` or `.to_pandas()`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use `pip install ibis-framework` (with relevant extras) or `conda install ibis-framework` to install the correct library.","message":"There is a different PyPI package also named `ibis` which is a web templating framework. Ensure you install `ibis-framework` for the dataframe library.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}