{"id":7982,"library":"bigquery-magics","title":"Google BigQuery Magics for Jupyter and IPython","description":"bigquery-magics provides IPython cell magics that allow users to run SQL queries directly within Jupyter and IPython notebooks against Google BigQuery. It simplifies data exploration by automatically converting query results into pandas DataFrames. The library is part of the actively maintained `google-cloud-python` monorepo and sees frequent updates, with the current stable version being 0.14.0.","status":"active","version":"0.14.0","language":"en","source_language":"en","source_url":"https://github.com/googleapis/google-cloud-python/tree/main/packages/bigquery-magics","tags":["bigquery","jupyter","ipython","google-cloud","data-analysis","sql"],"install":[{"cmd":"pip install bigquery-magics","lang":"bash","label":"Install latest stable version"}],"dependencies":[{"reason":"Core library for interacting with the BigQuery API.","package":"google-cloud-bigquery","optional":false},{"reason":"Query results are returned as pandas DataFrames by default.","package":"pandas","optional":false},{"reason":"Required by google-cloud-bigquery (>=3.x) for faster result downloads.","package":"google-cloud-bigquery-storage","optional":false},{"reason":"Required by google-cloud-bigquery (>=3.x) for efficient data handling, especially with storage API.","package":"pyarrow","optional":false},{"reason":"Optional dependency for displaying query progress bars.","package":"tqdm","optional":true}],"imports":[{"note":"Magics are loaded using the %load_ext command, not Python's 'import' statement.","symbol":"bigquery_magics","correct":"%load_ext bigquery_magics"}],"quickstart":{"code":"# Load the BigQuery magics extension\n%load_ext bigquery_magics\n\n# Set your Google Cloud Project ID\n# Replace 'your-gcp-project-id' with your actual Project ID.\n# Ensure your environment is authenticated (e.g., via gcloud auth application-default login)\nimport os\nos.environ['GOOGLE_CLOUD_PROJECT'] = os.environ.get('GOOGLE_CLOUD_PROJECT', 'your-gcp-project-id')\n\n# Run a BigQuery SQL query using the %%bigquery cell magic\n# The results will be stored in a pandas DataFrame named 'df_names'\n%%bigquery df_names\nSELECT\n    name,\n    SUM(number) AS count\nFROM\n    `bigquery-public-data.usa_names.usa_1910_current`\nGROUP BY\n    name\nORDER BY\n    count DESC\nLIMIT\n    5;\n\n# Display the DataFrame\nprint(df_names)","lang":"python","description":"This quickstart demonstrates how to load the `bigquery_magics` extension, set the Google Cloud project, and execute a BigQuery SQL query directly within a Jupyter cell. The `%%bigquery` cell magic runs the query and automatically saves the results into a pandas DataFrame, specified as a line argument."},"warnings":[{"fix":"Upgrade your Python environment to 3.9 or higher. The current version requires Python >= 3.10.","message":"Python 2 and older Python 3 versions (<= 3.8) are no longer supported. Ensure your environment runs Python 3.9 or newer.","severity":"breaking","affected_versions":"<=0.10.x"},{"fix":"Ensure you have authenticated your environment using `gcloud auth application-default login` for local development, or that your service account has the necessary IAM permissions (e.g., BigQuery User, BigQuery Data Editor) if running on Google Cloud.","message":"Authentication to Google Cloud and BigQuery is crucial. Without proper Application Default Credentials (ADC) or explicit credential setup, queries will fail with authorization errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Set the `GOOGLE_CLOUD_PROJECT` environment variable, or use the `--project` argument with the `%%bigquery` magic (e.g., `%%bigquery --project your-project-id`). Ensure billing is enabled for the specified project.","message":"Specifying the correct Google Cloud Project ID is essential. If not explicitly set via an environment variable or magic argument, the magic may fail or query against an unintended project.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Refer to the monorepo's documentation for the most up-to-date development and contribution information.","message":"The `bigquery-magics` library's source code was migrated into the larger `googleapis/google-cloud-python` monorepo. While generally not a breaking change for users, be aware that issue tracking and contribution guidelines are now centralized in the monorepo.","severity":"gotcha","affected_versions":"All versions (organizational change)"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Verify the Project ID in the Google Cloud Console. Ensure you have the `resourcemanager.projects.get` permission and that billing is enabled for the project. Set the correct ID via `os.environ['GOOGLE_CLOUD_PROJECT'] = 'your-project-id'` or `--project` argument.","cause":"The specified Google Cloud Project ID is incorrect, does not exist, or you lack permissions to view it.","error":"Project 'your-project-id' not found."},{"fix":"Grant the required IAM roles (e.g., `BigQuery User`, `BigQuery Data Editor`, `BigQuery Job User`) to your Google Cloud identity. If using `gcloud auth login`, ensure you've authenticated with scopes that allow BigQuery access.","cause":"Your authenticated user or service account lacks the necessary IAM roles or permissions to access BigQuery resources.","error":"403 Forbidden: Request had insufficient authentication scopes."},{"fix":"Check if your BigQuery project has sufficient quota, or if it's assigned to a reservation with available slots. Ensure the dataset/table you're querying exists in the region where the query is being run. Verify billing is enabled.","cause":"This usually indicates an issue with BigQuery reservations, project region mismatch, or insufficient quota/billing.","error":"Cannot run query: project does not have the reservation in the data region or no slots are configured."},{"fix":"Go to the Google Cloud Console, navigate to 'Billing', and ensure a valid billing account is linked and active for the specified project. BigQuery queries incur costs and require an enabled billing account.","cause":"The Google Cloud project associated with your queries does not have an active billing account linked.","error":"400 Bad Request: Billing account not enabled for project 'your-project-id'."}]}