Google Cloud BigQuery Client (Async)

Common errors

• RuntimeError: Event loop is closed

  cause Attempting to run asyncio tasks or access the event loop after it has been explicitly closed, often by calling `asyncio.run()` more than once, or mixing sync and async code improperly.
  fix Ensure `asyncio.run()` is called only once per application execution. If you need to run multiple async functions, wrap them in a single `async def main():` and call `asyncio.run(main())`.

• gcloud_aio_auth.exceptions.AuthError: Failed to obtain credentials

  cause Google Cloud authentication failed. This can be due to missing `GOOGLE_APPLICATION_CREDENTIALS` environment variable, an invalid service account key, or `gcloud` not being logged in.
  fix Verify your `GOOGLE_APPLICATION_CREDENTIALS` environment variable points to a valid service account JSON file, or ensure `gcloud auth application-default login` has been run and is active. For service accounts, ensure the path to the JSON file is correct.

• AttributeError: module 'gcloud_aio_bigquery' has no attribute 'BigqueryClient'

  cause Incorrect import statement for `BigqueryClient`.
  fix The correct import is `from gcloud_aio_bigquery import BigqueryClient`.

• google.api_core.exceptions.Forbidden: 403 The caller does not have permission

  cause The authenticated service account or user lacks the necessary IAM permissions or OAuth scopes for the requested BigQuery operation.
  fix Review the IAM roles assigned to your service account or user in the Google Cloud Console. Ensure they have sufficient permissions (e.g., `BigQuery Data Editor`, `BigQuery User`, etc.). Also, check that the correct OAuth scopes are requested during credential building.

Warnings

• breaking `gcloud-aio-auth` (a core dependency) dropped support for Python 3.9 in version 5.4.4. While `gcloud-rest-bigquery` technically supports Python 3.9, installing `gcloud-aio-auth>=5.4.4` will cause issues.
  Fix: Upgrade your Python environment to 3.10 or newer, or pin your `gcloud-aio-auth` dependency to `<5.4.4` in your `requirements.txt`.
• gotcha Incorrect `asyncio` event loop management (e.g., calling `asyncio.run()` multiple times in the same thread, or not awaiting coroutines) can lead to `RuntimeError: Event loop is closed` or deadlocks.
  Fix: Ensure `asyncio.run()` is called only once per thread. Use `async with` for client and credential objects to ensure proper resource cleanup. Always `await` coroutines and `async` functions.
• gotcha Default authentication credentials might not include all necessary OAuth 2.0 scopes for every BigQuery API call, leading to `403 Permission denied` errors.
  Fix: Consult the BigQuery API documentation for the required scopes for your specific operations (e.g., `https://www.googleapis.com/auth/bigquery`, `https://www.googleapis.com/auth/devstorage.read_only`). Pass these explicitly to `build_default_credentials` or `build_from_service_account_info`.

Install

• pip install gcloud-rest-bigquery Install latest version

Imports

• BigqueryClient
  wrong:

  from gcloud_rest_bigquery import BigqueryClient

  correct:

  from gcloud_aio_bigquery import BigqueryClient

  The library uses 'gcloud_aio_bigquery' as its top-level module name for imports, not 'gcloud_rest_bigquery'.
• build_default_credentials
  wrong:

  from gcloud_aio_bigquery.auth import build_default_credentials

  correct:

  from gcloud_aio_auth import build_default_credentials

  Authentication utilities are provided by the separate 'gcloud-aio-auth' package.

Quickstart

This quickstart demonstrates how to initialize the BigqueryClient using default Google Cloud credentials (e.g., `gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`), run a simple SQL query, and fetch its results asynchronously. Ensure the `GCLOUD_PROJECT` environment variable is set to your Google Cloud project ID.

import asyncio
import os
from gcloud_aio_auth import build_default_credentials
from gcloud_aio_bigquery import BigqueryClient

async def main():
    project_id = os.environ.get("GCLOUD_PROJECT", "")
    if not project_id:
        print("Please set the GCLOUD_PROJECT environment variable.")
        return

    # Using default credentials (e.g., from gcloud CLI, service account JSON in GOOGLE_APPLICATION_CREDENTIALS)
    async with build_default_credentials(
        scopes=["https://www.googleapis.com/auth/cloud-platform"]
    ) as credentials:
        async with BigqueryClient(
            project=project_id, credentials=credentials
        ) as client:
            # Execute a simple query
            print(f"Running query in project: {project_id}")
            query_job = await client.query(
                project_id=project_id,
                query="SELECT 1 AS one, 'hello' AS greeting",
                use_legacy_sql=False,
                location="US" # Or your preferred dataset location
            )

            # Wait for the job to complete and fetch results
            results = await client.get_job_results(
                project_id=project_id,
                job_id=query_job["jobReference"]["jobId"],
                location=query_job["jobReference"]["location"]
            )

            print("Query results:")
            if results and "rows" in results:
                for row in results["rows"]:
                    # Each 'f' element corresponds to a column, 'v' is the value
                    print([col["v"] for col in row["f"]])
            else:
                print("No rows returned or unexpected result format.")

if __name__ == "__main__":
    asyncio.run(main())