Google Cloud BigLake Client Library for Python

0.3.0 · active · verified Fri Apr 17

The `google-cloud-biglake` library provides a Python client for interacting with the Google Cloud BigLake API. BigLake allows you to unify data lakes and warehouses, enabling a consistent management plane and access control for data stored in various formats across Google Cloud (e.g., Cloud Storage) and other clouds. It is currently in a 0.x.x version, indicating early access or preview, and is part of the larger `googleapis/google-cloud-python` monorepo, receiving updates as the underlying API evolves.

Common errors

```
ModuleNotFoundError: No module named 'google.cloud.biglake'
```
cause Attempting to import the client directly from `google.cloud.biglake` instead of the versioned submodule.

fix Change your import statement to `from google.cloud.biglake_v1 import BigLakeServiceClient`.
```
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or provide a credentials object.
```
cause The client could not find valid Google Cloud credentials in the environment.

fix Ensure you have authenticated to Google Cloud. This can be done by running `gcloud auth application-default login` for user credentials, setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of a service account key file, or ensuring your code is running in a Google Cloud environment (e.g., GCE, Cloud Run, Cloud Functions) with appropriate service account permissions.
```
google.api_core.exceptions.NotFound: 404 Not Found: The requested resource was not found. Please ensure the project, location, and resource exist.
```
cause The specified project ID, location, or BigLake resource (catalog, database, table) does not exist or is misspelled.

fix Double-check the project ID, location, and the names of any BigLake resources you are trying to access. Verify that the BigLake API is enabled for your project in the Google Cloud Console.

Warnings

gotcha The library is in a 0.x.x version, meaning the API surface might not be fully stable. Breaking changes may occur in minor versions until a 1.0.0 release. Always pin to specific patch versions (e.g., `google-cloud-biglake==0.3.0`) in production environments.
Fix: Pin to specific patch versions in your `requirements.txt` or `pyproject.toml` to prevent unexpected breaking changes: `google-cloud-biglake==0.3.0`.
gotcha Google Cloud BigLake resources (catalogs, databases, tables) require specific permission roles for access. Common issues arise from missing 'BigLake Admin' (roles/biglake.admin) or 'BigLake Viewer' (roles/biglake.viewer) permissions on the project or specific resources.
Fix: Ensure the service account or user running the code has the necessary BigLake IAM roles assigned at the project or resource level via the Google Cloud Console or `gcloud iam` commands.
gotcha BigLake API methods often require resource names in a specific string format (e.g., `projects/{project}/locations/{location}/catalogs/{catalog}`). Incorrectly formatted resource paths will result in `google.api_core.exceptions.InvalidArgument` errors.
Fix: Always construct resource paths using the `client.catalog_path`, `client.database_path`, or `client.table_path` methods, or carefully follow the documented string format for each API call.

Install

pip install google-cloud-biglake Install stable version

Imports

BigLakeServiceClient
wrong:
```
from google.cloud.biglake import BigLakeServiceClient
```
correct:
```
from google.cloud.biglake_v1 import BigLakeServiceClient
```
The BigLake API is currently versioned as `v1`, so the client lives within the `biglake_v1` submodule. Importing directly from `biglake` will lead to a ModuleNotFoundError.

Quickstart

This quickstart demonstrates how to initialize the BigLakeServiceClient and list existing BigLake catalogs within a specified Google Cloud project and location. It assumes you have Application Default Credentials (ADC) configured or `GOOGLE_APPLICATION_CREDENTIALS` environment variable set.

import os
from google.cloud.biglake_v1 import BigLakeServiceClient

# Set your Google Cloud Project ID and Location
# You can also set GOOGLE_CLOUD_PROJECT environment variable
project_id = os.environ.get('GCP_PROJECT_ID', 'your-project-id')
location = os.environ.get('GCP_LOCATION', 'us-central1') # e.g., 'us-central1', 'europe-west1'

if project_id == 'your-project-id' or location == 'us-central1':
    print("Please set GCP_PROJECT_ID and GCP_LOCATION environment variables or replace placeholders.")
    exit()

client = BigLakeServiceClient()

# Construct the parent path for listing catalogs
# The parent must be in the format: projects/{project}/locations/{location}
parent_path = f"projects/{project_id}/locations/{location}"

try:
    # List BigLake catalogs within the specified project and location
    print(f"Listing BigLake catalogs in {parent_path}:")
    for catalog in client.list_catalogs(parent=parent_path):
        print(f"  Catalog: {catalog.name}")
except Exception as e:
    print(f"An error occurred: {e}")
    print("Ensure BigLake API is enabled and your service account has 'biglake.catalogs.list' permission.")

view raw JSON →