Google Cloud BigQuery Data Policies

0.9.0 · active · verified Fri Apr 17

The Google Cloud BigQuery Data Policies API client library provides programmatic access to manage data policies (e.g., column-level security or data masking) within BigQuery datasets. This library allows users to create, retrieve, update, and delete data policies, enforcing fine-grained access control on BigQuery data. It is currently at version 0.9.0 and is part of the `google-cloud-python` monorepo, receiving updates aligned with the underlying API and shared client library components.

Common errors

```
ModuleNotFoundError: No module named 'google.cloud.bigquery_datapolicies'
```
cause Attempting to import the client library without specifying the API version, which is required for this library.

fix Correct the import statement to include the API version, usually `_v1`. Use `from google.cloud import bigquery_datapolicies_v1` instead.
```
google.api_core.exceptions.NotFound: 404 Not found: Location 'global' is not supported for BigQuery Data Policy APIs.
```
cause Trying to perform operations on BigQuery Data Policies in a 'global' location, which is not allowed. Data policies are regional resources.

fix Specify a valid regional location (e.g., 'us-central1', 'europe-west2') for all API calls involving BigQuery Data Policies.
```
google.api_core.exceptions.InvalidArgument: 400 Invalid argument: Requested project 'your-project' does not exist.
```
cause The `project_id` provided in the API call (e.g., in the `parent` string) is incorrect, misspelled, or the authenticated principal does not have access to it.

fix Double-check the `project_id` for accuracy. Ensure your `GOOGLE_CLOUD_PROJECT` environment variable is set correctly or the project ID is explicitly passed correctly. Verify that the authenticated service account or user has access to that specific project.

Warnings

gotcha The library is in `0.x.x` versioning. While generally stable, this indicates that the API is not yet declared stable and may introduce breaking changes in minor versions, not just major versions.
Fix: Always pin to a specific `0.x.y` version in your `requirements.txt` (e.g., `google-cloud-bigquery-datapolicies==0.9.0`) and review release notes when upgrading.
gotcha BigQuery Data Policies are a regional resource. You must specify a valid `location` when interacting with the API, as operations on resources in a `global` location are not supported.
Fix: Ensure that the `location` parameter (e.g., 'us-central1', 'europe-west2') in your API calls is correct and matches the region where your data policies are defined.
gotcha PermissionDenied errors (HTTP 403) are common when your service account or user lacks the necessary IAM roles to manage BigQuery Data Policies. Required roles include 'BigQuery Data Policy Admin' or equivalent custom roles.
Fix: Verify that the principal (user or service account) used for authentication has the appropriate IAM roles on the BigQuery project or dataset. For creating/managing policies, 'BigQuery Data Policy Admin' is typically required.

Install

pip install google-cloud-bigquery-datapolicies Install stable version

Imports

DataPolicyServiceClient
wrong:
```
from google.cloud import bigquery_datapolicies
```
correct:
```
from google.cloud import bigquery_datapolicies_v1
client = bigquery_datapolicies_v1.DataPolicyServiceClient()
```
Google Cloud client libraries often include API version numbers (e.g., `_v1`) in their import paths to distinguish between different API versions.

Quickstart

This quickstart initializes the BigQuery Data Policies client and lists all data policies within a specified Google Cloud project and location. Remember to authenticate your application (e.g., via `gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`).

import os
from google.cloud import bigquery_datapolicies_v1
from google.api_core.exceptions import GoogleAPIError

def list_bigquery_data_policies(project_id: str, location: str):
    """Lists BigQuery data policies for a given project and location."""
    # Ensure GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS
    # or other authentication methods are set in your environment.
    client = bigquery_datapolicies_v1.DataPolicyServiceClient()
    parent = f"projects/{project_id}/locations/{location}"

    print(f"Listing data policies for {parent}:")
    try:
        response = client.list_data_policies(parent=parent)
        policies_found = False
        for data_policy in response:
            print(f"  - Data Policy: {data_policy.name}")
            policies_found = True
        if not policies_found:
            print("  No data policies found.")
    except GoogleAPIError as e:
        print(f"Error listing data policies: {e}")

# Example usage:
# Replace with your actual project ID and desired location
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT", "your-gcp-project-id")
LOCATION = os.environ.get("GCP_REGION", "us-central1") # e.g., "us-central1", "europe-west2"

if PROJECT_ID == "your-gcp-project-id":
    print("Please set the GOOGLE_CLOUD_PROJECT environment variable or replace 'your-gcp-project-id' in the script.")
else:
    list_bigquery_data_policies(PROJECT_ID, LOCATION)

view raw JSON →