Google Compute Engine Guest Environment Utilities
The `google-compute-engine` library provides core utilities, scripts, and daemons that run within Google Compute Engine (GCE) virtual machine instances. It enables VMs to properly interact with the GCE platform, primarily by reading from and writing to the metadata server. This package is part of the standard Google-provided OS images for Compute Engine, and its primary role is to facilitate the VM's lifecycle and configuration, rather than acting as a client library for managing GCE resources externally. The latest PyPI version is 2.8.13, but the public releases on GitHub are quite old (late 2019), indicating an infrequent, system-level release cadence.
Common errors
-
ModuleNotFoundError: No module named 'google_compute_engine'
cause The `google-compute-engine` package is typically pre-installed on Google-provided Compute Engine OS images as a system component, or installed manually for VM-internal use. It is not usually imported by standard Python applications for external API calls, and trying to do so often indicates a misunderstanding of its purpose.fixIf you intend to manage Compute Engine resources programmatically, install and use the `google-cloud-compute` library (`pip install google-cloud-compute`). If you need to interact with the VM's guest environment, ensure the package is installed on the VM and check the context of your import. Often, direct interaction with the metadata server (as shown in quickstart) is the intended pattern. -
requests.exceptions.ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))cause This error occurs when trying to access the metadata server URL (e.g., `http://metadata.google.internal`) from a machine that is not a Google Compute Engine VM, or if the metadata server is temporarily unavailable/unreachable within the VM.fixVerify that your code is running inside a Google Compute Engine VM. If it is, check network connectivity or potential firewall rules that might be blocking access to the metadata server (though this is rare for the default internal endpoint). Ensure the URL `http://metadata.google.internal` is correct and accessible. -
HTTPError: 404 Client Error: Not Found for url: http://metadata.google.internal/computeMetadata/v1/instance/non-existent-key
cause You are trying to retrieve a metadata key or path that does not exist on the VM or in the project metadata. This could be a typo in the path or an attempt to access a custom metadata key that has not been set.fixDouble-check the metadata path you are querying against the official Compute Engine metadata documentation. If accessing custom metadata, ensure the custom key has been correctly set for the instance or project via `gcloud compute instances add-metadata` or the Cloud Console.
Warnings
- gotcha This library (`google-compute-engine`) is distinct from the Google Cloud Client Library for Compute Engine (`google-cloud-compute`). This package provides low-level guest environment utilities for VMs, while `google-cloud-compute` is the idiomatic Python client to interact with the Compute Engine API for managing resources from an external application.
- deprecated The public GitHub releases for `google-compute-engine` have not been updated since late 2019 (v20191210). While the package remains functional and is included in GCE images, significant active development or new Python-facing features should not be expected. Users should rely on `google-cloud-compute` for modern API interactions.
- gotcha Metadata access relies on the HTTP metadata server (e.g., `http://metadata.google.internal`). This server is only accessible from within a Compute Engine VM instance. Attempting to query the metadata server from outside a GCE VM will result in connection errors.
Install
-
pip install google-compute-engine
Imports
- metadata_server
from google_compute_engine import ComputeEngineClient
import requests # This library's components are primarily system-level agents, # not typically imported directly by user applications for API access. # Interaction on a GCE VM is usually via the metadata server.
Quickstart
import requests
import json
import os
# This quickstart demonstrates how to access VM metadata from within a Google Compute Engine instance.
# The google-compute-engine package helps manage this interaction at a system level.
METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/'
HEADERS = {'Metadata-Flavor': 'Google'}
def get_metadata(path):
try:
response = requests.get(METADATA_URL + path, headers=HEADERS, timeout=1)
response.raise_for_status() # Raise an HTTPError for bad responses (4xx or 5xx)
return response.text
except requests.exceptions.RequestException as e:
# Check if running on a GCE instance or if URL is accessible
if 'metadata.google.internal' in str(e):
return f"Error accessing metadata. This code must run inside a Google Compute Engine VM. Details: {e}"
else:
return f"Error accessing metadata: {e}"
if __name__ == '__main__':
print('--- Instance Name ---')
instance_name = get_metadata('instance/name')
print(f'Instance Name: {instance_name}')
print('\n--- Project ID ---')
project_id = get_metadata('project/project-id')
print(f'Project ID: {project_id}')
print('\n--- Zone ---')
zone_full = get_metadata('instance/zone')
zone = zone_full.split('/')[-1] if 'zones/' in zone_full else zone_full
print(f'Zone: {zone}')
print('\n--- Custom Metadata (example) ---')
# To test custom metadata, add a key-value pair 'my-custom-key: my-custom-value'
# to your VM or project metadata via gcloud CLI or Cloud Console.
# Example: gcloud compute instances add-metadata my-instance --metadata=my-custom-key=my-custom-value --zone=us-central1-a
custom_key = 'instance/attributes/my-custom-key'
custom_value = get_metadata(custom_key)
print(f'Custom Metadata (my-custom-key): {custom_value}')
print('\n--- All Project Attributes (JSON format) ---')
all_project_attributes_json = get_metadata('project/attributes/?recursive=true')
try:
print(json.dumps(json.loads(all_project_attributes_json), indent=2))
except json.JSONDecodeError:
print(all_project_attributes_json)