{"id":1741,"library":"tensorflow-io-gcs-filesystem","title":"TensorFlow I/O GCS Filesystem","description":"TensorFlow I/O GCS Filesystem (tensorflow-io-gcs-filesystem) extends TensorFlow's capabilities to interact seamlessly with Google Cloud Storage (GCS) by enabling `tf.io.gfile` operations on `gs://` paths. It is part of the broader TensorFlow I/O project, providing a modular installation for GCS-specific functionalities. The library is actively maintained with frequent releases, often aligned with new TensorFlow versions or important bug fixes.","status":"active","version":"0.37.1","language":"en","source_language":"en","source_url":"https://github.com/tensorflow/io","tags":["tensorflow","gcs","google cloud storage","filesystem","io","cloud"],"install":[{"cmd":"pip install tensorflow-io-gcs-filesystem","lang":"bash","label":"Install GCS Filesystem"}],"dependencies":[{"reason":"Core dependency; this library extends TensorFlow's I/O capabilities.","package":"tensorflow","optional":false}],"imports":[{"note":"The `as _` convention is commonly used to indicate that the module is imported for its side effects (registering the GCS filesystem handler) rather than for direct symbol usage.","wrong":"import tensorflow_io_gcs_filesystem","symbol":"GCS Filesystem Registration","correct":"import tensorflow_io_gcs_filesystem as _"}],"quickstart":{"code":"import tensorflow as tf\nimport tensorflow_io_gcs_filesystem as _  # Registers the GCS filesystem handler\nimport os\n\n# Ensure you have GCS authentication set up, e.g.,\n# by running `gcloud auth application-default login`\n# or setting the GOOGLE_APPLICATION_CREDENTIALS environment variable.\n# For testing, you might use a public bucket or ensure permissions are granted.\n\nbucket_name = os.environ.get('GCS_TEST_BUCKET', 'your-gcs-bucket-name') # Replace with your bucket\nfile_name = \"registry_test_file.txt\"\nfile_path = f\"gs://{bucket_name}/{file_name}\"\n\nprint(f\"Attempting to interact with GCS path: {file_path}\")\n\ntry:\n    # 1. Write a file to GCS\n    with tf.io.gfile.GFile(file_path, 'w') as f:\n        f.write(\"Hello from TensorFlow I/O GCS Filesystem registry check!\")\n    print(f\"Successfully wrote to {file_path}\")\n\n    # 2. Read the file from GCS\n    with tf.io.gfile.GFile(file_path, 'r') as f:\n        content = f.read()\n    print(f\"Successfully read from {file_path}. Content: '{content}'\")\n\n    # 3. List files in the directory (or bucket prefix)\n    parent_path = f\"gs://{bucket_name}\"\n    print(f\"Listing contents of {parent_path}:\")\n    for item in tf.io.gfile.listdir(parent_path):\n        print(f\"  - {item}\")\n\n    # 4. Check if the file exists\n    exists = tf.io.gfile.exists(file_path)\n    print(f\"File {file_path} exists: {exists}\")\n\n    # 5. Delete the file from GCS\n    tf.io.gfile.remove(file_path)\n    print(f\"Successfully deleted {file_path}\")\n\n    # Verify deletion\n    exists_after_delete = tf.io.gfile.exists(file_path)\n    print(f\"File {file_path} exists after delete: {exists_after_delete}\")\n\nexcept tf.errors.FailedPreconditionError as e:\n    print(f\"Error: Failed to connect to GCS. Please ensure you have authenticated and your bucket '{bucket_name}' exists and is accessible. Error details: {e}\")\n    print(\"HINT: Try running `gcloud auth application-default login` or setting `GOOGLE_APPLICATION_CREDENTIALS`.\")\nexcept tf.errors.NotFoundError as e:\n    print(f\"Error: GCS path not found. Please check bucket name and permissions. Error details: {e}\")\nexcept Exception as e:\n    print(f\"An unexpected error occurred: {e}\")","lang":"python","description":"This quickstart demonstrates how to use `tensorflow-io-gcs-filesystem` to enable standard `tf.io.gfile` operations (write, read, list, delete) on Google Cloud Storage paths (starting with `gs://`). It requires prior GCS authentication and a GCS bucket."},"warnings":[{"fix":"Always install `tensorflow-io-gcs-filesystem` with a `tensorflow` version that is explicitly supported by the `tensorflow-io` release notes. For example, v0.37.0 supports TensorFlow 2.16. Check the release notes or the official TensorFlow I/O documentation for compatibility matrices.","message":"TensorFlow I/O (and its sub-packages like GCS Filesystem) are often tightly coupled to specific TensorFlow versions. Mismatches can lead to runtime errors or build issues.","severity":"breaking","affected_versions":"All versions, especially when TensorFlow major/minor versions update."},{"fix":"If you need broader TensorFlow I/O functionality beyond GCS filesystem operations, install the full `tensorflow-io` package (`pip install tensorflow-io`). If only GCS is needed, `tensorflow-io-gcs-filesystem` is sufficient and lighter.","message":"This package (`tensorflow-io-gcs-filesystem`) provides *only* GCS filesystem support. It does not include other features found in the full `tensorflow-io` meta-package (e.g., other file formats like Avro, or other filesystems like S3).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your environment has access to GCS. Common methods include `gcloud auth application-default login`, setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to a service account key file, or running on a GCE/GKE instance with appropriate service account scopes.","message":"Proper Google Cloud Storage authentication is crucial. Operations will fail with permission errors if authentication is not correctly configured in the environment where your TensorFlow code runs.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always include `import tensorflow_io_gcs_filesystem as _` at the beginning of your script or module where you intend to use `tf.io.gfile` with GCS paths.","message":"The GCS filesystem handler is registered by importing `tensorflow_io_gcs_filesystem`. If this import is omitted, `tf.io.gfile` will not recognize `gs://` paths.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your Python environment is within the supported range. Check the `requires_python` field on PyPI for the exact specification.","message":"The library requires Python versions >=3.7 and <3.13.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}