{"library":"snakebite-py3","title":"Snakebite-py3: Pure Python HDFS Client","type":"library","description":"snakebite-py3 is a Python library that provides a pure Python client for the Hadoop Distributed File System (HDFS). It communicates directly with the HDFS NameNode using protobuf messages and implements the Hadoop RPC protocol, offering a native Python alternative to calling Java-based `hadoop fs` commands. This fork, maintained by the Internet Archive, specifically targets Python 3 compatibility. The current version is 3.0.6, released in February 2025.","language":"python","status":"active","last_verified":"Sat May 16","install":{"commands":["pip install snakebite-py3"],"cli":{"name":"snakebite","version":"usage: snakebite [general options] cmd [arguments]"}},"imports":["from snakebite.client import Client","from snakebite.client import Client"],"auth":{"required":false,"env_vars":[]},"links":{"homepage":null,"github":"http://github.com/internetarchive/snakebite-py3","docs":null,"changelog":null,"pypi":"https://pypi.org/project/snakebite-py3/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null},"quickstart":{"code":"import os\nfrom snakebite.client import Client\n\n# Configure HDFS NameNode host and port\n# Default HDFS RPC port is 8020\nhost = os.environ.get('HDFS_NAMENODE_HOST', 'localhost')\nport = int(os.environ.get('HDFS_NAMENODE_PORT', '8020'))\n\ntry:\n    # Initialize the HDFS client\n    # It's recommended to set use_trash=False for non-interactive scripts\n    # Or explicitly set hadoop_version if not the default (9)\n    client = Client(host, port, use_trash=False)\n\n    print(f\"Connected to HDFS NameNode at {host}:{port}\")\n\n    # Example: List contents of the root directory\n    print(\"Listing /:\")\n    for item in client.ls(['/']):\n        print(item)\n\n    # Example: Create a directory\n    test_dir = '/user/test_snakebite_py3'\n    if not list(client.ls([test_dir])):\n        print(f\"Creating directory {test_dir}\")\n        list(client.mkdir([test_dir], create_parents=True))\n    else:\n        print(f\"Directory {test_dir} already exists.\")\n\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\n    print(\"Please ensure your HDFS NameNode is running and accessible at the specified host and port.\")\n","lang":"python","description":"This quickstart demonstrates how to establish a connection to an HDFS NameNode and perform basic file system operations like listing directories and creating a new directory. It uses environment variables for host and port for flexibility, defaulting to `localhost:8020`. Ensure your HDFS cluster is running and accessible from where you execute this code.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-16","installed_version":"3.0.6","pypi_latest":"3.0.6","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":2,"avg_import_s":0.34,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.27,"mem_mb":11.6,"disk_size":"20.2M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.1,"import_time_s":0.13,"mem_mb":7.5,"disk_size":"23M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.52,"mem_mb":12.1,"disk_size":"22.5M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":1.9,"import_time_s":0.45,"mem_mb":12.1,"disk_size":"23M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.4,"mem_mb":12.2,"disk_size":"14.3M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":1.7,"import_time_s":0.43,"mem_mb":12.2,"disk_size":"15M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.41,"mem_mb":12.7,"disk_size":"14.1M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":1.8,"import_time_s":0.42,"mem_mb":12.7,"disk_size":"15M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"noisy","install_time_s":null,"import_time_s":0.21,"mem_mb":10.9,"disk_size":"19.7M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"snakebite-py3","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.3,"import_time_s":0.13,"mem_mb":7,"disk_size":"23M"}]}}