SSH Filesystem for fsspec

2025.11.0 · active · verified Fri Apr 10

sshfs is an implementation of fsspec, providing a unified Pythonic interface for interacting with SFTP servers over SSH. It leverages the asyncssh library for its underlying secure shell operations, offering a fast and asynchronous way to manage remote filesystems. The library is actively maintained, with its current version being 2025.11.0, and typically sees several releases per year to incorporate updates and improvements. [3, 10]

Warnings

Install

Imports

Quickstart

This quickstart demonstrates how to connect to an SFTP server using `sshfs` with both password and SSH private key authentication. It shows direct usage of the `SSHFileSystem` class for programmatic file operations and integrates with `fsspec.open` for URL-based file access. It includes examples for listing, writing, and reading files, with cleanup operations. Remember to replace placeholder credentials with secure environment variables or a robust SSH configuration. [3, 10]

import os
from sshfs import SSHFileSystem
from fsspec import open

# Configuration (use environment variables for sensitive data)
SSH_HOST = os.environ.get('SSH_HOST', '127.0.0.1')
SSH_USER = os.environ.get('SSH_USER', 'user')
SSH_PASSWORD = os.environ.get('SSH_PASSWORD', 'password')
SSH_KEY_PATH = os.environ.get('SSH_KEY_PATH', '~/.ssh/id_rsa')

# --- Method 1: Using SSHFileSystem class directly ---
print("\n--- Using SSHFileSystem directly ---")
try:
    # Connect with password
    fs_pass = SSHFileSystem(SSH_HOST, username=SSH_USER, password=SSH_PASSWORD)
    print(f"Connected to {SSH_HOST} with password. Current dir: {fs_pass.pwd()}")
    # Example operation: list files
    print("Files in remote home (password auth):", fs_pass.ls('.'))
    fs_pass.close()

    # Connect with private key
    fs_key = SSHFileSystem(SSH_HOST, username=SSH_USER, client_keys=[SSH_KEY_PATH])
    print(f"Connected to {SSH_HOST} with key. Current dir: {fs_key.pwd()}")
    print("Files in remote home (key auth):", fs_key.ls('.'))

    # Example: Write and read a file
    remote_path = f'/tmp/test_sshfs_{os.getpid()}.txt'
    with fs_key.open(remote_path, 'w') as f:
        f.write('Hello from sshfs via key!')
    with fs_key.open(remote_path, 'r') as f:
        content = f.read()
    print(f"Read from {remote_path}: '{content}'")
    fs_key.rm(remote_path)
    print(f"Cleaned up {remote_path}")
    fs_key.close()

except Exception as e:
    print(f"SSHFileSystem connection/operation failed: {e}")

# --- Method 2: Using fsspec.open with URL ---
print("\n--- Using fsspec.open with URL ---")
try:
    # Using SSH URL with password
    # Note: fsspec.open requires storage_options for auth. Password direct in URL is less secure.
    # Better to pass via storage_options or rely on SSH agent/config.
    with open(f'ssh://{SSH_USER}:{SSH_PASSWORD}@{SSH_HOST}/tmp/fsspec_test_{os.getpid()}.txt', 'w') as f:
        f.write('Hello from fsspec.open!')
    print("Wrote file via fsspec.open (password).")

    with open(f'ssh://{SSH_USER}@{SSH_HOST}/tmp/fsspec_test_{os.getpid()}.txt', 'r',
              client_keys=[SSH_KEY_PATH]) as f:
        content = f.read()
    print(f"Read via fsspec.open (key auth): '{content}'")

    # Clean up (requires SSHFileSystem directly or a complex fsspec glob pattern)
    fs_cleanup = SSHFileSystem(SSH_HOST, username=SSH_USER, client_keys=[SSH_KEY_PATH])
    fs_cleanup.rm(f'/tmp/fsspec_test_{os.getpid()}.txt')
    print("Cleaned up fsspec.open test file.")
    fs_cleanup.close()

except Exception as e:
    print(f"fsspec.open connection/operation failed: {e}")

print("\nQuickstart finished.")

view raw JSON →