Type Annotations for s3transfer

Warnings

• gotcha It's crucial to install `s3transfer` alongside `types-s3transfer`. `types-s3transfer` provides only type stubs and does not contain the runtime implementation. Without `s3transfer` installed, your code will fail at runtime with `ModuleNotFoundError`.
  Fix: Ensure both `s3transfer` and `types-s3transfer` are installed: `pip install s3transfer types-s3transfer`.
• gotcha Ensure the version of `types-s3transfer` installed closely matches the version of `s3transfer` you are using. Mismatched versions can lead to incorrect type hints or false positive type errors, as the API might have changed between `s3transfer` versions but the stubs don't reflect that change.
  Fix: Consult the `types-s3transfer` release notes or `s3transfer` documentation to determine compatible versions. Ideally, keep them in sync, e.g., `pip install 's3transfer==X.Y.Z' 'types-s3transfer==X.Y.Z'`.
• breaking The return type of `TransferFuture.result()` was fixed in `types-s3transfer` version 0.6.0.post2 (and likely corresponding `s3transfer` versions) to correctly return `str`. Older versions of the stubs might have had incorrect `Any` types, potentially masking real type issues or causing confusion.
  Fix: Upgrade to `types-s3transfer` version `0.6.0.post2` or newer to get the correct `str` return type for `TransferFuture.result()`: `pip install --upgrade types-s3transfer`.
• gotcha Some users have reported `ImportError` for components like `RetriesExceededError` from `s3transfer.exceptions`, often due to version conflicts between `boto3`, `botocore`, and `s3transfer`. While `types-s3transfer` itself is stubs, these underlying runtime issues can affect how correctly the stubs are applied or whether the application even runs.
  Fix: Ensure a compatible set of `boto3`, `botocore`, and `s3transfer` versions. Often, `pip install boto3` will install compatible `botocore` and `s3transfer` versions. If issues persist, try `pip install 's3transfer==0.4.0'` as a known stable combination with certain `boto3` versions, or refer to `boto3`'s dependency matrix.

Install

• pip install types-s3transfer Install

Imports

• S3Transfer

  from s3transfer.manager import S3Transfer

• TransferConfig

  from s3transfer.manager import TransferConfig

• TransferFuture

  from s3transfer.futures import TransferFuture

Quickstart

This quickstart demonstrates how to use the underlying `s3transfer` library for uploading a file to S3. With `types-s3transfer` installed in your environment, your IDE and static type checker (e.g., MyPy, Pyright) will provide accurate type hints for `S3Transfer`, `TransferConfig`, and `TransferFuture` objects, improving development experience and helping catch type-related errors before runtime.

import boto3
from s3transfer.manager import S3Transfer, TransferConfig
from s3transfer.futures import TransferFuture
import os

def upload_file_to_s3(local_filepath: str, bucket_name: str, s3_key: str) -> TransferFuture:
    # Initialize S3 client
    s3_client = boto3.client(
        's3',
        region_name=os.environ.get('AWS_REGION', 'us-east-1'),
        aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID', 'DUMMY_KEY'),
        aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY', 'DUMMY_SECRET')
    )

    # Configure transfer (optional)
    transfer_config = TransferConfig(
        multipart_threshold=8 * 1024 * 1024, # 8 MB
        max_concurrency=10
    )

    # Create S3Transfer manager
    transfer_manager = S3Transfer(s3_client, transfer_config)

    # Perform upload
    future: TransferFuture = transfer_manager.upload_file(local_filepath, bucket_name, s3_key)
    print(f"Uploading {local_filepath} to s3://{bucket_name}/{s3_key}")
    return future

if __name__ == '__main__':
    # Example usage (replace with actual values for testing)
    dummy_filepath = 'temp_file.txt'
    dummy_bucket = 'your-test-bucket'
    dummy_s3_key = 'test-upload/my_document.txt'

    # Create a dummy file
    with open(dummy_filepath, 'w') as f:
        f.write('Hello from types-s3transfer quickstart!')

    print(f"Created dummy file: {dummy_filepath}")

    try:
        # The 'types-s3transfer' package ensures that 'future' is correctly typed as TransferFuture
        future = upload_file_to_s3(dummy_filepath, dummy_bucket, dummy_s3_key)
        future.result() # Wait for the upload to complete
        print(f"Upload successful: s3://{dummy_bucket}/{dummy_s3_key}")
    except Exception as e:
        print(f"Upload failed: {e}")
    finally:
        # Clean up the dummy file
        if os.path.exists(dummy_filepath):
            os.remove(dummy_filepath)
            print(f"Removed dummy file: {dummy_filepath}")

# To benefit from type hints, ensure 'types-s3transfer' is installed in your environment.
# Type checkers like MyPy or Pyright will then provide suggestions and error checking.