{"library":"maggma","title":"Maggma Data Pipeline Framework","description":"Maggma is a framework to build scientific data processing pipelines, handling data from diverse sources like databases, Azure Blobs, and local files, up to REST APIs. It provides core abstractions, `Store` and `Builder`, for modular ETL-like operations. The `Store` interface often mimics PyMongo syntax, enabling consistent data access across different backends. Actively developed by the Materials Project, it is currently at version 0.72.1 and requires Python 3.9+.","language":"python","status":"active","last_verified":"Sat May 16","install":{"commands":["pip install maggma"],"cli":{"name":"maggma","version":"sh: 1: maggma: not found"}},"imports":["from maggma.stores import MemoryStore","from maggma.stores import MongoStore","from maggma.builders import Builder","from maggma.core import Store"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import os\nfrom maggma.stores import MemoryStore\n\n# Sample data\nturtles = [\n    {\"name\": \"Leonardo\", \"color\": \"blue\", \"tool\": \"sword\"},\n    {\"name\": \"Donatello\", \"color\": \"purple\", \"tool\": \"staff\"},\n    {\"name\": \"Michelangelo\", \"color\": \"orange\", \"tool\": \"nunchuks\"},\n    {\"name\": \"Raphael\", \"color\": \"red\", \"tool\": \"sai\"}\n]\n\n# Create a MemoryStore (in-memory, data not persistent)\n# 'key' argument specifies the unique identifier for documents\nstore = MemoryStore(key=\"name\")\n\n# Connect to the store (for MemoryStore, this just initializes it)\nstore.connect()\n\n# Add data to the store using update\n# upsert=True means insert if not found, update if found\nstore.update(turtles, key_field='name', upsert=True)\n\n# Query the store\nprint(f\"Total documents: {store.count()}\")\nprint(f\"Blue turtle: {store.query(criteria={'color': 'blue'}).current()}\")\n\n# Find distinct values\nprint(f\"Distinct colors: {list(store.distinct(field='color'))}\")\n\n# Close the store connection (important for persistent stores)\nstore.close()\n\n# Example of using a persistent store (e.g., MongoStore)\n# Requires a MongoDB instance running and pymongo installed.\n# uri = os.environ.get('MONGO_URI', 'mongodb://localhost:27017/test_db')\n# from maggma.stores import MongoStore\n# mongo_store = MongoStore(collection_name='my_collection', database_name='test_db', host=uri, key='name')\n# try:\n#     mongo_store.connect()\n#     mongo_store.update(turtles, key_field='name', upsert=True)\n#     print(f\"MongoStore count: {mongo_store.count()}\")\n# finally:\n#     mongo_store.close()\n","lang":"python","description":"This quickstart demonstrates the core concepts of Maggma: defining data as a list of dictionaries, creating a `Store` (using `MemoryStore` for simplicity), connecting to it, adding data using the `update` method, and querying data. It highlights the use of a `key` field for unique document identification. A commented-out example for `MongoStore` is included to illustrate persistent storage.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-16","installed_version":"0.72.1","pypi_latest":"0.72.1","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":17.1,"avg_import_s":2.14,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"maggma","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.88,"mem_mb":37.6,"disk_size":"255.8M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"maggma","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":18.4,"import_time_s":1.43,"mem_mb":37.5,"disk_size":"246M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"maggma","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":2.5,"mem_mb":43,"disk_size":"277.3M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"maggma","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":16.7,"import_time_s":2.35,"mem_mb":43,"disk_size":"267M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"maggma","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":2.5,"mem_mb":42.1,"disk_size":"261.4M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"maggma","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":14.6,"import_time_s":2.56,"mem_mb":42.1,"disk_size":"252M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"maggma","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":2.4,"mem_mb":43.6,"disk_size":"260.3M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"maggma","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":14.4,"import_time_s":2.42,"mem_mb":43.6,"disk_size":"252M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"maggma","exit_code":0,"wheel_type":"sdist","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.74,"mem_mb":37.8,"disk_size":"264.0M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"maggma","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":21.6,"import_time_s":1.66,"mem_mb":37.8,"disk_size":"256M"}]}}