ECMWF Open Data Client

0.3.26 · active · verified Thu Apr 16

ecmwf-opendata is a Python package designed to simplify the download of ECMWF open data. It provides a request-based interface similar to ECMWF's MARS language for selecting meteorological fields. The library is actively maintained (current version 0.3.26) and receives frequent updates, with new releases typically addressing data source changes, bug fixes, and supporting new data streams. It allows users to retrieve data from ECMWF's direct servers as well as cloud replicas on AWS, Azure, and Google Cloud Platform.

Common errors

```
HTTPError: 404 Client Error: Not Found for url: ...
```
cause The requested URL or product name is incorrect, or the combination of parameters does not correspond to available data.

fix Verify all `retrieve` parameters (e.g., `date`, `time`, `step`, `stream`, `type`, `param`) against the official ECMWF Open Data catalogue and documentation. Small typos or incorrect parameter values are common causes.
```
HTTPError: 400 Bad Request / HTTPError: 500 Internal Server Error
```
cause Your request parameters were syntactically incorrect or resulted in an unprocessable request on the server side.

fix Carefully review the `retrieve` arguments, especially complex ones like `number` for ensemble members or `levelist`. Ensure correct data types and formats. Consult the ECMWF Open Data documentation for specific parameter requirements.
```
Download is extremely slow or hangs, or I'm stuck in a long queue.
```
cause High demand on the ECMWF Open Data Portal or exceeding its connection limits. This is a common issue, particularly during peak hours.

fix Switch the data `source` to one of the cloud providers: `client = Client(source="aws")`, `client = Client(source="azure")`, or `client = Client(source="google")`. These replicas offer better reliability and performance during high demand.

Warnings

gotcha The ECMWF open-data portal imposes a limit of 500 simultaneous connections, which can lead to 'Too many requests' errors, especially during peak demand. Using cloud-based sources (AWS, Azure, Google Cloud) is often more reliable.
Fix: Initialize the client with a specific cloud source: `client = Client(source='aws')`, `client = Client(source='azure')`, or `client = Client(source='google')`.
gotcha This package is optimized for downloading subsets of data. If you intend to download a large percentage or entire data files, it may be more efficient to download whole files directly and filter the data locally, rather than using the request-based interface.
Fix: For bulk downloads, consider alternative methods or adjust your workflow to download broader files and process them with tools like `earthkit-data` locally.
breaking The GRIB2 encoding for ECMWF Open Data products changed to use CCSDS compression starting July 2023. While `ecmwf-opendata` is designed to handle this, older tools or custom processing pipelines that expect pre-July 2023 GRIB2 formats might encounter issues.
Fix: Ensure your data processing tools and libraries are updated to handle GRIB2 files with CCSDS compression. Verify compatibility if using custom decoders or older versions of data analysis software.
gotcha All ECMWF Open Data is governed by the Creative Commons Attribution 4.0 International license (CC-BY-4.0) and the ECMWF Terms of Use. This requires appropriate attribution when using or redistributing the data.
Fix: Always include appropriate attribution to ECMWF when using or disseminating data obtained through this library.

Install

pip install ecmwf-opendata Install latest version

Imports

Client
```
from ecmwf.opendata import Client
```

Quickstart

This quickstart demonstrates how to instantiate the Client and use its `retrieve` method to download meteorological data. It highlights the importance of specifying a data `source` for optimal performance and reliability, especially given potential connection limits to the ECMWF portal. The parameters for `retrieve` follow the MARS request language to define the desired dataset.

from ecmwf.opendata import Client

# Initialize the client, specifying a source for reliability (e.g., 'aws', 'azure', 'google' or 'ecmwf')
# Using a cloud source is often recommended due to connection limits on ECMWF's direct portal.
client = Client(source="aws")

# Example: Download the latest available 10-day forecast for mean sea level pressure (msl)
# into a local file named 'data.grib2'.
# Parameters like 'step', 'type', 'param' are based on MARS language.
try:
    client.retrieve(
        step=240,       # Forecast step in hours (e.g., 240 for 10 days)
        type="fc",      # Type of data (e.g., 'fc' for forecast)
        param="msl",    # Parameter to retrieve (e.g., 'msl' for mean sea level pressure)
        target="data.grib2", # Output file name
    )
    print("Data downloaded successfully to data.grib2")
except Exception as e:
    print(f"An error occurred during data retrieval: {e}")

view raw JSON →