PyDAP

3.5.9 · active · verified Thu Apr 16

PyDAP is a pure Python implementation of the Data Access Protocol (DAP, also known as OPeNDAP), enabling access to scientific data over the internet. It acts as both a client for accessing remote OPeNDAP servers and a server for making local data available via OPeNDAP. A key feature is lazy data evaluation, where data is downloaded on-the-fly as needed, conserving bandwidth and time. The library is actively developed and maintained by the OPeNDAP community.

Common errors

```
AttributeError: '<class 'pydap.model.DatasetType'>' object has no attribute 'iteritems'
```
cause This error typically occurred with older versions of the `xarray` library when attempting to use `pydap` as an engine, due to `pydap.model.DatasetType` changing its internal iteration methods (e.g., from `iteritems` to `items`) in later PyDAP versions.

fix Upgrade your `xarray` library to a version compatible with `pydap` 3.x. Ensure both libraries are kept up-to-date to maintain compatibility. This issue is largely resolved in recent `xarray` releases.
```
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x__ in position __: invalid continuation byte
```
cause Attempting to open a URL containing characters that are not correctly encoded for the system's default locale or the URL's intended encoding, leading to a decoding failure.

fix Explicitly encode problematic URL components using `urllib.parse.quote` or verify the encoding of the URL source. Ensure your Python environment is configured for UTF-8 where possible, e.g., via `locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')` if applicable.
```
Request too big=XXXX.X Mbytes, max=YY.Y Mbytes (403 Forbidden)
```
cause The OPeNDAP server misinterpreted a subset request as a request for the entire dataset, exceeding its configured maximum download size. This can be due to how pydap constructs the OPeNDAP URL constraint expression for certain data types or server implementations.

fix Double-check that the subsetting (slicing) in your `pydap` code is correct. If the issue persists even with very small subsets, the problem lies with the specific OPeNDAP server or `pydap`'s interaction with it. Consider fetching data in even smaller chunks, if possible, or consulting the server's documentation.

Warnings

gotcha When accessing certain OPeNDAP servers, requesting even a small data subset can result in a 'Request too big' error. This often indicates a server-side parsing issue where the request is misinterpreted as a full array download, or a quirk in pydap's request building logic for specific grid maps.
Fix: If encountered, try reducing the requested subset further. If the issue persists with small requests, it might be a server-specific limitation or a fundamental difference in how pydap constructs requests compared to other clients. Consider reporting to the pydap issue tracker.
gotcha Some URLs, particularly those with complex query parameters or non-ASCII characters, may lead to a `UnicodeDecodeError` when used with `pydap.client.open_url`. This indicates an encoding problem during URL parsing or data retrieval.
Fix: Ensure that any problematic parts of the URL are properly URL-encoded (e.g., using `urllib.parse.quote`) before passing them to `open_url`. Also, verify the encoding of your environment if possible.
deprecated Older documentation, especially on third-party sites or unmaintained GitHub Pages, may contain outdated examples or instructions. Pydap has undergone updates, including dropping Python 2.7 support (and thus the `six` dependency).
Fix: Always refer to the official PyDAP documentation linked from its main GitHub repository or PyPI page for the most up-to-date and accurate information. Update to Python 3.10+ for full compatibility.

Install

pip install pydap Client Installation
pip install "pydap[server]" Server Installation (includes core)
conda create -n pydap -c conda-forge python=3.11 pydap numpy jupyterlab ipython netCDF4 scipy matplotlib && conda activate pydap Conda Installation (client with common extras)

Imports

open_url
```
from pydap.client import open_url
```
The primary function for accessing remote OPeNDAP datasets as a client.
DAPHandler
```
from pydap.handlers.dap import DAPHandler
```
Used for low-level interaction with DAP responses, typically for server implementations or advanced client use.
app
```
from pydap.wsgi import app
```
The WSGI application for running pydap as a server. Installed when using `pydap[server]`.

Quickstart

This quickstart demonstrates how to use PyDAP as a client to open a remote OPeNDAP dataset, inspect its variables, and retrieve a subset of data using lazy evaluation. Data is only downloaded when explicitly accessed or sliced.

from pydap.client import open_url

# Example OPeNDAP URL for a publicly available dataset
dataset_url = 'http://test.opendap.org/dap/data/nc/coads_climatology.nc'

try:
    # Open the remote dataset
    dataset = open_url(dataset_url)
    print(f"Dataset keys: {list(dataset.keys())}")

    # Access a variable lazily (no data downloaded yet)
    sst_variable = dataset['SST']
    print(f"SST variable shape: {sst_variable.shape}")
    print(f"SST variable data type: {sst_variable.dtype}")

    # Retrieve a small subset of data (data downloaded on-the-fly)
    # Example: first time step, subset of latitude/longitude
    data_subset = sst_variable[0, 10:14, 10:14]
    print("\nSubset of SST data (downloaded):")
    print(data_subset.data)

except Exception as e:
    print(f"An error occurred: {e}")

view raw JSON →