{"id":3444,"library":"dagster-docker","title":"Dagster Docker","description":"Dagster-docker is a Dagster integration that enables launching and executing Dagster runs or individual ops/assets within Docker containers. As part of the broader Dagster ecosystem, it follows a separate versioning scheme (currently 0.29.0) but is released in lockstep with the core `dagster` library, which sees frequent updates (typically minor releases weekly or bi-weekly, with major changes less often).","status":"active","version":"0.29.0","language":"en","source_language":"en","source_url":"https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-docker","tags":["dagster","docker","orchestration","etl","containerization"],"install":[{"cmd":"pip install dagster-docker","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core Dagster library, required for all Dagster functionality.","package":"dagster","optional":false}],"imports":[{"note":"Primarily configured in `dagster.yaml` for a Dagster deployment to launch runs in Docker.","symbol":"DockerRunLauncher","correct":"from dagster_docker import DockerRunLauncher"},{"note":"A pre-configured executor definition used when defining Dagster jobs programmatically to run ops/assets in Docker.","symbol":"docker_executor","correct":"from dagster_docker import docker_executor"}],"quickstart":{"code":"from dagster import Definitions, asset, define_asset_job\nfrom dagster_docker import docker_executor\nimport os\n\n# Define a simple asset\n@asset\ndef my_docker_asset():\n    \"\"\"An asset whose execution will be managed by a Docker executor.\"\"\"\n    message = os.environ.get(\"GREETING\", \"Hello from default asset!\")\n    print(message)\n    return message\n\n# Define a job that uses the docker_executor.\n# The configuration demonstrates how to specify Docker image and environment variables\n# for the container in which the asset will run.\nmy_docker_job = define_asset_job(\n    name=\"my_docker_job\",\n    selection=[my_docker_asset],\n    executor_def=docker_executor.configured(\n        {\n            \"container_kwargs\": {\n                # This image needs to have Python and your Dagster code installed.\n                # For a minimal example, we use a basic Python image. In production,\n                # you'd build an image with your specific Dagster code and dependencies.\n                \"image\": os.environ.get(\"DOCKER_IMAGE\", \"python:3.10-slim-buster\"),\n                \"environment\": {\n                    \"GREETING\": \"Hello from inside the Docker container!\"\n                },\n                # Optional: specify a Docker network if your Dagster instance\n                # is also running in Docker and needs to communicate.\n                \"network_mode\": \"bridge\"\n            },\n            # If you want to pass host environment variables directly to the container,\n            # list them here. The values will be taken from the environment where Dagster runs.\n            \"env_vars\": [\"HOST_VAR\"]\n        }\n    ),\n)\n\n# To make these definitions loadable by Dagster (e.g., `dagster dev -f my_file.py`)\n# you would typically expose them via a Definitions object:\n# defs = Definitions(\n#     jobs=[my_docker_job],\n# )","lang":"python","description":"This quickstart demonstrates how to define a Dagster job that utilizes `dagster-docker`'s `docker_executor`. When this job is launched within a Dagster deployment, the `my_docker_asset` will execute inside a Docker container configured as specified. For this to run successfully, a Docker daemon must be accessible, and the specified Docker image should contain Python and any necessary dependencies for your asset code. Remember to replace `python:3.10-slim-buster` with an image containing your Dagster code for real-world scenarios."},"warnings":[{"fix":"Ensure Docker is installed and running, and the Dagster user has permissions to interact with the Docker socket (e.g., `sudo usermod -aG docker $USER` on Linux systems).","message":"The machine running the Dagster daemon/webserver (for `DockerRunLauncher`) or the machine launching jobs (for `docker_executor`) must have access to a running Docker daemon.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Carefully configure the `network` or `network_mode` parameters in `container_kwargs`, ensuring that the container can reach the Dagster API server's host and port. Often, using a custom Docker network or host networking mode (if applicable) is required for seamless communication.","message":"Docker containers launched by `dagster-docker` need to be able to communicate back to the Dagster API server (e.g., Dagit, Dagster daemon). Incorrect network configuration can lead to runs hanging or failing to report status.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always install `dagster-docker` alongside the corresponding version of `dagster` you are using. For `dagster-docker==0.29.0`, this means installing `dagster==1.13.0` (or a compatible `1.x.x` version). For example: `pip install dagster==1.13.0 dagster-docker==0.29.0`.","message":"`dagster-docker` versions are tightly coupled with the `dagster` core library. Using `dagster-docker` with a significantly mismatched `dagster` core version (e.g., `dagster-docker==0.29.0` with `dagster==1.0.0`) can lead to runtime errors or unexpected behavior due to API changes.","severity":"breaking","affected_versions":"All versions when used with mismatched `dagster` core versions."}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}