Abstract

Pipeline-as-code approaches built on YAML - GitHub Actions workflows, GitLab CI definitions, Bitbucket Pipelines - give teams version control and reviewability for their CI configuration. What they do not give is local reproducibility: a pipeline that passes on GitHub may fail due to runner-specific environment differences, cached layer divergence, or API rate limits that behave differently on a developer laptop. Dagger, from the team that originally built Docker, attacks this problem by defining pipelines as real code (Go, Python, TypeScript, or any language with a generated SDK) executed inside a content-addressed container graph. The pipeline runs identically on a developer’s machine and on any CI platform, because Dagger itself abstracts the execution environment.

How Dagger’s Engine Works

A Dagger pipeline is written against the Dagger SDK, which exposes a graph-building API: you construct a DAG of container operations - from base image, run command, mount directory, export file - and the SDK serializes this into a GraphQL query sent to the Dagger Engine running as a local daemon. The engine executes the graph using BuildKit under the hood, leveraging BuildKit’s content-addressed cache so that unchanged intermediate layers are not rebuilt. Because the same GraphQL representation is sent whether the developer runs dagger run locally or a CI runner triggers the pipeline via dagger run, the execution graph is identical and cache hits are portable if a remote cache is configured. Dagger’s remote cache support works with any OCI-compatible registry, so teams can share a build cache between developer machines and CI runners by pointing both at the same registry endpoint.

Module Ecosystem and Reuse

Dagger Modules, introduced as stable in late 2024, let teams package pipeline logic as versioned, callable modules published to the Dagger Cloud registry. A module might encapsulate a multi-step Go test-and-lint pipeline, a Helm chart packaging workflow, or a container signing step using Cosign. Consuming a module in a pipeline is a single dagger install command that pins a version hash; the module is fetched and cached locally. This is the composability story that is genuinely difficult to achieve with YAML: in GitHub Actions, a reusable workflow requires a dedicated repository, a specific calling syntax, and version pinning via git SHA rather than a content-addressed module hash. Dagger modules are real code that can be imported, tested with unit tests, and debugged with standard language tooling, which lowers the maintenance burden for shared pipeline logic across many repositories.

Migration Patterns and Realistic Scope

Teams rarely migrate all pipelines to Dagger simultaneously. The practical pattern is to start with the most expensive, most frequently broken pipeline step - usually the integration test suite or the multi-arch container build - and replace it with a Dagger pipeline called from the existing YAML CI definition via a single dagger run ./pipeline.go ... invocation. This preserves the YAML skeleton that CI platforms require while moving the actual work into Dagger. Once developers can run dagger run locally and reproduce CI failures without pushing commits, the iteration speed on debugging flaky tests and environment-dependent failures typically improves enough to justify the conversion of additional pipeline steps. The Dagger CLI’s --debug flag traces the BuildKit graph execution step by step, which is more actionable during failures than scrolling CI runner logs looking for which ephemeral environment variable was missing.