Pipeline failures used to wake a human at 3am to read three dashboards. Dock for data analytics moves the reading and the first-pass write-up to an agent. The agent reads Airflow DAG state, dbt run results, and Datadog alerts, drafts an incident summary with a proposed remediation, and posts it to a Dock surface where the on-call data engineer approves or edits before anything reruns. The pager still fires. The first ninety minutes of triage do not.
Airflow + dbt + Datadog stay the system of record for the raw data. Dock is the system of record for what the AGENT INTERPRETS. Each Dock row carries a pointer back to the platform record, agent identity, decision, reviewer, and timestamp. The agent re-fetches platform data via fresh API reads when it needs current state.
The Dock surface: Pipeline Incidents
| incident_id | dag / model | airflow_run | dd_alert | agent_summary | proposed_fix | severity | reviewer | status |
|---|---|---|---|---|---|---|---|---|
| INC-4411 | revenue_daily / fct_orders |
run_2026_05_30_02 | dd-7714 | Source orders_raw row count dropped 41% vs 7d median. Upstream Fivetran sync failed at 01:47. |
Re-trigger Fivetran connector, then dbt run --select fct_orders+ |
P2 | maya.k | approved |
| INC-4412 | attribution_hourly |
run_2026_05_30_03 | dd-7720 | dbt test not_null_session_id failed on 0.3% of rows after schema change in events_v3. |
Hold downstream models. Open schema-change ticket with eventing. | P3 | rohan.s | approved |
| INC-4413 | ml_features_nightly / customer_embeddings |
run_2026_05_30_02 | dd-7731 | Task embed_batch_4 OOMed twice. Memory ceiling reached after vendor list grew 3x. |
Bump worker class to mem-xlarge for this task only. |
P2 | maya.k | edited |
Each row links back to the Airflow run URL, the Datadog alert ID, and the dbt artifact hash. None of that data lives in Dock. The interpretation does.
The workflow
- Datadog fires a monitor on a failed Airflow task or a dbt test failure.
- The on-call agent pulls the DAG run, the dbt
run_results.json, the failing test row sample, and the last seven days of run history for that task. - The agent drafts a Pipeline Incidents row: what broke, what changed, what the proposed fix is, and which downstream consumers are affected.
- The on-call data engineer opens the row in Dock, edits the proposal if needed, and clicks approve.
- Dock writes the remediation event with reviewer identity and timestamp. The agent re-fetches state from Airflow and executes the approved action.
- The closed row stays as the canonical record of who decided what, with platform IDs as pointers.
The agent never reruns a DAG without a human approval on the row. That is the whole point of agent identity being a first-class field.
Why it matters
Without Dock, the incident write-up lives in a Slack thread that nobody can audit six weeks later. With Dock, the agent-drafted summary, the reviewer, and the remediation are one queryable table. This is the same pattern we describe in Dock for DevOps and the broader Cloud 2.0 engineering thesis: platforms hold the artifacts, Dock holds the agent-readable narrative.
Attribution survives staff turnover. When the agent identity rotates per the identity lifecycle policy, historical rows still resolve. Auditors get the trail they need without screen-sharing through three SaaS consoles, which is the workflow covered in agent audit and compliance.
Monte Carlo's framing of data downtime as periods when data is incomplete or inaccurate (Monte Carlo) maps cleanly onto the Pipeline Incidents surface: every row is a downtime event with an owner. Datadog's data streams monitoring product locates the failing producer or consumer (Datadog); the agent reads that signal and converts it into a decision. Airflow's DAG run model with its logical date and task states (Airflow) gives the agent a stable identifier to point back to.
See the full pillar at Dock for data analytics.
FAQ
Q: Does the agent rerun pipelines on its own? A: No. The agent drafts a remediation and waits for a reviewer approval on the Dock row. Execution happens after approval, against fresh state pulled from Airflow.
Q: What if Datadog and Airflow disagree about whether a task failed? A: The agent records both signals on the row and flags the discrepancy in the summary. The reviewer decides which source is canonical for that incident.
Q: How is this different from PagerDuty plus a runbook? A: PagerDuty routes the alert. A runbook tells a human what to check. Dock holds the agent-drafted interpretation, the proposed action, and the approval, all with attribution.
Q: Where does agent identity come from? A: Each agent has a Dock-issued identity tied to scoped credentials. Rotation, revocation, and audit follow the lifecycle described under agent identity.
