CI Failure Watcher
watch it green.
For the red-X morning: the pipeline failed overnight and you want a calm, ordered watch — newest failure first, smallest fix that turns it green, repeat until the whole pipeline passes.
Typical run: 15–60 min typical · Copy any prompt box and hand it to your agent as-is.
Loops / CI Failure Watcher
When to use
- CI is red on main or on a release branch and merges are blocked.
- Failures look spread across jobs (tests, lint, build) and you need an order to work through them.
- You want the agent to watch and triage without force-pushing over other people’s commits.
- Re-running a single job is cheap, so polling every few minutes is practical.
The steps
Fetch the latest pipeline result for the branch. List each failed job with its name, failed step, and the last 20 lines of its log. Do not fix anything yet — stop after the list.
Label each failure: flake-or-infra (timeout, network, runner), test-failure (assertion), or config (lint, types, missing secret). Report the labels with one quoted log line of evidence each, then stop.
Retry ONLY the jobs labeled flake-or-infra, one time, with no code changes. Report which jobs turned green and which stayed red, then stop.
Fix ONLY the first remaining real failure. Quote the failing assertion or error, change the minimum code or config to address it, and open no new jobs. Explain the fix in two sentences, then stop without pushing.
Push the single fix, then poll the pipeline until that run finishes. Report per-job status (green/red) and paste the new failure log if anything is still red. Do not stack another fix until the run completes.
Wait for a full green run on the branch. Write a 5-line note: which jobs failed, what was flake vs real, what the fix changed, commit hash, and pipeline URL. Keep it under 120 words, then stop.
Stop conditions
- The full pipeline is green on a single run with no retries pending.
- Every failure from the first read is accounted for as flake (retried green) or fixed (commit linked).
- The watch note with commit hash and pipeline URL is recorded.
Failure modes
Stacking fixes on a moving pipeline
Symptom: three pushes race each other and nobody knows which fixed what. Guardrail: one fix per pipeline cycle — wait for the run to finish before the next push.
Force-pushing over teammates
Symptom: someone else’s commit vanishes mid-watch. Guardrail: never force-push on shared branches; rebase locally and push normally, or hand the loop to a human.
Retrying real failures into green
Symptom: a flaky test passes on retry and the bug ships. Guardrail: a test that fails then passes on retry gets a flake issue filed — retrying twice without a fix is the limit.