CI monitor reports job conclusions inherited from the enclosing workflow run, marking a passing required check as failed

Status Open
Maintainer reply None cached
Activity 0 comments · opened Aug 11, 2026

Invariant

A monitor must report job conclusions from job conclusions, not inherit the enclosing workflow-run conclusion.

When a workflow contains a required gating job and a deliberately non-blocking advisory job, the run-level conclusion is failure whenever the advisory job fails — even though the required job passed. A monitor that derives per-job status from the run-level conclusion then reports the required check as failed, which is materially false.

Why it matters

This is not cosmetic. The reported state drives the operational response:

  • The required job is the merge gate. Reporting it as failed says the branch is unmergeable when it is mergeable.
  • The advisory job is failing by design in this configuration. Telling the user "the required check failed" points remediation at the wrong job, and the natural next step someone takes is to weaken or suppress the advisory check to make the notification go quiet — defeating the reason the two jobs were split apart.

A gate split into "what must pass" and "what must report the truth" is a common CI arrangement. It only works if the reporting layer preserves the distinction.

Synthetic fixture A — mixed conclusions

Workflow conclusion: failure

Jobs:
  - name: required-check
    conclusion: success
    steps: 12

  - name: advisory-validation
    conclusion: failure
    steps: 7
    application verdict: NON_COMPLIANT

Expected: required-check passed; advisory-validation failed.
Observed: required-check failed — the run-level conclusion applied to the job.

Synthetic fixture B — zero execution

Workflow conclusion: failure

Jobs:
  - name: required-check
    conclusion: failure
    steps: 0
    annotation: <infrastructure block; job never started>

Expected: required-check NOT EXECUTED / BLOCKED, with no application verdict reported.
Must not: present this as a check result, or conflate it with fixture A's failure.

A job with an empty steps array carries no information about the code. The reason typically appears in the check-run annotation rather than the log, so log-based reporting shows nothing and the failure looks contentless rather than infrastructural.

Three axes the monitor should keep separate

| Axis | Values |
| --- | --- |
| execution state | executed / not executed |
| job conclusion | success / failure |
| application verdict | whatever the job's own tool reported, or none |

Fixture A and fixture B both have workflow conclusion failure and both contain a job with conclusion failure, and they mean entirely different things. Collapsing any two of these axes produces a wrong report for one of the two fixtures.

Provenance, stated honestly

The defect was observed on a private repository with this gate arrangement: the workflow was red, the required job succeeded with real execution, and two advisory validation jobs failed with real execution; the monitor named the required job among the failed checks.

The fixtures above were constructed from the invariant, not derived from that run, so they contain no identifiers from it. I have not executed the monitor against them — I have no way to feed it synthetic input — so they are offered as a specification of the expected behaviour and as the shape a regression test should take, not as a reproduction I have run. Treating them as verified would overstate what I checked.

Happy to supply more detail through a private channel if the synthetic description is not sufficient to locate the code path.

View original on GitHub ↗