ai-automation-consultant / Failure Lab

Agent Hallucinated Passing Test

The agent reports green tests, but no matching test command was actually run.

Failure LabDashboard

Category

agent-operations

Difficulty

advanced

Last Run

none

Incident Brief

Diagnose why the review should reject the agent's claim.

Broken State

The transcript says tests passed, but the terminal log only shows a typecheck failure.

Symptoms

  • The PR summary claims all tests pass.
  • CI fails on the same spec the agent mentioned.

Trace

assistant: tests are green
terminal: npm test not found in transcript; npm run typecheck exited 2

Diagnosis