ai-automation-consultant / Failure Lab
No Human Review Boundary
AI output is written directly to customer-facing messages without review or confidence checks.
Category
architecture
Difficulty
advanced
Last Run
none
Incident Brief
Diagnose the missing architecture boundary.
Broken State
The document workflow sends AI-generated permit summaries directly to customers even when confidence is low.
Symptoms
- A customer receives a summary with a made-up inspection date.
- Low confidence records are not routed to the review queue.
Trace
confidence=0.42 route=send_to_customer review_queue_skipped reason=missing_threshold