Last year I watched a demo of an AI system built for clinical decision support. The vendor walked through the architecture. Ingestion layer, policy engine, model inference, output filter. Clean diagram. Good talk. Then I asked what happens when the policy engine goes down.
Pause.
“The system continues processing with reduced policy checks until the engine recovers.”
That’s fail-open. And in a system that processes patient health data, it means your compliance boundary disappears the moment something breaks.
What fail-open and fail-closed mean
The terms come from physical security. A magnetic door lock that releases when power is cut fails open. People can walk through. An electromagnetic bolt that engages when power is cut fails closed. The door stays locked. You choose one or the other based on what’s worse: unauthorized access or people trapped inside.
In software, the same question applies. When a component responsible for access control, policy enforcement, or content filtering goes down, does the system keep running without that component? Or does it stop?
Fail-open means it keeps running. The guardrail is gone but the pipeline continues. Fail-closed means the system halts. No output is produced until the safety component is restored.
Fail-Open
✓ All components operational
Fail-Closed
✓ All components operational
Interactive: click 'Simulate failure' to compare how each architecture handles a component outage.
Most AI systems I’ve looked at fail open. The policy engine times out, so the request bypasses it. The content filter is unreachable, so the response goes through unfiltered. The credential vault throws an error, so the system falls back to cached (possibly stale) credentials. Each of these is an engineering decision, and each one says: availability matters more than safety.
In a regulated environment, that tradeoff is wrong.
Why this keeps happening
Fail-open is the default in most web architectures because the web was built around availability. If a CDN node goes down, you route around it. If a cache layer fails, you hit the origin. The instinct is always to degrade gracefully and keep serving requests. Engineers trained in this tradition carry the same instinct into AI systems. When the policy engine doesn’t respond within 200ms, the natural move is to add a timeout and proceed without it.
The problem is that in a regulated AI system, the policy engine isn’t a cache. It’s the component that decides whether the system is allowed to produce a given output at all.
Skipping it doesn't degrade the experience. It removes the compliance boundary.
An AI system processing health records without policy enforcement isn’t operating in degraded mode. It’s operating in violation.
EU AI Act · Articles 9 & 14
Risk management system that functions across the AI system's lifecycle. Human oversight mechanisms for high-risk systems.
Effective: 2 August 2026
View sourceThe EU AI Act’s Article 14 requires human oversight mechanisms for high-risk systems. 1 Article 9 mandates a risk management system that functions across the AI system’s lifecycle. 2 If your oversight mechanism is a software component that gets bypassed when it fails, you’ve built a compliance architecture that evaporates under load.
What fail-closed looks like in practice
Fail-closed means that when a safety-critical component is unavailable, the system stops producing outputs. The request enters a queue. The user gets an explicit error. Nothing downstream executes until the component recovers.
Three things make this work in production.
Circuit breakers. The circuit breaker pattern (borrowed from electrical engineering, formalized by Michael Nygard in Release It!) 3 monitors calls to a dependency. When failures exceed a threshold, the breaker trips open and stops sending requests. In a fail-closed AI system, the circuit breaker on the policy engine doesn’t reroute traffic around the engine. It halts the pipeline.
The system returns an explicit failure to the caller: “Policy enforcement unavailable. Request not processed.” The breaker periodically lets a test request through to check recovery. When the engine is back, the breaker closes and normal operation resumes.
Invariant testing in CI. A fail-closed architecture is only as reliable as the code that enforces it. You need automated tests that verify the halt behaviour. At Hypereum, we run invariant tests in CI that simulate component failures and assert that the system refuses to produce output. If someone writes a code path that bypasses the policy engine on timeout, the test catches it before it merges. We have 71 security invariants in our CI pipeline. Several exist specifically to verify fail-closed behaviour: policy engine unreachable, credential vault timeout, audit trail write failure. Each one asserts that the system returns an error, not a result.
Graceful halt with explicit signalling. Fail-closed doesn’t mean crash. It means the system communicates clearly that it has stopped and why. The caller gets a structured error response: which component failed, when, and what the recovery path looks like. Logs record the halt event with the same fidelity as any other auditable action. The system doesn’t silently drop requests or return ambiguous errors.
”But what about uptime?”
The pushback I hear most often is: “If the system stops every time a component hiccups, we’ll have unacceptable downtime.” This is true if your components are unreliable. The answer is to make them reliable, not to remove the safety constraint.
Redundancy, health checks, automatic failover, load balancing across replicas. These are solved problems. The policy engine should be highly available. If it’s a single point of failure with no redundancy, that’s an infrastructure problem, and you fix it with infrastructure.
That's trading a visible problem for an invisible one. The invisible problem is worse.
What you don’t do is compensate for unreliable infrastructure by letting the system run without its safety components. Downtime is recoverable. A compliance violation that produces a record of uncontrolled outputs is not.
Where we landed
Hivemind, our multi-agent orchestration engine, is fail-closed by default. If the policy engine is unreachable, the mission halts. If the audit trail can’t be written, the mission halts. If credential decryption fails, the mission halts. There is no fallback mode that produces output without full safety enforcement.
This was an unpopular decision early on because it made the system feel fragile during development. Every misconfigured test environment, every flaky network connection, every cold-start delay in the policy engine caused a halt. We spent time we could have spent on features making infrastructure reliable instead.
That was the right tradeoff. When I sit down with a compliance officer at a bank or a hospital and they ask “what happens when something breaks?”, the answer is clean: the system stops. It tells you what broke. It waits until the problem is fixed. There is no mode in which Hivemind operates without its safety guarantees.
I'd rather explain downtime than explain why an AI system produced uncontrolled output with patient data.
