Governance and risk

What happens when AI fails

Every AI system will fail. The question is not whether, but what happens when it does. A board without an off switch is a board without control.

By Zakir Hoosen, Director, Fuzzelogic Solutions. Board-level guidance in plain English.

AI will fail. Not might, will. The only question is how often, how badly, and whether anyone can stop it quickly enough. Most boards approve AI projects without asking one simple question: what happens when this gets it wrong? The answer matters more than the feature list.

This guide explains what failure looks like, why it happens more often than vendors admit, and the off switch every board should insist on before approving anything.

What failure actually looks like

Failure is not always dramatic. It is not always a system crash or a headline. Most AI failure is quiet. A chatbot gives a customer the wrong answer. A recommendation engine pushes a product that does not exist. An automated process sends an email to the wrong person. Nobody notices for weeks.

Then there is the kind of failure that is not quiet. A system makes a decision about a customer that violates a regulation. A tool exposes data that should never have been accessible. An automated process makes a financial decision that costs real money. Those are the ones that end up in board papers.

Four in ten projects cancelled. That is not a failure rate. That is a pattern. The projects are not failing because the technology does not work. They are failing because nobody planned for what happens when it does not.

Why AI fails more than other technology

Traditional software fails in predictable ways. A line of code either runs or it does not. A database either stores the data or it loses it. AI is different. It does not fail in the code. It fails in the output. The system runs perfectly, does exactly what it was trained to do, and gives you the wrong answer with complete confidence.

This is the core problem. AI does not throw errors when it gets something wrong. It produces an output that looks right. A human has to check it, and humans get tired, bored, or trusting. The more reliable the system seems, the less likely anyone checks. That is when mistakes compound.

There is also the problem of gradual failure. The system works well at launch. Months later, the data changes. The world moves. The AI does not. It keeps producing outputs based on a version of reality that no longer exists. Nobody notices because the system is still running. Nobody checks because it was fine last time they looked.

The off switch question

Every AI system needs an off switch. Not a theoretical one. A practical, tested, immediate one. The question a board should ask before approving any AI project is simple: who can turn this off, how fast, and what happens to the business when they do?

The Anthropic guide makes a point that boards should read carefully. The most effective control is not a better algorithm. It is a human saying yes or no before something consequential happens. The off switch is not a last resort. It is a first line of defence.

If a system cannot be turned off without breaking something else, the system was designed wrong. If nobody can name who has the authority to stop it, the governance was designed wrong. If the off switch has never been tested, it is not an off switch. It is a hope.

How Google handles it internally

It is worth looking at how a major technology company governs its own AI use.

Google does not trust its own AI to make significant changes without human review. If the company that builds the technology still insists on a human checking the output, a board should ask why its own governance would be any less careful.

This is not about being slow. It is about being sure. Google's approach is a reminder that the companies building these systems do not treat them as infallible. They build checks precisely because they know failure is part of the design.

The five things to test before you approve

When an AI project comes to the board, five questions separate a prepared team from an optimistic one.

1. What is the failure mode? Not can it fail, but how does it fail? What does a wrong answer look like?
2. Who checks the output? Not the team that built it. Someone independent, with the authority to stop it.
3. What is the off switch? Who has it, where is it, and how fast does it work?
4. What happens to the business if it stops? If the answer is nothing, it is probably not important enough to automate. If the answer is significant, the off switch matters more.
5. When was the off switch last tested? A switch that has never been pulled is a switch you do not know works.

These five questions should be part of every AI project approval. If a team cannot answer them, the project is not ready for a board vote. It is ready for more work.

What the research tells us about cost

The Gartner cancellation figure is not just about governance. It is about money.

AI projects cost more than people expect. Not because the technology is expensive, though it often is. Because the hidden costs pile up. Data preparation takes longer than planned. Integration with existing systems is harder than the demo suggested. The team needs skills that do not exist in-house. The monitoring costs are ongoing and nobody budgeted for them.

When a project is cancelled, the money spent is gone. But the disruption is worse. Teams built processes around the system. Customers got used to a feature. The business reorganised to accommodate it. Removing it is more expensive than building it was.

This is why the off switch matters. Not just as a safety measure. As a financial one. If you can turn it off cleanly, you save the second cost. If you cannot, you are locked in.

The honest version

Fuzzelogic works with boards across banking, insurance, healthcare, retail, manufacturing, and government. In nineteen years, we have never seen a system that could not fail. We have seen plenty that nobody planned to fail.

Your systems were built for a world before AI. Most can get there. We tell you which ones cannot.

Start with the assessment. Two to four weeks, fixed price, and you own the verdict and the roadmap whether or not we build any of it. When you are ready to talk AI, call Fuzzelogic Solutions and ask for Zak. www.FuzzelogicSolutions.com | info@FuzzelogicSolutions.com | +44 (0)1624 618950

For context, read AI governance for boards first, then shadow AI. The full library is on our index. Our site explains how Fuzzelogic approaches AI for business. You can reach Zak directly via our contact page.