Technically Correct, Operationally Wrong

AI is getting better at producing answers. That doesn’t mean it’s getting better at knowing which answers matter.

It can summarize a thousand pages, write software, draft a campaign, analyze a dataset, propose an architecture or automate a workflow in a fraction of the time it once took. That capability is real. But in most businesses, producing an answer was never the whole job.

Someone still has to understand the problem and know what good looks like. Someone has to notice when an output is technically correct and operationally wrong. And someone has to be accountable for the decision.

So we don’t find “human versus machine” a useful frame. The more useful question is where each one is strongest.

AI is very good at accelerating work

In almost every knowledge workflow, there are parts where AI removes real friction. It can synthesize large amounts of information, spot patterns, generate alternatives and first drafts, translate between formats and systems, and carry out bounded tasks that used to take hours of manual effort.

We see it in software, where the path from requirement to tested code is getting much shorter. In marketing, where research, content and campaign iteration move at a pace that would have been hard to imagine a few years ago. And in operations, where information buried across systems, documents and people’s memories becomes something a team can act on.

But speed isn’t the goal. The question is whether the work is getting better.

The harder part of AI decision-making is judgment

Most important business decisions carry context that doesn’t reduce neatly to a prompt.

Should this process be automated at all?Is the customer problem framed correctly?Will this architecture fit the organization that has to maintain it?Is the recommendation sound, but impossible to implement?Is the model optimizing the metric that matters, or the one that’s easiest to measure?Should we move quickly here, or deliberately?

Those are judgment questions. They depend on experience, trade-offs and an understanding of consequences. AI can help illuminate them. It shouldn’t quietly answer them on the organization’s behalf.

Human-in-the-loop is not enough

“Human in the loop” has become a standard phrase, and we think it undersells the role. A person placed at the end of an automated process to approve whatever it produced is closer to a rubber stamp than to oversight.

The human role should start much earlier, with questions like:

• What problem are we actually solving?
• What context does the system need?
• Which decisions can AI make safely, and which need to stay with a person?
• How will we know the work is improving?

In that model, people aren’t a checkpoint at the end. They shape the system from the start.

AI should amplify expertise, not flatten it

One of the better effects of AI is that sophisticated capability reaches more people. A product manager can explore technical approaches. A developer can get up to speed on an unfamiliar business domain. A marketer can work more deeply with data. An operations team can interrogate its own information without waiting weeks for a report.

But wider access to expertise isn’t the same as no longer needing it. In our experience, AI usually makes experienced people more valuable. A strong architect evaluates generated designs faster because they already know the trade-offs. A good marketer can test a dozen ideas without losing sight of the customer. Someone who knows the operation can tell an elegant workflow from one that will fall apart on the floor.

AI increases leverage. Experience determines how well that leverage is used.

Context is where people add the most

AI does its best work with good context. Deciding which context matters is a human problem.

Take a request as simple as “improve this workflow.” Doing it well can mean understanding the customer, the operating model, the systems involved, the regulatory constraints, decisions made years ago, the incentives at play, the exceptions nobody wrote down, who actually does the work, and what happens when something breaks.

None of that appears just because a model is capable. Someone has to know the environment well enough to supply the right context, and to notice what’s missing. That’s why AI implementation is usually less about the model than organizations expect. The hard work sits around it.

Where AKS draws the line on AI governance

1. Start with the work

We don’t begin by asking where AI could go. We begin by asking what the organization is trying to accomplish, then look at where AI would meaningfully improve how that work gets done. Sometimes the answer involves AI. Sometimes it doesn’t.

2. Keep people where judgment matters

AI can take on repetitive analysis, information retrieval, generation and bounded execution. Decisions involving significant trade-offs, risk, customer impact, architecture or accountability should stay deliberately human-led.

3. Make the boundaries explicit

Every AI system needs clear limits on:

• what information it can access
• what actions it can take
• what it can decide on its own
• when it has to escalate
• the standard its output has to meet

Defining those boundaries is as much a part of the work as choosing the technology.

4. Design for correction

AI will sometimes be wrong, and the system should assume it. People need to be able to see what happened, correct it and improve the process. Observability, review and feedback can’t be afterthoughts.

5. Measure the work, not the novelty

“We deployed AI” isn’t a result. Results look more like this: the process takes less time, decisions are better informed, errors fall, customers get answers sooner, people spend less time hunting for information, or the business can do something it couldn’t before.

AI is useful when the work gets better.

The goal isn’t less human

AI transformation is often described in terms of replacement: fewer steps, fewer handoffs, fewer people. Sometimes those are outcomes. They’re rarely a good place to start.

A better question is what people should be able to do now that they couldn’t before. AI can give them better information, take routine work off their plate, help experts move faster, surface patterns that would otherwise stay hidden, and put sophisticated capability where the work actually happens. That’s a more interesting kind of transformation than taking a person out of a process.

AI is new. Human judgment isn’t.

The technology will keep improving. Models will get more capable, agents will handle longer sequences of work, and more decisions will become automatable. The underlying questions will stay familiar:

What are we trying to achieve?What matters to the customer?Which trade-offs are we willing to make?What should we trust the system to do?Where does a person need to decide, and who is accountable when it matters?

Those aren’t AI questions. They’re business questions. Getting them right still takes people who understand the work.