When AI produces faster than an organization can verify

For years we asked whether AI could produce anything valuable. Now, in some processes, a different problem shows up: it can produce more than an organization can verify with judgment.

The question is no longer only:

Can it generate a result that's valuable enough?

But rather:

Can the organization verify, integrate and act on everything AI produces?

What we're seeing in more and more organizations is that the bottleneck is moving. It's no longer only in production. It's also starting to sit in control.

When producing more doesn't solve the problem

A preprint published in July 2026 analyzed 196,212 pull requests from 802 developers at a company aiming to double its output. A pull request proposes code changes that are reviewed before being merged into a project.

By April 2026, the average number of pull requests created per active developer reached 2.09 times the previous level: it went from 21.2 to 44.3 per month.

But review capacity didn't grow at the same rate. Total volume rose 3.1 times, while the number of people reviewing grew only 1.5 times. The load per reviewer practically doubled.

It's a single case, at a company prepared to adopt AI, and the authors don't causally attribute the entire increase to the technology. But we do read it as a signal of a problem that can repeat itself in other processes:

when AI speeds up production, the limit shifts toward verification.

The most interesting part is how the organization responded.

According to the study, the share of changes that received at least one human review fell from 89% to 68%. In parallel, automated review went from roughly 19% to 84%. The same change could receive both types of review, but the direction is clear: automation started to replace part of the human review.

The same study shows that substantive human review, the kind that included written comments, fell from 39% to 21%. Silent approvals stayed relatively stable.

When verifying became the limit, part of that verification was delegated back to the AI that was expanding the volume to be verified.

The bottleneck didn't disappear. It became less visible.

The limit of human in the loop

The concept of human in the loop was useful for introducing automation while keeping a person inside the flow, able to approve or reject an action. It was also reassuring: AI could assist, but the human was still "in control."

The problem is that this model becomes hard to scale if every output requires human approval. When frequency rises or processes run persistently, the human can turn into a signature at the end of the circuit. The approval is still there. The judgment, not always.

On top of that, if the system routes to people only what it can't resolve, the nature of their intervention changes. Routine cases stop arriving. What's left are the ambiguous, exceptional or conflicting ones. The human intervenes fewer times, but each intervention can demand more judgment, critical thinking and ability to handle exceptions.

It's no longer enough to decide at which step we place a human approval. We have to design how control is distributed across the system.

The proposal: a five-layer control architecture

When redesigning a workflow, at Kintara we classify what AI can do now, what it can do later and what must stay manual. But that still leaves open who sets the limits, receives the exceptions, reviews quality and changes the rules.

That's why we propose a control architecture in five layers:

1. Human in command

Defines the objective, the limits, the forbidden outcomes and the acceptable level of risk. Doesn't approve every operation. Designs the frame within which AI can act.

2. AI in execution

AI produces, analyzes or coordinates the operational sequence within those limits, bounded by rules, permissions, data and authorized actions.

3. Human on exception

Steps in on events defined in advance: deviations, low confidence, missing data, conflicts between sources or high-impact consequences.

4. Human in review

Evaluates samples, outcomes and patterns. No longer reviews every case, but whether the system is still making good decisions and whether its limits remain well calibrated.

5. System learning

Errors, complaints, incidents and exceptions feed changes in rules, thresholds, data, prompts or models.

These layers don't necessarily have to sit with one person. Whoever defines the objective, whoever resolves exceptions and whoever evaluates quality perform different functions and require different judgment and expertise.

What it looks like in a procurement process

Think about invoice management.

Human in command. Finance, Procurement and Risk define which invoices AI can process without intervention and which cases can never be approved automatically.

AI in execution. For authorized invoices, AI extracts the data, compares invoice, purchase order and delivery note, detects differences and routes the case to the right circuit.

Human on exception. A person steps in for a new supplier, a missing order, an amount above the threshold, a suspected duplicate or deviations from the contract.

Human in review. Internal Control reviews a sample of automatically processed invoices. It no longer asks only "is this invoice correct?", but "are the thresholds still well calibrated?" and "what patterns show up in the exceptions?".

System learning. The team analyzes disputed, duplicated, reprocessed or audit-flagged invoices, determines which layer should have caught the problem and adjusts the system.

The new critical capability

During the first stage of adoption, many organizations built the capacity to use AI. The next one demands governing systems that produce more than a person can review case by case.

That requires new metrics. Not only how much AI produces, but also:

• what share is reviewed substantively;
• which exceptions reach people, and with what context;
• how many errors escape each layer;
• how long it takes to recalibrate rules and thresholds;
• and which decisions keep a clear human owner.

That part of the review gets automated isn't the worrying part. It would be contradictory to ask for scale and demand that one person check every output.

What's worrying is mistaking an automatic approval for a real verification capability. Productivity can keep growing while the capacity for control weakens.

The question is no longer how much work AI can absorb. It's what architecture the organization needs in order to preserve judgment, accountability and the ability to learn while AI scales.

Sources

• Hao He et al., AI Writes Faster Than Humans Can Review: A Longitudinal Study of an Enterprise "2×" Mandate, arXiv preprint, July 2026.
• GitHub Docs, About pull requests.