AI risks are project risks

The most visible AI failure is often not the most dangerous one.

A fabricated source, a non-existent court ruling or an obviously wrong number can be noticed with enough attention. The harder case is an answer that is 95 percent plausible but uses one wrong premise, confuses a term, skips an intermediate step, or inserts stale information into an otherwise correct argument so smoothly that the error travels unnoticed.

That is the point where a model error becomes a project risk.

Professional quality control does not only ask whether an AI hallucinates. It asks which error classes matter in this project, how they can propagate, which controls detect them early, and how much verification the project can realistically afford.

Hallucination is only one error class

“Hallucination” is a useful popular term, but it is too broad for project management. It directs attention toward invented facts while many operational quality problems look different.

A professional risk catalogue should distinguish at least:

The question changes from “Is the model good?” to: Which errors must this workflow never let through?

The most dangerous error often sounds reasonable

Obvious hallucinations have one advantage: they can trigger suspicion.

Subtle errors do not. They fit the language, tone, format and structure of the argument. A model can provide ten correct facts and derive the wrong priority. It can use the right regulation but apply it to the wrong organizational role. It can generate working code that mishandles an edge case. It can cite every source in a market analysis and still confuse correlation with causation.

The more professional the surface, the stronger the trust effect can become.

Quality control therefore has to identify not only obvious falsehoods but the points at which plausible language becomes decision-relevant.

An important source correction: Deloitte, not “Lloyd”

The underlying project material refers to an Australian consulting incident as a “Lloyd scandal.” The reliably documented public case is the Deloitte Australia report for the Australian Department of Employment and Workplace Relations.

The report included, among other problems, non-existent academic references and an incorrect quotation attributed to a court decision. Deloitte issued a corrected version and refunded part of the fee.^[5]

For this article, the brand is less important than the control failure:

A professional-looking deliverable reached the client even though critical assertions had not been adequately verified against primary sources.

That is a project-management failure, not merely a language-model failure.

Quality belongs in the risk register

Traditional projects manage schedule, budget, suppliers, resources, scope and dependencies as risks.

AI projects add quality risks such as:

False facts, unsupported claims, inconsistent versions, missed edge cases, faulty tool calls, incomplete reviews, model or provider changes, unstable outputs, context loss, flawed evaluations — and human verification capacity as a bottleneck.

These risks should not live in a footnote. They need an owner, likelihood, impact, detection mechanism and response.

Surface correctness is not semantic correctness

An output can look formally perfect and still be wrong.

Two quality dimensions should therefore be separated.

Surface Correctness

Format is valid. Tables are complete. Citations are syntactically present. Code compiles. Language is professional. Schema is respected.

Semantic Correctness

The statement is actually true. The source really supports the claim. The conclusion follows from the evidence. Reported process state matches real state. Edge cases are adequately considered. The decision is valid for the actual context.

Automation is often excellent at surface correctness. Professional quality begins when semantic correctness is controlled as well.

The claim is the smallest verifiable unit

Long AI documents are often reviewed as wholes: “looks good,” “sounds plausible,” “has sources.”

That is too coarse for critical content.

A better approach is claim-level verification. Treat a decision-relevant claim as an object:

What exactly is being asserted? Is the claim descriptive, causal, normative or predictive? What evidence is supposed to support it? Does the source support that exact statement? Is the source current and authoritative enough? What uncertainty remains? Which project decision depends on it?

This turns “source checking” into a reproducible quality process.

Not every statement needs the same verification depth

Verifying every sentence completely would make large projects unaffordable.

Quality therefore needs risk classification of claims.

The question is not “How much do we verify?” but where do we spend scarce verification capacity?

The review-budget error

A common project mistake is scaling AI output without scaling human review capacity.

If a system generates 50 pages in minutes, it has not produced a 50-page result. It has produced a 50-page verification task.

This is where verification fatigue emerges:

Reviewers read only headings. Long stretches of correct material reduce attention. Plausible prose creates routine trust. Time pressure shrinks samples. Hard claims are skimmed more than easy ones. “Reviewed” becomes a workflow status rather than a real review action.

A human-in-the-loop process can therefore exist formally while being operationally ineffective.

Verification Capacity is a project resource

Alongside budget, GPU time and tokens, an AI project should plan Verification Capacity.

It consists of:

Available expert time, suitable automated checks, independent model review, test data, reference sources, reproducible evals and time for disputed cases.

A team that can generate 100 units of output but genuinely verify only ten does not possess a 100-unit system. It possesses a 90-unit quality deficit.

Verification Debt

When output is created faster than it can be verified, Verification Debt accumulates.

Typical symptoms include:

Open claims without evidence, unchecked agent results, old evals used for new model versions, growing review queues, partially reviewed reports, “verify later” notes, ambiguous approvals.

Verification Debt is dangerous because it rarely explodes immediately. It silently becomes input to the next project phase.

Error propagation: a small mistake becomes project state

In agentic workflows, one early mistake can be amplified repeatedly.

Example:

A research agent adopts a wrong market figure. A planning agent uses it as a budget assumption. A risk agent evaluates the business case around it. A presentation agent visualizes the number convincingly. A reviewer checks the presentation but not the origin. The decision is approved.

The initial error no longer appears as an uncertain statement. It has become consolidated project state.

Quality control should therefore happen as close as possible to the creation of critical information.

Error Propagation Graph

For complex projects, a simple dependency map is useful:

Source → Claim → Assumption → Decision → Artifact → Action

A claim with high downstream centrality deserves more verification than an isolated side note.

This is often more efficient than “double-check everything.”

Cross-model checks help — but they are not ground truth

A second model can find errors, expose blind spots or provide alternative arguments. A third model can make disagreement visible.

But model consensus is not proof of truth.

Several models can:

Share the same popular misinformation, consult the same weak source, misinterpret the same ambiguous task, or be anchored by the first output they are shown.

Cross-model triangulation is a disagreement detector, not a truth oracle.

The reviewer must be independent enough

A reviewer that inherits exactly the same context, assumptions and reasoning as the generator has limited independence.

Depending on the task, a stronger reviewer may receive:

The original task, the evaluation criteria, primary sources, possibly the output — but not necessarily the full reasoning path of the generator.

This reduces anchoring and creates genuine counter-review.

Generator and reviewer need different objectives

The generator optimizes for creating.

The reviewer optimizes for finding failures.

That architectural difference matters. If the same prompt says “create a persuasive recommendation” and then “critically review your recommendation,” the model often remains inside its own trajectory.

A stronger reviewer receives a different mission:

Falsify rather than confirm. Search for counterexamples. Mark missing evidence. Construct edge cases. Raise uncertainty when justified. Prefer finding one critical defect over ten stylistic improvements.

Three types of graders

Modern evaluation architectures typically combine several grader types.

1. Deterministic graders

Unit tests, schema checks, regex and string checks, database state, calculation checks, static analysis.

They are fast and reproducible but work only where criteria can be operationalized precisely.

2. Model-based graders

They can evaluate coherence, coverage, groundedness, tone, rubrics or open-ended quality criteria.

Their weakness: the reviewer is itself a probabilistic model.

3. Human / domain graders

They remain essential for normative, domain-specific or high-impact questions, but they are slower and more expensive.

Mature quality systems combine all three rather than declaring one grader to be truth.

Outcome beats self-report

An agent can say “the file was saved,” “the booking was made,” or “the test passed.”

That is only text output.

Where possible, quality should be checked against the real end state:

Does the file actually exist? Does the database contain the record? Was the order really created? Does the code pass the tests? Was the correct dataset modified?

A real system state is often stronger evidence than the agent’s statement about that state.

Execution trace and outcome test different things

The outcome asks: Did it work?

The trace asks: How did it happen?

Both can matter.

An agent may reach the correct outcome through an impermissible path. Or it may follow a sound process but fail because of an external outage.

Critical evaluations should therefore separate process quality from outcome quality.

Non-determinism requires multiple trials

A single successful run proves little in a stochastic system.

If the same task is run five times and succeeds only three times, “it worked” is not a sufficient quality statement.

Important workflows therefore need multiple trials, seeds or scenarios.

Two questions are different:

Can the system solve the task at least once?

Can it solve the task reliably and repeatedly?

Creative exploration may care about the first. Customer-facing processes usually care about the second.

Capability evals and regression evals

These evaluation types serve different purposes.

Capability evals ask which difficult tasks the system can now solve.

Regression evals ask whether the system still performs tasks that worked yesterday.

A new model, prompt update or tool can improve one benchmark and break an old workflow at the same time.

Regression testing therefore belongs in every relevant change process.

Real failures should become test cases

The most valuable eval dataset often comes from real project failures rather than theoretical questions.

Every relevant incident can become a regression case:

Document the failure. Preserve a minimally reproducible input. Define expected behavior. Mark the failure mode. Build a grader. Add the case to the suite. Rerun it after model, prompt or tool changes.

Project history then becomes quality infrastructure.

Groundedness is more than “there is a source next to it”

A claim is not grounded merely because a link appears after it.