Human oversight that can intervene

An AI prepares a response. A person sees it before it is sent and clicks “Approve”. An organisational chart may therefore say “human oversight”. In actual work, however, that click can be almost empty: the person cannot see the case information needed to decide, is not allowed to stop the dispatch, has no time to check, does not know a reliable fallback route or has to carry an uncertainty alone. A person is then present in the workflow, but unable to change it.

The decisive question is therefore not: Is there a person in the process? It is: Can that person identify a specific erroneous output, stop its effect before it occurs, choose a defensible alternative or involve a clearly designated role, and make the reason traceable afterwards? A role label cannot answer that question. It requires a test under conditions that make intervention possible in the first place.

This article develops such a test for a small team. The case, all roles, all products and all results are entirely fictional. The exercise sends nothing to customers and measures no performance of a production system. Its result is not a quality seal for an organisation either. It only shows whether a previously defined oversight route could be carried out in practice for a known error in teaching or an internal dry run.

The scope matters. “Who bears responsibility when AI prepares the work?” addresses how responsibility and accountability are distributed across roles. “Judging Under Time Pressure” examines how time pressure and workload can alter judgement. “Shadow AI and the Dignity of Data” concerns unclarified AI use and data flows. A later article in this series will build on a limited pilot with a baseline, harm indicators and a stopping threshold. This article has a narrower focus on an observable oversight test before an external effect: what must be in place for a person to stop an incorrect proposal effectively?

A click is not oversight

Oversight is not a character trait. It is a relationship among a person, a decision and a technical-organisational route. For it to be more than symbolic presence, at least six things must come together:

1. Visibility: The reviewing person sees the AI output and the context that makes it reviewable.

2. Independent reference: They can compare the output with current, accessible professional information instead of merely sensing plausibility.

3. Authority: They may stop, override or hand over the matter before it takes effect; a warning without the ability to act is insufficient.

4. Time and attention: The time window permits a review proportionate to the significance of the decision.

5. Capability and support: The person understands the task, typical error boundaries and the escalation route, or can clarify them without delay.

6. Safe fallback: After a stop, there is a permissible next action, such as a manual response, a follow-up question or an expert review.

This list is neither a legal checklist nor an empirically confirmed minimum formula. It merely makes visible which prerequisites may be absent in a test. That is precisely its function: a team should not infer from an overlooked error that the reviewing person “did not pay close enough attention” if the information or authority chain made intervention impossible.

The underlying philosophical point is simple but demanding. Responsibility does not require omniscience. It does, however, require that someone can make a difference and that the reasons for their action do not have to be invented after the fact. A person who formally approves a decision without access to its decisive reason carries a burden without corresponding power to act. A person who identifies an error but is not allowed to stop the dispatch is a witness, not oversight. Fair design therefore asks not only who signs at the end, but also who has which options and what support at the decisive moment.

This view avoids two reductions. The first shifts every duty onto the individual reviewing person: an error is then explained as individual inattention, even though the display, deadline, access or permission was poorly designed. The second denies people any role because AI systems are complex. Complexity makes human review harder; it does not remove the need for assigned intervention. The aim is not a person who detects every output perfectly. The aim is a process that does not force an external effect prematurely when there is recognisable uncertainty.

For high-risk AI systems, the EU AI Act sets out a particularly concrete standard: depending on the circumstances, the responsible natural persons must, among other things, be able to correctly interpret, refrain from using, disregard, override or reverse an output, and intervene in or interrupt the system’s operation. Deployers of such systems must assign oversight to persons with the necessary competence, training, authority and support.¹ This rule does not apply broadly to every AI assistant and every small business. Whether a specific system falls within this scope depends on its role, use and risk classification.

Short legal box, current as of 29 September 2026: The following case cards do not claim that the fictional text assistant is high-risk AI. Under the consolidated version, the relevant requirements in Chapter III, Sections 2 and 3 apply to systems under Article 6(2) in conjunction with Annex III from 2 December 2027, and to systems under Article 6(1) in conjunction with Annex I from 2 August 2028. Regulation (EU) 2026/1744 amending the Act and the consolidated text are therefore part of the material to be examined in real cases. A specific legal question also requires the system, role, jurisdiction, date, contract and facts.^1,2

The idea is useful outside this legal framework as well: an organisation can use a simulated error to test whether its own claim of reviewable approval is met technically and organisationally. That is a design choice, not a statement about a legal obligation or effectiveness.

What the research suggests – and what it does not

Three original studies give reason not to reduce oversight to an explanation button. Each addresses limited experimental tasks, not real spare-parts decisions. For that very reason, they must not be used to derive a general promise of success.

A study on cognitive forcing functions examined 199 online participants choosing a lower-carbohydrate ingredient from meal images. The simulated AI was deliberately wrong in one quarter of the cases. The study compared, among other conditions, simple explanations with processes that first prompted independent reflection or a justification. For incorrect proposals, those proposals were adopted less often in the conditions with such interventions; at the same time, trust and preference for those conditions were lower.⁴ This is not proof that every prompt placed before a decision prevents errors in practice. It does show why “show the proposal immediately, then require approval” should not be treated as neutral design.

Another experimental study with 3,800 participants in total used a simplified apartment-price prediction. A transparently presented linear model was easier to understand, but did not automatically lead to more appropriate correction. For large errors on unusual data points, participants in a clear-model condition were worse at identifying and correcting the errors.⁵ The authors discuss overload as a possible explanation. The finding does not automatically apply to every visualisation or field. It does, however, point to a practical caution: more information displayed is not the same as greater ability to intervene.

A third online study asked 600 participants to classify short biographies. None of the explanation conditions examined caused incorrect recommendations to be overridden selectively more often than correct recommendations. Certain explanations nevertheless changed override behaviour; this did not produce selectively correct correction.⁶ An explanation on screen can therefore influence people without reliably improving their search for errors in this task.

The shared conclusion is limited: visible proposals and visible explanations are insufficient evidence of effective oversight. For designing a test, it is reasonable to let the person first review the case file, then compare the AI output with an independent reference and observe the actual stop or escalation route. This sequence is a recommendation based on limited evidence and practical caution, not a tested universal rule.

The voluntary NIST AI Risk Management Framework fits this modesty. It does not require a particular stop button for every use. It does, however, describe roles and responsibilities for human-AI configurations, documented oversight processes, and procedures for feedback, complaints, overrides, decommissioning and change management.³ The framework also points out that it must be examined whether people are actually able and willing to challenge AI outputs. That is a good reason not only to publish a policy, but also to practise its intervention route on an erroneous output.

The entirely fictional A21/A12 case

The business Nordwerk Spare Parts is fictional. It sells no real products and has no real customers. In this exercise, the team uses an approved text assistant solely in an isolated test environment. The assistant may generate a response draft, but may neither send a message nor change inventory.

A fictional case file with the identifier F-217 contains the request: “Please confirm the availability of part type A21 for maintenance.” The exercise’s internal reference card lists A21 as the requested part type and records the status “available” for this fictional test case. A deliberately incorrect AI draft instead replies: “Part type A12 is available and will be reserved.” A21 and A12 are invented character strings. The difference is deliberately small, so that the test does not merely test obvious nonsense, but the link among attention, source and the right to intervene.

The erroneous output is not intended to expose the AI or test the reviewing person as in an examination. It tests the route between recognition and action. Four conditions are therefore set before the start:

The exercise separates three decisions that easily merge in everyday work. Stop means: the specific draft must not take effect. Override means: the overseeing role has sufficient reference and authority to replace it, with reasons, by a different response. Escalation means: the reference or expertise is insufficient; the next action is a limited clarification request. Escalation is not failure. It is the correct decision when a safe replacement is unavailable.

This distinction also guards against a common misunderstanding: oversight does not require every person to solve every error themselves. It requires the person to recognise when they cannot safely make a decision and to move the matter into a responsible review without external effect.

Sample run of the fictional oversight test

What follows is a fully played-through sample run of the fictional classroom simulation. The times are fictional educational worksheet data, not measurements from a business, not personal performance data and not a comparison among people. The role “reviewing person” refers to a fictional role; no real person took part.

Before the test, the reviewing person received the brief oversight card, but not an indication of which card contained an embedded error. The moderator ensured that Stop, Override and Escalation were accessible in the test interface. The test consisted of three separate cards:

The record shows only that the planned route could be used in this controlled exercise. It shows neither that the person would act the same way under work pressure, nor that the assistant would reliably be recognisably wrong in other cases, nor that the organisation thereby reduces risks. The times serve subsequent reflection: Was the review time sufficient in the scenario? Was the reference easy to find? Did the stop produce a sensible follow-up action? They are not a metric for a production target time.

The most important observation is on card 2. Mere error recognition would fall short there. The draft contains no visible A21/A12 mix-up, but the basis for an availability statement is missing. Had the reviewing person been forced to correct quickly, they might have invented a claim. In this exercise, oversight consists precisely in recognising uncertainty and limiting the decision. That distinguishes responsible control from the expectation that an answer must always be supplied immediately.

After each card, a short trace is recorded:

Which reference was present or missing? Which specific contradiction, uncertainty or confirmation was seen? Which effect was still pending? Was the matter stopped, overridden, escalated or approved – and why? Was the intended route technically and organisationally accessible? What residual uncertainty remains, and who will address it next?

A complete trace must not become mere self-documentation. Its purpose is to revise the design. If, for example, Stop had worked only through an unreachable administrator role, the test would not have been “just barely passed”; the oversight route would be inadequate. If the reviewing person had noticed A21 and A12 but could not find a reference card, that would not be merely an attention problem. The case design would then need to change information access, display or responsibility.

Applying K: the four-stage core cycle

The K consultation model provides a structure here for a reasoned oversight route. Its core cycle has four stages: exploration, reflection/analysis, decision/recommendation, and feedback/evaluation. It is not evidence that the test is effective. It helps prevent a decision from being reduced to the surface of a draft.

1. Exploration: What should remain under human control?

Nordwerk Spare Parts first determines that the assistant produces text drafts only. An availability claim takes effect only when a person approves it in the simulation. In the fictional case, those affected are the requesting person, the expert role, the role processing the response and the business, which should not make a false commitment. The exercise also records the actual intervention points: hold back the draft, replace the text, open an expert question and change the rule later.

Exploration does not ask whether the tool is “good”. It asks which decision is being prepared, which reference supports it and where an error would have consequences. In the A21/A12 case, the verifiable reference is the case card. In a real case, it could be an approved professional source; if that source is inaccessible, the process must not demand a merely apparent review.

2. Reflection and analysis: What conflict is in the quick draft?

Fast responses can help a requesting person. An unverified commitment can impair maintenance, planning and trust. Both sides must be taken seriously. The team therefore records not only “incorrect part designation”, but also: What happens if no one intervenes? How long is the real time window? What information does the reviewing role need? When is a follow-up question fairer than a smooth but unfounded response?

This phase also brings power and burden into view. If management requires a very short response time but holds the reviewing person accountable for errors, a conflict arises between expectation and working condition. “Judging Under Time Pressure” examines precisely this tension between time pressure and loss of control in more depth. For the oversight test, the concrete implication is sufficient: a time requirement must not silently eliminate the ability to stop when in doubt.