Who bears responsibility when AI prepares the work?

This article examines responsibility in the moment between a plausible AI draft and an effective decision. What matters is not who clicks last, but who can know, change, stop and later explain.

A draft can make a decision look as if it has already been made. It contains friendly wording, a clear recommendation and perhaps even a reference to an internal rule. The service worker then appears to have only a small task left: check, confirm, send. Responsibility is often described too quickly at precisely this point. “A human approved it” sounds like control. It answers neither what that person knew, nor what action they could actually prevent, nor who would have to deal with a later objection.

The following case is wholly fictional. A small spare-parts business supplies machinery and vehicle parts to workshops. In an exercise using only invented tickets, it tries a language model that prepares a reply draft for transport-damage complaints: proposing a replacement, requesting evidence or formulating a rejection. The service worker reviews the draft only in the simulated case; neither this draft nor any other message is sent. The article makes no claim about a real business, real customers or the typical reliability of such a system.

In case R-184, a workshop reports that a component arrived with damaged packaging. The case material includes the expressly fictional stop marker “possible safety relevance”. The model text proposes: “Please submit photos; we will decide within five working days.” The message contains something else as well: an installation appointment has already been agreed for the workshop. The ticket does not say whether the box has been opened. Is the part usable, is a replacement needed, must the goods be held back, or does the workshop first need different information? The text draft does not know. It merely makes the uncertainty easy to overlook.

The guiding question is therefore: With what information, rights and evidence may a service worker confirm an AI recommendation for a transport-damage complaint?

This question shifts the focus. The article treats responsibility as organisational accountability. In this teaching matrix, the model is not an accountable actor; accountability, review and correction tasks are assigned to human or organisational roles. The article makes no statement about employment, liability or contractual duties in any concrete situation. Nor does it merely ask whether humans are involved. People can be present in a process and still lack an effective role. Here, responsibility is treated as an organisational connection among four conditions: sufficient case-specific knowledge, a practical right to intervene, an attributable decision, and an accessible route to correction.

A click is not an accountability architecture

A person may confirm a proposed text and still be scarcely able to decide responsibly. Perhaps they see only a summary, not the original ticket. Perhaps they may change the text but are judged by a processing target that makes every follow-up question look like a delay. Perhaps they can report an error, but the person who could change rules or access rights never learns about it. Perhaps the later record only says “reviewed”, while the reason, missing information and escalation are absent. In all four cases, the organisation bears responsibility differently from what the final click suggests.

This does not excuse the person at the screen. A service worker can also act professionally wrongly with an AI draft if they ignore available information or disregard a clear stop rule. An organisation must not, however, make that person the sole bearer of error when it withholds the necessary file, time, cover and authority. The philosophical work of Santoni de Sio and van den Hoven describes meaningful human control through “tracking” and “tracing”: by their account, a system must be able to respond to relevant human moral reasons and environmental facts (tracking), and every system action must be traceable to the technical and moral understanding of at least one person in the design or deployment chain (tracing). The authors develop these terms in the debate on autonomous systems; they are neither an empirical effectiveness study nor a legal rule for this fictional business. For R-184, they nevertheless sharpen the question: can a responsible role understand what the system does and causes in this case, and can the decision be traced back to that role?

A workable answer needs several levels.

Authorisation responsibility belongs to the role that sets the purpose, boundary, roles and resources for use.

Case responsibility belongs to the role that makes a professional decision on these facts or consciously defers the permitted decision.

Fact responsibility belongs to the role that checks a particular fact, such as packaging condition, delivery status or product identification.

System and rule responsibility belongs to the role allowed to change the prompt, rule set, access and stop condition of the system.

Review responsibility belongs to the role that receives an objection, has the decision checked again and communicates a correction intelligibly.

These are not five categories of legal fault. They are a teaching separation of tasks so that not everything disappears into “human oversight” in one case. Several tasks may be held by the same person in a small business. In that event, it must be documented which role that person is exercising at which moment and who covers for them. A table of five names is inadequate if all are overloaded at once or nobody may stop the process.

The NIST AI Risk Management Framework 1.0 treats clear roles, responsibilities and communication routes as a governance task (GOVERN 2.1–2.3, printed p. 23). It also describes collecting, considering, prioritising and incorporating feedback from external stakeholders about possible impacts (GOVERN 5.1, p. 24). The framework is voluntary, sector-neutral guidance. It does not determine how many review steps R-184 needs and does not assign legal responsibility to any particular person in the example. The article draws only one practical requirement: a role must be assessable by its action and its evidence.

The UNESCO Recommendation on the Ethics of Artificial Intelligence calls on Member States to ensure that ethical and legal responsibility can be attributed to natural persons or existing legal entities throughout the AI life cycle and in remedy (para. 35). This too is not a single business instruction. It does, however, support the distinction between a tool that produces suggestions and the people and organisations that decide about its use and effects.

The case starts with a gap, not an answer

R-184 contains three distinct questions that the smooth text draft draws together. First: what actually happened? Damaged packaging does not yet show whether the part itself is affected. Second: what professional action is appropriate while that is clarified? That may include holding the part, inspection, replacement or a reasoned follow-up question. Third: what may the business promise the workshop? That communication may depend on a contractual or warranty question, a goodwill rule or a delivery condition. A well-written draft replaces none of these checks.

The supplied F model already changes the case at step B. Here, B does not mean gathering still more arguments for the first draft. It requires competing explanations for why responsibility could be lost. The following three hypotheses look similar, but require different repairs:

F-B thus changes the starting question. Without this step, the answer might be: “The service worker must read more carefully.” With it, three testable organisational questions emerge: which documents must be present? How is a stop made practically possible? What is recorded so that a later objection can be dealt with? A fourth option also belongs on the table: no model text, but a professional checklist with text modules. This non-AI option is neither a step backwards nor a penalty for the project. It is the comparison that prevents the draft from counting as help simply because it is new.

The following F step, D, keeps open facts open instead of turning the most plausible assumption into a decision. For R-184, neither the opening of the box nor the condition of the part is established. This does not automatically imply a replacement, a rejection or a photo request as the final solution. It implies a stop: the draft must not be sent as an ordinary complaint response while the information is missing and the fictional safety marker has not been professionally clarified. F-C then formulates a narrower rule: A service worker may confirm only when the required case information is present, no stop rule is triggered, and the draft remains within an approved rule. Otherwise, they refer the case onward.

This formulation is not a “smart” automation rule. It identifies when automation does not help further. In a small business, that can be more valuable than the ability to draft very many routine cases quickly. The F model used here establishes neither the effectiveness of the rule nor an appropriate review period. It orders the questions; the answers need a file, roles and observation.

What a visible human does not yet achieve, according to research

The claim that “a human checks at the end” is not an empirical safeguard. In a laboratory simulation of an aviation-monitoring task, Skitka, Mosier and Burdick compared 80 students with and without automation assistance. Participants knew the assistance could be imperfect. In this study, accuracy for unreported events was 59 percent with assistance and 97 percent without it; for conflicting recommendations, the authors counted an average of 3.92 compliance errors in six opportunities. At the same time, assistance improved performance for correct directives. The finding therefore shows no general superiority or inferiority of AI prompts. For this exact simulation, it shows that known unreliability alone does not make a final human review reliable. The original study concerns an older simulated task with a student sample; it yields no error rate for current generative AI or this fictional customer service.

A second study makes the trade-off harder rather than easier. In an online study, Buçinca, Malaya and Gajos examined 199 analysed US MTurk participants in a simulated nutrition task with AI intentionally correct only 75 percent of the time. Cognitive-forcing designs reduced adoption of wrong suggestions compared with simple explanation designs, but did not remove the errors. Participants also experienced those designs as more complex; there was no significant difference in overall performance between the design categories studied. The study does not examine professional oversight, contractual decisions or a high-risk setting. It does, however, support the counterargument to simple recipes: an additional demand for thought or review can reduce wrong adoptions while also costing usability.

This article therefore does not call for universal double review. The study determines neither a review period for R-184 nor a suitable interface. It only makes clear why the matrix must ask about information, intervention rights and evidence, rather than treating the human click as evidence of effective control.

K governance: a role gains weight only through rights

The supplied K model is not used in the case as a discussion round beside the process. It changes the accountability matrix. In exploration, the original ticket, delivery record, photo, product identification, model proposal and missing details are made visible separately. In reflection and analysis, conflicting concerns come to the table: the workshop needs a reliable next action; the warehouse specialist must not verbally smooth away a possibly affected part; the service worker must not be reduced to a mere signatory by a time metric; management must take responsibility for capacity, rule and pilot boundary. In the decision, cases are not all treated alike; rights, stops and evidence are assigned. Feedback and evaluation later consider exactly the cases in which a draft was stopped, changed or escalated.

K therefore changes the matrix substantially compared with a minimal version. A minimal version would merely say: “Service checks, management approves, technology operates.” The completed version below assigns knowledge, action, cross-checking, escalation, evidence and feedback for R-184. It describes a teaching target for a later use that must be examined separately; R-184 itself remains an exercise with no external effect. It is not a certified governance model.

The matrix makes two important points visible. First, the service worker in R-184 is not “without responsibility” because they do not themselves decide replacement or warranty. Their responsibility is concrete: they must not turn the draft into a decision when the stop rule applies, and they must hand the case file to the right roles. Second, responsibility does not disappear during escalation. Management must provide time, cover and the possibility of pausing. The warehouse specialist must document a finding as a finding, not as a customer commitment. Service management must explain the communication decision. A complaint office must enable real reopening.

K thus changes the decision itself. The first response to R-184 is no longer: “The text sounds reasonable, so someone will check it.” It is: “The case class changes from routine to stop; the service worker may not send; the warehouse review and management receive clear tasks; missing evidence keeps the decision open.” The matrix creates extra work. Precisely for that reason, it is not a decorative governance document, but a statement about the work that responsible handling of the recommendation actually requires.

The matrix distinguishes a cross-check by another case role from a second signature. The reviewing role does not assess its own initial decision, but performs a different case task. A deputy replaces the original role, but is not a cross-check. For routine cases, this can be a sample check by management. For R-184, it is the specialist factual assessment by the warehouse and the communication review by service management. An organisation must limit access to information to the relevant purpose. “Everyone sees everything” would not solve responsibility; it would raise a new question about data access and confidentiality.

Accountability means: the route back can be described

A decision trail is more than a log for a later question of blame. It should enable the workshop and the business to recognise and correct an error. For a later pilot that would require separate approval, the minimum trail would include: case ID; permitted purpose and boundary of AI use; permitted inputs; model, prompt and rule version with time; original records; draft; human change, adoption or rejection with reason; escalation and outcome; intended or later actually sent message; feedback or correction. In the exercise, the message remains unsent.

This list does not prove that a decision was correct. A well-kept file can contain an error. It does, however, make three evasions harder: presenting a model text as a source when decisive facts are absent; claiming that an organisation gave personal approval without showing the intervention route; and ending a complaint in a system where nobody knows which rule would need changing. Raji and colleagues propose connecting documents, stakeholders and decision-capable roles across the life cycle for internal AI audits (pp. 1–3). This is a conceptual proposal, not evidence that the trail designed here actually remedies an error.

Under para. 38 of the UNESCO Recommendation, affected persons should receive reasons for an AI-assisted decision and be able to request review and correction by a responsible person. The OECD Recommendation on AI names transparency information intended to enable adversely affected people to challenge an output, and context-appropriate mechanisms to override, repair or decommission safely (§ 1.3(iv), § 1.4(b), p. 8). These texts establish in this article neither a generally enforceable right to complain nor a duty to have every response reviewed by several people. They support a design rule: when a decision can have an external effect, a person must be able actually to find and use the route to human review.

For R-184, the exercise requires a draft follow-up question or interim message that does not pretend a final review has occurred and explains what remains open. It is not sent. Which information must be accessible in an individual case depends on context and, where applicable, the law. For the exercise, it is enough that the feedback must not pretend to close the matter and must name an understandable route to renewed human review.

A serious counterargument: does this matrix slow exactly the help that is needed?

The strongest counterargument is not that responsibility is unimportant. It is this: in a small business, people, warehouse access and management capacity are limited. Every stop rule, second review and record takes time. The workshop with an urgent appointment in particular could wait longer for a useful reply because of an elaborate procedure. A later deployment might prepare simple follow-up questions; whether it actually frees time would need separate observation. If every deviation triggers half a committee, governance itself becomes a source of delay, frustration and poor accessibility.

This argument holds. A responsible organisation must not pretend that reviews are free or waiting times ethically neutral. Workload is also an impact of use. The right conclusion, however, is proportionality, not a return to the symbolic click. For a complete routine file, the service worker can decide within a clear rule; a sample is sufficient as a cross-check. R-184 follows the more demanding trail because the fictional safety marker, unclear facts and possible commitment are precisely not routine. The case does not add complexity through ethics. It exposes the complexity already present in the complaint.

A lean organisation can work with three thresholds:

1. Routine: Complete file, no stop rule, clear rule. The service worker reviews and decides within the defined boundary.

2. Stop: Decisive information is missing, there is a safety marker, the draft cannot be explained, or a commitment is disputed. No external communication as a finished decision; hand over to the named role.

3. Rule review: A recurring stop, missing cover, rule deviation or pattern of complaints. Management adjusts the scope, pauses it, or returns to a non-AI alternative.

These thresholds are a fictional role rule, not a measurement of speed, workload or errors. The article “AI ethics begins before the first prompt” clarifies comparison with the non-AI alternative; a later article in this series examines workload; another later tests the actual quality of an override. Here, the counter-question remains limited to responsibility: is it also visible for the checklist who reviews its proposal professionally, may stop it and responds to an objection?

R as a narrow specialist-review request, not a legal stamp

The matrix contains a trigger for specialist review, but no legal advice. R is not run in full for every ticket. In R-184, R1, preparation or anamnesis, begins because the case material connects possible safety relevance, possible replacement costs and customer communication with an open factual core. The review request receives only the question that actually needs answering and the records required for it.

The R request guards against two errors. It prevents a text draft from silently answering a contractual, warranty or liability question. And it prevents a specialist role from having to give a general “compliant” verdict for a process whose purpose, data flow and individual case are still unclear. A concrete specialist review can request records and can return the outcome that no reliable commitment is presently possible. That is a useful outcome, not a mishap.

90 minutes: “The click has no name”

The lesson turns the matrix into a test task. The completed matrix, stop and escalation card and blank template are available in Worksheet. Learners do not have to make the same customer decision. They must show whether a role knows enough, can intervene effectively and leaves a trail that can later be checked. The learning material contains only the fictional case R-184: ticket, model text, delivery status, an incomplete photo reference and the case marker “possible safety relevance; installation appointment pending”. One element is deliberately open: whether the box has been opened.

Learning product: Each group submits a completed accountability matrix, two stop triggers, a narrowly worded R review request and a minimum decision trail. Assessment tests the connection between knowledge, intervention and evidence. Agreement on the same replacement or rejection decision is not a criterion.

The teacher need not know the correct spare-parts decision. They facilitate whether groups clearly separate facts, the model proposal and the rule. One group may reasonably say that without additional information it recommends no customer commitment. Another may formulate an interim message with a prompt follow-up question. Both can work well, provided no group invents missing facts and the matrix does not delegate the stop to a role label with no effect.

The negative test is deliberately concrete: one group enters “service worker stops”; the reviewing group asks whether the matrix names an information source, time and cover for this. If any is missing, the fictional role description is incomplete. The group adds file access, cover or a smaller deployment class. This paper exercise does not test whether a stop succeeds under actual work pressure; that remains for later articles in this series.

What this article decides — and what must be examined later

This article does not develop a general answer to whether AI should be used in customer service. The article “AI ethics begins before the first prompt” addresses the prior decision about problem, purpose and a bounded test. This article follows it by assigning responsibility for a prepared individual case. A later article in this series will explore workload and time pressure as matters for examination: this version identifies time and cover as conditions for a real intervention right, but does not measure workload. Another will perform the intervention test on faulty output: this version creates the case file, stop, roles and evidence for it, but does not yet test the override.

The article therefore makes no claim that the matrix reduces errors, resolves complaints or makes a model safe. It offers a narrower institutional thesis: a person takes responsibility for an AI draft only when they can examine the relevant facts, change or stop the proposal with practical effect, and document their decision so that another responsible role can check and correct it. Where one of these conditions is absent, the organisation must identify the gap and either equip the role, escalate the case or narrow the deployment boundary. The counter-test in this article likewise checks only the completeness of a fictional matrix, not the performance of a production override.

The outcome in R-184 is therefore neither a dramatic act of heroism nor a claim that AI is prohibited. The proposal is stopped. The warehouse specialist checks the facts, service management decides on communication only on a more complete basis, and any necessary specialist question remains narrowly bounded. The decisive progress is that no one can pretend that fluent text has replaced the missing fact and the responsible decision.

Sources and limits of application

NIST (2023), Artificial Intelligence Risk Management Framework (AI RMF 1.0), especially GOVERN 2.1–2.3 and 5.1 (printed pp. 23–24). Voluntary framework; no specific accountability matrix, legal norm or effectiveness study.

UNESCO (2021), Recommendation on the Ethics of Artificial Intelligence, especially paras. 35–38, 47 and 50–53. International normative recommendation; no individual legal advice and no prescribed complaint form for the fictional business.

OECD (2019, amended 2024), Recommendation of the Council on Artificial Intelligence (OECD/LEGAL/0449), § 1.3(iv) and § 1.4(b), p. 8. Guidance on contestability and safe override mechanisms; no blanket requirement for multiple human reviews.

Skitka, Mosier & Burdick (1999), “Does automation bias decision-making?”, International Journal of Human-Computer Studies 51, pp. 991–1006, especially PDF pp. 5–9 and 12–13. Laboratory simulation with 80 students; no estimate for generative AI or customer service.

Buçinca, Malaya & Gajos (2021), “To Trust or to Think”, DOI: 10.1145/3449287, especially PDF pp. 1–2 and 16. Online experiment with 199 analysed US MTurk participants and simulated AI; no study of professional oversight or organisational liability.

Santoni de Sio & van den Hoven (2018), “Meaningful Human Control over Autonomous Systems: A Philosophical Account”, DOI: 10.3389/frobt.2018.00015, especially PDF pp. 8–12. Philosophical conceptual work; no empirical effectiveness study and no legal rule.

Raji et al. (2020), “Closing the AI Accountability Gap”, DOI: 10.1145/3351095.3372873, especially PDF pp. 1–3. Conceptual proposal for internal algorithmic audits; no guarantee of the effectiveness or independence of internal reviews.

Supplied internal method materials: F (A → B → D → C), K (exploration; reflection/analysis; decision/recommendation; feedback/evaluation) and R (preparation; analysis; solution design; implementation/documentation; evaluation). They structure the teaching casework; they establish neither effectiveness, an appropriate review period nor legal compliance.

The matrix and the proportionality rule are expressly identified as teaching, fictional syntheses. A later production use would need case-specific specialist review of jurisdiction, role, data flow and system type.