Shadow AI and the Dignity of Data

A quick response turns into an unclarified data flow

In an entirely fictional small continuing-education provider, an employee is responding to feedback about how a course is run. A response is due today. The designated writing assistant is unavailable; it is also unclear whether the new internal tool is approved for texts of this kind. To smooth the wording, the employee opens a freely accessible chatbot. The draft includes a name, the course context and specific feedback. The wording comes back improved. No one at the organisation knows exactly which account was used for the input, whether it was stored or which rules apply to later use.

The first reaction might be: “Who did this?” A second question is more urgent: “What information flowed where, and who needs to act now?” A third goes deeper: “Why was the fast, permitted work route unclear or unavailable?” None of these questions cancels out the others. The data subject needs protection and, where appropriate, an understandable route for follow-up. Employees need to know and be able to follow the rules. The organisation must investigate both the specific incident and the conditions that enabled or encouraged the use.

The case is entirely fictional. It illustrates a decision question, but does not depict any real organisation, person or event.

This article uses shadow AI as a working term for AI applications used in an organisation without being documented, assessed or approved through the intended work route. The term is not a legal category. An application can be unknown even when its users have good intentions. A use can be permitted in an individual case even though the organisation does not yet have an orderly process. Conversely, a tool does not become safe merely because it appears on an internal list.

People, not datasets, are at stake

The phrase “dignity of data” is a deliberate shorthand. Data themselves do not possess human dignity. People possess dignity; information about them can, however, affect decisions, relationships and their ability to act. Moving a message, a performance note, health information or a confidential draft to another service may change the circle of actors, purposes, storage locations and later recipients. The person concerned can neither see nor correct that change.

The Charter of Fundamental Rights of the European Union places human dignity, private life and the protection of personal data alongside one another. The GDPR adds a framework for personal data, including principles such as purpose limitation, transparency and data minimisation, as well as duties of privacy by design and appropriate security [1, 2]. These rules are not interchangeable: human dignity is a normative reference point; data protection law is a legal framework with its own requirements and responsibilities.

A helpful philosophical lens is Helen Nissenbaum’s concept of contextual integrity. On this account, privacy does not depend solely on whether information is secret or public. It also depends on who collects it in what context, for what purpose it is used and under what expectations it is passed on [4]. A person may entrust information to a continuing-education provider for course feedback. That does not by itself mean they expect that information to end up in any external writing service.

Contextual integrity is not an additional legal rule and does not on its own decide consent or lawfulness. But it makes visible a question that is often overlooked in practice: Does the data flow fit the purpose and relationship in which the information was collected? Removing a name can reduce risks. It does not automatically answer whether a rare combination of role, course, timing and event still makes the person identifiable, whether the new service processes data or how the use is classified contractually.

Dignity becomes concrete here when people are not treated merely as material for faster writing. It is expressed in clarity of purpose, appropriate limits, traceable responsibility and a way to raise concerns about incorrect or unexpected processing. The data subject should not have to work out which chatbot was used or whom they can contact. That responsibility belongs in the work process.

The real question is: what happens to the data?

With an AI chat, the interface often looks like a single conversation. Organisationally, it is a data path: Who provides the account? Which input is transferred to which service? What logs are created? What storage, retention, onward use or sub-processing do the contract and settings provide for? Who can access inputs and outputs? What human review follows? And how is an incorrect output corrected?

For an initial assessment, not every technical detail has to be established immediately. Unknown points are marked as unknown and passed to a responsible person. A marketing statement such as “Data protection matters to us” or “Your data are not used for training” does not automatically answer every question about account type, retention periods, security measures, logs, roles and contractual terms. The appropriate assessment depends on the specific product, the nature of the use and the roles of those involved.

The GDPR defines personal data broadly as information relating to an identified or identifiable natural person [2, Art. 4(1)]. A spelled-out name is therefore not the only possible identifier. An abbreviation, a combination of function and date, a quoted message or a rare circumstance can also be identifying in context. Conversely, not every internal item of information is personal data. Trade secrets, customer contracts and confidential plans can merit protection independently of the GDPR. Data protection, information security, contractual protection and confidentiality overlap, but are not the same.

For personal data, the case may require, among other things, a clear purpose, an appropriate legal basis, transparency, appropriate security and assessment of the roles of controllers and processors. Special categories of personal data require an additional assessment. A data protection impact assessment is not automatically required for every chatbot input; it may be necessary where there is likely to be a high risk to rights and freedoms. These points depend on the actual processing operation and do not decide the fictional case in this article.

In Opinion 28/2024, the European Data Protection Board addresses questions about personal data in the context of AI models, including anonymity, legal bases and the consequences of unlawful processing [3]. The document is not a general product assessment for every chat service. In particular, it does not support the claim that every input to a model remains permanently in that model. It does, however, remind us that roles and responsibilities must be clarified before or in connection with processing, and that merely claiming “the model is anonymous” does not replace a case-specific assessment.

The scientific evidence also needs to be read closely. Wagman, Dearing and Chetty studied employees at one US scientific organisation that introduced an internal generative AI system. Their work combined a survey of 66 staff members, interviews with 22 people and eight months of usage data. Reported concerns included reliability, privacy and security, publication practice, and possible effects on work and employment [5]. This shows that such issues were perceived in a specific organisational context. It neither measures the prevalence of shadow AI in SMEs nor demonstrates that time pressure automatically leads to unapproved use.

Why blanket bans rarely amount to complete governance

A ban can mark a necessary boundary, especially where data are highly sensitive or a service cannot be assessed. For many organisations, it is also economically easier to set a narrow framework at first. A limited “do not use until approved” can be more responsible than unclear tolerance.

The problem begins when the ban remains the only answer. The rule then says what is not allowed, but not how an employee should complete the actual task safely. If the approved tool is missing, too slow, does not support the needed language or has incomprehensible input limits, the work need remains. The person may still have to finish the text. Whether this results in a workaround has to be assessed in the organisation concerned; it must not be asserted as a general psychological law.

This is the practical balance: the organisation needs binding boundaries while also providing a viable alternative route. That can be an approved enterprise instance, a non-AI procedure, a template, human expert support or an exception assessed by an expert. The alternative must fit the work problem and be usable within actual working time. A policy that no one understands or cannot follow under ordinary conditions is not a robust protection process.

A serious objection is that a non-punitive reporting route could encourage recurrence: if a report always leads only to a process conversation, it may fail to deter deliberate disregard of clear rules. The objection is justified. “Without blame” does not mean “without accountability.” It means directing the first response to a report towards protection, containment and clarification. Deliberate or repeated conduct can then be assessed differently under transparent rules from a good-faith error under unclear requirements. A fair process must not treat these differences according to sympathy, hierarchy or visibility.

Three models, three different tasks

The work in this article can be structured more precisely with the SAKIZLI models themselves. They are neither legal sources nor evidence of effectiveness. The difference between their tasks prevents a good questioning model from being mistaken for an approval or a governance cycle from being confused with specialist advice.

F v5: from “ban it” to an assessable question

The F v5 questioning model structures the work as A→B→D→C. The assignment “How do we ban shadow AI?” already contains a diagnosis: use is primarily a rules problem, and a ban is the appropriate answer. F-A therefore begins with context and objective. What specific work was carried out? What information was involved? Who might be affected? What was the intended route? Which deadline, technical limitation or missing responsibility is evidenced, and what is still only assumed?

In A, 5 Whys can help as a causal heuristic. The technique asks step by step about conditions, but must not present a chain of assumptions as the cause. Each answer needs a cross-check: What observation would support this explanation? What other explanation also fits? The question “Why did the person circumvent the rule?” can end in blame. More useful would be: “What working conditions made the known approved route harder than the unknown route?” This wording, too, is only a beginning, not a pre-empted answer.

F-B keeps several hypotheses open at the same time: perhaps an approved tool was inaccessible; perhaps the approval rule was unknown; perhaps an appropriate function or language was unavailable; perhaps the workload could not be managed in the available time; perhaps there was in fact deliberate disregard. These explanations require different evidence and different responses. F-D uses follow-up questions, feedback and iteration to examine the hypotheses with the affected work roles and available records. Missing facts are not replaced by trust in the model.

Finally, F-C records the improved question, the transparent trace of steps A, B and D, and a GROW action plan: What objective should a safe process achieve? What does the current reality look like? What options are available? Who does what next, and by when? The result might be: “How do we create an accessible, approved route for recurring writing tasks that prevents sensitive inputs, enables questions and brings good-faith reports into rapid triage?” This question improves the assignment. It does not yet approve a tool or decide whether processing is lawful.

K: make the governance decision visible and capable of change

The K model is the governance and consultation architecture. Its core cycle has four steps: Exploration, Reflection/Analysis, Decision-making/Recommendation, and Feedback/Evaluation. Exploration does not merely inventory a tool. The team maps the actual work route, the groups affected, the data types and the points at which responsibility or approval becomes unclear. Reflection weighs benefits, dignity, protection, effort and possible power asymmetries together.

The decision sets out traceable roles, an initial data zoning, a safe reporting channel and available alternatives. Feedback/Evaluation asks later: Can employees find the approved route? Do they understand the boundaries? Can they report without unnecessary disclosure? Is there feedback for the data subject? Which cases remain unresolved? A percentage figure alone would not answer these questions. The feedback loop must be allowed to change the decision.

The model materials also include a five-stage K case-scenario extension, adding Implementation/Pilot and Final Evaluation/Optimisation after the decision. It must be kept separate from the four-step core cycle. For a small team, it can be useful where the new reporting route is first tested in a clearly bounded work area. A pilot then shows whether procedures work and which questions arise anew. It does not prove that all departments or affected groups were reached.

R: hand a specific specialist problem to the appropriate expertise

The R consultation model is a separate, five-phase specialist route. It becomes relevant when the general question turns into a specific point: What role does the company have in this data flow? Which contract applies to the account used? Is a data protection impact assessment required? Which information obligation needs to be assessed in the specific case? R begins with preparation and assessment, analyses gaps and risks, designs a solution with qualified expertise, plans implementation and later reviews the changes.

An AI system must not itself claim the applicability of law from a rough description. The R handoff should contain a concise description of the facts, open questions, missing documents and relevant roles. Once a specialist has assessed the conditions, the results return to K so that the governance decision can be adjusted. R is neither a fifth step of K nor a guarantee that an organisation is “GDPR-compliant.”

Data zones as an initial, understandable guide

A small team needs a rule it can recognise in the work situation. The following zones are a locally adaptable starting model. They are not legal categories. They can be set more strictly depending on the sector, data types, contracts, jurisdiction and technical environment.

The colours are design aids, not a risk assessment. A public text can still have copyright or contractual limits. Internal data can become identifying through their combination. Deleting a line with a name does not automatically anonymise a rare story. Where classification is unclear, the higher protection level applies until a responsible role has clarified the question.