Good AI consulting starts with a defensible system assessment

Consultants who present solutions too early often advise an imagined organisation. A defensible assessment first reveals how work, data, systems, decisions and responsibility actually connect.

An initial AI workshop quickly produces a wish list: knowledge assistant, automated support, document analysis, agent team, forecasting and content production. A roadmap may seem possible after one hour. Yet the list describes opportunities, not a dependable system. Process variants, data quality, informal approvals, legacy software, worker participation, security boundaries and actual decision authority remain hidden.

Good discovery does not slow innovation. It prevents speed in the wrong direction. Its product is neither a transcript archive nor a colourful maturity slide, but a verifiable view of current state and a reasoned decision about which use case should proceed under which conditions.

Consulting begins with a hypothesis, not an answer

An engagement may start with hypotheses: „Finding valid work instructions causes rework" or „The handoff from customer request to expert review could be partly automated." A hypothesis is useful while it remains visibly provisional.

It is tested against several forms of evidence: interviews, process observation, system configuration, data samples, policies, logs, incident tickets and existing controls. No source is automatically authoritative. A documented process may differ from practice; an interview may omit exceptions; a log shows behaviour but not always professional intent.

The assessment record separates claim, evidence and judgement. „Everyone reviews the output" becomes defensible only when role, timing, criteria, recorded overrides and technical authority are visible. Where evidence is missing, the status is „unresolved," not „compliant."

Three maps describe the same organisation

A useful assessment builds at least three connected maps.

The work map shows triggers, steps, roles, waits, exceptions and decisions: how is an outcome produced today? The system and data map shows applications, interfaces, storage, sources, transformations, identities and flows: what technical reality supports the process? The governance map shows owners, approvals, policies, risks, control evidence, affected people and escalation: who may decide, who is accountable and how is impact controlled?

Separated, these maps are incomplete. A process step without a system link hides shadow IT. A database without work context does not explain which decision it influences. A policy without technical enforcement may be an aspiration. Shared identifiers connect process, system, dataset, use case, risk and control.

Interviews reveal knowledge and blind spots

Interviews are essential but not neutral. Leaders often describe the target process, practitioners the exceptions, and IT the officially supported systems. Staff may consider workarounds normal and never mention them. Consultants hear more easily what fits their preferred solution.

Questions are therefore role-specific. Process owners explain objectives and acceptance. Practitioners demonstrate real cases. IT and security explain integrations, identities and constraints. Privacy, legal, compliance and worker representatives identify review paths. Customer-facing or affected roles expose consequences.

Good questions request examples: „Show the last case in which the standard route failed." „Which input would make the result unusable?" „When was a recommendation last overridden?" „Which file is authoritative, and how can a system recognise its current version?" These questions produce inspectable objects instead of abstract agreement.

A data-flow map is more than arrows

A flow diagram becomes useful when every arrow has meaning. Where does data originate? Who controls quality? In which region and service is it processed? Which transformation occurs? What is logged? Which retention applies? Where does output go, and which action does it trigger?

Generative and agentic systems add system prompts, retrieval sources, embeddings, vector stores, tool parameters, model providers, caches, telemetry and human feedback. A field hidden in the interface may still appear in logs, traces or backups.

The assessment does not catalogue every byte. It prioritises data influencing purpose, decisions, security or rights. For sensitive or business-critical paths, provenance, authorisation, version, transfer, storage, deletion and possible disclosure remain reconstructable.

The actual decision path matters

Organisations like to describe AI as assistance. The word says little about real impact. A recommendation may be effectively binding when deadlines, interface design, targets or leadership expectations suppress deviation. Conversely, an automated step may be low impact where it is reversible, does not materially affect people and is reliably controlled.

The assessment marks every point at which output changes priority, access, price, service, selection, safety or communication. Each point records the decision owner, human authority, required information, possible bias, reversal and complaint path.

„Human in the loop" is not accepted as a control checkbox. The review asks whether that person has time, competence, information, independence and technical power to challenge and stop the output meaningfully.

Maturity without theatre

Maturity models can structure dialogue but easily create false precision. An organisation is not simply „level three." Data governance may be strong, model testing weak and incident management absent. An average score hides the exact gap that can sink a pilot.

A defensible assessment rates specific capabilities through observable criteria. Is there a complete system register? Are owners assigned? Are data sources versioned? Do acceptance tests exist? Can users report errors? Are changes reassessed? Is stopping technically possible? Is there evidence from real cases?

Each rating includes evidence, scope and confidence. „Partial" is explained. Uncertainty remains visible. The goal is not an attractive score but decisions about investment and controls.

Use cases are not ranked by value alone

A good portfolio separates problem, target group, expected value, decision impact, data readiness, technical feasibility, integration effort, risk, reversibility and evidence needs. Excitement and model novelty are not prioritisation criteria.

High value with weak data is not a quick win. Low risk without an owner is not a pilot. A small assistant can be expensive if it requires ten integrations. A technically ambitious case may still be suitable when it can be tested in a narrow, reversible and measurable scope.

Prioritisation is a portfolio decision. Some ideas are rejected, some require prerequisites, and few enter pilots. Every candidate gets a „why now," a „why not" and a stop criterion.

From assessment to controlled pilot

A pilot is not miniature production. It is an experiment with decision value. It tests defined hypotheses within a limited scope and produces evidence to continue, change or stop.

Before launch, teams define baseline, test cases, quality metrics, risk thresholds, user group, data scope, duration, budget and owners. Critical failures, group differences, false confidence, user behaviour and human-review effort matter alongside average quality. A successful demo on selected examples is insufficient.

Go-live is a separate gate. Architecture, privacy, security, operations, training, support, monitoring and rollback must be ready before a pilot becomes a service. The decision is recorded; residual uncertainty does not disappear behind a green indicator.

AI literacy is contextual

Article 4 of the AI Act requires measures for sufficient AI literacy in light of knowledge, experience, deployment context and affected people. One generic introduction does not automatically meet every need.

Roles require different capabilities. Users need limitations, data rules and escalation. Process owners need evaluation and control skills. Developers need testing, security and documentation practice. Leaders must govern risk acceptance, resources and accountability. Control functions need access to technical evidence.

The assessment therefore doubles as a learning-needs analysis. Training is linked to real decisions, failure modes and tools and verified through exercises, not attendance certificates alone.

The boundary of good consulting

AI consulting connects technology, organisation and governance without pretending to replace missing expertise. It can document systems and flows, identify duty interfaces, collect evidence and prepare questions for legal, privacy, security and sector specialists. Binding professional judgement remains with the competent role.

That boundary increases value. Legal review becomes faster when purpose, roles, data, decision points, provider information and open questions arrive in a structured package. Security can assess concrete attack surfaces rather than product names. The organisation receives decisions instead of disconnected documents.

Conflicts of interest must also be visible. A party selling a particular tool should not frame its selection as a neutral assessment. Assumptions, methods, limitations and commercial relationships belong in the engagement and result documentation.

The result is a decision and implementation package

A defensible assessment ends with a connected package:

1. System and use-case register with owners and status.

2. Linked work, data and governance maps.

3. Evidence register with facts, sources, gaps and confidence.

4. Risk and control register with accountable owners.

5. Prioritised use-case portfolio with rejection reasons.

6. Pilot charter with baseline, tests, boundaries and gates.

7. Specialist questions and handoff packages.

8. Implementation roadmap with dependencies and reviews.

The organisation can then make a reasoned decision. It knows not only which ideas sound attractive, but which state exists, which evidence is missing, which control must come first and which next step is reversible.

Worksheet: Conduct a mini system assessment

Choose a real or proposed AI use case.

1. State three hypotheses about problem, value and risk.

2. Identify interview roles and request one real example from each.

3. Draw work, system/data and governance maps.

4. Link every important claim to evidence or a gap status.

5. Mark decision points and actual human authority.

6. Rate data readiness, feasibility, impact and reversibility separately.

7. Define pilot baseline, tests, stop criteria and go-live gate.

8. Build a handoff package for at least one specialist function.

All materials to download — the topic overview and the worksheet:

Scope: This article presents a cross-functional method for discovery, system assessment and implementation planning. It does not replace legal, data-protection, information-security, worker-participation or sector-specific review. Standards and frameworks provide orientation; their use or certification is not automatically a legal requirement. Editorial review date: 16 July 2026.