Good AI consulting starts with a defensible system assessment

Consultants who present solutions too early often advise an imagined organisation. A defensible assessment first reveals how work, data, systems, decisions and responsibility actually connect.

An initial AI workshop quickly produces a wish list: knowledge assistant, automated support, document analysis, agent team, forecasting and content production. A roadmap may seem possible after one hour. Yet the list describes opportunities, not a dependable system. Process variants, data quality, informal approvals, legacy software, worker participation, security boundaries and actual decision authority remain hidden.

Good discovery does not slow innovation. It prevents speed in the wrong direction. Its product is neither a transcript archive nor a colourful maturity slide, but a verifiable view of current state and a reasoned decision about which use case should proceed under which conditions.

Consulting begins with a hypothesis, not an answer

An engagement may start with hypotheses: „Finding valid work instructions causes rework" or „The handoff from customer request to expert review could be partly automated." A hypothesis is useful while it remains visibly provisional.

It is tested against several forms of evidence: interviews, process observation, system configuration, data samples, policies, logs, incident tickets and existing controls. No source is automatically authoritative. A documented process may differ from practice; an interview may omit exceptions; a log shows behaviour but not always professional intent.

The assessment record separates claim, evidence and judgement. „Everyone reviews the output" becomes defensible only when role, timing, criteria, recorded overrides and technical authority are visible. Where evidence is missing, the status is „unresolved," not „compliant."

Three maps describe the same organisation

A useful assessment builds at least three connected maps.

The work map shows triggers, steps, roles, waits, exceptions and decisions: how is an outcome produced today? The system and data map shows applications, interfaces, storage, sources, transformations, identities and flows: what technical reality supports the process? The governance map shows owners, approvals, policies, risks, control evidence, affected people and escalation: who may decide, who is accountable and how is impact controlled?

Separated, these maps are incomplete. A process step without a system link hides shadow IT. A database without work context does not explain which decision it influences. A policy without technical enforcement may be an aspiration. Shared identifiers connect process, system, dataset, use case, risk and control.

Interviews reveal knowledge and blind spots

Interviews are essential but not neutral. Leaders often describe the target process, practitioners the exceptions, and IT the officially supported systems. Staff may consider workarounds normal and never mention them. Consultants hear more easily what fits their preferred solution.

Questions are therefore role-specific. Process owners explain objectives and acceptance. Practitioners demonstrate real cases. IT and security explain integrations, identities and constraints. Privacy, legal, compliance and worker representatives identify review paths. Customer-facing or affected roles expose consequences.

Good questions request examples: „Show the last case in which the standard route failed." „Which input would make the result unusable?" „When was a recommendation last overridden?" „Which file is authoritative, and how can a system recognise its current version?" These questions produce inspectable objects instead of abstract agreement.

A data-flow map is more than arrows

A flow diagram becomes useful when every arrow has meaning. Where does data originate? Who controls quality? In which region and service is it processed? Which transformation occurs? What is logged? Which retention applies? Where does output go, and which action does it trigger?

Generative and agentic systems add system prompts, retrieval sources, embeddings, vector stores, tool parameters, model providers, caches, telemetry and human feedback. A field hidden in the interface may still appear in logs, traces or backups.

The assessment does not catalogue every byte. It prioritises data influencing purpose, decisions, security or rights. For sensitive or business-critical paths, provenance, authorisation, version, transfer, storage, deletion and possible disclosure remain reconstructable.

The actual decision path matters

Organisations like to describe AI as assistance. The word says little about real impact. A recommendation may be effectively binding when deadlines, interface design, targets or leadership expectations suppress deviation. Conversely, an automated step may be low impact where it is reversible, does not materially affect people and is reliably controlled.

The assessment marks every point at which output changes priority, access, price, service, selection, safety or communication. Each point records the decision owner, human authority, required information, possible bias, reversal and complaint path.

„Human in the loop" is not accepted as a control checkbox. The review asks whether that person has time, competence, information, independence and technical power to challenge and stop the output meaningfully.

Maturity without theatre

Maturity models can structure dialogue but easily create false precision. An organisation is not simply „level three." Data governance may be strong, model testing weak and incident management absent. An average score hides the exact gap that can sink a pilot.

A defensible assessment rates specific capabilities through observable criteria. Is there a complete system register? Are owners assigned? Are data sources versioned? Do acceptance tests exist? Can users report errors? Are changes reassessed? Is stopping technically possible? Is there evidence from real cases?

Each rating includes evidence, scope and confidence. „Partial" is explained. Uncertainty remains visible. The goal is not an attractive score but decisions about investment and controls.