AI system design: from process map to production

The most important work in an AI system happens before a model call. It is deciding what the system should own, which information it may use, how its output will be judged and what happens when confidence is low.

I treat AI implementation as systems design rather than prompt writing. A production result normally combines ordinary software, business rules, integrations, a model, an interface and an operating process. The model handles the parts where language or flexible interpretation creates value. Everything around it makes that capability safe, repeatable and useful.

This is the thought process I use, presented as a full implementation path.

Phase 1: find work worth changing

Start by observing a process, not collecting AI ideas. Ask a team member to complete the job with a real case. Record where they search, copy, wait, compare, decide, ask for approval and repair errors. The official procedure rarely contains every step; the workaround reveals the actual system.

Good candidates tend to have four properties. They happen often enough to matter. They consume skilled attention in reading, classification or coordination. The quality of the output can be inspected. A failure can be contained.

Do not begin with a process whose desired result nobody can define. If two leaders disagree about what qualifies a lead, an AI score will automate the disagreement. Resolve the policy or design the system to surface the decision rather than pretending it is objective.

Estimate the baseline. Measure cases per period, active handling time, waiting time, rework, error types and business impact. The estimate can be rough, but it gives the system a reason to exist. “We built an agent” is not a result. “Review time fell while factual correction stayed below the agreed threshold” is.

Phase 2: model the decisions

Draw the process as states and transitions. What causes a case to enter? What information is required? Which states are reversible? Which action changes an external record? Who owns an exception?

Then label each decision:

• Rule: exact conditions produce an exact result. Use code or workflow logic.
• Lookup: information must be retrieved from an approved source. Use a database query, resource or tool.
• Interpretation: meaning must be inferred from language, images or messy input. A model may help.
• Judgement: context and responsibility matter. The model can prepare evidence, but a person may own the decision.

This classification is a secret ingredient of reliable systems. It prevents the model from becoming the default answer to every box in the diagram. A deterministic validator is cheaper and more dependable than asking a model whether a date matches a format.

Define uncertainty explicitly. What missing input should stop the process? When may the system ask a question? When should it produce a draft with a warning? When must it route to a human? The user experience around uncertainty often determines whether people trust the system.

Phase 3: design the data boundary

List every data source and field the system would like to use, then remove what it does not need. Identify personal, confidential, financial and regulated information. Document where data travels, which provider processes it, how long logs retain it and how deletion works.

Choose a source of truth for each field. A company name may appear in CRM, billing and a spreadsheet, but one system needs ownership. When sources disagree, the workflow should follow a rule or surface the conflict. Letting the model quietly choose whichever value sounds plausible creates invisible data corruption.

Convert unstructured inputs into a stable internal schema early. Keep the original reference, extraction status and confidence or warning. Downstream steps should not repeatedly parse the same document. Schema validation at boundaries catches failures close to their cause.

Treat retrieved text as untrusted. A document, website or CRM note can contain instructions, accidental or malicious, that conflict with the system’s role. The model should know it is evidence, not authority. Tool permissions and action rules must be enforced outside that text.

Phase 4: choose the minimum architecture

Use the smallest architecture that meets security, volume and operational needs.

A simple classified inbox may be a managed automation calling a model with structured output and placing uncertain cases in a review queue. A research assistant may be a focused web application, an MCP client and several read-only tools. A high-volume document system may need queues, workers, separate storage and detailed tracing.

Static websites on cPanel should remain static. They can host the public interface and small PHP forms, but they should not run a persistent agent or orchestration service. Sensitive AI work belongs in a managed automation platform, serverless function or properly operated application environment. This separation keeps the marketing site fast and the execution boundary governable.

Avoid adding a vector database simply because the project uses AI. Use one when semantic retrieval across a changing corpus is genuinely needed and exact filters alone are insufficient. For a small set of current policies, a controlled document store with keyword or direct addressing may be simpler and easier to verify.

Phase 5: define tools as product interfaces

Whether tools use MCP, function calling or internal APIs, design them for the business job. A good tool name expresses one action. Its description says when to use it and what it will not do. Inputs have types, limits and stable identifiers. Outputs distinguish results from warnings and errors.

Prefer get_campaign_performance_summary to query_warehouse. Prefer create_draft_contact_note to update_crm_object. The narrow tool enforces approved fields and gives operators a meaningful log.

Separate reads from writes. Make mutations idempotent. Require an approval token for high-impact changes. Provide a dry-run operation that returns the proposed effect. Timeouts and retry behaviour should match the action; a timed-out email send must be checked before retrying because it may already have succeeded.

Tool descriptions are part of model behaviour, but tool code owns policy. The description can say “use only after approval”. The server must still verify approval.

Phase 6: create the evaluation before the polished interface

Collect cases that represent the real distribution, including difficult edges. Remove or synthesise sensitive details when a safe development dataset is needed. For each case, define expected facts, required structure, prohibited claims, allowed tool calls and escalation behaviour.

Evaluation has layers:

1. Schema tests confirm valid structured output.
2. Deterministic assertions check identifiers, calculations, permissions and required sources.
3. Rubrics assess relevance, completeness and clarity.
4. Human comparison checks whether the result supports the actual work.
5. Adversarial cases test injection, data leakage, ambiguous instructions and tool misuse.

Use a baseline. Compare the system with the current manual output or a simple non-agentic approach. A more complex agent should earn its complexity through a meaningful quality or flexibility gain.

Store model, prompt, tool and dataset versions with results. Re-run the set after any significant change. A model upgrade is a software change, not a neutral switch.

Phase 7: prototype the riskiest assumption

Do not build the full interface first. Identify what could make the project fail: poor extraction from source documents, insufficient API access, ambiguous matching, unacceptable correction rate or latency. Build a thin experiment around that risk.

Use representative inputs and the intended constraints. A prototype with hand-cleaned data and administrator credentials proves little about the production path. Measure quality and failure, not how impressive the ideal example looks.

At the end of the prototype, make a decision. Continue, narrow the scope, change architecture or stop. Stopping a weak idea after a focused test is a successful outcome compared with operating an unreliable system.

Phase 8: engineer the surrounding workflow

Once the core assumption holds, build the deterministic frame. Validate input. Retrieve context. Call the model with a clear schema. Validate output. Apply business rules. Route uncertainty. Request approval. Execute allowed actions. Record status.

Each state should be visible to an operator. Use durable identifiers so retries resume rather than duplicate work. Store the minimum data necessary for debugging and reporting. Secrets stay in credential stores. Development, staging and production use separate connections.

Add a dead-letter or exception queue. A failed case should include category, safe diagnostic context, owner and next action. Do not rely on someone opening a workflow history to notice. Define service expectations for the queue so automation does not merely create a new hidden backlog.

Build the manual fallback too. If the model provider is unavailable, can a person complete urgent cases? If the CRM is down, are approved actions queued safely? Resilience is part of the workflow, not an infrastructure afterthought.

Phase 9: design the human interface

The interface should make responsibility easy. Show the source evidence, generated result, warnings and proposed action together. Distinguish model-generated text visually from verified system fields. Let the reviewer edit, approve, reject or ask for missing information.

Capture rejection reasons in categories that can improve the system. Free-text comments are useful, but a category such as wrong source, unsupported claim, missed exception or poor tone can be aggregated. Do not measure approval rate alone; a reviewer who rubber-stamps everything creates a misleading success signal.

Accessibility and speed matter. Keyboard navigation, clear focus, useful error messages and a responsive review screen improve both inclusion and operating throughput. The AI may be sophisticated, but users judge the whole system through this surface.

Phase 10: release by increasing authority

Release to a small, informed group. Begin with observation or read-only assistance. Then allow draft creation. Add approved writes only after quality and permission controls hold. Autonomous low-risk actions can come last, if the measured benefit justifies them.

This staged authority model is more useful than calling a system “human in the loop” without defining the loop. Specify which action needs which role, what evidence is shown and how approval expires.

Set volume and cost caps during early operation. Monitor latency, provider errors, tool failures, correction categories and business outcome. Keep a mutation kill switch. Review a sample of successful cases, because failures are not the only place quality drifts.

Phase 11: operate it as a product

Assign an owner for quality, permissions, source freshness and incident response. Maintain a change log. Review unused tools and stale credentials. Re-run evaluation sets on a schedule and after provider changes.

Watch for process change. Teams adapt around a system, source fields evolve and product policy moves. A prompt that was correct six months ago can now encode an obsolete rule. Connect important policies to maintained resources rather than copying them permanently into instructions.

Measure the complete outcome: active time saved, elapsed time, correction, exception backlog, user adoption, customer effect and cost per completed case. An AI call can be cheap while the review process is expensive. Optimise the system, not the token bill in isolation.

Composite example: a multi-system campaign briefing app

Consider a team that prepares regional campaign briefs. The manual process involves CRM account data, analytics, product documents, market notes and a template. The work takes hours and quality depends on who remembers which source.

The redesigned system uses an internal form for region, segment and objective. Deterministic code validates inputs. Read-only tools retrieve selected CRM and analytics summaries. Approved resources provide current product and market context. A model synthesises a structured brief with source references and open questions. Rules reject unsupported numerical claims. A regional reviewer resolves warnings and approves the result. A workflow stores the approved brief and creates tasks, but it never launches a campaign automatically.

The model is one part. The value comes from making the sources, structure, validation and handoff consistent. The speed can exceed manual research dramatically because retrieval and synthesis run together. The system remains accountable because a reviewer sees evidence and owns publication.

The implementation principles that matter most

The strongest AI systems I have worked on follow a small set of principles:

• begin with observable work and a measurable outcome;
• use deterministic logic wherever interpretation is unnecessary;
• minimise data and permissions;
• make tools narrow, typed and observable;
• evaluate difficult cases before expanding authority;
• show evidence and uncertainty to the user;
• design retries, exceptions and manual fallback from the start;
• release gradually and operate the system as a maintained product.

The model will improve. APIs and protocols will evolve. These principles endure because they address the harder problem: turning flexible machine intelligence into dependable organisational capability.