Agentic Architecture

This document describes di-factory's reference architecture for agentic AI systems in production. It is the design we deploy when a client asks the honest question: "can we let AI actually do things in our stack — call our APIs, update our records, close our tickets — without breaking anything?" Twelve stages, grouped into four phases, cut by two cross-cutting planes and one Governance band. It runs on any channel the client already uses (WhatsApp, Telegram, Web, Voice, Email, API), calls any tool the client can expose (REST or MCP), and does not lock the client into a single model or a single cloud.

The audience is technical. If you are evaluating whether di-factory can put a grounded, multi-tenant, MX-compliant agentic system into production on your stack — or whether "agentic" in our vocabulary means the same thing it means in yours — this is the document.

Design principles

Four commitments shape every decision below.

1. Actions are effectful, therefore verified. Every state-changing tool call passes a verify checkpoint before it reaches the outside world. Agentic systems don't just answer wrong — they do things wrong. Wire an unverified agent into your ERP and the failure mode isn't a hallucinated paragraph, it's a duplicate invoice. Stage 11's verify pass and the retry loops at stages 9–10 are load-bearing, not optional.

2. Model-agnostic. The reasoning engine at stage 5 is a configuration parameter, not a hardcoded dependency. Claude, GPT, Gemini, self-hosted Llama, Mistral, Qwen — swap is a config change, not a code change. Same rule as our RAG architecture. Same reason: pricing shifts, rate limits bite, compliance forces on-prem, a new release beats the incumbent on your eval set. The client keeps the choice.

3. Tools are contracts, not code. Every tool the agent can call is declared as a JSON schema in a registry (MCP-native where possible), not hardcoded into the reasoning path. Adding a new capability — a SAT CFDI validator, a new CRM, a client's internal ERP endpoint — is a registry entry, not an agent redeploy. As more of the client's stack exposes MCP endpoints, the tool count grows without touching agent code.

4. Multi-tenant SaaS by default, single-tenant on request. Each tenant gets its own memory namespaces, its own tool registry, its own config, its own observability slice, its own retention policy. Adding a tenant is a namespace and a config file — not a deployment. Clients who need dedicated infrastructure (regulated banking, health) get single-tenant deployments; the code path is identical.

The 12 stages

The system is a request pipeline organized into four phases — SENSE (ingest & understand), THINK (reason, plan, orchestrate), ACT (take real-world actions), and VERIFY & SHIP (check, explain, deliver). A channel event comes in from the left; a verified, cited response with any state-changes committed goes out on the right. Two feedback loops cut back from ACT and VERIFY & SHIP to THINK — retry, self-correct, re-plan.

Phase 1 — SENSE (ingest & understand)

Stage 1 — Input. API, webhook, voice, document, event trigger. Channel adapters (FastAPI · LiteLLM · Whisper) normalize inbound payloads into a canonical request envelope. This is where PII redaction begins — the adapter strips CURPs, RFCs, and account numbers before the request enters the reasoning path, replacing them with reversible tokens. The Governance band re-applies the original values only at stage 12 render, and only after verify signs off. This means the reasoning model never sees raw PII, which is what CNBV asks for when they ask about "data minimization."

Stage 2 — Safety & Access Checks. Permissions, content filters, PII detection, rate limits (Guardrails AI · NeMo · OPA · Presidio). Before the request enters the reasoning path, the safety gate confirms: does this user have permission to invoke this intent on this tenant? Is the input free of prompt-injection patterns? Are we inside the rate envelope? A request that fails safety never reaches stage 5 — it returns a scoped refusal, and it produces a trace regardless. Safety failures are observable events, not silent drops.

Stage 3 — Understanding Layer. Intent classification, entity extraction, structured-output shaping (Instructor · Outlines · spaCy · Pydantic). This is where we short-circuit. A "hola / horarios / dónde queda" hits an FAQ shortcut and skips stages 4–12 entirely. Only requests that require reasoning enter the full pipeline. This is the single largest driver of cost control at scale — 60–80% of traffic in typical banking / retail deployments never touches an LLM.

Phase 2 — THINK (reason, plan, orchestrate)

Stage 4 — Memory. Short-term (per-session) plus long-term (RAG-backed) memory (Redis · Postgres/pgvector · Qdrant). Short-term holds what was said in this conversation; long-term holds what is true about the tenant's corpus and past history. Memory is per-user, per-tenant, with a configurable TTL. Conflating short-term with long-term is how agentic systems start inventing facts — we don't. When long-term memory is the primary answer surface and no tools are involved, the architecture reduces to our RAG reference; when tools are involved, memory is one input among several.

Stage 5 — Reasoning (Model-Agnostic). This is where the LLM lives, and where the "no lock-in" story is a real design decision, not marketing copy. Provider is a config parameter, exposed at the tenant level. The prompt shape, tool schema, and response contract are provider-agnostic. Provider swaps happen in production without redeploy — Claude, GPT-4o, Gemini, Llama 3.3, Mistral Large, Qwen, DeepSeek, whatever the client's policy or budget allows this quarter. We say model-agnostic. We do not say the models are equivalent — the abstraction layer at stage 5 owns the translation cost so you don't.

Stage 6 — Decision Router. Simple? Answer directly. Complex? Plan (LangGraph state · DSPy routing). Not every reasoning turn needs a plan — most single-turn questions resolve in stage 5. The router keeps the fast path fast, and reserves the planner for genuinely multi-step work. Getting this split wrong is how "chatbot with plugins" systems end up 4× slower and 10× more expensive than they need to be.

Stage 7 — Planning Module. Task decomposition, DAG construction, dependency resolution (LangGraph · CrewAI · AutoGen). For any request that requires more than a single retrieve-and-answer turn, the reasoning model emits a task plan: an ordered (or DAG-shaped) list of tool calls, sub-queries, or downstream actions. The plan is a first-class artifact — persisted, inspectable, replayable. If the client asks "why did the agent do it in this order?" we answer with a plan ID, not a shrug.

Stage 8 — Task Queue. Prioritize, retry, durable state (Temporal · Celery · Redis Q). The persistence layer for planned tasks. Default is a Postgres-native message queue with lightweight workers for the ~90% of deployments where throughput is measured in requests-per-second, not thousands. The scale path is Temporal or Celery + Redis when a specific tenant genuinely needs it. We choose the boring option by default because we run these systems, and every extra piece of infrastructure is another 3AM page.

Phase 3 — ACT (take real-world actions)

Stage 9 — Action / Tool Layer. API calls, database queries, code execution, SaaS integrations (LangChain tools · LlamaIndex tools · custom Python). Tools get called. Results come back. This is where the first feedback loop lives: if a tool call fails, times out, or returns an unexpected shape, the executor re-enqueues the task back to stage 8 with an incremented retry counter and a failure reason attached to the trace. The reasoning model at stage 5 sees the failure on the next tick and can either retry with different parameters, fall back to a different tool, or escalate to human. Tool retry is not exception handling bolted on top — it's an architectural loop.

Stage 10 — MCP (Model Context Protocol). File system, GitHub, Slack, Jira, client's ERP/CRM/core system (Anthropic MCP SDK · FastMCP servers). MCP is the durable interface — as more of our clients' internal systems expose MCP endpoints, the tool count at stage 7's registry grows without touching agent code. This is the second feedback loop: MCP call failures also re-enqueue at stage 8. The pattern is identical to stage 9's tool retry; the difference is what's on the other end of the wire (an internal service vs. an external API).

Phase 4 — VERIFY & SHIP (check, explain, deliver)

Stage 11 — Result Check + Explainability. Goal met? Retry? Self-correct? LLM-as-judge? Human checkpoint? (DSPy assertions · Guardrails validators · Captum · SHAP · InterpretML). Every state-changing action passes verify before it commits. Every factual claim in the draft response is traced back to an evidence chunk from memory or a tool result. Hallucinated numbers, dates, entities are caught here — not by the user. The explainability sub-band persists which evidence backs which claim alongside the response. If the client asks "why did the agent say X?" we answer with citation IDs, not adjectives. If verify fails, the retry loop cuts back to Phase 2 THINK — this is the self-correct arrow on the diagram.

Stage 12 — Output. Formatted response, streamed to user or downstream system (FastAPI streaming · SSE · Jinja templates). PII tokens are un-tokenized where authorized. The trace ID, evidence chunk IDs, tool call IDs, plan ID, and verification verdict are stored — always — for audit. Human escalation is a first-class branch here, not an error path: if verify rejected the answer, if the user asked for a human, if confidence is low, or if the query hit a policy-restricted topic (medical diagnosis, legal advice, financial recommendation on a retail channel), the conversation hands off — with the full trace attached — to a human agent through the client's existing ticketing / helpdesk system.

The 3 cross-cutting planes

Observability. Every request produces a trace: input, intent, memory hits, retrieved chunks with rerank scores, reasoning tokens (in/out), tool calls made with args and results, verify verdict, latency per stage, cost per stage (Langfuse · OpenLLMetry · Arize Phoenix · Grafana · Prometheus). Traces are queryable — by tenant, by intent, by outcome, by cost. This is not "we have logs." This is "we can answer why did this specific request cost 40 cents and take 8 seconds" in one query. Without this plane, the system is uncontrollable in production. Also: quality metrics (tool success rate, verify pass rate, escalation rate, cost per resolved intent) roll up here so we can tell whether the agent is getting better or worse over time.

Configuration. Every stage exposes its knobs as config: intent taxonomy, embedding model, top-K, rerank threshold, LLM provider, temperature, planner depth, tool timeouts, retry counts, verify strictness, PII policy, escalation policy. Config is versioned, per-tenant, and hot-reloadable. A tenant onboarding is a config file. A model swap is a config change. A policy update is a config change. Adding a tool is a registry entry. If it required a redeploy, we'd be redeploying weekly. We don't redeploy weekly.

Data Governance & Compliance. LFPDPPP (Ley Federal de Protección de Datos Personales), CNBV cir. 20/2021 (data protection for regulated financial entities), NOM-024 (health data interchange), IFT (telecoms). The band runs the length of the pipeline — every stage that touches user data touches the governance plane on the way through. PII detection and redaction at stage 1. Access control on retrieval and tool sources at stages 4, 9, 10. Provider-region enforcement at stage 5 (a CNBV tenant cannot route to a US-hosted provider without an explicit override). Immutable audit log across all stages. This is not a compliance appendix — it is the reason we win in regulated MX verticals. Most reference architectures were designed for a jurisdiction that isn't ours.

The Feedback Loop — verify failures, tool retries, user-facing 👍/👎 at stage 12, and escalation outcomes — feeds back into three places: prompt evolution (which system prompts under-perform per intent), planner tuning (which task shapes over-decompose or under-decompose), and tool registry cleanup (which tools fail silently or return unusable shapes). This is what makes the system get better week over week instead of decaying.

Where it plugs into your stack

Three integration surfaces.

Inbound (channels). Stage 1's channel adapter is parametric. WhatsApp Business API, Telegram Bot API, web widget, voice (SIP → STT), REST endpoint, or an existing helpdesk (Zendesk, Freshdesk, ServiceNow). If you already have a channel layer, we consume from it. If you don't, we run one.

Outbound (tools). Stages 9 and 10. Any REST- or MCP-exposed system on your side is a candidate tool. We do not require access to your primary databases. We require access to the specific tools we call, on the scopes we call them with. New tools are a registry entry, not a redeploy.

Outbound (escalation). Stage 12's human handoff plugs into your existing ticketing / helpdesk system. We do not replace your agent tooling — we hand off to it, with full conversation context, plan, tool trace, and verify verdict attached.

Each tenant gets isolated memory namespaces, isolated tool registries, isolated config, isolated traces, and isolated retention policy. Adding a tenant is a config file plus a namespace. Multi-tenancy is a first-class primitive.

Business cases we deploy this for

Same architecture, different intent set, different tool registry, different escalation policy. Six shapes we see repeatedly:

• Service-desk agent that acts. WhatsApp/Web front, ticket + product-doc corpus, tool registry = (create-ticket, check-order-status, issue-refund up to a policy limit, update-address, escalate). RAG answers what; this executes do. Deflection climbs past 60% because the second-tier "OK but can you actually fix it?" no longer needs a human.
• Sales-ops agent. Web/CRM-embedded front, prospect + case-study corpus, tool registry = (LinkedIn intel, CRM update, calendar hold, proposal-doc scaffolding, send-outreach-draft-for-review). Moves the SDR from "8 hours of research per lead" to "8 minutes of review before hitting send." All send actions pass verify + human approval.
• Financial ops (invoicing / reconciliation). Email/API/portal front, invoice + policy corpus, tool registry = (SAT CFDI validation, ERP posting, exception-flag, escalate). Every state-changing action passes verify before it commits. Duplicate invoices are the recurring nightmare here — verify catches them before they hit the ledger.
• HR self-serve + workflow. WhatsApp/Web front, policies + benefits + payroll FAQ corpus, tool registry = (PTO submission, benefits enrollment, expense pre-approval up to a limit, escalate to HRBP). PII redaction and access control are load-bearing — an employee cannot see another employee's record even through a prompt-injection attempt.
• IT / incident responder. PagerDuty/Slack front, runbooks + past-incident corpus, tool registry = (log query, service restart, config rollback, page on-call). Autonomous only for the classes of incident where the runbook explicitly authorizes it; everything else escalates with the investigation already done and the trace attached.
• Multi-agent orchestrator (research / due-diligence / M&A). Internal front, structured deliverable, tool registry = (search, extract, tabulate, compare, draft-memo, route-to-reviewer). One supervising agent, multiple worker agents at stages 6–7, one critic at stage 11. This is the shape our VC-partnership LOB is built around.

The architecture is the same. The corpus, the channel, the tool registry, the escalation target, and the policy are configuration.

What this architecture is not

Anticipating the questions we're asked in every CTO evaluation:

• This is not a RAG system. RAG answers questions from a corpus. Agentic acts — it calls tools, updates state, changes the outside world. The two compose (RAG becomes a memory source at stage 4 and a retrieval tool at stage 9), but they are different systems with different failure modes. When a client only needs "answer questions from our docs," we deploy the RAG architecture, not this one — same reason a scalpel isn't a chainsaw.
• This is not RPA. RPA scripts a fixed click-path. When the underlying UI changes, RPA breaks silently. The agent at stage 5 reasons about its tools, retries with different parameters when they fail, and asks for help when it doesn't know — it adapts to the shape of the failure, not to a static happy-path.
• This is not a chatbot with plugins. A "chatbot with plugins" fires tools directly from the LLM without a planner, without a verify pass, without a retry loop, and without observability. When it works it looks the same as this architecture. When it fails, no one can tell you why. This one, we can.
• This is not fine-tuned. We don't fine-tune the reasoning model on the client's corpus. Fine-tuning couples the client to a specific base model, and the model-agnostic contract at stage 5 is more valuable than the marginal quality gain. Where domain specificity matters, we invest in retrieval quality (chunk strategy, embedding choice, rerank) and tool design — the gains are larger and portable.
• This is not autonomous-fire-and-forget. Human checkpoints are load-bearing: verify can insert one, the escalation path always exists, and policy-restricted intents route to human by default. Autonomy is a per-intent, per-tenant configuration — not the default. The agent is a colleague on probation, not an operator with root.
• This is not multimodal-native — yet. Text-first by design. Voice comes in through STT, images through OCR at stage 1. Native multimodal reasoning (visual understanding inside stage 5) is a roadmap item, activated per-tenant when the use case requires it.

Latency and cost envelope

Realistic numbers, not marketing numbers.

• FAQ short-circuit (stage 3 intent hit, no reasoning): 150–400 ms end-to-end. This is the majority of traffic in retail / SMB deployments.
• Single-turn reasoning, no tools: 2–4 seconds. LLM inference dominates. Provider choice matters — hosted frontier model vs. self-hosted 70B has a 3–5× latency delta.
• Multi-hop agentic path (3–5 tool calls, plan → execute → verify): 8–20 seconds. This is workflow automation, not chat. Users tolerate it because the alternative was a 40-minute human handoff.
• Multi-agent orchestration (supervisor + workers + critic): 1–10 minutes; users perceive it as "working" because stage 12 streams progress updates instead of blocking.

Cost per request ranges from ~$0.001 (short-circuit, self-hosted embeddings) to ~$0.20 (frontier model, multi-hop, long context). The observability plane lets you enforce cost ceilings per intent, per tenant, per channel.

Closing

This is the architecture we deploy for agentic AI. It is opinionated. Each opinion is a scar. If any of the design principles conflict with your environment — you can't touch a hosted model, your compliance office rejects tenant-shared infrastructure, your latency target is sub-500 ms on the full pipeline, verify slows you down more than it saves you — we adjust the specific stage, not the shape. The shape is load-bearing.

For a technical deep dive on any single stage, or a scoping conversation on your specific tool registry and deployment envelope, the follow-up is a 30-minute call. Growgy (CGO) schedules it; Teky (CTO) runs it.