Conversational RAG — Parametric 1:1 Channel Assistant Runtime

This document describes di-factory's reference architecture for Retrieval-Augmented Generation systems in production. It is the design we deploy when a client asks the honest question: "we have a lot of documents — how do we let people ask them questions and trust the answer?" Ten stages, four clusters, three cross-cutting planes. It runs on any channel the client already uses (WhatsApp, Telegram, Web, Voice), ingests any file type the client can produce, and does not lock the client into a single model or a single cloud.

The audience is technical. If you are evaluating whether di-factory can put a grounded, multi-tenant, MX-compliant RAG system into production on your stack — or whether "RAG" in our vocabulary means the same thing it means in yours — this is the document.

Design principles

Four commitments shape every decision below.

1. Grounded, not generative. Every user-facing answer traces back to a retrieved chunk. If retrieval finds nothing above threshold, the system says so — it does not improvise. This is the difference between a RAG system and a chatbot with a knowledge disclaimer.

2. Model-agnostic. The generation engine at stage 9 is a configuration parameter, not a hardcoded dependency. Claude, GPT, Gemini, self-hosted Llama, Mistral — swap is a config change, not a code change. Same rule as our agentic architecture. Same reason: pricing shifts, rate limits bite, compliance forces on-prem, a new release beats the incumbent on your eval set. The client keeps the choice.

3. Corpus is a living system, not a one-shot import. Documents get added, updated, retired, republished. Retrieval quality depends on the index reflecting reality today, not the state of the world on ingestion day. The Corpus Management plane treats versioning, deletion propagation, and re-indexing as first-class operations — because in every deployment we've done, the second-worst failure mode is "the answer is confidently wrong because the source got replaced six months ago."

4. Multi-tenant SaaS by default, single-tenant on request. Each tenant gets its own corpus namespace, its own config, its own observability slice, its own retention policy. Adding a tenant is a namespace and a config file — not a deployment. Clients who need dedicated infrastructure (regulated banking, health) get single-tenant deployments; the code path is identical.

The 10 stages

The system has two request paths: an async ingestion path (Cluster A) that keeps the corpus fresh, and a real-time query path (Clusters B → C → D) that answers user questions. The stages describe each path in order.

Cluster A — Corpus (async ingestion side)

Stage 1 — Multi-format Ingest. PDF, DOCX, XLSX, PPTX, HTML, plain text, images with OCR, audio with STT. Plus first-class connectors for Google Drive, SharePoint, Notion, Confluence, S3, and internal document stores over API. Ingest is event-driven — a new file lands, a webhook fires, or a scheduled sync detects a delta — and each ingestion event carries provenance (source, author, timestamp, tenant, permission scope) that propagates all the way to the citation at stage 10. If we can't tell the user where an answer came from, we won't show them the answer.

Stage 2 — Parse & Chunk. Layout-aware parsing (tables stay tables, headings stay headings, footnotes attach to their paragraph) followed by semantic chunking — not fixed-size windows. Chunk boundaries respect document structure because retrieval quality collapses when a table row gets split across two chunks or a citation gets orphaned from the sentence it modifies. This is the single most under-invested stage in most "our RAG doesn't work" audits we run.

Stage 3 — Embed & Index. Embeddings are computed and written to the vector store with the chunk, its metadata, its provenance, and its tenant namespace. Default embedding model is bge-large or bge-m3 — self-hosted, multilingual (Spanish-first), no per-token cost. Managed alternatives (Bedrock Titan, Vertex, OpenAI) are per-tenant config for clients whose data can leave the VPC and whose ops team prefers managed. The Corpus Management plane handles versioning here: when a source document changes, the old chunks are marked deprecated (not deleted immediately) so citations already in-flight resolve, then garbage-collected on a schedule.

Cluster B — Query (real-time, per-request)

Stage 4 — Query Understanding. The user question — arriving from WhatsApp, Telegram, Web, or Voice via a normalized channel adapter — is classified, expanded, and reshaped for retrieval. Intent detection routes trivial queries (greetings, hours, escalation requests) to short-circuit paths that never touch retrieval. Query expansion (HyDE, multi-query, sub-question decomposition) is applied conditionally, not by default — expansion is expensive and hurts as often as it helps on short, well-formed questions. This stage also enforces out-of-scope detection: if the question is not answerable from the corpus, the system knows this before retrieval runs.

Stage 5 — Session Memory. Multi-turn conversation state, scoped to the current cycle. "El cliente que mencioné hace un rato" resolves. Follow-ups without repeated context work. Memory is per-user, per-tenant, with a configurable TTL (default: end of session, or 30 minutes of inactivity). Memory is not the corpus — memory holds what was said in this conversation, corpus holds what is true about the world. Conflating the two is how RAG systems start making things up.

Stage 6 — Hybrid Retrieval. Dense embedding search plus BM25 (keyword, sparse), fused. Dense-only retrieval fails on the exact terms that matter in Spanish enterprise corpora — product SKUs, MX legal citations (LFPDPPP art. 8), medical acronyms (NOM-024-SSA3), internal codenames. BM25 catches them. Dense catches the paraphrases BM25 misses. The fusion score outperforms either signal alone; we have not seen a production corpus where this generalization breaks.

Stage 7 — Re-rank & Filter. Cross-encoder rerank on the top ~50 hybrid candidates, yielding ~5–10 evidence chunks. Then permission filtering — a chunk the user is not authorized to see is dropped before it enters the prompt, not redacted after. Skipping rerank is the single most common failure mode in "our RAG doesn't work" audits. Top-K by cosine similarity is not the same as top-K by relevance, and the gap is usually the difference between "the answer is here somewhere" and "the answer is at position 1."

Cluster C — Generate

Stage 8 — Prompt Assembly. The reranked chunks, the session-memory context, the system prompt (tenant-specific, versioned), the user question, and the response contract (citation format, tone, length, safety policy) are assembled into the final prompt. Templates are declarative, versioned, and hot-swappable per tenant. When a client says "the answers are too long" or "never use the word 'ciertamente'" — that's a config change, not a redeploy.

Stage 9 — Generation (Model-Agnostic). This is where the LLM lives, and where the "no lock-in" story is a real design decision, not marketing copy. Provider is a config parameter, exposed at the tenant level. The prompt shape, tool schema, and response contract are provider-agnostic. Provider swaps happen in production without redeploy — Claude, GPT-4o, Gemini, Llama 3.3, Mistral Large, Qwen, DeepSeek, whatever the client's policy or budget allows this quarter. We say model-agnostic. We do not say the models are equivalent — the abstraction layer at stage 9 owns the translation cost so you don't.

Cluster D — Deliver

Stage 10 — Output & Feedback. The final response is formatted for the channel (WhatsApp markdown, Telegram HTML, Web streaming, voice TTS), citations are attached (source document, page/section, timestamp), and the response goes back to the user through the same channel adapter that received the question. Human escalation is a first-class branch here, not an error path: if retrieval returned nothing above threshold, if the user explicitly asked for a human, if the generation confidence is low, or if the query hit a policy-restricted topic (medical diagnosis, legal advice, financial recommendation on a retail channel), the conversation hands off — with the full context, chunks retrieved, and reason-for-escalation — to a human agent through the client's existing ticketing / helpdesk system. Feedback capture is also here: 👍/👎 or a "was this helpful?" prompt writes back to the Feedback Loop plane, which is what closes the improvement cycle.

The 3 cross-cutting planes

Observability. Every request produces a trace: input, intent, retrieved chunks with rerank scores, prompt tokens (in/out), generation cost, latency per stage, citation IDs, feedback verdict. Traces are queryable — by tenant, by intent, by outcome, by cost. This is not "we have logs." This is "we can answer why did this specific answer take 6 seconds and cost 4 cents" in one query. Without this plane, the system is uncontrollable in production. Also: retrieval quality metrics (hit@k, MRR, chunk-utilization rates) roll up here so we can tell whether the corpus is getting better or worse over time.

Configuration. Every stage exposes its knobs as config: connector list, chunk strategy, embedding model, top-K, rerank threshold, LLM provider, temperature, session TTL, escalation policy, feedback prompts, PII policy. Config is versioned, per-tenant, and hot-reloadable. A tenant onboarding is a config file. A model swap is a config change. A policy update is a config change. If it required a redeploy, we'd be redeploying weekly. We don't redeploy weekly.

Data Governance & Compliance. LFPDPPP (Ley Federal de Protección de Datos Personales), CNBV cir. 20/2021 (data protection for regulated financial entities), NOM-024 (health data interchange), IFT (telecoms). PII detection and redaction on ingest and on user input. Access control on retrieval sources — a chunk the user is not authorized to see is unreachable, not just unrendered. Provider-region enforcement at stage 9 (a CNBV tenant cannot route to a US-hosted provider without explicit override). Immutable audit log across all stages. This is not a compliance appendix — it is the reason we win in regulated MX verticals. Most reference architectures were designed for a jurisdiction that isn't ours.

The Feedback Loop — 👍/👎 at stage 10, plus escalation outcomes and low-confidence patterns — feeds back into three places: retriever tuning (which chunks over-retrieve for their utility), corpus flagging (which documents are wrong, stale, or ambiguous), and prompt evolution (which system prompts underperform per intent). This is what makes the system get better week over week instead of decaying.

Where it plugs into your stack

Three integration surfaces.

Inbound (channels). Stage 4's channel adapter is parametric. WhatsApp Business API, Telegram Bot API, web widget, voice (SIP → STT), REST endpoint, or an existing helpdesk (Zendesk, Freshdesk, ServiceNow). If you already have a channel layer, we consume from it. If you don't, we run one.

Inbound (corpus). Stage 1's connector list is parametric. Google Drive, SharePoint, Notion, Confluence, S3, direct upload, scheduled scrape, webhook push, or a database read. New connectors are a config entry, not a redeploy.

Outbound (escalation). Stage 10's human handoff plugs into your existing ticketing / helpdesk system. We do not replace your agent tooling — we hand off to it, with full conversation context and retrieved evidence attached.

Each tenant gets isolated corpus namespaces, isolated config, isolated traces, and isolated retention policy. Adding a tenant is a config file plus a namespace. Multi-tenancy is a first-class primitive.

Business cases we deploy this for

Same architecture, different corpus, different channel, different escalation policy. Six shapes we see repeatedly:

• Service desk (external customers). WhatsApp/Web front, product docs + FAQ + past-ticket corpus, escalation to human agent when confidence is low or the user asks for one. First-line deflection of 60–80% of repetitive tickets. This is the shape most first-time buyers ask for.
• Internal knowledge assistant. Slack/Teams front, wiki + policy PDFs + resolved-tickets corpus, no external escalation (routes to the internal SME instead). Cuts "where is the doc for X?" round-trips from hours to seconds.
• Sales enablement copilot. Web/CRM-embedded front, product catalog + case studies + pricing + contract templates corpus, escalation to solutions engineer for custom scoping. Prep time per proposal drops materially; proposals get more accurate.
• HR self-serve. WhatsApp/Web front, policies + benefits + PTO rules + payroll FAQ corpus, escalation to HRBP for personal-data queries. PII redaction and access control are load-bearing here — the wrong employee cannot see the wrong record.
• Technical documentation Q&A. Web/Slack front, engineering docs + runbooks + architecture decisions corpus, escalation to on-call. Onboarding acceleration for new engineers; incident-response speed for existing ones.
• Compliance / regulatory advisor. Internal-only front, regulation text (LFPDPPP, CNBV, NOM, IFT) + internal policy + prior interpretations corpus, mandatory human review on any answer used for a binding decision. Not a legal opinion — an evidence-first draft that a human can approve, edit, or reject in seconds instead of hours.

The architecture is the same. The corpus, the channel, the escalation target, and the policy are configuration.

What this architecture is not

Anticipating the questions we're asked in every CTO evaluation:

• This is not a fine-tuned model. We don't fine-tune the generation model on the client's corpus. Fine-tuning couples the client to a specific base model, and the model-agnostic contract at stage 9 is more valuable than the marginal quality gain. Where domain specificity matters, we invest in retrieval quality (chunk strategy, embedding choice, rerank) — the gains are larger and portable.
• This is not GraphRAG. For the corpora our clients bring (product docs, policy PDFs, ERP tickets, internal wikis), hybrid retrieval + rerank consistently outperforms graph-based retrieval at 1/10 the operational complexity. If a specific tenant has a genuinely graph-shaped corpus (drug interactions, org charts, dependency trees), we add graph as a supplementary tool. It's not the default.
• This is not agentic. RAG answers questions from a corpus. Agentic executes tasks by calling tools. When a client asks "can it also send the email / update the CRM / trigger the workflow?" — that is our agentic architecture, not this one. The two compose cleanly (RAG becomes a tool in an agent's stage-7 registry), but they are different systems with different failure modes.
• This is not multimodal-native — yet. Text-first by design. Images come in through OCR at stage 1, voice through STT. Native multimodal reasoning inside stage 9 (visual question-answering on document layout) is a roadmap item, activated per-tenant when the use case requires it.

Latency and cost envelope

Realistic numbers, not marketing numbers.

• Short-circuit path (out-of-scope, greeting, direct handoff): 100–300 ms end-to-end. No retrieval, no generation.
• Standard RAG path (hybrid retrieval + rerank + generation, ~5 chunks, no session context): 2–4 seconds. Generation dominates. Provider choice matters — a hosted frontier model vs. a self-hosted 70B has a 3–5× latency delta.
• Complex path (multi-query expansion, longer context, streamed generation): 5–10 seconds, of which the user perceives ~1 second (time-to-first-token) if streaming is enabled at stage 10.

Cost per query ranges from ~$0.001 (short-circuit, self-hosted embeddings) to ~$0.10 (frontier model, long context, expanded query). The observability plane lets you enforce cost ceilings per intent, per tenant, per channel.

Closing

This is the architecture we deploy for RAG. It is opinionated. Each opinion is a scar. If any of the design principles conflict with your environment — you can't touch a hosted model, your compliance office rejects tenant-shared infrastructure, your latency target is sub-500 ms on the full pipeline — we adjust the specific stage, not the shape. The shape is load-bearing.

For a technical deep dive on any single stage, or a scoping conversation on your specific corpus and channel mix, the follow-up is a 30-minute call. Growgy (CGO) schedules it; Teky (CTO) runs it.