AI Chatbot Architecture.

• Content pipeline — ingesting and continuously syncing documentation, product data, and ticket history.
• Retrieval — hybrid semantic and keyword search plus re-ranking, tuned per content type.
• Synthesis — answer generation constrained to retrieved context, with citations and explicit refusal on thin evidence.
• Handoff — escalation logic based on confidence, detected frustration, and topic sensitivity.
• Evaluation — a growing set of real questions with known correct answers, run before every change.

Teams that skip straight to synthesis, picking a model and writing a good system prompt, without building the first two components properly end up with a system that sounds confident and answers wrong. The model was never the weak link.

A common mistake is treating retrieval as a solved problem after initial setup. Documentation changes, product behavior changes, and what counted as a good chunk boundary for last quarter's content may not hold for this quarter's additions. Retrieval quality needs the same ongoing evaluation as the synthesis step, not a one-time tuning pass.

• Use a managed vector database or search platform for the retrieval infrastructure itself, building this from scratch rarely earns its cost.
• Build the chunking and content-type-specific tuning custom, this is where the actual differentiation in answer quality comes from.
• Build the evaluation set custom and specific to your domain, generic benchmarks don't reflect what your users actually ask.
• Buy channel integrations (website widget, WhatsApp, Slack) where mature connectors exist, build custom only for unusual internal channels.

• One chunking strategy applied uniformly across content types with very different structures.
• No real evaluation set, quality assessed informally rather than measured against known-correct answers.
• Escalation logic bolted on late instead of designed alongside the confidence scoring from the start.

See our AI chatbot guide for how these five components fit into the full retrieve-synthesize-answer pipeline.