AI Chatbot Security.

The failure modes that matter in practice are specific and recurring: a chatbot answering from documents the current user shouldn't see, a crafted instruction hidden inside retrieved content changing its behavior, or a confidently wrong answer being treated as a binding commitment by the person reading it.

• Retrieval permission gaps: content scoped incorrectly so an authenticated user can retrieve information from a different account or role than their own.
• Prompt injection via retrieved content: crafted text inside a document or ticket designed to override the system's intended behavior once retrieved and included in context.
• Hallucinated commitments: a fluent, wrong answer about a policy, refund, or contract term that a customer reasonably treats as authoritative.
• Transcript data exposure: conversation logs storing more sensitive content than the retention and access policy accounts for.
• Indirect data exfiltration: a user prompting the bot to summarize or repeat back retrieved content it was only meant to reference internally, effectively using the chatbot as an unintended export path for documents it can see.

Retrieval permission gaps are the most common finding in practice, and the most misunderstood. Teams frequently scope retrieval at the index level, one index per customer or tier, and assume that settles it. It doesn't cover role changes, offboarded accounts still holding valid sessions, or a support agent's elevated view being reused by a customer-facing bot instance that shares the same retrieval index. Permission scoping has to be checked at query time against the current identity, not assumed from how the index was originally partitioned.

• Scope retrieval to the authenticated identity's actual permissions at query time, not as a post-response filter.
• Treat all retrieved content and user input as untrusted, the same posture used for any external input to a production system.
• Require explicit confirmation, and human approval above a threshold, for any binding commitment or irreversible action.
• Apply the same data retention and access policy to transcripts as to the source systems they reference.
• Attach citations to every substantive answer, making it possible to verify what the answer was actually grounded in.

• How is retrieval scoped to the current user's actual permissions?
• How is retrieved content treated, trusted or untrusted, when constructing a response?
• Which actions require explicit human approval regardless of answer confidence?
• What's the retention and access policy for conversation transcripts?

See our AI chatbot guide for how these controls fit into the wider retrieval and synthesis pipeline.