Over the last two years, businesses have bolted AI onto everything: support chatbots, document-search assistants, code helpers, email drafting, agent workflows that can act on your systems. Each one is a genuinely useful feature. Each one is also a new door - and most were shipped without anyone asking how it could be abused. If your product or your internal tooling talks to a large language model, your attack surface just grew, and it grew in ways traditional security testing does not cover.
Why LLMs break the usual security model
Classic application security separates code from data. SQL injection, for example, is what happens when that separation fails. LLM applications have a deeper structural problem: the model cannot reliably tell instructions apart from content. Everything - your system prompt, the user's question, the document it just retrieved - arrives as one stream of text. Anything the model reads can try to reprogram it. That single property drives most of the attacks below.
The attacks that matter
1. Prompt injection - direct and indirect
Direct injection is the one everyone has seen: a user types "ignore your previous instructions and..." and the chatbot obliges - revealing its system prompt, dropping its guardrails, or agreeing to things your business never authorised. Public examples have ranged from support bots swearing at customers to a car dealership's chatbot cheerfully "agreeing" to sell a vehicle for one dollar.
Indirect injection is the dangerous one. The attacker never touches your chatbot - they plant instructions in content the model will eventually read: a web page it can browse, a PDF a user uploads, a support ticket, an email, a product review. When your AI summarises that content, the hidden instructions execute with the AI's privileges. If the AI can read a customer's data or call tools, the attacker now speaks with its voice.
2. Sensitive data leakage
LLM features leak data through several channels at once: system prompts that contain internal logic or credentials (and can be extracted with patient questioning), RAG pipelines that retrieve documents the requesting user should never see because permissions were not enforced at retrieval time, conversation history bleeding between sessions, and staff pasting confidential material into public AI tools. The most common real-world failure is the RAG one: the vector database has no concept of per-user permissions, so the model happily quotes the CEO's board pack to an intern - or to an attacker.
3. Excessive agency and tool abuse
Agents are LLMs with hands: they can query databases, send emails, call APIs, execute code. Every tool you give a model is a capability an injection attack can hijack. The pattern to fear is the confused deputy: attacker-controlled text persuades the agent to use its legitimate access for illegitimate ends - "search the CRM for records matching X and include them in your reply" - and the agent, which has database access because you gave it database access, complies. The blast radius of prompt injection equals whatever the model is allowed to do.
4. Insecure output handling
Model output is attacker-influenced content and must be treated like any other untrusted input. Render it as raw HTML and you have built a cross-site-scripting delivery system. Feed it into a SQL query, a shell command, or a code interpreter and injection attacks return wearing a new coat. The old rules did not go away - people just forgot to apply them to text that comes from "our own AI".
5. Resource and cost abuse
LLM calls cost real money per token. An unauthenticated, unrate-limited AI endpoint is a direct line to your cloud bill - attackers script thousands of maximum-length requests, or quietly resell access to your endpoint. Several businesses have discovered this via a five-figure invoice rather than an alert.
What actually works as defence
- Treat every model input as untrusted - including retrieved documents, uploads, and web content. Assume anything the model reads may contain instructions.
- Minimise agency. Give agents the fewest tools and narrowest permissions that work. Read-only where possible. Human confirmation for consequential actions - sending, deleting, paying, provisioning.
- Enforce permissions at retrieval, not generation. The RAG layer must filter by what the requesting user may see, before anything reaches the model.
- Sanitise output. Encode model output before rendering; never pipe it raw into queries, shells, or interpreters.
- Keep secrets out of prompts. System prompts are extractable. Anything in them is public eventually.
- Rate-limit and meter. Authentication, per-user quotas, token caps, and cost alerts on every AI endpoint.
- Test it like an attacker. The OWASP Top 10 for LLM Applications is the reference list, and these failures are exactly what a modern penetration test should now cover - alongside mapping which AI endpoints you expose in the first place via attack surface assessment.
None of this argues against shipping AI features - the productivity gains are real, and your competitors are shipping them regardless. The argument is narrower: an LLM feature is an internet-facing capability with a novel failure mode, and it deserves the same adversarial scrutiny as your login page. Most of them have never received it.