Security Bros - Episode 6 - OWASP Top 10 LLM Vulnerabilities: Real AI Security Risks
A lawyer submitted fictional court cases to a federal judge, sourced directly from ChatGPT. Samsung engineers leaked proprietary source code within 20 days of lifting their ChatGPT ban. Air Canada tried to argue its chatbot was a separate legal entity when sued over wrong information it gave a customer.
These are real events, and every one of them maps to a vulnerability on the OWASP Top 10 for LLM Applications.
The OWASP Top 10 for LLM Applications (2025 edition) is the closest thing the security world has to a peer-reviewed standard for AI security risks. Over 600 researchers across 18 countries contributed. If you’re building on top of any language model, or if your organization’s employees are using AI tools, this list tells you where things go wrong. Here’s what each vulnerability means, and what the breach stories teach us.
What Is the OWASP Top 10 for LLM Applications?
OWASP (Open Web Application Security Project) has maintained its famous Top 10 for web application security for years. SQL injection, broken authentication, cross-site scripting: those categories shaped a generation of secure software practices.
The LLM-specific list follows the same model, but for AI. The first version shipped in May 2023, shortly after large language models went mainstream. The 2025 update reflects how fast the threat surface expanded as agentic AI tools, RAG pipelines, and enterprise LLM deployments multiplied. The list isn’t exhaustive; it’s a prioritized snapshot of where organizations are getting hurt right now.
The 10 Vulnerabilities, Explained in Plain English
LLM01: Prompt Injection
Prompt injection sits at number one because LLMs have no clean separation between their instruction set and their data. An attacker who can feed input to the model can override its behavior. That includes invisible text embedded in a webpage the AI is asked to summarize, or malicious instructions buried in a calendar invite that an agentic AI reads and acts on without the user knowing.
The threat grows as AI agents gain more capability. An agent that can send emails, read files, or call APIs becomes a high-value target. Prompt injection against that agent is an indirect way to do all of those things without the user ever clicking anything.
LLM02: Sensitive Information Disclosure
LLMs connected to proprietary data can surface that data in their outputs. The Samsung incident is the canonical example: engineers pasted semiconductor source code and meeting notes into ChatGPT to speed up their work. None of them were trying to cause harm, but the data left the building and Samsung couldn’t get it back.
Most sensitive information disclosures aren’t malicious. They’re productivity-seeking behavior that runs headlong into an AI policy that hasn’t been backed by technical controls.
LLM03: Supply Chain Risk
You pull an open-source LLM from Hugging Face. You build your application stack around it. The problem: you can’t fully inspect what’s inside it. Supply chain attacks on AI models follow the same pattern as supply chain attacks on software, introduce a backdoor before the victim ever downloads anything. A poisoned model could behave perfectly under normal use, then act on a specific trigger phrase. Blocking ChatGPT and letting developers download local models instead can trade one risk for a worse one.
LLM04: Data and Model Poisoning
If bad data gets into the training process, the model’s behavior gets corrupted downstream. The sleeper agent scenario is the most concerning: a model that behaves normally until a specific input triggers malicious behavior baked in during training. At foundation model scale, poisoning is extremely difficult to execute. For smaller, purpose-built models with less data and less scrutiny, the risk is meaningfully higher.
LLM05: Insecure Output Handling
AI generates code, scripts, and structured content. If that output goes directly into production without review, any vulnerability in the output becomes a vulnerability in your system. AI-assisted development is fast. “We’ll review it later” has a way of meaning never. Build review steps into the workflow, not just the policy.
LLM06: Excessive Agency
Agentic AI tools are over-permissioned by default. Developers in a hurry grant broad access to get things working, and users approving OAuth prompts grant everything because they don’t understand the individual scopes.
This is the principle of least privilege problem in a new context. An over-permissioned AI agent can act on those permissions autonomously, without a human making a deliberate decision at each step. Pair excessive permissions with prompt injection and the attacker has a full execution path.
Lack of monitoring compounds the problem. Most personal and small-team AI deployments have no alerting for unusual or high-privilege actions. If the agent does something unexpected, you may not find out until the damage is visible.
LLM07: System Prompt Leakage
System prompts often contain details developers assume are invisible: internal instructions, database names, API behavior, sometimes credentials. With the right prompting, model outputs can reflect or reconstruct those instructions. Treat anything in a system prompt as potentially visible. API keys in prompts are the same mistake developers were making on GitHub a decade ago.
LLM08: Vector and Embedding Weaknesses
RAG systems retrieve information from external sources before generating a response. That retrieval layer can be poisoned. If an attacker can modify the documents or database your AI queries, they can influence what the AI says. Even without active poisoning, data from one user’s retrieval context can leak to another. As more organizations build internal AI tools on top of file shares and databases, this attack surface grows.
LLM09: Misinformation (Hallucination)
LLMs generate plausible-sounding text, not verified facts. They’re weakest at the extremes: rare edge cases, recent events, or situations where the training data is thin. A health AI that handles 95 percent of cases well may still give dangerous advice on the 5 percent that fall outside its competence, and it won’t flag the uncertainty. It just sounds confident.
The legal cases in this episode are direct products of this vulnerability.
LLM10: Unbounded Consumption
Stolen API keys can drain your budget fast. An attacker who gets hold of your AI provider credentials can run queries at your expense until you notice or until your account runs out. “Denial of wallet” is the right framing: it’s a resource exhaustion attack that hits your bank account instead of your CPU.
The DeepSeek situation illustrates the larger version of this problem: systematic querying of a competitor’s model to reverse-engineer its behavior and replicate it at scale, without paying for the compute that made it possible.
Three Breach Stories That Prove the List Is Real
Samsung’s Data Leak
After briefly banning ChatGPT, Samsung lifted the restriction under pressure from engineering teams. Within 20 days, three incidents had been reported: source code from a faulty semiconductor pasted into ChatGPT for analysis, internal diagnostic code sent the same way, and meeting notes uploaded for transcription. All of it ended up on external servers. Samsung banned the tool again. Engineers built their own internal AI, Samsung Gauss.
The lesson isn’t that Samsung employees were careless. The lesson is that blocking a tool without replacing it with a secure alternative doesn’t fix the underlying need. As Rocky put it on the show: “Security policy is not going to beat convenience without technical controls. You have to make the secure path the easy path.”
Air Canada’s Chatbot Ruling
A passenger asked Air Canada’s chatbot about bereavement fares before booking a flight for a family funeral. The chatbot told him he could buy the ticket at full price and apply for a retroactive discount within 90 days. He did. Air Canada denied the claim and argued the chatbot was “a separate legal entity responsible for its own actions.”
The tribunal disagreed. The judge ruled that Air Canada was responsible for all information on its website, including what its chatbot said. The passenger received a partial refund. The precedent matters more than the dollar amount: companies are liable for what their AI systems tell customers.
Mata v. Avianca: ChatGPT in Federal Court
A lawyer suing Avianca Airlines for a knee injury used ChatGPT to research and draft a legal brief. ChatGPT cited six supporting cases. None were real. The case names, citations, airlines, and rulings were fabricated. When the opposing counsel couldn’t locate the cases, the lawyer asked ChatGPT whether they were real. ChatGPT confirmed they were, citing Westlaw and LexisNexis as sources. The lawyer submitted the brief under oath.
The court sanctioned the firm $5,000 and required the lawyers to write personal apology letters to every judge whose name appeared in the fabricated rulings. LLMs generate confident-sounding text. They don’t verify facts. That’s how the architecture works.
Protecting Your LLM Applications: What to Do Now
The OWASP list isn’t a list of unsolvable problems. Most vulnerabilities have documented mitigations. The pattern that runs through all of them is familiar from traditional security work:
• Apply least privilege to AI agents and integrations. If a tool only needs to read files, don’t give it write access. If a workflow needs one API scope, don’t approve all of them.
• Log and monitor AI actions, especially privileged ones. This gap is absent from the OWASP list but shows up in every post-incident analysis. Know what your AI tools are doing.
• Implement DLP controls that understand AI context. Traditional DLP rules weren’t built for how people use LLMs. You need controls that catch sensitive data before it leaves the prompt.
• Treat system prompt contents as potentially visible. Credentials and API keys in AI systems carry the same risk as hardcoded keys in source code: don’t do it.
• Validate AI-generated outputs before using them. The speed advantage of AI is real. Unreviewed outputs in production are a real risk. Build review steps into workflows, not just policies.
Resources:
OWASP Top 10 for LLM Applications (2025)
FAQ
What are the OWASP Top 10 LLM vulnerabilities?
The OWASP Top 10 for LLM Applications (2025) lists: (1) Prompt Injection, (2) Sensitive Information Disclosure, (3) Supply Chain Risk, (4) Data and Model Poisoning, (5) Insecure Output Handling, (6) Excessive Agency, (7) System Prompt Leakage, (8) Vector and Embedding Weaknesses, (9) Misinformation/Hallucination, (10) Unbounded Consumption. The full list is maintained at the OWASP Gen AI Security Project.
What is prompt injection in AI?
Prompt injection is an attack where malicious instructions are embedded in content the AI processes, overriding its intended behavior. It can happen through user inputs, documents the AI reads, or web pages it accesses. Agentic AI tools are especially vulnerable because prompt injection can cause the agent to take real-world actions on behalf of the attacker.
What is “denial of wallet” in AI security?
Denial of wallet is a resource exhaustion attack targeting AI API costs. An attacker who obtains your API credentials can run queries at your expense, generating large bills before you detect the unauthorized use. It’s the AI equivalent of crypto-mining attacks on cloud environments.
Can a company be held liable for its AI chatbot’s mistakes?
Yes. The Air Canada case (2022) established that companies are responsible for information provided by chatbots on their websites. A chatbot is part of the company’s service offering, not a separate legal entity. The same liability logic applies to health advice, financial guidance, and any other domain where incorrect AI output causes harm.
How does sensitive information get leaked through LLMs?
Most leaks happen when employees paste proprietary data, source code, or meeting content into an AI tool to get help with a task. If the AI provider uses input for training or stores it on external servers, that data may be permanently exposed. The Samsung case is the most-cited example: three incidents in 20 days, none of them malicious, all involving engineers trying to work faster.
What is excessive agency in LLM security?
Excessive agency occurs when an AI agent is granted broader permissions than it needs to perform its function. Over-permissioned agents can take high-privilege actions autonomously. Combined with prompt injection, those permissions become accessible to an attacker. The fix is the principle of least privilege, applied to AI integrations from the start, not revisited later.
How do you secure a RAG system against embedding attacks?
Limit write access to the data sources your RAG system retrieves from. Monitor those sources for unexpected changes. Apply access controls so one user’s retrieval context doesn’t expose data intended for another. Validate outputs against known-good data where possible, particularly in high-stakes applications.
Conclusion
The OWASP Top 10 for LLM Applications maps real attack patterns to real consequences. The Samsung leak, the Air Canada ruling, and the Mata v. Avianca sanctions all happened because organizations deployed AI without matching technical controls to the new risk surface. The list isn’t a one-time checklist; it’s a framework for ongoing evaluation as the AI landscape shifts.
If you’re building AI applications or managing an organization’s AI usage, start with the list. Work backward to your actual controls. The gap between what you’ve deployed and what you’ve secured is where these stories happen.
Listen to the full episode: