People tell AI chatbots things they have never told another human being. Health symptoms they’re frightened of. Marriages that are failing. Debts nobody knows about. Then, periodically, those conversations end up on the open internet.
In one incident this year, a security researcher found an AI chat application leaking roughly 300 million messages tied to around 25 million users. Separately, several AI companion apps — the ones marketed as virtual partners and confidants — spilled millions of private conversations. And a chatbot used by McDonald’s for recruitment exposed the personal details of more than 64 million job applicants, after researchers found they could get in using a password so weak it barely deserves the name.
None of the people in those datasets were told in advance. Most still don’t know.
This article explains why AI chats leak so reliably, what protections you do not have when you type into one, and what actually reduces your exposure.
What people actually put into chatbots
To understand why this matters more than an ordinary breach, it helps to be honest about what these conversations contain.
Chatbots occupy a strange psychological position. There’s no human on the other end to judge you, no receptionist, no waiting room, no bill. That combination makes people extraordinarily candid. Surveys of workplace AI use found that the proportion of prompts containing sensitive information rose from roughly one in ten in 2023 to nearly four in ten by 2026 — and that’s just what people admit to at work.
The personal conversations are more intimate still: symptoms people are too scared to search for under their own name, relationship breakdowns, mental health crises, money problems, immigration status, sexuality. The things you would tell a doctor, a lawyer or a therapist.
Which brings us to the first uncomfortable fact.
A chatbot conversation has no legal privilege
When you speak to a doctor, a lawyer or a licensed therapist, that conversation carries legal protection. It generally cannot be demanded in court, and the professional is bound by confidentiality rules with real consequences behind them.
A conversation with an AI chatbot has none of that. It is data on a company’s servers. It can be subpoenaed. It can be handed to law enforcement on request. It can be read by employees during debugging or moderation. It can be retained after you delete it. And it can be sold or transferred if the company is acquired or goes bankrupt — which is exactly how a great deal of sensitive data changes hands.
People routinely use chatbots as a substitute for therapy or legal advice. That’s an understandable response to cost and waiting lists. But the protections that make those professions safe to confide in simply don’t transfer.
Why these apps leak so often — the part nobody explains
Ordinary data breaches usually involve someone breaking in. A striking number of AI chat leaks involve nothing of the sort. The data was simply sitting where anyone could reach it.
1. Unsecured databases with no password at all
The most common cause by a distance. A developer spins up a cloud database during development, leaves authentication switched off to move quickly, and ships. The database stays exposed. Researchers who scan the internet for these find them constantly — no hacking involved, just an address and a browser.
2. Conversations stored in plain text
Even when a database is protected, the messages inside are frequently unencrypted. Encryption at rest costs a little engineering effort and provides no visible feature, so on a rushed timeline it’s the first thing dropped. The result is that a single mistake exposes everything in readable form.
3. The apps are built extremely fast by very small teams
The AI app boom rewards shipping in weeks. Companion apps, therapy bots and “AI girlfriend” products are often built by tiny teams with no security specialist at all, and no budget for an audit. They are handling some of the most sensitive material on the internet with a fraction of the care a bank would apply to a mailing list.
4. The chatbot itself becomes the way in
This is the newest category and the most interesting. In the McDonald’s case, the exposure ran through the recruitment chatbot’s own administration system, reachable with credentials that were trivially guessable. Attackers have also abused chatbot features directly — in one incident, Meta’s AI assistant was manipulated into sending account password-reset codes to attacker-controlled addresses, which was then used to hijack Instagram accounts.
An AI assistant with access to accounts and the willingness to act on instructions is a genuinely new kind of attack surface, and the industry is still working out how to secure it.
What “delete” usually doesn’t do
Most services offer a delete button for conversations. It’s worth understanding what it typically does and doesn’t cover:
- Deleting a chat usually removes it from your view. Retention periods on the company’s side often continue for a set number of days — sometimes considerably longer for abuse-monitoring purposes.
- Backups persist independently. Data removed from the live system can survive in backups for months.
- Training data is the big one. If your conversations were used to train a model before you deleted them, that cannot be unpicked. Deletion doesn’t reach into a trained model.
- Third parties who already received the data — analytics providers, cloud vendors, subprocessors — are not covered by you clicking delete in the app.
What actually reduces your exposure
You don’t have to stop using these tools. But a few habits meaningfully change what’s at stake when the next leak happens.
Change what you type
- Never include identifying details. Describe the situation without names, employers, addresses, account numbers or dates of birth. “A friend of mine” is genuinely useful framing here.
- Never paste credentials, API keys or financial details. These appear in leaked datasets constantly.
- Assume anything you type may one day be public. Not because it’s likely on any given day, but because the cost when it happens is asymmetric.
Change the settings
- Turn off training on your data. Most major providers offer this in privacy settings, and most people never look. It’s usually on by default.
- Use temporary or incognito chat modes where offered — these typically carry much shorter retention.
- Check the retention policy of any smaller app before confiding in it. If you can’t find one, that tells you what you need to know.
Be sceptical of small apps
Companion apps, mood trackers, AI therapists and niche assistants collect the most sensitive material and have the least capacity to protect it. The pattern in nearly every leak covered here is the same: a small team, moving fast, handling data far more sensitive than their security budget.
The wider point
We have spent two decades learning to be careful about what we post publicly. Chatbots feel like the opposite of posting — private, ephemeral, addressed to nobody. That feeling is the problem. Structurally, typing into a chatbot is closer to writing in a notebook that a company owns, stores indefinitely, may train on, and can be compelled to hand over.
None of which means don’t use them. It means write as though the notebook might be read — because periodically, for millions of people at a time, it is.
See what a website learns about you before you type anything: our live browser reveal shows exactly what your device hands over the moment you open a page — no signup, and we store none of it. You can also remove yourself from data brokers with our free opt-out directory, or read AI and Your Privacy in our plain-English guide series.
Reporting on the incidents referenced here: Malwarebytes, Enterprise Security Tech, TechCrunch and tech.co, 2026.