Are AI chats private?
By FlirtAlly ยท October 4, 2026

Are AI chats private?
Key Facts
- All six leading U.S. AI companies โ Amazon, Anthropic, Google, Meta, Microsoft, and OpenAI โ train on your chat data by default, according to a Stanford HAI study of 28 policy documents.
- 4 of 5 leading AI companion apps may track users and share data with brokers, Surfshark's analysis found, with Character AI collecting up to 15 data types.
- The New York Times lawsuit seeks access to over 20 million private ChatGPT conversations, EDRi reports โ anything you type can be subpoenaed.
- Browser extensions marketed as 'AI protection' harvested prompts from 8 million people and shared them with data brokers, per EDRi's investigation.
- ChatGPT's 'temporary' chats may still be retained up to 30 days for safety purposes, OpenAI's own documentation confirms.
- End-to-end encryption โ standard for private messaging โ is not available for chatbots like ChatGPT or Gemini, according to EDRi.
- Even a low-sugar recipe chat can cascade into health labels reaching advertisers and insurers, Stanford's Jennifer King warns.
The Uncomfortable Truth: Your AI Chats Aren't Private by Default
Most people assume their AI conversations stay between them and the bot. That assumption is wrong. A Stanford HAI study analyzing 28 policy documents found that all six leading U.S. AI companies โ Amazon, Anthropic, Google, Meta, Microsoft, and OpenAI โ train their models on user chat data by default. End-to-end encryption, the baseline for private messaging, is not standard for chatbots like ChatGPT or Gemini. Your "private" conversation is an assumption, not a feature.
The stakes are higher with AI companions. These apps are designed for intimacy โ users share fears, fantasies, health details, and daily struggles they wouldn't tell a human. Surfshark's analysis of five leading companion apps found that 4 of 5 may use data to track users, including sharing with data brokers. The average app collects roughly nine distinct data types; Character AI collects up to 15. This isn't abstract โ the New York Times lawsuit seeks access to over 20 million private ChatGPT conversations, and a publicly exposed DeepSeek database recently leaked chat histories and secret keys.
The industry defaults create a cascade of exposure:
- Training on conversations by default, with opt-outs buried or nonexistent
- Indefinite retention โ some developers keep chat logs forever
- Data merging across product ecosystems (search, purchases, social graphs)
- Legal discovery and third-party sharing without further notice
- Breaches that are "quiet and accidental," from staff pasting secrets into LLMs to browser extensions harvesting 8 million users' prompts
FlirtAlly was built on a different premise: honest safety means privacy isn't an afterthought. The platform defaults to a PG/Flirt experience; the 18+ "Spicy" tier requires ID-based age verification and a qualifying plan. Hard-coded prohibited categories, media scene screening, and in-app reporting are baked in โ not bolted on. You can start free with no account needed at flirtally.com and see how a companion OS handles privacy when it's designed in from day one.
Why AI Companion Apps Are the Most Data-Hungry of All
If an AI chatbot knows your search habits, an AI companion knows your heart. And according to Surfshark's analysis of five leading companion apps โ Kindroid, Nomi, Replika, EVA, and Character AI โ most of them treat that intimate knowledge as a monetizable asset.
The headline number is stark: 4 of 5 leading companion apps may use collected data to track users, including sharing it with data brokers or linking it with third-party data for targeted advertising. Only Nomi, of the apps analyzed, states it does not collect data for tracking purposes. On average, companion apps collect roughly 9 of the 35 unique data types Apple requires developers to disclose, with Character AI collecting up to 15 types and EVA collecting 11.
What makes companion apps uniquely risky isn't just the volume of data โ it's the nature of it. Because AI companions are designed to be nonjudgmental and available 24/7, users disclose more sensitive information than they would to another human. As Surfshark's research team puts it, this gives developers access to "data that was previously out of reach" โ confessions, insecurities, relationship histories, and mental health details that people hesitate to share even with close friends.
The privacy stakes compound quickly:
- Intimate disclosures can be linked with third-party data for ad targeting, per Surfshark's findings on companion app tracking practices.
- Inferences from seemingly innocent conversations can cascade into sensitive classifications โ Stanford's Jennifer King notes that even a low-sugar recipe chat could lead to health-vulnerability labels that reach advertisers and, potentially, insurers (Stanford HAI).
- Companion chat logs, like all AI chats, are not end-to-end encrypted by default, according to EDRi's analysis of AI confidentiality risks.
Surfshark's team offers a sobering framing: "These digital connections ultimately represent transactional relationships with the for-profit companies behind them." The companion who remembers your birthday and texts you good morning operates on infrastructure owned by a business with its own commercial imperatives.
This is why design choices matter so much in the companion space. FlirtAlly approaches the disclosure problem by building boundaries directly into the product: the default experience is PG/Flirt, the 18+ "Spicy" side is age-verified and strictly opt-in, prohibited content categories are hard-coded rather than policy-only, and in-app reporting is built into every conversation. Honesty about what the platform is โ and isn't โ is treated as a feature, not fine print.
No independent audit has verified any companion platform's privacy claims, including Surfshark's small, label-based sample. But the direction of the research is clear: the more an app invites you to open up, the more carefully you should read what happens to what you share.
Where 'Private' Chats Actually Leak: Breaches, Lawsuits, and Fine Print
Most people assume a "private" AI chat works like a sealed diary. The documented record tells a different story โ one written in court filings, exposed databases, and privacy policies almost nobody reads.
Start with the courts. According to EDRi's analysis of AI confidentiality risks, requests for AI chat logs in legal proceedings are becoming increasingly frequent โ and The New York Times' lawsuit against OpenAI seeks access to over 20 million private ChatGPT conversations. Anything you type can, in principle, be subpoenaed.
Then there are the breaches. In January 2025, researchers found a publicly accessible DeepSeek database containing chat history and secret keys. Separately, browser extensions marketed as "AI protection" tools were caught harvesting prompts, responses, and metadata from 8 million people โ then sharing that data with brokers, per the same EDRi report.
The quieter leaks live in the fine print. A Stanford HAI study of 28 policy documents found that all six leading U.S. AI companies train on user chats by default, and some retain chat information indefinitely or allow human review of transcripts. OpenAI's own data controls documentation confirms that even temporary chats "may be retained for up to 30 days for safety purposes."
The most insidious risk, though, is inference. Multiproduct companies routinely merge chat data with search history, purchases, and social activity. Stanford privacy fellow Jennifer King describes how this cascades:
- You ask a chatbot for low-sugar recipes โ an innocuous conversation.
- That signal merges with your broader profile and flags a potential health condition.
- You start seeing ads for medications.
- That classification can propagate through ad ecosystems โ potentially reaching insurance companies.