Signal chief says careless AI use, not horror tales, is danger
Meredith Whittaker says AI doom stories distract from the real problem: people trusting faulty systems, and governments pushing to weaken the encryption that keeps chats private.
Signal Foundation president Meredith Whittaker says the real AI danger is careless use of flawed systems, not machines turning on humanity.
She cites a reported case where a US intelligence chatbot invented details about a Chinese ship carrying nuclear material.
Whittaker says Signal would leave the UK or any other country that forced it to weaken its end-to-end encryption.
In a late October podcast she added that AI agents able to see everything on a device threaten private messaging itself.
Quick read · 2 min
Signal Foundation president Meredith Whittaker says the real danger from AI is not machines rising up. It is people trusting faulty systems too quickly. Her example: a reported case in which a US intelligence chatbot invented details about a Chinese ship carrying nuclear material, which she says brought the US close to a military response.
She traces this worry back to 2017, when she helped lead protests at Google against Project Maven, a Pentagon drone-targeting AI contract that Google later dropped. Today she is fighting a second battle, over encryption. Whittaker says Signal would leave the UK, or any country, rather than weaken the protection that keeps chats private, even as Britain prepares new child-safety laws.
Her core point: bad data plus fast decisions is the real AI risk.
Signal says it would quit markets that demand weaker encryption.
Her newest warning: AI agents that can see your screen threaten private messaging.
The scariest AI story Meredith Whittaker tells is not about robots. It is about a chatbot used by US intelligence that reportedly made up details about a Chinese vessel carrying nuclear material, a mistake she says nearly pushed the US toward a military response. Her verdict on careful AI use is blunt: a system that does that should not be called super-intelligent by anyone.
Whittaker runs the Signal Foundation, the nonprofit behind the encrypted messaging app Signal. Her argument matters to anyone who uses AI tools or private chat apps, because it moves the worry away from science fiction and toward everyday choices: who trusts AI output, what data it was built on, and whether your messages stay private when governments and AI assistants both want a look.
01
Why careful AI use matters more than doom stories
Whittaker says the risk from AI is real, but it is not the dramatic tale of machines turning on people. She calls those warnings a fear with no grounding in science and dismisses them as ghost stories. The more pressing danger, in her view, is the technology being used carelessly, with bad data and too little checking.
The chatbot case is her favorite example. The incident was first reported by CNN and has not been publicly confirmed by US officials, so treat the details as alleged. But her point does not depend on one case. When an AI system produces confident, wrong answers and people act on them quickly, the harm is very ordinary and very human.
That is a useful way to think about the AI tools you already use. A chatbot that writes a wrong date in an email is a small problem. The same kind of mistake inside a bank, a hospital or a government office becomes a big one, because the speed that makes AI attractive also cuts the time anyone spends checking its work.
02
The Google protest that shaped her view
Whittaker did not start out as an engineer. She studied literature, then joined Google in 2006 as an entry-level temp without a computer science degree. Time magazine later named her one of the most powerful people in AI.
Her turning point came in late 2017, when she learned from a coworker about Project Maven. It was a confidential Pentagon deal for Google to build drone-targeting AI. She saw it as a way to shorten the gap between a machine flagging a target and a person deciding to fire. In other words, less time to think. She warned publicly that it would launder bad intelligence and bad data.
Thousands of Google workers pushed back. A year later she helped organize a walkout of some 20,000 staff over several issues, including what they saw as rushed technology. Google dropped the Maven contract, and Palantir picked it up.
Whittaker now links that system to a US missile strike on a school in Minab, Iran, this year, which she says relied on outdated intelligence. Palantir disputes this. It says there is no evidence its software was at fault and that it is not responsible for the underlying data. The US military has not yet released its investigation findings.
03
Would Signal really leave the UK?
Yes, according to Whittaker. Asked whether Signal would quit any country that forced it to weaken its encryption, the UK included, she answered with an emphatic yes and said her position does not shift with mood or consultants. She says exposing people who depend on the app, sometimes in life-or-death situations, to surveillance would leave Signal no choice but to exit that market.
The pressure is real. Governments, including Britain’s, want more access to conversations on messaging apps and say it is needed to protect women and children. Keir Starmer, then prime minister, used London Tech Week 2026 to tell tech firms they must spot and stop nude images on kids’ phones, or new laws would force them to. The Department for Science, Innovation and Technology says it is preparing legislation.
Whittaker argues children would find ways around such measures and calls the plan an unprecedented system of communication controls. Child safety groups see it differently. The NSPCC’s head of child safety online, Rani Govender, says children who are groomed are often moved onto encrypted platforms, and that tech firms are responsible for the risks their design choices create.
Signal is already fully or partly blocked in Russia, China and Iran, so leaving a market would not be new territory.
04
What end-to-end encryption means for your chats
End-to-end encryption means only the people in a conversation can read the messages. The company running the service cannot. Signal goes a step further and hides who sent a message from its own servers, which leaves very little data for hackers or outside observers to collect.
Signal is small by comparison, with an estimated 40 million to 70 million users against WhatsApp’s billions. But its design is the reference point in the privacy debate, and Whittaker insists it has to work the same way for everyone. When asked about Canadian police officers whose offensive Signal group chat leaked after one member shared it, she compared it to a private kitchen conversation: nobody blames the kitchen when a friend repeats what was said.
She also points to a US woman who was jailed after Meta handed over messages showing she helped her daughter get an abortion in a state that had banned it. For Whittaker, that is the case for privacy in a nutshell. Messages that a company can read can be handed over.
Whittaker’s warnings have only sharpened since. On a podcast in late October, she said private messaging faces an existential threat from two directions at once. One is governments asking for backdoor access to chats. The other is AI agents, the assistants that can act for you, which can see everything on a device.
Her concern is simple. Encryption protects a message on its way between phones. If an AI assistant built into your phone can read the screen, your messages may be exposed before they are ever encrypted or after they arrive. She warned that this could remove the possibility of offering privacy at all.
For ordinary users, the practical steps are modest but real. Check which AI features on your phone have permission to read notifications, messages or your screen, and switch off any you do not use. Be wary of acting on AI answers about money, health or legal matters without checking them yourself. And if private conversations matter to you, an end-to-end encrypted app is still the strongest option available.
06
Quick answers
What does Meredith Whittaker say is the real AI risk?
She says the main danger is people using AI carelessly and trusting systems built on bad data, not machines turning against humanity.
Will Signal leave the UK?
Whittaker says Signal would leave the UK or any country that forced it to weaken its encryption. The UK government says it is preparing new legislation, but no such order has been made.
Why are AI agents a privacy problem?
Assistants that can see your screen or read your messages sit outside the protection of encryption, so private chats could be exposed on the device itself.
An open-source framework that hands most of a break-in to AI agents is still circulating. Here's who built it, what it hit, and what the FBI says to do.
Some Android beta testers can now skip Google Drive and save chats on WhatsApp's servers, locked by default. Here's how it works and how to protect yours today.
6 min read
We use cookies to understand how readers use Talk With Tech, so we can make it better. Is that OK? Privacy policy
The Daily Brief
Today's biggest tech stories, in 5 minutes
Every morning, the news that matters from AI, phones, apps and the people shaping tech. Explained in plain English. Free.
One email a day. No spam, unsubscribe anytime. Privacy policy