AI refusals are stopping some useful questions and hiding risks
Chatbots are trained to say no to dangerous requests, but nobody fully understands how that works, it can be tricked, and companies alone decide where the line sits.
Modern chatbots are trained to refuse a huge range of prompts, and refusal has become the main pillar of AI safety.
Researchers say nobody fully understands how a model decides to say no, and the same risky question can get different answers on repeat tries.
Companies stack extra filter models around chatbots, an approach called the Swiss cheese model, but jailbreaks such as poetic prompts still get through.
Private companies set the refusal rules today, and governments will soon set their own, which raises free speech worries.
Quick read · 2 min
Refusal, the habit of saying no to dangerous requests, has become the main safety tool in modern chatbots. Models are trained to turn down a vast range of prompts, and companies add extra filter models that screen both your question and the AI's answer.
The catch is that nobody fully understands how a model decides to refuse. Ask the same risky question many times and the major chatbots usually decline, but not always. Jailbreaks still work too: researchers broke two dozen popular models just by writing requests as poems. Filters also cost money, with Anthropic saying one kind raised computing costs by nearly a quarter.
Private companies decide where the line falls today, which is why some harmless questions get blocked. Governments will soon set their own lines, which could stop real threats but could also be used to silence legitimate speech.
A refusal is a statistical guess, not a judgment.
Rephrase harmless questions with context.
Check sensitive answers with a real expert.
Ask a chatbot the same risky question over and over, and most of the time it will refuse. Every so often, it won’t. That random gap sits at the heart of AI refusals, the main way the safety of tools like ChatGPT and Claude is enforced, and researchers say we trust it far more than we should.
This matters to anyone who uses AI for work, school or health questions. Refusals decide which of your questions get answered and which get a polite brush-off. They are also the main thing standing between powerful models and people who want to misuse them. Yet the rules are set privately by the companies, the system can be tricked, and even the people who build it can’t fully explain how it works.
01
Why AI refusals became the core of AI safety
AI refusals became central because a model trained on billions of web pages learns harmful knowledge along with everything else, and it doesn’t naturally keep that knowledge to itself. In 2021, a team at Anthropic set out the idea that language models should be helpful, honest and harmless, which included politely turning down requests to help with dangerous acts.
Early systems had no such instinct. Steven Adler, a former OpenAI safety staffer (he was there from 2020 to 2024), says the company’s first models would “blab on about anything.” A Harvard researcher who studies AI and mental health, Ryan McBain, recalls that early chatbots would readily answer questions about self-harm.
Today the picture is very different. Models are trained to say no to a vast list of prompts, and anything that looks statistically close enough to one of them is likely to be refused too. Companies also run training exercises that reward a model for refusing harmful questions and penalize it for refusing harmless ones, often using other AI models to grade the results.
02
How a chatbot actually decides to say no
It isn’t moral reasoning. When a model sees words that resemble prompts it was trained to refuse, patterns called activations fire somewhere inside its billions of internal settings, a bit like neurons lighting up in a brain. Those patterns steer the model toward a refusal.
The trouble is that nobody can map them completely. Researchers who studied refusal inside models describe it as a loose bundle of directions rather than one clear switch. Jannes Elstner, an author of a recent study on the topic who now works on AI safety at Apollo Research, explains that even once you think you’ve found every part of a model behind a refusal, other hidden parts may still play a role. In his view the industry needs refusal anyway, whether or not anyone understands it.
Because of that, companies wrap their chatbots in smaller filter models called classifiers. Some screen what you type and block dangerous prompts before they reach the main model. Others read the model’s answer and stop harmful content before it reaches you. Each filter has gaps, so the industry stacks several in the hope that the holes never line up. Insiders call this the Swiss cheese model.
03
The hidden cost and the holes in the cheese
All this filtering is expensive. Anthropic has said one type of classifier raised its chatbots’ computing costs by nearly a quarter, which means more electricity and water. Companies are now moving to a cheaper kind of filter called a probe, which watches the model’s inner activity directly, a little like putting it in a brain scanner.
Even so, the filters are still based on statistics, not certainty. Anthropic writes its rules in a document it calls a “constitution” and OpenAI calls its version a “model spec,” but the judgment on any single question still comes down to probabilities. McBain’s latest experiments found the big chatbots usually refuse repeated risky mental health questions, but not every time.
Determined users find ways through. Tricking a model into ignoring its training is called jailbreaking. This year, Italian researchers broke the safeguards of two dozen popular models just by phrasing requests as poems. Another team showed an attack where a model says sorry it can’t help, then gives the forbidden answer anyway.
04
Who decides where the line is drawn
Right now, AI companies draw the line, and they do it largely in secret. There is no formula. Some virologists have good reasons to study dangerous viruses, and some security researchers want to know about system weaknesses so they can fix them. Zico Kolter, an OpenAI board member and cofounder of the testing firm Gray Swan, calls the question of where to draw it a huge one.
The result is that chatbots sometimes refuse things that seem harmless, such as counting to a million or telling a racy joke. Meanwhile governments are about to set their own lines. That could help block real threats, but it could also let repressive governments use the same machinery to suppress legitimate speech. The better AI gets at refusing harm, the better it could get at refusing ideas that only threaten whoever writes the rules.
There is a deeper bind too. The knowledge that makes AI useful can’t easily be separated from the knowledge that makes it dangerous. Adler says removing those core abilities would make a model much less smart. A model that could help fight cancer needs to understand genetics well enough that, in the wrong hands, it could help modify germs.
For everyday users, the takeaway is practical. A refusal is not a verdict on whether your question was wrong. It is a statistical guess made by software, tuned by a private company, and it can change from one day or one app to the next.
If a harmless question gets refused, rephrase it with more context about why you’re asking, or try another reputable tool.
Don’t treat a refusal as a safety guarantee. If a chatbot does answer a sensitive health or safety question, check it with a qualified human source.
For mental health concerns, talk to a doctor or a crisis line rather than relying on a chatbot’s answer either way.
The bigger debate, over who gets to set these rules and how open they should be, is only starting, and it will shape what AI will and won’t tell you for years.
06
Quick answers
What is an AI refusal?
It is when a chatbot declines a request because its training or its safety filters flag the prompt as harmful or against company rules.
Why do chatbots refuse harmless questions?
Refusals work on statistical similarity, so a harmless prompt that looks like a risky one can get blocked. Companies try to reduce this over-refusal in training, but it still happens.
Can AI safety filters be bypassed?
Yes. Jailbreaks, such as hiding a request in a poem, have broken the safeguards of many popular models, which is why researchers warn against relying on refusals alone.
Cambridge Dictionary picked "human" over "agent" and "analogue". The reason says a lot about how AI is changing everyday speech, and what people now want to protect.