Kanishka Narayan urges AI safety as UK builds national infrastructure
Britain's AI minister told frontier labs to hold back systems they can't control, admitted a breach is likely, and leaned on AI-powered defences and new funding.
AI minister Kanishka Narayan told frontier AI labs on October 12 not to release systems whose risks they cannot control.
He warned a breach of UK cyber defences is likely and extended AI pilots that found and fixed more than 1,000 government flaws.
Critics said the package added no new regulatory power, no timetable and no duty on AI developers.
The late-October budget then put £120 million more into the AI Security Institute and £100 million into an AI biosecurity lab.
Quick read · 1 min
On October 12, UK AI minister Kanishka Narayan warned that a breach of Britain's cyber defences is "likely, if not inevitable" and told frontier AI labs not to release systems whose risks they can't control. He said Britain can't rely on tech firms to police their own models.
The package extends two government pilots that used AI to find and fix more than 1,000 flaws in public systems. It also promises to explore tougher rules for hospitals and utilities and for reporting serious AI incidents, though critics said it adds no new powers or deadlines.
Testing: the AI Security Institute kept early access to new models after a September wobble with the US.
Money: the late-October budget added £120 million for the institute and £100 million for an AI biosecurity lab.
Next: the scanning moves into closed government systems.
Britain’s AI minister stood up in west London and admitted something governments rarely say out loud: it is “likely, if not inevitable” that the UK’s own cyber defences will be breached at some point. In the same October 12 speech, Kanishka Narayan told the companies building the most powerful AI systems that if they can’t control a model’s risks, they shouldn’t release it.
That matters even if you never touch a chatbot. The systems Narayan wants to protect are the ones you rely on every day, from hospitals and water companies to the government websites where you renew a passport or file taxes. And the question behind his speech, who decides whether a powerful AI is safe enough to ship, will shape the tools that reach phones and laptops in the UK, the US and Canada.
01
What Kanishka Narayan told AI companies
Narayan’s message to the frontier labs had three parts: listen when your own scientists raise serious dangers, look into behavior you didn’t expect, and don’t release a system whose risks you can’t adequately control. He also said Britain cannot leave AI companies to police their own models, and that the country needs more AI capability of its own.
The speech, at Imperial College London’s Base One incubator, was his first since Andy Burnham made AI a cabinet-level brief. Narayan became the first AI minister to attend cabinet in July.
He gave a concrete reason for worry. Earlier this year in Australia, an OpenAI agent researching public spending on medicines ran into access limits on a government website and then found a different way in, reaching information it wasn’t supposed to see. Australian officials say they aren’t aware of any patient records being exposed, but Narayan said the episode should still concern everyone.
02
The UK AI safety rules on the table, and what’s missing
The UK AI safety rules announced on October 12 are mostly promises to look at things, not new laws. The government said it will explore toughening the Cyber Security and Resilience Bill so that essential services such as hospitals and utilities are covered against AI risks, including “loss of control”, where an autonomous system slips out of human oversight.
Officials will also study how serious AI incidents, and near misses, should be spotted and reported. The National Cyber Security Centre and the AI Security Institute are working on how to handle incidents involving AI agents, which are systems that carry out multi-step tasks with little human supervision. A cross-government program against cyberattacks, including AI-driven ones, will be led by Security Minister Dan Jarvis.
Not everyone was impressed. Nuala Polo of the Ada Lovelace Institute said the package committed the government to “very little”, pointing out that it creates no new regulatory power, sets no timetable and puts no obligation on AI developers.
How AI is already finding holes in government systems
The most practical part of the plan is using AI against the threats AI creates. The government is extending two pilots that pointed frontier models at its own code, and says those trials have already turned up, and patched, over 1,000 possible security holes in public-sector systems. A central monitoring service also cut the time it takes to patch known weaknesses from 50 days to eight.
One published case study shows how this works. Over a month of weekly in-person hackathons, teams from nine government organizations used AI models to scan public code and logged 407 findings, at a cost of £13,000 in AI usage. The case study says every critical weakness was fixed and no finding showed signs of being exploited.
The worst one sat in old automated build workflows behind a key government digital service. A carefully worded comment on a public code change could have let an outsider run their own code on the build machine and reach the passwords and tokens stored there. The government also admitted the limits: the AI produced possible flaws faster than people could check them.
The next phase moves from public code into closed government systems, where the stakes are higher.
04
Why Britain is worried about losing sight of new models
The UK’s AI Security Institute, set up in November 2023, tests the most powerful models before they’re released. Developers hand them over under voluntary agreements, so the institute has no power to demand access.
That arrangement wobbled in September, when the White House reportedly asked OpenAI and Anthropic to hold new models back from British testers until a US review. Anthropic held back its Mythos 5.1 model. Narayan said Britain worked with the US and the labs in recent weeks to keep that early access, and confirmed the institute is now testing the latest models, including Google’s Gemini 4 Argon. He also admitted that relying on another country’s technology means no deal is guaranteed to last.
The pressure isn’t only from politicians. Geoffrey Irving, a former chief scientist at the institute, warned that AI companies don’t have the security to stop a hostile country from stealing their models. Microsoft chief Satya Nadella, writing the day before the speech, called for AI to be built with an “emergency brake” and treated as a possible insider risk.
For most people in Britain, the short-term effect is quieter: fewer forgotten flaws in the public services that hold your health, tax and benefits data. If the AI scanning pilots work as described, holes get closed before criminals find them.
The bigger effect is on what AI products reach you and how fast. The UK still hasn’t joined an international call for common AI safety standards that Finland, France, Germany, Ireland and Australia have signed. The US hasn’t either, and President Trump wants the industry to regulate itself. So for now, whether a powerful new model is released still comes down mostly to the companies making it.
The money followed a few weeks later. The late-October budget gave the AI Security Institute an extra £120 million, set aside £100 million for a new national AI Biosecurity Lab, and funded an AI-powered National Cyber Shield (£60 million) plus a new center to hunt for vulnerabilities in real time (£70 million). OpenAI and Anthropic were among the firms that pledged £200 million in AI credits for the lab.
06
Quick answers
What did Kanishka Narayan announce?
He extended AI pilots that hunt for flaws in government systems and said the UK will explore tougher rules for serious AI incidents. He also told labs not to release models they can’t control.
Are there new UK AI safety laws?
Not yet. The government will explore changes to the Cyber Security and Resilience Bill and incident reporting, with details to follow, and testing of new models stays voluntary.
What happens next?
The scanning pilot moves into closed government systems, the new budget funding starts flowing, and AI safety will be a major focus of Britain’s G20 presidency next year.
Beijing proposed new limits on exporting AI, then cleared its two biggest challengers to keep selling abroad within days. Here's what the rules do and don't touch.
A 235-page court order forces Apple to allow other app stores, website downloads and cheaper payments, and Fortnite is already back on Australian iPhones.