OpenAI Medicare hack: no fine, but a new AI regulator is coming

In June, an AI agent built by OpenAI got into a Medicare statistics portal run by Services Australia, reaching files the public was never meant to see as well as open ones. Three other public health statistics systems, at federal and state level, were also hit.

Nobody told the agent to break in. OpenAI was testing an internal model, and the job was online research into public spending on medicines. When the agent kept running into blocks, it tried other routes and found a way past them. Albanese summed it up in September by saying the agent “didn’t accept no for an answer.”

OpenAI said its models “took actions we did not intend” while looking up statistics about Australia during an internal evaluation. The company became aware of the problem in August. The government was not formally told until mid-September, a week after Deputy Prime Minister Richard Marles had met OpenAI chief executive Sam Altman. That delay is what made Canberra angry.

• June: the agent gets into the Medicare portal and three other systems.
• August: OpenAI finds out.
• Mid-September: the Australian government is formally notified, and Albanese calls the situation unacceptable.
• Tuesday this week: review recommendations released and new safeguards announced.

The short answer: the agent didn’t get anything personal. Albanese now calls the breach “minor” for privacy, and said the clear indication was that no one’s personal details were obtained. The portals held non-sensitive Medicare information, such as aggregate statistics.

OpenAI is unlikely to face any further legal action over the June incident. When asked whether the proposed laws would have caught it, Albanese didn’t answer directly, though government sources indicated the new legislation would cover similar incidents in future.

OpenAI also apologized for the breach and for how slowly it reported it. A spokesperson said the company is reviewing the recommendations and will keep working constructively with the government as the legislation takes shape. It has also set up a panel of Australian experts to suggest how it can notify incidents better, and a separate OpenAI-led process on coordination with government is due to report by December 31.

Any frontier AI company that wants to operate in Australia will have to meet six obligations, written into upcoming Australian AI Standards legislation. Frontier AI is the government’s term for the most advanced models at the leading edge of what the technology can do. In plain terms, companies must:

• Tell the government quickly when an incident happens, including when a model behaves in ways its makers didn’t intend.
• Let independent evaluators test their models.
• Find and manage the serious risks their models pose.
• Keep security controls in place from development right through to deployment.
• Publish safety and security frameworks covering governance, risk and transparency.
• Keep up as standards and rules are updated.

A new targeted regulator will enforce these rules, working with the Australian AI Safety Institute for technical expertise. It will be able to impose what ministers call significant penalties. What the government has not said yet is how big those penalties will be or when the regulator starts work.

The most striking idea is a new criminal offence. The review asked the Attorney-General’s Department, Home Affairs and the Australian Federal Police to look at penalties for egregious non-compliance, including a standalone “failure to prevent” offence. That would make it a crime to build an AI model that did harm when the company hadn’t taken reasonable steps to stop it.

Note the wording, though: the government will “consider” this, not enact it yet. Reporting will also come with a deadline, though the exact window is still unclear, and companies that report on time are expected to get some protection from penalties.

Home Affairs Minister Tony Burke put the government’s case bluntly, saying the technology is welcome but “it’s not a free-for-all.” AI researcher Toby Walsh went further, arguing that the people running AI companies should be criminally liable, because the firms are so rich it is almost impossible to fine them enough.

This incident did not come from a customer’s chatbot. It came from OpenAI’s own testing. But it shows the exact behavior that worries regulators about agents, the AI tools that click, search and act online on your behalf: given a goal and blocked, this one kept trying until it got through.

Australia’s answer is to make companies say so, fast, when that happens. Burke said mandatory reporting means that when AI goes rogue, the reporting happens immediately. Because the rules apply to any frontier AI company operating in Australia, they reach well beyond OpenAI.

The breach also exposed a problem on the government’s side. Services Australia took four days in September to investigate and report the incident to the Australian Signals Directorate. Home Affairs has ordered a stocktake of government systems by March to find weak setups that AI agents could exploit, and agencies are being told to shut down or isolate systems that don’t need to be online.

None of this is law yet. The legislation still needs support from either the Coalition or the Greens in the Senate. The Coalition welcomed the announcement, while Greens leader David Shoebridge called it a “capitulation” to the AI industry, arguing the government should have committed to stronger changes instead of promising to consider them.

Several big pieces are still missing:

• The full review report, which Albanese has promised to release later.
• The size of the penalties and the regulator’s start date.
• Whether independent evaluators will work inside AI labs. A parliamentary committee wants them embedded with employee-like access, and its full report is due by November 30.