An OpenAI agent broke into Australia's Medicare portal. Here's why the UK is just as exposed

This month, Australia's prime minister, Anthony Albanese, revealed something few people thought they would hear so soon. An artificial intelligence model built by OpenAI had got into a government health website without permission.

The model had not been released to the public. It was being tested inside OpenAI in June when it reached the Medicare statistics portal, and possibly three other government websites, while trying to find answers to questions about Australia. OpenAI says its models "took actions we did not intend." The company found the activity in August but did not tell the Australian government until 10 September. Albanese called the delay "unacceptable".

The portal held aggregate statistics, not patient records, and no personal data has been confirmed as accessed. But the incident is widely described as the first known case of an autonomous AI agent getting into a government system. British readers may be tempted to see it as a distant problem. They shouldn't.

The same companies run through British public services

The UK government keeps a public register of the algorithms and AI tools used by public bodies, called the Algorithmic Transparency Recording Standard. I went through all 152 records currently published. In 111 of them, an outside company builds, hosts or supplies the technology.

A small group of US technology firms dominates that list. Microsoft appears in around 40 records, Amazon Web Services in 29, OpenAI in 25, Google in 16 and Anthropic in nine. Most government use of OpenAI's models runs through Microsoft's cloud, and most use of Anthropic's through Amazon's.

These are not obscure back-office systems. In Newcastle, an AI tool records and summarises conversations between social workers and adults receiving care, and an OpenAI model is among the services that process them. In the London borough of Sutton, sensors in about 2,000 residents' homes flag unusual behaviour, and OpenAI's GPT-4o writes a weekly summary on each person. The Ministry of Justice uses Amazon's facial recognition to check the identity of people on probation during remote check-ins, and immigration judges use Microsoft's speech recognition to transcribe their decisions.

Could it happen here?

The Australian incident did not involve a government tool going wrong. It involved an AI company's own experimental agent going out onto the internet and getting into a system it had no right to enter. Britain has plenty of public health and statistics portals that such an agent could reach in exactly the same way.

The more important question is whether we would find out, and how quickly. Here the UK's position looks uncomfortably like Australia's: it has no AI act, relying on existing regulators such as the Information Commissioner's Office and the medicines regulator to apply existing laws. Data protection law requires a breach to be reported within 72 hours, but only when personal data is involved. An incident like Medicare's, where an agent reaches aggregate data, may not trigger that duty at all.

The main new security law, the Cyber Security and Resilience Bill – now before the House of Lords – puts duties on the organisations that use AI rather than on the companies that build it. During the bill's scrutiny, Baroness Kidron put it bluntly: "the NHS must protect itself, but the AI that is attacking it has no requirement… to check itself." The binding rules for the most powerful AI companies that Labour promised in its 2024 manifesto have not appeared. OpenAI itself urged the UK government this month to legislate.

The register is a real strength. Few countries publish anything like it. But it is only government policy for central departments. Councils and most of the NHS publish records voluntarily, so the true scale of AI use in British public services is unknown.

Three things the UK could do now

First, AI developers should have a legal duty to report serious incidents to government within a fixed deadline, whether or not personal data is involved. Seventy-two hours would match existing data breach rules. Australia learned of its breach almost three months late.

Second, the transparency register should be compulsory across the whole public sector, including councils and NHS bodies. The public cannot judge risks it cannot see.

Third, any AI tool used in health, social care or justice should publish how often it makes mistakes, and for whom. "A human checks it" is not enough when people are reviewing hundreds of mostly correct outputs a day.

None of this would stop an AI agent from misbehaving. It would mean that when one does, Britain finds out quickly and knows exactly which of its services depend on the company responsible. Australia has shown what happens when a government has to learn about an AI incident second-hand and late. The UK still has time not to repeat it.

Dr Sebastian Smart, Senior Research Fellow, School of Economics, Finance and Law