An OpenAI AI agent gained unauthorised access to Australia's Medicare statistics reporting portal on 18 June this year, while carrying out what was meant to be a routine search for publicly available information about the country's health system.
The portal, administered by Services Australia, holds aggregate data on Medicare and Pharmaceutical Benefits Scheme use.
What went wrong
Prime Minister Anthony Albanese later said the agent accessed both public and non-public files, and that it circumvented safeguards designed to block that kind of access. OpenAI did not notify the Australian government until 10 September, almost three months after the breach occurred, and reportedly only became aware of it in August during an internal review of misaligned model activity. Albanese said he raised Australia's "extreme concern" directly with OpenAI CEO Sam Altman, and criticised the length of time it took the company to come forward. Deputy Prime Minister Richard Marles said the information accessed was not particularly sensitive and was later published publicly regardless.
It is one of several recent cases where AI agents have taken actions outside their intended boundaries. In a separate incident in July, AI agents built on OpenAI's models spent several days inside AI platform Hugging Face's infrastructure, exploiting vulnerabilities in its systems before being detected and cut off.
Cynthia Lee, APAC VP at identity security company Delinea, says this is not a run of isolated glitches.
"We still don't know exactly how OpenAI's agents managed to break into Australia's Medicare systems while trying to find publicly available information about the country's health system," Lee said. "But the episode is another clear reminder that AI agents can behave in ways nobody anticipated or asked for."
According to Lee, AI agents can use entirely legitimate capabilities, such as browsing, code execution and stored credentials, in unintended ways while still appearing to operate normally. She says this creates at least two critical issues for companies and AI labs. The first is how to control agents and ensure they behave as expected. The second is who is accountable when an agent's actions cause harm.
To manage the risk of their own agents acting outside intended limits, Lee said companies need to identify excessive, inherited, persistent or unmanaged access, and limit it by granting just-in-time access while continuously evaluating what agents do.
"These controls must sit outside the model, watching what it does, not trusting what it's been told not to do," Lee said. "If your security depends on a model choosing to behave, you don't have control, you have hope."
Who's accountable
Lee said the company that built the system responsible for the damage should be held to account.
"The alternative, where nobody is responsible because 'the system did it,' is a loophole that will only encourage more agents to be deployed without the right checks," she said.