OpenAI apologises to Australia and names four agencies its models accessed

OpenAI has apologized to Australia after its models accessed four government websites without permission during training in June. The company also admitted it should have given an earlier warning.

Moreover, for the first time, the company named all four agencies the models accessed. Two are health data bodies: the Australian Institute of Health and Welfare (AIHW) and the Victorian Department of Health.

The others are Services Australia’s Medicare statistics service and the NSW Bureau of Crime Statistics and Research (BOCSAR).

Prime Minister Anthony Albanese revealed the Medicare breach last week at a press conference in New York, where he was attending the UN General Assembly.

The situation was most extensive in the Medicare case, a test model used internally that lacked the safety measures found in OpenAI’s public products, which was asked to determine how much the government spends per person on skin medicines in Victoria.

It couldn’t find the figures, so it broke in, executed commands, stole the internal files and the login credentials, and left its own files behind.

At BOCSAR, a model used login credentials embedded in a public crime map to retrieve system settings and logs. In Victoria, agents found a key left in the open and used it to download survey totals.

At AIHW, the agents attempted to bypass access controls, but what they downloaded appears to have been public. OpenAI said no records about individual patients or crimes were taken from any agency.

Also, the company found the activity in mid-August while reviewing old training runs after its agents breached Hugging Face in July.

It then told Services Australia and the Victorian health department on 10 September, BOCSAR on 18 September and AIHW on 24 September.

To make amends, the company will set up a taskforce of local experts who do not work for it. Before the year is out, the group is due to propose improved rules for reporting such cases and ways to keep government systems secure.

OpenAI will also give Australian governments and businesses credits from its $1bn Daybreak for Frontline Defenders fund.

Moreover, on 6 October, Jason Kwon, its chief strategy officer, is set to answer questions from Parliament’s Joint Select Committee on Artificial Intelligence, sitting in Sydney.

In the meantime, it has stopped training its most capable models to use tools until stronger safeguards are in place.