Australia Paid for Robodebt and Still Chose Voluntary Rules

Australia has no AI-specific statute. After Robodebt, the automated welfare debt scheme a Royal Commission called "crude and cruel," the government chose voluntary AI guidance, and the one new binding duty on automated decisions starts this December: an organization’s privacy policy must say which kinds of significant decisions about people it lets a computer program make.

My argument cuts against both sides of the Australian debate. Choosing voluntary rules was less reckless than its critics say, because Robodebt broke laws that already existed and used no AI at all. The choice still lets nobody off. Robodebt ran for years because nobody explained its decisions and the people with the power to stop it chose not to, and voluntary guidance fixes neither. I wrote this for the executive in Australia who signs off a system that makes decisions about people. The duty that binds you in December is a first step on the first failure. The rest is yours to build.

What Robodebt Did

The mechanism was arithmetic. From a pilot in 2015 and an online system rolled out in 2016, the Commonwealth matched welfare records against the annual income employers reported to the tax office, spread that income evenly across the year, and raised a debt wherever the averaged figure exceeded what a person had declared. Anyone whose earnings rose and fell through the year could be told they owed money they did not owe. The Royal Commission wrote that "no-one could understand how the debts were calculated, because the debt notices gave no explanation."

When Justice Murphy approved the first settlement in June 2021, he called the case "a shameful chapter in the administration of the Commonwealth social security system and a massive failure of public administration." The figures above are the Court's: the Commonwealth withdrew the debts, refunded what it had recovered and paid A$112 million on top. In June 2026 the Court set those orders aside and approved a second settlement of A$548.5 million, A$475 million of it compensation, after group members reopened the case on evidence from the Royal Commission. The Commission recorded that some people with debts raised against them died by suicide, and called the scheme "a crude and cruel mechanism, neither fair nor legal."

The Law Was Already There

Robodebt is usually told as a story about rules arriving too late. The Royal Commission's report tells a different one. Inside the Department of Social Services, a 2014 advice had already pointed out "the inconsistency of averaging with the legislative framework." As run, the scheme "neither produced accurate results nor complied with the income calculation provisions of the Social Security Act 1991." Some tribunal members were ruling against averaged debts. By the end of 2016 the scheme faced heavy public criticism, "but was nonetheless persisted with until November 2019."

Commissioner Catherine Holmes put the turning point years before the end: "The beginning of 2017 was the point at which Robodebt’s unfairness, probable illegality and cruelty became apparent. It should then have been abandoned or revised drastically." Instead, she wrote, "the path taken was to double down."

Read it as a governance case and the missing piece is plain. The law existed, the advice existed, the complaints and the tribunal rulings existed. Nobody explained a single debt to the person who received it, and the people with the power to stop the system chose not to. I have argued in Nobody Tells You When to Stop an AI Deployment that this second gap is the one most organizations still have.

What Australia Chose Instead

Australia's answer has been guidance. The AI Ethics Principles of 2019 were voluntary. In September 2024 the government published a Voluntary AI Safety Standard with ten guardrails and, beside it, a proposals paper on mandatory guardrails for AI in high-risk settings. In October 2025 the Guidance for AI Adoption folded the guardrails into six practices, from deciding who is accountable to maintaining human control. The National AI Plan of 2 December 2025 then said regulation would continue to build on existing legal and regulatory frameworks, and committed A$29.9 million to an AI Safety Institute that "will support existing regulators with independent advice." The mandatory guardrails were not taken forward.

The government holds itself to more than it asks of business. Since 15 December 2025 its policy for the responsible use of AI in government has required every non-corporate Commonwealth entity to begin impact assessments of its AI use cases within 12 months and to name an accountable owner for each. And in September 2023 it agreed that people should have "a right to request meaningful information about how automated decisions with legal or similarly significant effect are made." The 2024 privacy amendments did not enact that right.

The Duty That Starts on 10 December 2026

The Privacy and Other Legislation Amendment Act 2024 received Royal Assent on 10 December 2024. Its Part 15 adds Australian Privacy Principles 1.7 to 1.9, which start on 10 December 2026. From that day, if an organization has arranged for a computer program to make a decision, or to do something "substantially and directly related to making" one, and the decision "could reasonably be expected to significantly affect the rights or interests of an individual" whose personal information the program uses, its privacy policy must list three things: the kinds of personal information used, the kinds of decisions made solely by the program, and the kinds where the program does the closely related work.

Three details are easy to miss. The examples in the Act include a decision under a law "to grant, or to refuse to grant, a benefit," which is close to the Robodebt case, and decisions affecting rights under a contract or "access to a significant service or support." The duty covers decisions made after 10 December 2026 even when the system was built years earlier. And APP 1.7 joins the short list of privacy principles whose breach is a civil penalty provision, for which the regulator can issue infringement or compliance notices.

Set the new duty beside what the Royal Commission asked for in its recommendations on automated decision-making, and the gap shows.

The recommendations were written for government services, and APP 1.7 reaches businesses too, so the comparison is not exact. The direction is still clear. The law now asks you to say that a computer decides. It does not ask you to say how, to let anyone challenge the result, or to stop. Even the explaining is harder than a policy line suggests. In 2025 a Deloitte review meant to explain the employment department's automated welfare compliance system was published with errors from generative AI, including in its summary of the Amato Robodebt case. I tell that story in I Do Not Fully Understand the AI I Help Govern.

Where the Binding Rules Already Sit

Voluntary at the top does not mean unregulated underneath. APRA's prudential standards on information security and operational risk already reach AI systems in banks, insurers and super funds, and in April 2026 APRA wrote to every regulated entity warning that "governance, risk management, assurance and operational resilience practices are not keeping pace with AI adoption." ASIC reviewed 624 AI use cases at 23 licensees in 2024 and found that in some cases "governance and risk management lagged the adoption of AI." Discrimination law, consumer law and the TGA's medical device rules apply whatever the technology. The regulator-by-regulator detail, with dates, is in the Australia section of my AI regulation tracker.

What to Build Before December

Start with the duty and go past it where Robodebt shows the law is thin.

1. List the decisions. Find every system that makes, or does the work closely related to making, a decision that could significantly affect a person: benefits, contracts, credit, prices, access to a service. Include the ones inside software you buy, which The AI That Hurts You Is Not the AI You Own covers.
2. Write the disclosure in plain words. APP 1.7 asks for kinds of information and kinds of decisions. Add a sentence on how each kind of decision is made, which the Royal Commission asked of government and no law yet asks of you. Your lawyers will prefer a vague line. A Robodebt debt notice was a vague line.
3. Give people a route to a human. Nothing in APP 1.7 requires a review path. Robodebt’s recipients had one, through the tribunal, but the Commission found that using it took an understanding of what had happened that was “virtually impossible to ascertain” from the department’s documents.
4. Name who can stop each system. One person per system, with the authority to pause it and a written test for when they must, triggered by signals from outside the team: complaints, overturned decisions, review findings. That person should not own the system’s savings target, and each system needs a manual fallback, or nobody will pull the lever.

If you sell into Europe, build to the stricter regime once. The EU AI Act lists as high-risk any AI system a public authority uses to "grant, reduce, revoke, or reclaim" public assistance benefits, along with credit scoring and hiring, and after the Digital Omnibus those duties apply from 2 December 2027. An AI version of Robodebt would be on that list.

Where This Argument Is Weakest

The strongest objection is that the law I praise worked slowly and only for people with somewhere to go. Robodebt was ended by tribunal appeals and a Federal Court test case, paid for through a class action and explained by a Royal Commission, years after the harm. A private company faces regulators and courts too, but nothing that looks specifically at how its automated decisions are made, and voluntary guidance gives no regulator a reason to start. The Royal Commission asked for an audit body, and nothing described above creates one.

The second objection is the counterfactual. I cannot show that an AI statute would have changed nothing. A law forcing published business rules or audits might have exposed the averaging in months. My answer is that what Robodebt lacked was an outside power to audit and stop automated decisions, AI or not. That argues for the audit body the Royal Commission proposed, which an AI statute would not by itself supply. Until one exists, build the nearest thing inside your organization, staffed by people who do not own the system’s savings.

Sources and Changes

• Federal Court of Australia, Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634, Murphy J, 11 June 2021.
• Federal Court of Australia, order of Justice Beach, 23 June 2026, VID1252/2019, and the Court-ordered notice of proposed settlement, 31 October 2025 (both hosted by the applicants' solicitors).
• Royal Commission into the Robodebt Scheme, Report, Volume 1, 7 July 2023: pages v, xvi, xxvii to xxix and 181 (recommendations 17.1 and 17.2 on page xvi).
• Privacy and Other Legislation Amendment Act 2024 (No. 128, 2024), section 2 and Schedule 1, Part 15; assented 10 December 2024.
• Attorney-General's Department, Government Response to the Privacy Act Review Report, 28 September 2023, proposal 19.3.
• Department of Industry, Science and Resources: AI Ethics Principles (2019); Voluntary AI Safety Standard (5 September 2024); Guidance for AI Adoption (21 October 2025); National AI Plan (2 December 2025).
• Digital Transformation Agency, Policy for the responsible use of AI in government, version 2.0, effective 15 December 2025.
• APRA, Letter to industry on artificial intelligence, 30 April 2026.
• ASIC, Report 798, Beware the gap: Governance arrangements in the face of AI innovation, 29 October 2024.
• Regulation (EU) 2024/1689 (the AI Act), Annex III, points 4 and 5, as amended by Regulation (EU) 2026/1744, Article 113.

All sources checked on 6 October 2026.

Changes on 6 October 2026. Cut from about 6,300 words to an argument under a new headline (formerly "AI Governance in Australia After Robodebt"). The A$1.87 billion settlement figure was replaced with the Federal Court’s own figures, and the second settlement approved in June 2026 was added. The Privacy Act section now quotes APP 1.7 and no longer says the law requires explanations on request. Bank, mining and sector detail moved to the Australia section of the AI regulation tracker. A download box pointing to a missing file was removed.