Disadvantage and the automated decision

Automated decision-making is becoming increasingly prevalent in Australia. Familiar examples include MyGov for tax and social security benefits, and the use of SmartGates when arriving in Australia. Yet vulnerable populations have been detrimentally affected by the Australian government’s use of automated processes. This is illustrated by the Robodebt debacle, where errors of methodology of government decision-making resulted in incorrect or inflated debt calculations for over 470,000 individuals. This article focusses on the impact of automated decision-making on vulnerable individuals. It will closely examine automated decision-making in the context of social security, focussing on the Robodebt and ParentsNext programs, as well as the recent incursions of automated decision-making into income management for vulnerable youth and changes proposed under the National Disability Insurance Scheme. Further, this article will also consider the related issues raised by automated decision-making in the context of criminal justice, from the investigatory stage — including facial recognition technology and police databases to identify offenders — through to sentencing. Finally, this article will develop guiding principles to protect the rights of vulnerable populations. These include safeguards at all stages of the automated decision-making process, including the design, implementation and evaluation of new technologies.