AI, public governance and the cult of innovation: history, problems and possibilities

In human history, new technologies seem always to be attended by utopian hopes, and equally dark fears – in the case of AI, the fear that it will make existing inequalities worse, as well as increase vulnerable peoples’ fear that they are governed and scrutinised in the cold gaze of a machine. Just as thinkers and writers have worried about technology’s effect on the human spirit, policy-makers have grappled with the question of how to regulate and manage AI technologies, which are frequently adopted by bureaucracies without adequate scrutiny, and at the behest of private companies and consultants sheltering their algorithms in the ‘black box’, and behind trade secrets laws. This chapter surveys the history of problems in automated governance in Europe, the United States and Australia, from the UK’s Post Office scandal, to the SyRI episode in Holland, MiDAS and COMPAS in the United States, and Robodebt in Australia. It identifies common features and differences, before looking ahead at potential solutions to the issues raised, and more generally at the future of automated governance.