Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of algorithm registers in providing transparency within sociotechnical systems, specifically through a case study of a Dutch city's decision-support tool for welfare benefits eligibility. By employing interviews, surveys, and participatory system mapping workshops with various stakeholders, the authors reveal that the algorithm register alone obscures critical safety hazards, such as wrongful denial of benefits and system performance issues. The findings underscore the importance of incorporating diverse stakeholder perspectives to enhance system-level transparency and accountability in algorithm governance.
Algorithm registers can obscure critical safety hazards that only emerge when diverse stakeholder perspectives are integrated into the analysis.
Algorithm registers have been championed as a means of providing transparency on the use of algorithms in public services. Yet potential publics differ in their expectations of what should be made transparent and how, as well as in their interest in and ability to parse the information currently published in the registers. Moreover, it remains unclear how these instruments can represent the sociotechnical systems in which these algorithms are embedded, and how system-level transparency can facilitate accountability. In this paper, we ask, what do algorithm registers reveal (and occlude) about the sociotechnical systems governing algorithmic systems, and how can diverse stakeholder perspectives inform a more pluralistic system-theoretic safety analysis? To do this, we probe the municipal algorithm register of a Dutch city through a case study of a decision-support tool for caseworkers'assessment of citizens'welfare benefits eligibility based on legal automation through a business rule engine. Through interviews, surveys, and participatory system mapping workshops (with municipal staff, civil society organisations, and ombudsmen, N=8), we seek to understand to what extent the register allows stakeholders to map the algorithmic system in question. These maps inform a System-Theoretic Process Analysis (STPA) that situates the register within a wider sociotechnical governance structure. Participants'contributions allow us to identify potential safety hazards which would not have been possible to see using the algorithm register alone, including benefits eligibility denial, system performance deterioration, and inability to contest wrongful decisions. By engaging both direct and indirect stakeholders, we reflect on the normative dimensions of algorithm governance efforts and how politics shape the practice of system safety analysis.
