Algorithmic Harms in Child Welfare: Uncertainties in Practice, Organization, and Street-level Decision-Making

by   Devansh Saxena, et al.

Algorithms in public services such as child welfare, criminal justice, and education are increasingly being used to make high-stakes decisions about human lives. Drawing upon findings from a two-year ethnography conducted at a child welfare agency, we highlight how algorithmic systems are embedded within a complex decision-making ecosystem at critical points of the child welfare process. Caseworkers interact with algorithms in their daily lives where they must collect information about families and feed it to algorithms to make critical decisions. We show how the interplay between systemic mechanics and algorithmic decision-making can adversely impact the fairness of the decision-making process itself. We show how functionality issues in algorithmic systems can lead to process-oriented harms where they adversely affect the nature of professional practice, and administration at the agency, and lead to inconsistent and unreliable decisions at the street level. In addition, caseworkers are compelled to undertake additional labor in the form of repair work to restore disrupted administrative processes and decision-making, all while facing organizational pressures and time and resource constraints. Finally, we share the case study of a simple algorithmic tool that centers caseworkers' decision-making within a trauma-informed framework and leads to better outcomes, however, required a significant amount of investments on the agency's part in creating the ecosystem for its proper use.


A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare

Algorithms have permeated throughout civil government and society, where...

Designing Human-Centered Algorithms for the Public Sector: A Case Study of the U.S. Child-Welfare System

The U.S. Child Welfare System (CWS) is increasingly seeking to emulate b...

Digitizing Municipal Street Inspections Using Computer Vision

"People want an authority to tell them how to value things. But they cho...

Algorithmic Monoculture and Social Welfare

As algorithms are increasingly applied to screen applicants for high-sta...

Rethinking "Risk" in Algorithmic Systems Through A Computational Narrative Analysis of Casenotes in Child-Welfare

Risk assessment algorithms are being adopted by public sector agencies t...

A Conceptual Framework for Using Machine Learning to Support Child Welfare Decisions

Human services systems make key decisions that impact individuals in the...

Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis

Caseworkers are trained to write detailed narratives about families in C...

Please sign up or login with your details

Forgot password? Click here to reset