Search papers, labs, and topics across Lattice.
This paper investigates the use of Reasoning Language Models (RLMs) to assess parental cooperation in Child Protective Services (CPS) interventions using complex, ambiguous case reports. A four-stage workflow was developed, incorporating case report collection, reasoning-based assessment, automated category extraction, and case labeling. The largest RLM (255B parameters) achieved 89% accuracy in classifying parental cooperation, outperforming a baseline approach and demonstrating the potential of RLMs for complex social work assessments.
Language models can achieve near-human accuracy in assessing complex social factors like parental cooperation from messy case reports, but also inherit human biases in the process.
Purpose: Reasoning language models (RLMs) have demonstrated significant advances in solving complex reasoning tasks. We examined their potential to assess parental cooperation during CPS interventions using case reports, a case factor characterized by ambiguous and conflicting information. Methods: A four stage workflow comprising (1) case reports collection, (2) reasoning-based assessment of parental cooperation, (3) automated category extraction, and (4) case labeling was developed. The performance of RLMs with different parameter sizes (255B, 32B, 4B) was compared against human validated data. Two expert human reviewers (EHRs) independently classified a weighted random sample of reports. Results: The largest RLM achieved the highest accuracy (89%), outperforming the initial approach (80%). Classification accuracy was higher for mothers (93%) than for fathers (85%), and EHRs exhibited similar differences. Conclusions: RLMs' reasoning can effectively assess complex case factors such as parental cooperation. Lower accuracy in assessing fathers' cooperation supports the argument of a stronger professional focus on mothers in CPS interventions.
