Search papers, labs, and topics across Lattice.
This study investigates the trustworthiness ideals in AI-powered information seeking within maternal and reproductive health by conducting focus groups with birthing individuals, clinicians, and health workers. The research highlights that in high-stakes health contexts, stakeholders emphasize the necessity for inspectable trustworthiness rather than mere assertions of credibility, revealing a consensus on the importance of transparency and recourse in AI systems. The findings culminate in four governance themes that inform the design of AI tools, advocating for a mistrust-aware approach that enhances user confidence and system accountability.
In high-stakes health contexts, stakeholders demand that trustworthiness in AI systems be inspectable, not just asserted, reshaping how we design health information tools.
AI-powered tools increasingly promise to fill information gaps in health, especially in domains like maternal and reproductive health that demand timely, accurate, and actionable information. This is extremely important, as the United States leads peer nations in preventable deaths, with stark racial disparities. However, current AI and NLP-powered systems aim to improve access to vetted maternal health information by routing user queries to a factual response while under-specifying the socio-technical governance structures that shape trust, use, and harm in practice. We report findings from four synchronous focus groups ($n=24$) with three stakeholder groups central to peripartum information support: birthing people, clinicians, and health workers (e.g., doulas, social workers, community health workers) exploring topics around information seeking, experience with current clinical infrastructure, misinformation, and an AI-enabled factual answering tool design probe. Our inductive analysis surfaces a central finding: in high-stakes health contexts shaped by historical inequities, trustworthiness must be inspectable and not asserted. While stakeholders diverge on what makes information credible, they converge on the need for transparency, recourse, and ecosystem complementarity. Based on the discussions, we identify four themes and governance requirements: (1) support for social and identity-based sensemaking, (2) pluralistic verification practices, (3) inspectable governance with recourse mechanisms, and (4) ecosystem-aware integration that avoids shifting burden. Building on these findings, we propose design artifacts that are mistrust-aware and promote principled governance mechanisms for transparent, pluralistic AI systems. Finally, we discuss the implications of our findings for expanding human-AI evaluations and improving the transparency of deployed AI systems.
Google Research