Search papers, labs, and topics across Lattice.
The paper introduces Syn-STARTS, a framework leveraging LLMs to generate synthetic triage scenarios for evaluating AI models in mass casualty incidents, addressing the scarcity of real-world MCI data. They validated Syn-STARTS by demonstrating that its generated cases are qualitatively similar to manually curated data and by showing stable LLM evaluation accuracy across various triage categories. This suggests synthetic data can effectively support the development of AI for critical medical decision-making.
Synthetic data generated by LLMs can effectively mimic real-world triage scenarios, offering a scalable solution to the data scarcity problem in AI-driven mass casualty incident management.
Triage is a critically important decision-making process in mass casualty incidents (MCIs) to maximize victim survival rates. While the role of AI in such situations is gaining attention for making optimal decisions within limited resources and time, its development and performance evaluation require benchmark datasets of sufficient quantity and quality. However, MCIs occur infrequently, and sufficient records are difficult to accumulate at the scene, making it challenging to collect large-scale realworld data for research use. Therefore, we developed Syn-STARTS, a framework that uses LLMs to generate triage cases, and verified its effectiveness. The results showed that the triage cases generated by Syn-STARTS were qualitatively indistinguishable from the TRIAGE open dataset generated by manual curation from training materials. Furthermore, when evaluating the LLM accuracy using hundreds of cases each from the green, yellow, red, and black categories defined by the standard triage method START, the results were found to be highly stable. This strongly indicates the possibility of synthetic data in developing high-performance AI models for severe and critical medical situations.
