Search papers, labs, and topics across Lattice.
This study compared the readability of online information about carpal tunnel syndrome (CTS) from the American Academy of Orthopaedic Surgeons' OrthoInfo website with responses generated by ChatGPT-4 and Google Gemini 2.0 Flash. Using six prompts related to CTS, the readability of information from each source was assessed using multiple indices. The study found no significant differences in overall readability scores between OrthoInfo and the AI platforms, although all sources exceeded recommended reading level thresholds.
AI chatbots provide carpal tunnel syndrome information at a similar readability level to AAOS OrthoInfo, but all sources are written above the recommended reading level.
INTRODUCTION Patients have become increasingly reliant on the internet to seek health-related information (HRI). The newfound popularity of artificial intelligence (AI) search engines has created interest in their ability to provide HRI. This study aimed to quantify and compare the readability of carpal tunnel syndrome (CTS) HRI from the American Academy of Orthopaedic Surgeons OrthoInfo and AI search engines. METHODS Six prompts were developed using the OrthoInfo page on CTS. These prompts were entered to ChatGPT-4 and Google Gemini 2.0 Flash to generate AI responses. The readability of this information was calculated using the Flesch-Kincaid Reading Ease Index, Coleman-Liau Index, Flesch-Kincaid Grade Level, FORCAST Readability Formula, Gunning Fog index, and Simple Measure of Gobbledygook Index. Statistical testing was performed using the Kruskal-Wallis nonparametric One-Way Analysis of Variance test. RESULTS The mean grade level readability score across all platforms, questions, and testing metrics was 12.6. No significant differences were observed between the overall mean grade level readability scores of OrthoInfo, ChatGPT, and Gemini, nor were they observed for any specific prompt. The only significant differences were found using the Flesch-Kincaid Grade Level test, for which ChatGPT had the lowest scores. CONCLUSION The readability of carpal tunnel syndrome health-related information from OrthoInfo, ChatGPT, and Gemini is similar. Physicians should advise patients to continue using OrthoInfo as a primary source of carpal tunnel syndrome information, although artificial intelligence search engines are useful to supplement when patient concerns require more tailored responses. Notably, no text included in this study was at recommended reading level thresholds.
