Search papers, labs, and topics across Lattice.
This study assessed the accuracy of Google AI Overview in identifying correct CPT codes for 100 common hand and upper extremity procedures using 150 simplified search phrases. The overall accuracy was 90%, with tendon and nerve procedures showing lower accuracy (78% and 71%, respectively) compared to bony procedures (88%) and other categories. The authors conclude that Google AI Overview can be a helpful tool but should not be considered perfectly accurate for CPT coding.
Google AI Overview provides accurate CPT codes for hand surgery procedures 90% of the time, but accuracy varies by procedure type, suggesting caution when relying on it for coding, especially for tendon and nerve procedures.
PURPOSE As the Google Artificial Intelligence (AI) Overview function becomes engrained in the search engine, it is important to understand the accuracy of these search results as they pertain to Current Procedural Terminology (CPT) codes. We hypothesized that the identified codes will not be 100% accurate and that there will be subgroups of procedures with relatively lower accuracy than others. METHODS One hundred common CPT codes used in hand and upper-extremity surgery were selected and searched in Google using 150 different simplified phrases describing the procedure. The additional 50 phrases were intended to provide more detail on the importance of phrasing variation, for example, "arthrodesis" versus "fusion". The accuracy of the codes was assessed using American Academy of Orthopaedic Surgeons CodeX and the American Society for Surgery of the Hand Coding App. The accuracy was calculated as a percentage. RESULTS Google AI Overview's response provided an accurate CPT Code 90% of the time. Tendon procedures (78%) and nerve procedures (71%) had lower accuracy than the remainder of the groups. Bony procedures had an 88% accuracy rate, and the remainder of the categories had over 90% accuracy. Ten percent of queries resulted in multiple codes; 14 of these 15 queries contained the correct code. Fifty-nine acronyms were used, and searches with acronyms resulted in accurate codes at a similar rate to those without acronyms. CONCLUSIONS This study demonstrates that providers and billers can use Google AI Overview as a tool but should not rely on it as being 100% accurate. CLINICAL RELEVANCE As AI becomes more involved in everyday clinical practice, hand surgeons should be aware of the potential strengths and weaknesses as it pertains to coding.
