Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SurgSkill-Bench, a novel benchmark dataset designed for the automated assessment of surgical skills, comprising 214 surgical simulation videos, six-dimensional OSATS scores, and expert feedback. The authors evaluate two settings: a video-only prediction model and a comment-assisted prediction model, demonstrating that content-adaptive key-frame sampling enhances performance in the video-only setting, while expert comments significantly improve score prediction accuracy. The best-performing model achieves a mean AUROC of 0.88, highlighting the potential of integrating multimodal inputs for surgical skill assessment.
Expert comments can dramatically enhance automated surgical skill assessments, achieving an impressive AUROC of 0.88 when combined with video data.
Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on labor-intensive expert review. Existing automated approaches primarily focus on visual inputs and provide limited support for jointly studying operative performance, structured skill scores, and evaluator feedback. We introduce SurgSkill-Bench, an initial video-score-text benchmark-style dataset containing 214 surgical training simulation videos, six-dimensional OSATS scores, and expert free-text comments. We define two evaluation settings: video-only OSATS prediction for automated assessment and post hoc expert-comment-assisted prediction, where evaluator comments are available as auxiliary information. We provide controlled baseline experiments using representative frozen visual backbones, content-adaptive key-frame sampling, and a simple video-text co-attention fusion module. Under internal video-level validation, content-adaptive sampling improves video-only performance in this dataset, while evaluator comments provide additional score-related signal in the assisted setting. The best mean AUROC reaches 0.88 under dataset-specific median dichotomization. We further discuss evaluation constraints related to dataset scale, metadata completeness, and the interpretation of comment-assisted prediction. Code will be released publicly at a later date.