Search papers, labs, and topics across Lattice.
The paper introduces MeowCrophone, a voice-controlled interface for the Scratch programming environment designed to improve accessibility for children with motor disabilities. MeowCrophone uses a multi-modal voice user interface with numerical overlays and label reading to enable code editing and workspace navigation without physical input. A multi-stage matching pipeline, incorporating regular expressions, phonetic matching, and a custom grammar, significantly improves speech recognition accuracy, achieving 82.8% overall success compared to a 46.4% baseline.
Voice control, previously insufficient for block-based programming, can now enable children with motor disabilities to effectively use Scratch, thanks to a novel multi-stage speech recognition pipeline.
Block-based programming environments like Scratch have become widely adopted in Computer Science Education, but the mouse-based drag-and-drop interface can challenge users with disabilities. While prior work has provided solutions supporting children with visual impairment, these solutions tend to focus on making content perceivable and do not address the physical interaction barriers faced by users with motor disabilities. To bridge this gap, we introduce MeowCrophone, an approach that uses voice control to allow editing code in Scratch. MeowCrophone supports clicking elements, placing blocks, and navigating the workspace via a multi-modal voice user interface that uses numerical overlays and label reading to bypass physical input entirely. As imperfect speech recognition is common in classrooms and for children with dysarthria, MeowCrophone employs a multi-stage matching pipeline using regular expressions, phonetic matching, and a custom grammar. Evaluation shows that while free speech recognition systems achieved a baseline success rate of only 46.4%, MeowCrophone's pipeline improved results to 82.8% overall, with simple commands reaching 96.9% accuracy. This demonstrates that robust voice control can make Scratch accessible to users for whom visual aids are insufficient.
