Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces FUTO Swipe, a layout-agnostic neural swipe decoder that can accurately interpret swipe gestures across various mobile keyboard layouts without needing retraining for each specific layout. By employing geometric augmentations during training, the model learns to predict character locations based on swipe characteristics rather than being tied to a specific keyboard configuration. The resulting model not only generalizes effectively to non-QWERTY layouts but also achieves accuracy that sometimes surpasses that of models trained on specific layouts, supported by the release of the largest MIT-licensed swipe dataset to date.
Swipe decoding just got a major upgrade—FUTO Swipe can accurately interpret gestures across any keyboard layout without retraining.
Neural swipe decoders are typically tied to the keyboard they were trained on, requiring a new corpus and training run for each layout. In this report, we document our approach toward training models that can function on any contiguous mobile keyboard layout. At each point along the swipe, our encoder predicts whether the user is indicating a character and where on the keyboard that character lies. The keyboard layout is supplied at inference time and used to map the spatial and temporal prediction to a logit at each key, rather than being learned during training. Training neural models requires substantial data, but public swipe data is limited, particularly for non-QWERTY layouts. We release swipe.futo.org, the largest MIT-licensed swipe corpus we are aware of, containing over 1M donated swipes from more than 12k donor sessions. To generalize beyond the English QWERTY layout, we apply geometric augmentations to both the swipe trajectory and the keyboard layout at every training step, forcing the model to make predictions based on characteristics of the swipe gesture rather than the training layout. The model generalizes to layouts absent from training, in some cases more accurately than the layout it was trained on. This combines the layout-flexibility of an algorithmic decoder with the accuracy of a neural model. Trained models are publicly available.