Search papers, labs, and topics across Lattice.
This paper introduces RISTER, a novel end-to-end Rotation-Invariant Scene Text Recognition network that integrates rotation equivariance in the encoder and rotation invariance in the decoder. By leveraging a rotation-invariant cross-attention mechanism and a rotation-equivariant local-global extraction network, RISTER addresses the limitations of existing methods that suffer from error accumulation and high computational costs. The results demonstrate that RISTER achieves state-of-the-art performance, outperforming the second-best model by 4.0 percent in accuracy on multi-oriented text recognition benchmarks, while providing theoretical guarantees for rotation invariance.
RISTER achieves state-of-the-art accuracy in multi-oriented scene text recognition by embedding rotation invariance with theoretical guarantees, eliminating reliance on data-driven orientation correction.
Multi-oriented text is ubiquitous in real-world scenes and remains a major challenge for scene text recognition (STR). Existing rotation-aware methods explicitly estimate text orientation. However, due to the lack of theoretical guarantees, they are prone to error accumulation, increased computational cost, and strong reliance on data. In this work, we incorporate rotation invariance into the STR framework to address these limitations. Specifically, we adopt an encoder-decoder architecture, embedding rotation equivariance in the encoder and rotation invariance in the decoder to construct a fully rotation-invariant network. On the decoder side, we first identify and prove the rotation-invariant property of the cross-attention mechanism and use it to formulate a rotation-invariant text decoder that maps visual features to output text in a rotation-invariant manner. On the encoder side, we propose a rotation-equivariant local-global extraction network that integrates deep equivariant convolutions with self-attention, enabling rotation-equivariant feature extraction while modeling inter-character dependencies and preserving fine-grained visual details. By integrating the encoder and decoder, we obtain an end-to-end Rotation-Invariant Scene Text Recognition network (RISTER). RISTER provides rotation invariance with theoretical guarantees, enhancing robustness on multi-oriented samples without introducing additional inference computation or relying on data-driven orientation correction. Experiments show that RISTER achieves state-of-the-art performance on both standard and multi-oriented benchmarks, surpassing the second-best model by 4.0 percent in accuracy on the general multi-oriented dataset.
