Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper introduces WHTMix, a novel method for stereo depth estimation that replaces the computationally expensive global self-attention in transformer-based models with a data-independent Walsh-Hadamard token mixer, achieving log-linear complexity. By retaining data-dependent cross-attention for left-right correspondence, the method matches the performance of traditional attention mechanisms while significantly reducing compute requirements and inference latency. Experimental results on synthetic driving data demonstrate a 2.46x reduction in model compute and a 2.65x decrease in single-image inference latency, highlighting the method's efficiency in high-resolution stereo matching scenarios.
Achieving state-of-the-art stereo depth estimation with 2.65x faster inference and 2.46x lower compute costs using a novel token mixing approach.
Stereo depth estimation for driving, robotics and augmented reality must run at high resolution under tight latency budgets, yet in transformer-based matchers the global self-attention that aggregates scene context grows quadratically with the number of pixels and comes to dominate runtime. We show that the joint self-attention stage of a stereo transformer, whose role is to spread context across both views, can be replaced by a data-independent Walsh-Hadamard token mixer that mixes tokens globally in the transform domain at log-linear cost, while the data-dependent cross-attention that performs left-right correspondence is retained. On synthetic driving data the mixer matches the attention baseline in end-point error while reducing model compute by a factor of 2.46 and single-image inference latency by a factor of 2.65. A complexity analysis shows the benefit is governed by the ratio of sequence length to channel width, which explains why high-resolution stereo matching is a particularly favorable setting and why classification transformers are not; we confirm this token-to-channel scaling on non-stereo long-sequence benchmarks. Furthermore, we introduce a hybrid log-disparity loss function designed to up-weight small-disparity pixels corresponding to long-range objects. This approach reduces the error on distant objects without incurring any additional computational overhead.