Search papers, labs, and topics across Lattice.
This study uncovers a significant correlation in the variance fields of path-traced synthetic stereo data, revealing that while Monte Carlo noise streams are independent, their variance fields exhibit a strong Pearson correlation when aligned by ground-truth disparity. The research, conducted across 20 scenes using Mitsuba 3, shows that this correlation is particularly pronounced in Lambertian surfaces, suggesting that it could serve as a valuable cue for disparity estimation. The findings indicate that disrupting this correlation leads to a substantial decrease in performance, highlighting its potential as a shortcut in sim-to-real applications for disparity estimation networks.
The variance fields in synthetic stereo data reveal a hidden correlation that could drastically enhance disparity estimation performance in neural networks.
Path-traced synthetic stereo data underlie a large fraction of modern disparity-estimation training pipelines. We report a previously unrecognised property of such data: while the Monte Carlo (MC) noise streams of the two cameras are statistically independent, the underlying \emph{variance fields} -- deterministic per-pixel functions of the rendering integrand -- are highly correlated once aligned by the ground-truth disparity warp. Across 20 scenes rendered with Mitsuba~3, the warped Pearson correlation reaches $蟻{=}0.754{\pm}0.016$ across 20 scenes at $\mathrm{SPP}{=}512$, and on a representative scene remains essentially invariant ($蟻{=}0.778{\pm}0.001$) over a $16\times$ range of samples per pixel. The effect is strongest in Lambertian regions ($蟻{\approx}0.78$) and substantially weaker in glass ($蟻{\approx}0.30$), as predicted by an integrand decomposition into view-independent and view-dependent components. A residual-shuffle intervention that breaks the cross-view alignment while preserving the clean image degrades the GT cost margin by $33\%$ on non-glass and the variance-based winner-take-all accuracy on glass by $4.3\times$, confirming the structure functions as a matching cue. This signal is unique to MC-rendered data and constitutes a candidate sim-to-real shortcut whose impact on trained networks remains to be quantified.
