Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces TouchTherm, a framework for creating digital twins of real-world objects that capture both tactile and thermal properties for simulation. By combining structured-light scanning with photometric stereo for micro-height fields and physics-regularized dynamic thermal field reconstruction from IR videos, TouchTherm enables high-fidelity visuo-tactile-thermal rendering. Experiments demonstrate the framework's ability to reconstruct detailed surface structures and accurately predict transient temperature dynamics, leading to successful tactile object recognition and realistic thermal feedback in VR.
Realistic tactile and thermal object properties can now be reconstructed from real-world scans, enabling physics-accurate simulation of touch and temperature in virtual environments.
Robotic simulation and virtual reality increasingly require object assets that capture not only visual geometry but also the physical cues underlying tactile and thermal interaction. Existing 3D datasets and reconstruction methods primarily represent object-scale geometry and visual appearance, overlooking microscale surface structure for high-fidelity haptic rendering and transient temperature dynamics for temperature-aware interaction. We present TouchTherm, a framework for constructing simulation-ready visuo-tactile-thermal object assets from real-world objects. For visual and tactile reconstruction, we combine structured-light scanning with multiview normal maps obtained from photometric stereo. The normal maps are registered to the scanned geometry and transformed into tangent space to recover local micro-height fields for optical tactile rendering, while the coarse mesh handles collision detection. For thermal reconstruction, we capture synchronized multiview infrared videos of natural cooling following controlled heating and reconstruct a physics-regularized dynamic thermal field. Experiments on 20 objects show that the reconstructed micro-height fields preserve dominant surface structures and recover higher-frequency details beyond the coarse geometry, while the thermal fields achieve held-out surface-temperature MAEs of 0.465 degrees C and 0.592 degrees C at 30 s and 45 s, respectively. The resulting tactile assets support synthetic-to-real object recognition from tactile observations, while a glove-based VR system demonstrates spatially and temporally varying thermal feedback. These results highlight the potential of TouchTherm for multimodal sensory simulation and temperature-aware virtual interaction.