Search papers, labs, and topics across Lattice.
This paper introduces a language-guided system for authoring multimodal textures, coordinating haptic feedback (sliding vibrations and tapping transients) with a text-prompted visual preview. The system uses a shared, language-aligned latent space to ensure semantic consistency between the haptic and visual modalities, enabling users to design textures using natural language prompts. User studies demonstrate that the system allows for coherent cross-modal experiences and efficient prompt-based iteration, replacing manual parameter tuning with text-guided refinement.
Imagine designing a texture by simply *describing* it – this system lets you feel and see your words come to life as coordinated haptic and visual signals.
Authoring realistic haptic textures typically requires low-level parameter tuning and repeated trial-and-error, limiting speed, transparency, and creative reach. We present a language-driven authoring system that turns natural-language prompts into multimodal textures: two coordinated haptic channels - sliding vibrations via force/speed-conditioned autoregressive (AR) models and tapping transients - and a text-prompted visual preview from a diffusion model. A shared, language-aligned latent links modalities so a single prompt yields semantically consistent haptic and visual signals; designers can write goals (e.g.,"gritty but cushioned surface,""smooth and hard metal surface") and immediately see and feel the result through a 3D haptic device. To verify that the learned latent encodes perceptually meaningful structure, we conduct an anchor-referenced, attribute-wise evaluation for roughness, slipperiness, and hardness. Participant ratings are projected to the interpretable line between two real-material references, revealing consistent trends - asperity effects in roughness, compliance in hardness, and surface-film influence in slipperiness. A human-subject study further indicates coherent cross-modal experience and low effort for prompt-based iteration. The results show that language can serve as a practical control modality for texture authoring: prompts reliably steer material semantics across haptic and visual channels, enabling a prompt-first, designer-oriented workflow that replaces manual parameter tuning with interpretable, text-guided refinement.
