Search papers, labs, and topics across Lattice.
Automatic music transcription models struggle with pop music, as evidenced by a mere 38% Onset F1 score on the new MulTTiPop dataset.
Multimodal unlearning could revolutionize how we handle sensitive data in AI, enabling targeted removal without sacrificing model performance.
Achieving 80% throughput with Classifier-Free Guidance challenges the assumption that CFG drastically reduces efficiency in multimodal audio processing.
MoVA achieves superior video-text alignment by disentangling evolving visual concepts from static textual descriptions, outperforming existing models in handling long sequences.
Even top-performing AI models struggle with PowerPoint tasks, achieving only 45% success rates despite a robust evaluation framework that rewards nuanced performance.
ELASTIC reduces wall-clock latency by 34% while matching the success rates of the best-performing models in real-world robot manipulation.
HTT enables tactile learning across diverse sensors, achieving adaptability that was previously unattainable in contact-rich manipulation tasks.
Touch is not just an add-on; it fundamentally enhances object representation, leading to dramatic improvements in physical property estimation and manipulation tasks.
Current multimodal conversational models miss critical emotional cues, revealing a significant gap in their ability to engage in nuanced human-like dialogue.
Lift4D achieves unprecedented accuracy in 4D reconstruction of dynamic objects, even in the presence of severe occlusions and complex motions.
Local names boost retrieval accuracy, but models still fail to generate images that faithfully represent specific street segments.
Instruction blindness in VLA models can be mitigated by optimizing for flatter loss landscapes, leading to over 60% better adherence to language instructions.
Rule-grounded reasoning can cut average distance errors in driving VLAs by nearly half, fundamentally enhancing their decision-making transparency and reliability.
T2I models can be effectively probed for identity memorization without any access to training data, revealing surprising differences in how they handle famous versus less recognized names.
LLMs can outperform humans in predicting the next speaker in meetings, even without audio or visual data.
Dixtral achieves up to 29% absolute improvement in speaker-attributed transcription accuracy by leveraging diarization masks without risking catastrophic forgetting.
TuneJury achieves superior music preference alignment with a single frozen reward model that adapts efficiently to new audio generators.
Humanoid robots can learn to distinguish themselves from others purely through proprioceptive-visual cues, enabling advanced social interactions without predefined identities.
Reducing inter-utterance silence from 9.6 seconds to 0.3 seconds transforms the quality of real-time game commentary, making it feel more natural and engaging.
Joint image-depth generation can be achieved with a single model trained on sparse data, outperforming existing methods by a significant margin.
Unstructured human videos can unlock scalable robot skill acquisition, enabling zero-shot transfer across diverse tasks with minimal supervision.
Behavioral INR reveals that self-supervised learning can effectively disentangle complex, overlapping policies from unlabeled behavioral data, outperforming traditional methods in high-dimensional settings.
A conformalized language feedback policy can boost VLA performance by over 65% while ensuring safe and reliable task execution in novel environments.
Achieving robust sim-to-real transfer, this framework allows robots to learn dexterous skills directly from human videos, outperforming traditional methods.
AHEAD transforms static VLA models into dynamic manipulators, achieving up to 97% success in scenarios where traditional methods fail spectacularly.
Navigating with fewer than 8 VLM calls per episode, Goal2Pixel redefines efficiency in vision-language navigation tasks.
Imagine generating photorealistic 3D head avatars from a single 2D image, no multi-view data needed – MVCHead makes it a reality.
UMMs struggle with cross-modal consistency not from a lack of shared representations, but from misaligned latent space transformations, which LatentUMM fixes.
VLMs can be significantly boosted on embodied tasks by mid-training on a carefully curated subset of VLM data that is highly aligned with the VLA domain, rivaling the performance of much larger models.
Mismatched visual elements torpedo design harmony, but GIST offers a training-free fix that stylistically blends components, boosting aesthetic quality in existing pipelines.
Dramatically improve multimodal recommendation accuracy without any training by initializing user embeddings with item modality features and user cluster information.
Iterative visual refinement lets agents navigate dense coding IDEs with superhuman precision, outperforming single-shot methods and paving the way for more reliable software engineering agents.
Unlock 20x faster and more accurate 3D human-object contact estimation in complex, multi-person scenes with Pi-HOC, a framework that doesn't require object meshes.
Stop reimplementing multimodal models: TorchUMM offers a unified codebase for evaluation, analysis, and post-training, streamlining research across diverse architectures and tasks.
Achieve sub-centimeter robotic placement accuracy from compositional language instructions by decomposing the task into visual goal representation and goal-conditioned execution.
Imagine populating any 3D environment with digital humans that spontaneously navigate and interact, driven only by visual input and goals.
Video diffusion models already contain implicit multi-view knowledge, making them surprisingly effective for novel view synthesis when adapted to ignore temporal coherence.
Forget simulated manipulation—ManipulationNet offers a global infrastructure for benchmarking robots in the real world, complete with standardized hardware and software, to finally measure progress toward general manipulation.
Finally, digital humans can have realistic, socially aware conversations: DyaDiT generates dyadic gestures that users strongly prefer over existing methods.
Forget monolithic models: pMoE shows that ensembling diverse expert prompts within a single model framework yields surprisingly large gains in visual adaptation across a wide range of tasks.
By decomposing long-horizon manipulation into transport and object-centric interaction, LiLo-VLA achieves state-of-the-art zero-shot generalization and robustness, outperforming end-to-end VLA models by a large margin.
Forget cloud GPUs – a new model brings unified multimodal understanding and generation to your iPhone, running 6x faster than alternatives.
MLLMs struggle with multi-turn chart editing, forgetting context and accumulating errors, especially when the edits involve data transformations, not just styling.
Forget slow text-based communication: Vision Wormhole unlocks faster multi-agent reasoning by turning VLMs into telepathic hubs, slashing runtime without sacrificing fidelity.
Stop treating generated images like real ones: GMAIL aligns them as separate modalities in a shared latent space, unlocking significant gains in vision-language tasks.
VLMs that ace RGB images completely fail at thermal imagery, revealing a critical gap in their ability to reason about temperature and physical properties.
Forget static datasets – RL-based co-training unlocks +20% real-world VLA performance by interactively leveraging simulation while preserving real-world capabilities.
Synthetic data generated by fine-tuning Stable Diffusion on multi-region satellite imagery boosts small object detection accuracy by 20%, even when real labeled data is scarce.
Forget tedious manual annotation: FlexDataset crafts customized, high-fidelity annotated datasets with 5x faster annotation times using a composition-to-data approach.
