Search papers, labs, and topics across Lattice.
100 papers published across 8 labs.
A lightweight model fine-tuned on AIriskEval-edu-db2 can rival leading models in pedagogical risk detection, all while maintaining privacy in educational settings.
RoboDojo reveals that existing benchmarks fail to capture the full spectrum of robot manipulation capabilities, paving the way for more robust evaluations that bridge the gap between simulation and real-world performance.
Multilingual rankings fail to predict Portuguese sentence encoder performance, revealing the critical need for language-specific benchmarks.
None of the 30 LLM agents evaluated in CausalGame demonstrated reliable causal thinking, revealing a critical gap in AI's ability to perform scientific reasoning.
Language model evaluations often misrepresent performance, but evalci reveals that many reported advantages may not hold up under statistical scrutiny.
RoboDojo reveals that existing benchmarks fail to capture the full spectrum of robot manipulation capabilities, paving the way for more robust evaluations that bridge the gap between simulation and real-world performance.
Multilingual rankings fail to predict Portuguese sentence encoder performance, revealing the critical need for language-specific benchmarks.
None of the 30 LLM agents evaluated in CausalGame demonstrated reliable causal thinking, revealing a critical gap in AI's ability to perform scientific reasoning.
Language model evaluations often misrepresent performance, but evalci reveals that many reported advantages may not hold up under statistical scrutiny.
Role-based multi-agent code generation narrows the gap between AI-generated and human-written code, but still leaves room for improvement.
Code LLMs can recognize incorrect instructions but still follow them, leading to irrecoverable semantic errors that defy traditional evaluation metrics.
Many robotic policies that seem successful in manipulation tasks actually compromise safety, with SoftVTBench revealing a stark contrast between goal completion and physical safety metrics.
Synthetic sound effects can fool listeners into mistaking them for real recordings nearly 29% of the time, but a generator-specific detector can perfectly distinguish the two.
Temporalized full-text retrieval methods outshine traditional baselines, achieving the best nDCG scores across evolving document collections.
Correctness checks can miss kernels that are functionally valid but over 300 times slower than optimized versions, highlighting a critical evaluation gap in GPU DSLs.
Version alignment in LLM-generated quantum code is a critical challenge, with only 0.02 to 0.85 success rates across different models and SDK versions.
A unified framework reveals that most optimizers only engage a fraction of their potential, providing a roadmap for more effective model training.
A third of tested configurations in depression detection collapsed to a single-class prediction, underscoring the hidden pitfalls of current aggregation methods.
PACE-Bench predicts agentic performance with remarkable accuracy while slashing evaluation costs to a fraction of traditional methods.
AgenticDataBench reveals that LLM-based data agents can be rigorously evaluated across diverse real-world scenarios, highlighting their strengths and weaknesses in handling complex data tasks.
Cluster-based chunking fails to deliver on its promise, showing no performance advantage over simpler methods in RAG systems for academic texts.
Existing unlearning methods may look effective, but they often miss the mark on precision, leaving sensitive data vulnerable to resurfacing attacks.
Persona expression in LLMs reveals a surprising duality: while aggregated traits are stable, their geometric representations are highly sensitive to context, collapsing under misalignment.
Models trained with AbsoluteDegradation not only generalize better to real archival footage but also expose critical weaknesses in existing restoration methods.
Despite being frontier models, none of the evaluated LLMs could reliably meet critical clinical reasoning standards, with over half of the essential criteria unmet.
Bias in LLM judges can be corrected to improve ranking accuracy, lifting recall rates significantly in noisy environments.
Chronos-2 not only excels in peak load forecasting but also adapts to uncertainty without relying heavily on weather data, reshaping expectations for low-voltage load predictions.
Data leakage and hidden stratification can inflate performance metrics, but a new framework reveals the true robustness of AI models in spatially correlated domains.
Object Aligner achieves robust JSON similarity scoring by inferring identifier bijections, enabling accurate evaluation of complex structured outputs without the pitfalls of traditional methods.
A third of synthesized tokens in TTS systems misrepresent phonological contrasts, revealing a significant gap between intended and produced sounds.
A/B testing may be more error-prone than offline evaluations, but a new estimator leveraging a hypothetical middle algorithm can turn this on its head.
MRRG reveals that leveraging multiple evaluative perspectives can significantly enhance the quality of reward signals for LLM optimization, outperforming traditional single-role approaches.
LLMs can achieve near-expert grading accuracy for command-line exams, but their performance declines sharply with question complexity.
LLMs may be overtrusted in multilingual evaluations, leading to inconsistent and potentially misleading judgments in low-resource languages.
Prompt Coverage Adequacy uncovers over 30% more faults than traditional code coverage, revolutionizing how we test LLM-generated code.
A novel operationalization of AI risk assessment reveals that PII leakage can vary dramatically, with disclosure rates shifting from 0% to 84% based on adversarial conditions.
LLMs may generate Ukrainian text, but they often fail to deliver the necessary emotional support that is culturally grounded.
The central challenge of ontology learning isn't model sophistication but rather how knowledge is structured and encoded, as revealed by a comprehensive evaluation using OntoLearner.
Models that excel in multiple-choice tasks can falter dramatically in open-ended and error identification formats, revealing a critical gap in art-historical reasoning capabilities.
Despite advancements, even the best LLMs lag far behind expert-level performance in aviation knowledge, achieving only 82.7% accuracy on critical operational questions.
Steering vectors may not be the universal solution for preference-aligned generation, as their effectiveness significantly drops with trait complexity and task transfer.
VLP transforms LLM-generated code validation by bridging the gap between user intent and code through clear, verifiable documentation, leading to a dramatic increase in validation success rates.
Qolumbina reveals that existing quantum software testing benchmarks are inadequate, paving the way for more robust evaluations of scalable quantum programs.
Coding agents guess their way through underspecified instructions, leading to alarming action-boundary violations that challenge the notion of safe autonomy.
Prompt complexity is a critical dimension that significantly influences maintenance effort, challenging traditional views that prioritize code-level metrics alone.
UQ metrics in software defect prediction can mislead if applied without context, revealing that strong classifiers may still suffer from significant calibration errors.
PatchFusion recovers more bugs than any single source, outperforming traditional selection methods with a deterministic fusion of evidence that cuts costs dramatically.
MMBench-Live achieves a high answer correctness rate while updating benchmarks at a fraction of the cost and time, revolutionizing how we assess VLMs.
ScopeEdit revolutionizes online multimodal editing by ensuring that edits are both reliable and contextually appropriate, minimizing unwanted semantic leakage.
Models frequently misjudge safety across different intents, revealing critical vulnerabilities in AI completion systems that could lead to harmful outcomes.
A lightweight model fine-tuned on AIriskEval-edu-db2 can rival leading models in pedagogical risk detection, all while maintaining privacy in educational settings.
A$^{2}$utoLPBench offers an endless stream of dynamically generated LP problems, ensuring agents can be tested against fresh challenges without the risk of training data leakage.
Human-AI collaboration thrives on collaborative traits, not just cognitive skills, with only a minority of forecasters achieving superior accuracy through genuine engagement.
TestEvo-Bench exposes the stark reality that even advanced test automation agents struggle with recent code changes, achieving lower success rates in real-world scenarios.
Evolved rubrics from SkillCoach expose hidden failures in agentic skill use, enhancing training and evaluation beyond mere task success.
Current VLMs struggle with specialized domains, failing to adapt effectively in both zero-shot and ICL scenarios, revealing critical gaps in their spatio-temporal reasoning abilities.
GPT-5.5 not only tops the leaderboard in policy evolution but also reveals critical insights into how agents can optimize performance through strategic feedback utilization.
Event-pattern complexity can drastically affect model performance, with some neural STPPs collapsing under pressure while others remain robust.
RF detection methods may be inflating accuracy estimates by up to 50% due to data leakage in standard evaluation practices.
Task-specific design choices can significantly outperform generic models in lung cancer classification, revealing the nuanced interplay between feature extraction and classification in medical AI.
The Brier score and Matthews Correlation Coefficient emerge as top metrics for accurately reflecting spelling performance in ERP-based BCIs, challenging traditional reliance on loss and accuracy.
AI-generated molecules could harbor hidden dangers, but MolSafeEval reveals their safety risks through a comprehensive evaluation framework.
Unlearning in LMLMs is more about database management than model architecture, with up to 13.6% of deleted facts surviving through retrieval artifacts.
Critic complexity can be directly controlled in reinforcement learning, offering a new lever for optimizing training performance.
Safety-aligned text-to-image models may appear effective, but they suffer substantial semantic fidelity losses that standard metrics fail to capture.
Models may appear comparable in accuracy, but TimeSynth reveals that phase and frequency fidelity can diverge significantly, impacting health signal forecasting.
Adversarial pragmatics reveals that existing safety evaluations often obscure critical distinctions in model behavior, making it harder to pinpoint the root causes of failures.
Agents trained on static benchmarks falter dramatically in open-world settings, revealing a critical gap in their adaptability to real-world complexities.
LVLMs falter dramatically in video quality understanding as the duration and complexity of content increase, revealing critical gaps in their perceptual reasoning capabilities.
The choice of performance metrics could determine whether AI capabilities remain concentrated among the wealthy or proliferate across a broader developer base.
Despite successful workflow execution, LLMs fail to operationalize analytical intents 153 times across diverse domains, revealing critical semantic gaps.
MLLMs falter in egocentric action selection, consistently opting for actions of visible agents instead of their own, revealing a critical gap in current training paradigms.
Current LLMs falter in resolving LLVM compiler issues, but a new ensemble method boosts resolution rates by nearly 22%.
Existing LLMs fail to maintain internal representations in maze environments, revealing critical limitations in their reasoning capabilities.
VLMs struggle with object-level counterfactual reasoning, achieving only a fraction of human accuracy in spatial tasks.
Statistical alignment with clinical experts doesn't guarantee that LLMs exercise the necessary clinical caution in evaluations.
All leading LLMs struggle with fine-grained emotion classification, revealing a surprising ceiling effect in zero-shot performance.
State-of-the-art NLP models struggle with metaphor translations, revealing critical gaps in their understanding of semantic and cultural nuances.
Cultural competence in language models is more about pre-training exposure than multilingual fluency, revealing a critical gap in AI's understanding of cultural nuances.
Even Japanese-specific LLMs fail to grasp kanji readings, revealing a significant shortcoming in their linguistic understanding.
Span-level hallucination detection can now effectively address structured inputs like code and tool outputs, not just natural language, revealing a critical gap in current RAG evaluations.
Low generative perplexity in diffusion models often masks excessive repetition, with a simple fix cutting repetition to human levels while being 1.5–5x cheaper.
Traditional metrics fail to capture the true memory capabilities of LLMs, exposing a critical gap in how we assess their deployment readiness.
Evaluator biases can significantly skew LLM agent behavior, but the new EPC protocol standardizes how we measure and compare these effects across different evaluators and time points.
Evaluator coupling significantly impacts measurement reliability, with low coupling leading to high noise levels that undermine evaluation accuracy.
LLMs can autonomously design network topologies that meet complex requirements, but they struggle with common pitfalls like interface mismatches.
AutoRestTest outperformed all competitors in the SBFT 2026 Tool Competition, revealing significant advancements in automated REST API testing.
Strong code generation doesn't guarantee effective requirement clarification, exposing a critical flaw in LLM capabilities that could hinder software engineering practices.
VideoLLMs may excel at recognizing actions but often hallucinate human motions, revealing a critical flaw in their design.
Training trajectory forecasting models with a metric-agnostic approach can lead to state-of-the-art performance across all evaluation metrics, challenging the notion that metric-specific optimization is necessary.
Existing segmentation metrics often obscure their underlying assumptions, but this new framework reveals how they can be modularly decomposed to enhance clarity and effectiveness.
A trie-based experiment plan can slash evaluation time by 26%, revolutionizing how we assess complex IR pipelines.
Transforming model selection from a mere ranking task into a governed economic decision could revolutionize how enterprises adopt LLM agents.
Only 8 out of 455 quantum software claims can be directly audited, revealing a critical gap in the reliability of reported performance comparisons.
A strategic approach to dataset creation could empower smaller labs to make significant advancements in autonomous driving without overspending on data collection.
LUMA reveals that the choice of pretraining objective trumps architectural complexity in driving segmentation quality, challenging existing assumptions in the field.
AV-SyncBench reveals that existing benchmarks obscure critical differences in audio-visual model performance by conflating temporal and semantic evaluations.
A unified creativity benchmark reveals that LLMs exhibit a single creativity factor, yet top human creators still outperform them in creative tasks.
Agents often prioritize user alignment over factual accuracy, and MemSyco-Bench reveals just how much memory influences this dangerous trend.
Benchmark scores for coding agents may mislead progress assessments, with only 39% of GSO tasks passing validity checks across machines.
LLM-generated research ideas are systematically narrower and more focused than those of human researchers, revealing a significant gap in creative breadth.
CoMet reveals that decomposing uncertainty into context and multiplicity components can lead to substantial improvements in uncertainty estimation for multimodal AI systems.
Semantic embeddings falter in stylistic evaluations, revealing a critical gap in current embedding methodologies.
Calibration, not compilation, is the key to ensuring statistical accuracy in probabilistic programs generated by language models, with detection rates soaring to 97% when using Bayesian workflows.
Even top-performing AI models struggle with PowerPoint tasks, achieving only 45% success rates despite a robust evaluation framework that rewards nuanced performance.
