Natural Language Processing - Weekly Roundup

Generalization after fine-tuning is dictated by the linear structure of knowledge in pretrained models, not just factual memorization.

Dual-view time-series representations unlock significant gains in temporal reasoning by preserving both absolute measurements and invariant temporal structure, outperforming dedicated TSLMs.

Simultaneously evaluate $K$ policies in linear contextual bandits with near-optimal sample complexity by sharing information across evaluations via a unified ridge regression estimator.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

Downstream QA metrics hide critical memory failures: agents correctly classify memory operations less than 20% of the time, a flaw invisible to current benchmarks.

Generalization after fine-tuning is dictated by the linear structure of knowledge in pretrained models, not just factual memorization.

Dual-view time-series representations unlock significant gains in temporal reasoning by preserving both absolute measurements and invariant temporal structure, outperforming dedicated TSLMs.

Simultaneously evaluate $K$ policies in linear contextual bandits with near-optimal sample complexity by sharing information across evaluations via a unified ridge regression estimator.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

Downstream QA metrics hide critical memory failures: agents correctly classify memory operations less than 20% of the time, a flaw invisible to current benchmarks.

Exponential Hopfield networks learn exponentially more archetypes than pairwise models by leveraging softmax attention, but risk memorizing noisy examples over true archetypes.

Language model states are not static points but possess dynamic, context-adaptive reference frames that can be approximated to achieve near-perfect interpretability and steerability.

A simple byte reordering strategy slashes multilingual boundary prediction errors by over 70% in scripts like Korean and Chinese, fundamentally altering how byte-level models perceive codepoint transitions.

Standard MIL suppresses secondary findings in coexisting lesions; this novel framework recovers them by modeling the dominant-label annotation process.

Frozen LLMs reject sender KV caches even when geometrically similar, failing due to competitive attention, not fidelity.

LLMs' financial forecasts are more swayed by commentary tone and momentum than by the actual earnings surprise itself.

A novel temporal graph framework achieves over 250x fewer FLOPs and 20x fewer parameters than Transformers/LSTMs for early gastrointestinal cancer risk stratification from sparse EHRs.

Causal direction can be identified by minimizing the Dirichlet energy of one variable treated as a signal on a kernel graph of the other, bypassing regression and independence tests.

Asymmetric temporal dynamics and expanded receptive fields are unlocked in spatio-temporal graph forecasting by a unified spectral operator on cellular sheaves.

Native desktop agents can finally bridge the gap between visual GUI interaction and programmatic code execution by learning a unified action space, achieving state-of-the-art performance on complex real-world tasks.

Brain activity can decode naturalistic speech segments with 42% accuracy within the top 10 of 2,000 options, a feat achieved by aligning fMRI with multimodal stimulus embeddings.

Learned token visibility control dramatically boosts LLM reasoning accuracy, achieving state-of-the-art performance on GSM8K and MATH-500 with a tiny, frozen inference-time policy.

LLM agents spontaneously form and dissolve fluid, polycentric governance structures, revealing a fundamental tension between economies of scale and accountability in delegated decision-making.

Multimodal clustering can now adapt to arbitrary instructions, achieving near-perfect V-measures by directly embedding images, text, and instructions.

Full-page document parsing models can now pinpoint and correct local errors by assigning credit directly to tokens, outperforming baselines across six benchmarks.

Generative models can achieve significantly higher output diversity and corpus alignment by decoupling sampling from rendering at test time.

Even the best AI agents fail to deliver substantial business value in realistic customer delivery scenarios, struggling to uncover unstated constraints and achieve high impact.

Persistent interactive worlds that evolve with history and agent actions are now possible through a novel coupling of a computable world engine and egocentric audiovisual generation.

Agentic markets can achieve near-optimal welfare in first-price auctions, with the price of anarchy driven to zero by competitive floors and approaching $1 - 1/e^2$ under near-independent value distributions.

LLMs can permanently "remember" factual edits and skill gains from context, retaining 99.8% of original performance after 50 sequential updates.

Heterogeneous LLMs can share internal factual states without fine-tuning by exchanging word indices mapped to receiver-local vector directions.

Simple feature propagation and label smoothing on relational graphs match much of the performance of complex GNNs, suggesting architectural complexity might be over-emphasized.

Novel disease trajectories discovered from EHRs can be effectively prioritized using a two-level semantic sparsity and reliability framework, leading to significant improvements in clinical forecasting accuracy.

Branching motifs in EHRs offer structured trajectory context that primarily enhances long-range temporal forecasting, rather than uniformly improving endpoint discrimination or survival prediction.

Disagreement in pairwise preferences isn't just noise; it reveals distinct latent semantic structures, with only aggregate directional preference being truly identifiable from binary feedback alone.

Asynchronous EEG event detection is solved by a novel multiscale discrete spatiotemporal tokenization that captures both brief and sustained neural dynamics, outperforming prior methods across diverse datasets.

A novel heterogeneous graph representation and message-passing mechanism unlocks 18.69% better routability prediction by jointly encoding circuit topology, layout, and routing grids.

LLMs can be steered to generate diverse, relevant responses by explicitly searching for unoccupied semantic "gaps" in their output space, reducing output collision by over 10%.

Standard OCR metrics treat a typo and a wrong name equally, but LLM judges reveal severity differences that humans care about, leading to better evaluation.

Language models encode task information in hidden states *before* their unembedding matrix can express it, suggesting current developmental timelines are misleading.

GUI agents forget their way: dynamic memory modality selection unlocks significant performance gains over static approaches.

State-of-the-art deep research agents fail to achieve even 65% accuracy on realistic life and productivity tasks, revealing a critical gap between current capabilities and real-world utility.

Generate graphs that *exactly* realize a target persistence diagram, moving beyond approximate topological matching.

LLMs can now learn to progressively optimize molecules across multiple turns by explicitly conditioning on past feedback, achieving substantial gains in similarity-constrained success rates.

Latent thoughts are now inspectable and tool-integrable, achieving 10.4% better performance with 3x faster inference and 70% fewer tokens than frontier LLM agents.

Explanations from Natural Language Autoencoders can identify their target activation but fail to discriminate it from plausible rivals, suggesting current training paradigms may not yield robust oversight tools.

Adversarial image perturbations and decision-based attacks can render state-of-the-art OCR models nearly useless for academic integrity, even against advanced transformer architectures.

Fine-grained, foundation-model-derived semantic representations unlock state-of-the-art whole-body PET/CT registration without task-specific fine-tuning.

Transformers can be quantitatively predicted by a Bayesian reference, revealing their failure modes on complex, interdependent sequential data.

Closed neighborhoods in planar graphs can be taught with at most two non-clashing examples, resolving a long-standing conjecture, yet distinguishing between one and two teaching examples is NP-hard.

Explicitly modeling cross-layer information flow as a memory retrieval problem unlocks a linear-time attention alternative that outperforms existing methods.

A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.

Emotions' hierarchical structure is finally captured in multimodal recognition by fusing Euclidean token-level fusion with hyperbolic core routing, achieving state-of-the-art results.

Deterministic forecasting embedded within generative models is unstable; DV-Flow decouples it as a parallel velocity, unlocking stable end-to-end training and superior probabilistic forecasts.

A novel entropy-based framework reveals hidden nonlinear predictive couplings that traditional error-magnitude metrics miss, even in stochastic, partly non-invertible settings.

Multilingual LMs encode a universal color-language geometry, but not a universal color-perception geometry, with perceptual grounding fading in later layers.

AI-generated mathematical proofs can now be verified at the step-level without human annotation, thanks to a novel framework that leverages formal proof assistants.

Explicitly modeling copy operations unlocks substantial improvements in LLM efficiency and performance, even for models already adept at in-context learning.

AI-powered scoping reviews can uncover novel plant trait combinations for ecosystem services, outperforming purely manual efforts.

Generative AI and learner-facing applications are now the dominant themes in medical education research, but empirical evaluation and responsible integration remain critical unmet needs.