Computer Vision - Weekly Roundup

Aligning video generation models to human intent and physical constraints is significantly harder than for text or images, necessitating a diverse suite of post-training techniques beyond simple fine-tuning.

Directly learning robot action policies from raw video without explicit action labels unlocks scalable world action models.

Autoregressive video models can now perform explicit, goal-directed reasoning by jointly predicting the final outcome and intermediate steps, achieving state-of-the-art performance with 75% fewer training steps.

Current omnimodal video embeddings struggle with physical reasoning, yet specific physics can be recovered and leveraged to significantly improve the physical fidelity of generated videos.

Composing aesthetic preference with explicit rubric rewards unlocks substantial gains in text-to-image generation, outperforming existing open-source models by a significant margin.

Directly learning robot action policies from raw video without explicit action labels unlocks scalable world action models.

Autoregressive video models can now perform explicit, goal-directed reasoning by jointly predicting the final outcome and intermediate steps, achieving state-of-the-art performance with 75% fewer training steps.

Current omnimodal video embeddings struggle with physical reasoning, yet specific physics can be recovered and leveraged to significantly improve the physical fidelity of generated videos.

Composing aesthetic preference with explicit rubric rewards unlocks substantial gains in text-to-image generation, outperforming existing open-source models by a significant margin.

State-of-the-art text-to-image models, despite photorealism, still exhibit pervasive world-consistency failures that existing metrics miss.

Generative 3D reconstruction now scales from objects to cityscapes with arbitrary resolution, matching per-scene optimization quality without per-scene tuning.

Imperfect diffusion models can now achieve exact target distribution sampling at inference time by correcting inherent path and endpoint errors.

Persistent world models can now track objects even when they leave the agent's view, thanks to a novel actor-observer decoupling mechanism.

Real-time 3D Gaussian avatar animation is now achievable at 60fps on mobile by distilling complex neural inference into a linear blendshape approximation, reducing CPU cost by 1000x.

Explicitly disentangling and structuring spatial geometry latents via CR-GEO and routed optimization unlocks significantly more effective 3D reasoning from 2D images, pushing SPAR-Bench and SPBench performance to new heights.

Adversarial distillation achieves state-of-the-art few-step visual generation by learning log-density ratios directly via classification, eliminating the need for auxiliary score fitting.

Standard UAD evaluations for brain MRI are brittle, with a single axis-order mismatch capable of collapsing voxel AUROC from 0.873 to 0.583, while slice-level AUROC remains largely unaffected.

Achieve precise, intuitive 3D control over video generation by moving rigid object meshes, eliminating depth ambiguity and viewpoint inconsistencies.

Dynamically recomputing embeddings at each denoising step in diffusion transformers halves training compute while achieving state-of-the-art FID scores.

Topology-preserving 3D generation now achieves 98% fewer tokens and 58.5% faster inference by encoding shapes as overlapping slice latents instead of expensive voxels.

Multimodal sampling with Wasserstein gradient flows and forward diffusion is fundamentally limited by slow mixing, requiring exponential time to bridge well-separated modes.

A single CatBoost model detects wildfires 26 times earlier than GOES FDC and VIIRS, and achieves significantly higher F1 scores, even at night.

Adversarial domain adaptation that explicitly conditions on predicted weather unlocks robust street-view perception even with non-street-view training data.

Malware analysis just got a new tool: S4 models unlock long-range dependency detection in execution flows, outperforming traditional deep learning for multi-class classification.

A novel Four-Dimensional Architecture Selection Framework correctly recommends 70% of deployed autonomous driving architectures, offering critical guidance for system design.

Adversarial training on Fourier phase perturbations unlocks over 20% domain generalization gains in medical imaging by decoupling spatial structure from domain-specific textures.

Discrete neural samplers achieve fixed-point convergence and drastically reduce mode collapse by leveraging Gibbs sampling within a masked diffusion framework.

Diffusion models can achieve fairness and diversity simultaneously without sensitive attribute labels by using a frozen model adapter and a vision-language embedding space.

Geographic context unlocks precise PCS anomaly detection, revealing infrastructure health issues previously missed by purely visual methods.

Artists can now directly sculpt 3D object motion relative to a moving camera, overcoming the inherent ambiguity of 2D controls in video generation.

Most video generation models fail to render text accurately, with even top performers exhibiting a 25% word error rate on a new benchmark designed for this critical oversight.

Current egocentric video generation models fail to execute multi-step manipulation tasks realistically, often skipping crucial actions and exhibiting poor physical dynamics.

Transformer and RNN tractography models achieve state-of-the-art performance by introducing a novel generation-validation phase for streamline-level supervision, overcoming the limitations of local loss functions.

End-to-end learning of temporally predictable latent spaces for world models unlocks superior future scene understanding without the need for separate representation and prediction training stages.

Human mobility generation can now be achieved without target trajectories by learning relative movement transitions, not absolute destinations, across cities.

Diffusion models can lose diversity when only later timesteps are updated during reward optimization, a finding that challenges prior assumptions and motivates a new training paradigm.

A data-driven weather model now rivals operational NWP for 1 km resolution forecasts over complex Alpine terrain, but struggles with localized wind extremes.

Diffusion models can plan collision-free, multi-agent trajectories without any training data by leveraging analytical local scores derived from geometric constraints.

Predict PDE dynamics on submanifolds with zero-shot generalization to moving sources and mild nonlinearities, achieving 4-8x lower error than surrogates at millisecond query times.

Discrete diffusion models can achieve optimal sampling complexity for categorical MRFs by decomposing the score into time- and target-dependent multiplicative components, allowing for a single, weight-shared score network.

A single denoising process now unifies discrete token generation and continuous latent trajectories, enabling diffusion models to jointly reason about token dependencies and global structure.

A unified proactive generation framework enables LLMs to form reusable factual memories from streaming video without compromising real-time perception.

VideoLLMs forget up to 40% of temporal context by the output layer, but a simple inference-time injection can recover it.

Robots can now plan complex, long-horizon tasks in novel environments by predicting sparse visual sub-goals, achieving 70% success in zero-shot real-world experiments.

LLM/VLM agents significantly outperform non-agentic methods in preserving unedited parts during iterative 3D asset modification, but current agentic edits are prohibitively slow.

StreamMAE unlocks self-supervised learning on continuous video streams, matching i.i.d. MAE performance by tackling intra-batch similarity with novel regularization.

Autonomous driving diffusion planners now learn to intrinsically adhere to driving rules and explain their decisions, significantly improving safety in challenging scenarios.

Sonar-only SLAM now achieves robust topological mapping by verifying loop closures with a sequence-aware approach, overcoming the inherent ambiguity of acoustic echoes.

Radar novel-view synthesis finally unlocks Doppler, achieving 90.7% object detection recovery by separating scene geometry from sensor spread and enabling zero-shot configuration transfer.

Physics-constrained generation now achieves comparable fidelity to multi-step methods in a single neural transport evaluation.

Visual perception agents can achieve significant performance gains by actively seeking information beyond a single glance, outperforming static models by up to 30 points on challenging benchmarks.

A 260M-parameter looped diffusion model achieves performance surpassing a 6.5x larger non-looped counterpart with significantly less inference compute.

Standard image ViTs, when adapted with masked Siamese learning on "super images," unlock state-of-the-art video representation learning with orders of magnitude less pretraining.

Current VLMs, despite impressive general capabilities, fail to grasp the nuanced social dynamics required for robot navigation, lagging behind simple rule-based systems.

A novel decoupled autoencoder design unlocks 32x spatial compression for diffusion models while simultaneously boosting reconstruction fidelity and accelerating training convergence.

Diffusion language models can now generate high-quality text with up to 49% fewer network evaluations by matching output distributions.

Kilometer-scale extreme heat downscaling accuracy scales linearly with climatological distance, not simulation volume, revealing a path to cost-effective urban adaptation.

Language, when self-evolving and treated as a physical representation, can unlock state-of-the-art physical plausibility in video world models, even surpassing proprietary systems.

Parallel diffusion language model generation can match autoregressive diversity and efficiency by strategically falling back to sequential sampling only at high-entropy decision points.

State-of-the-art precipitation downscaling models struggle to generalize across continents, revealing a critical gap in current ML-based climate forecasting.

Existing SF-UniDA methods fail catastrophically on time series due to a highly sensitive inference threshold, but a novel auto-thresholding module offers a universal fix.

A single video tokenizer can achieve superior semantic alignment and fidelity compared to much larger models by directly conditioning on frozen visual features.

Reconstructing revisited scenes in streaming world models is now possible with constant memory, outperforming full-softmax attention by 30% on peak memory.

Most multimodal models fail to truly understand tool use in real-world egocentric videos, with even Gemini-3.1-Pro struggling to ground actions in visual evidence.

A single visual interface, action views, bridges the gap between video generation models and diverse robotic embodiments, enabling unified action modeling and execution.

Future needs, not current context, are the key to unlocking effective long-horizon video generation.

Aligning video generation models to human intent and physical constraints is significantly harder than for text or images, necessitating a diverse suite of post-training techniques beyond simple fine-tuning.

A single continuous flow model now unifies language and vision generation, outperforming hybrid and discrete models under matched budgets and achieving strong benchmarks with drastically less data.

Immersive AR/VR experiences can finally have stereo audio that dynamically tracks visual sources, thanks to a novel framework that binds visual motion to sound localization.

Visual grounding can now be achieved at flash speed with precise localization by treating it as parallel evidence extraction, not sequential generation.

Unsupervisedly discover and generate specific subtypes of visual data, like distinct eyewear styles or retinal diseases, by disentangling salient factors from common content within foundation model latents.

Enforcing uniform load balancing across video tokens fundamentally breaks visual coherence, but bifurcating experts into semantic and generic tiers resolves patch fragmentation while driving emergent coarse-to-fine denoising.

Timing human supervision across a test stream proves far more impactful than intra-batch sample selection, matching full active test-time adaptation performance while querying only a fraction of test batches.

Robot policies can now test and refine manipulation plans entirely within a learned visual simulator across ten distinct embodiments, gaining up to 21.4 percentage points in task success without any target-domain fine-tuning.

Streaming video assistants face an unresolved utility–latency–storage trilemma the moment interactions are interrupted across sessions, exposing acute failures in long-term recall and missing-evidence awareness.

High-resolution 4K video upscaling no longer requires iterative diffusion: a single denoising step can outperform multi-step refiners while slashing latency by nearly 9×.

Fully factorizing self-attention across separate temporal, spectral, and spatial dimensions slashes compute while dominating SITS segmentation benchmarks—and uncovers that common tile-size discrepancies have quietly rendered most published baseline comparisons invalid.

Pixel diffusion’s chronic high-frequency blur is not an architectural bottleneck: a lightweight adversarial post-training phase restores natural spectral power laws without causing mode collapse or requiring changes to the sampling pipeline.

Append-only memory guarantees attention collapse in long-horizon video reasoning, but actively rewriting a bounded working memory from an external filesystem boosts hard-subset task accuracy by over 20%.

Generative models can only rely on efficient local denoising up to the exact moment they commit to global semantics, after which both phenomena collapse into a single sharp phase transition as system size grows.

Marginal conformal guarantees often mask severe error spikes on hard examples, but inverting localized risk curves delivers tight conditional error control without sacrificing distribution-free finite-sample validity.

Chronological text logs routinely confuse distinct physical objects across multi-day video, a failure mode resolved by explicitly clustering grounded visual instances into persistent entity biographies rather than relying on episodic text search.

Probing a frozen diffusion relighter with paired background queries extracts complex scene shadows and ambient lighting directly onto synthetic inserts, bypassing the need for banner-specific retraining.

A model trained solely on file metadata—with zero medical imaging data—hits 0.992 balanced accuracy on a flagship pediatric x-ray benchmark, proving that evaluation artifacts and pipeline choices dominate architectural gains.

Scaling model size fails to fix fundamental cross-video reasoning deficits in frontier MLLMs, but targeted supervision from physics simulators reliably transfers zero-shot to real-world multi-view video tasks.

CLIP's cross-modal gap makes text-derived classifier weights actively harm class-incremental learning—discarding the text branch entirely in favor of closed-form visual SVMs delivers superior continual recognition.

Decoupling student rollout generation from teacher supervision kernels enables stable on-policy distillation for few-step flow models, outperforming multi-reward GRPO while accelerating convergence.

Image editors secretly encode 3D projective geometry accurate to 0.8° of pitch—outperforming specialized calibration networks—yet completely fail if explicitly prompted to identify the horizon or vanishing points.

Earth observation streams and foundation model embeddings can be compressed 1,800× into continuous neural fields while retaining 90%+ downstream performance and cutting data retrieval latency tenfold.

Sparse neuron dictionaries are not just passive interpretability tools: actively steering frozen VLM tokens through query-guided concept activation delivers up to a +9.5 point accuracy boost on complex spatial reasoning without retraining.

Continual learners no longer need oracle task IDs to dynamically scale capacity, as loss-driven width expansion coupled with persistent key-value attention prevents catastrophic forgetting across unsegmented task streams.

Decoupling group-level interaction dynamics from individual body kinematics is the missing ingredient for multiperson generative models to generalize across arbitrary agent counts and severely corrupted observations.

Closed-loop vision verification is no longer bottlenecked by intractable generative surrogates: compact stochastic world models cut parameter counts by 130× while resolving over 80% of previously unverified RGB state spaces.

Non-factorized diffusion language models achieve superior few-step generation by leveraging a discrete latent space learned via Mixture-of-Experts routing, bypassing posterior collapse issues.

Independent chunk scoring in autoregressive video generation unlocks sustained visual quality over extended horizons, overcoming the temporal consistency trade-offs of prior methods.

Video world models can now maintain persistent scene memory without ever-growing storage by warping and fusing low-rank features into a fixed-size HexMemory.

Video agents can now autonomously discover and fix their own reasoning flaws by re-watching videos and evolving their evidence-gathering strategies.

Synthesizing historical street views is now possible with 17% better realism and consistency by treating it as a reward-guided editing problem.

Unified OCR foundation models achieve state-of-the-art visual intelligence by integrating recognition, parsing, and reasoning through instruction-following and a novel competence-guided policy optimization.

Decoupling structured semantics from frame-level actions and distilling autoregressive rollouts via shared, compressed memory tokens solves the long-standing trade-off between real-time responsiveness and multi-step drift in interactive world models.

Downstream classifier accuracy can actually improve when neural compressors intentionally distort reconstruction distributions away from mismatched source domains toward a target distribution.

Collecting more errors doesn't guarantee better models: targeting only the high-impact failure modes signaled by a network's own internal representations outperforms standard rarity-based active learning while saving annotation budget.

Robots can master delicate tool manipulation directly from unpaired human videos, achieving a 73% performance leap over SOTA by anchoring control in object-centric 3D pose priors rather than raw end-to-end pixels.

Categorical sampling cripples discrete diffusion by discarding intermediate uncertainty—diffusing directly over the probability simplex fixes this information collapse, enabling an 8-step distilled model to beat 128-step discrete baselines on GSM8K.

Human hair priors can substitute for non-existent animal fur datasets, enabling dense, editable 3D animal grooming with an order-of-magnitude faster optimization.