Architecture Design (Transformers, SSMs, MoE) - Weekly Roundup

The DeepSeek-V4.1-Flash model, a multimodal Mixture-of-Experts model with 552B backbone parameters and support for contexts of up to one million tokens, is introduced, substantially improving cost efficiency for agentic workloads and pushing the limits of KV cache compression.

DirtyMoCap is introduced, a robust, marker-layout-free framework that consistently outperforms state-of-the-art configuration-specific baselines in both joint and vertex reconstruction accuracy, while the custom CUDA solver achieves up to a 100x speedup over standard PyTorch implementations.

This work introduces physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction, and presents ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization.

A query optimization framework that integrates a Graph Transformer with the model-based reinforcement learning (RL) algorithm MuZero, which converts SQL queries into heterogeneous graphs containing tables, predicates, and join operators and demonstrates its practical effectiveness and scientific value for building high-performance, adaptive intelligent database systems.

A query optimization framework that integrates a Graph Transformer with the model-based reinforcement learning (RL) algorithm MuZero, which converts SQL queries into heterogeneous graphs containing tables, predicates, and join operators and demonstrates its practical effectiveness and scientific value for building high-performance, adaptive intelligent database systems.

TriDim, a reusable block that preserves the representation shape and keeps three EEG axes explicit: channel, sample position within each patch, and patch position across the recording, is proposed, a reusable block that preserves the representation shape and keeps three EEG axes explicit.

This work implements two representative architectures --- the Simplified Graph Convolution (SGC) and Linear Graph Convolution (LGC) models and evaluates them on open benchmark graph datasets and semi-supervised learning tasks using quantum simulation and presents a cost gradient analysis that identifies the tasks for which the models are trainable.

This work introduces a novel surface-based diffusion bridge framework DB-SUiT for MRI-to-PET translation that operates natively on the cortical manifold and demonstrates high-fidelity synthesis that substantially outperforms other baselines.

An architecture extending unanticipated-fault recovery from detection to mitigation and an ensemble methodology for evaluating LLM-assisted mission management on low-power AUVs are made.

This work formalises a unified view of block redundancy that decouples the geometry from specific surrogate interventions and introduces Transformer-Within-Transformer (TWT), a post-hoc method that fuses contiguous groups of redundant layers into a single learned surrogate layer.

This paper presents PART, a unified transformer-based framework for 3D part retrieval and assembly: given a target shape and a part library, PART automatically selects the appropriate parts and predicts their 6-DoF poses to reconstruct the target.

Initial evidence is provided that greCAPTCHA can assess manuscript-specific understanding under proctored conditions and the appropriate construct validity for author understanding is remarked on, while also suggesting important changes to be made before deployment.

FAMOS, a feed-forward model that predicts movable-part segmentation and joint parameters from a sparse, unordered set of partial point clouds, is presented and a procedural data generator that synthesizes self-annotated assets during training is introduced to overcome the limited scale and diversity of existing datasets.

This work instantiates VDN on MiniMax H3, which combines local Softmax attention with bidirectional linear memory for long-range video context and introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens.

Relational BabyLM is a system submission to the BabyLM 2026 challenge that combines two cognitively motivated inductive biases in a single decoder-only Transformer, and introduces a novel symbol-retrieval mechanism that matches learned symbol libraries while adding no parameters.

TetrisCNN is introduced, a convolutional architecture with parallel branches of differently shaped filters, reminiscent of Tetris blocks, that learns sparse, interpretable latent representations directly in terms of spin correlators, and opens the way to integrating interpretable neural networks with quantum simulators to uncover and understand new phases of matter.

This paper introduces multi-horizon multimodal prediction in ABBR, an agile tactile World Action Model for contact-rich robot control, which provides supervision for visual latent at a larger temporal offset while predicting the tactile latent in the next frame to capture fine-grained contact dynamics.

The results support the conclusion: foundation-model supervision can amplify a fixed manual annotation budget into a substantially larger training set and yield a compact, deployable edge model.

This paper proposes a conceptual Agentic AI architecture for AI systems that can plan, access data sources, execute tools, and act autonomously and audibly, aimed at supporting decision-making across the three Brazilian Armed Forces.

Results indicate that recursive quantum feature transformations can improve stability and out-of-sample performance for compact hybrid quantum--classical temporal models.

Conservative Adaptive Ranking and Screening, Selector-Evaluator Value Assessment, and Dynamic Adaptive Risk-aware Enhancement form CARE-VI, an evidence-regulated target construction framework that preserves the backbone interfaces for critic regression and actor updates.

This work proposes a framework for locally masked SAE-based anomaly detection, supported by theoretical justifications, and demonstrates their ability to capture meaningful safety information while using only 1-2% of SAE neurons for computation.

SETTer is introduced, a transformer-based model that addresses challenges of long-term multivariate time series forecasting by incorporating novel techniques for decoupled self-attention and hybrid masking and outperforms state-of-the-art models in 88% of the scenarios.

Experiments on long-horizon autoregressive ORAS5 forecasting show that OceanMoE lowers aggregate forecasting error and maintains lower geometric-mean normalized RMSE than the corresponding baselines over most later rollout months, which support structured conditional computation as a modeling strategy for balancing shared ocean context with adaptive specialization.

This tutorial aims to walk the reader through the journey from words to network traffic, shedding light on how language is translated into communication flows within HPC training systems, and tries to demystify how words travel across the network.

STR-Agent is proposed, an LLM-driven framework for QoS-aware routing in LEO satellite networks that significantly outperforms conventional baselines, and results demonstrate the potential of LLM-driven agent architectures to enable serviceaware and adaptive QoS routing in future LEO satellite networks.

The model presented here is a step toward a more rigorous formalization of cognitive phenomena and may provide a basis for further theoretical analysis, computer simulation, and implementation in artificial-intelligence systems inspired by biological processes.

FedeRICo, a federated traffic forecasting framework that combines gradient-level collaboration with boundary-aware residual communication, is proposed that consistently outperforms state-of-the-art federated spatial-temporal baselines while maintaining competitive training runtime.

A discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone, gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM.

The Depth Control Protocol (DCP) is proposed, a diagnostic suite that disentangles three quantities of depth: truncating depth at inference time, plotting quality against retained depth fraction, and reading off the slope.

Astronex-World 1.0 is presented, an open controllable video world-model foundation that provides a bidirectional model for full-context generation and a causal model with block-causal attention and cross-block KV caching for persistent generation, both built on the Wan2.2-TI2V-5B prior.

Findings show that semantic reference-text overlap should not be treated as a proxy for temporal localization, and that AVTrace can identify task-specific weaknesses while providing a testbed for temporal post-training.

Experiments on a real robot platform demonstrate that MaskHarness-WAM substantially outperforms limited-horizon policies on sequential multi-object manipulation, showing its effectiveness in extending local manipulation skills to reliable long-horizon execution.

UniPolicy is proposed, an objective-aware multi-policy alignment framework that supports parallel, business-customizable multi-policy beam search, flexibly allocating candidate quotas across objectives under a fixed retrieval budget and outperforming single-objective reinforcement learning and naive reward-fusion baselines.

A Retrieval-Augmented Generation framework for scientific image quality assessment, designed to simultaneously address both the understanding track (SIQA-U) and the scoring track (SIQA-S) of the SIQA challenge, is proposed.

Age, digital literacy, and interaction modality as correlates of psychomotor and cognitive-flexibility performance in immersive virtual reality (VR) are examined and age was associated with slower task times across modes, but controller direct touch carried the steepest relative age gradient.

Value Faces, a system that analyzes a person's existing chat histories from their everyday messaging platforms using Schwartz's ten basic human values and produces separate value profiles for their different relationships, finds that the resulting value profiles distinguished participants' national contexts with twice the odds of guessing, while system-inferred differences across relationships aligned with participants' perceptions of those differences.

Zarya is introduced, a family of hybrid language models that jointly optimizes an autoregressive (AR) objective and a masked-diffusion objective within a single architecture, demonstrating performance on standard benchmarks while offering a principled integration of autoregressive and diffusion paradigms.

Mental distance, a loss of belief that the work is worthwhile, is the only symptom unrelated to operational problems, and among posts with a single symptom it is accompanied by a stated intention to leave roughly twice as often as any other.

The approach enabled low-error NWU quantification from baseline NCCT without advanced imaging, supporting further prospective clinical evaluation and targeting NCCT-only infarct segmentation across heterogeneous external cohorts, although performance varied across domains.

This work presents the first systematic characterization of attention sparsity in autoregressive image generation across diverse workloads and representative open-source models and reveals several distinguishing properties, including a pronounced prefill-decode asymmetry, strong attention concentration on prompt and local tokens, and a unique diagonal attention sparsity pattern arising from the spatial locality of visual tokens.

Beyond instance-level understanding, few methods have explored fine-grained classification for urban buildings, and thus this work puts forward a fine-grained classification model for urban buildings with a spatially-supervised contrastive loss.

FP-Sketch is presented, a verifier that runs after an unmodified tensor core GEMM whose half precision operands are accumulated and delivered at FP32, and its constant depends on the BLAS and the operand format and must grow as $n^{2.57}$ for a square product.

This work presents LABELSENSE-Pilot, a reproducible prototype that generates eight compass candidates per feature, scores candidates with a multilayer perceptron over graph-context summaries, adds a previous-placement bonus, and selects a layout through mixed-integer optimization.

A design-oriented evaluation of PFAL in 16nm FinFET and a motivation to exploit adiabatic logic for future low-power system architectures are provided.

SeetaPsych v1.0 is presented, an open source, unified and extensible computer vision toolkit designed to extract psychologically relevant signals from facial images and/or face based videos to offer an integrated and accessible visual measurement platform for research in psychology, behavioral science, human computer interaction, and related fields.

This work investigates whether hybrid backbones can become effective DLMs by adapting Qwen3.5 at 0.8B, 2B, 4B, and 9B scales, and finds that they can be efficient starting points for adaptation.

SlugTrails is introduced, a floor plan localization benchmark for large indoor spaces under realistic egocentric sensing, and five representative geometric and learned systems under their native sensing configurations are evaluated, finding that stock checkpoints are near zero on SlugTrails and evidence that floor plan localization is currently limited by indoor data rather than by architecture.

Inspired by human cognition, CRCD reformulates the task as counterfactual latent change captioning to bypass affirmation bias, and contrasts a synthesized safe expectation with reality to identify semantic omissions.

This work proposes FreqDINO++, a frequency-guided multi-task routing vision foundation model for universal ultrasound analysis and demonstrates that FreqDINO++ consistently outperforms strong baselines and recent foundation models across 27 diverse clinical task scenarios, while also showing promising generalization to unseen data.

This work presents, to the best of the knowledge, the first online Gaussian SLAM framework that tracks and maps directly on single-exposure 16-bit linear HDR imagery and transfers seamlessly to SplaTAM, Gaussian SLAM, and DROID-W, eliminating all tracking failures these systems suffer on challenging illumination sequences.

This study presents a deep learning framework to distinguish between resting and cognitive states through EEG records that outperform the evaluated comparative methods and illustrates the efficacy of integrating signal processing with deep learning methodologies to discriminate resting from cognitive states utilizing EEG signals.

This survey examines the state of the RISC-V ISA in machine learning applications, analyzing current capabilities, challenges, and future directions based on recent research, and proposes four research directions to address current limitations.

A Transformer-based Quantum State Characterizer model is proposed for noisy RSP experiments that enables accurate tomographic characterization under dynamic noise and provides physically grounded post-hoc insights, holding promise for intelligent quantum information processing applications.

A unified conditional model for controllable field-to-field synthesis, built on the framework of conditional latent bridge matching, achieves highly competitive results across the validation phase for all three tasks of the MRIxFields2026 challenge without task-specific architectures or training.

The proposed E-AVI framework improves predictive performance while providing inspectable support for assessment, feedback, and interactive analysis, and these results demonstrate that E-AVI consistently outperforms fine-tuned multimodal baselines in rank correlation.

Adaptive Uncertainty-Routed Analysis is introduced, a multimodal email threat detection system that analyzes both the content of an email and its embedded URLs and quantifies prediction uncertainty from a URL classifier.

It is shown that steering creates a predictable prompt-injection attack surface whose vulnerability increases monotonically with steering magnitude, relevant to agent systems deploying steered classifiers.

The proposed scalable trust discovery architecture incorporates a registry-suffix-anchored composite identity scheme, which binds an agent native identifier to a trusted registry suffix to generate a globally discoverable identity.

FreqCondNorm is proposed, a Transformer-based architecture that introduces a FiLM-style frequency-conditioned normalization layer to unify heterogeneous time-series within a single model, demonstrating strong transfer across sampling frequencies.

Comparing the latest models from four language-model families against expert assessments, it is found that current models fall short of human expertise in predicting classroom outcomes; the strongest models reach 58% accuracy compared to 64% of human experts.

Teacher Prediction Refinement Distillation (TPRD), a plug-and-play module that refines teacher predictions before distillation by exploiting stage-wise prediction information, and Maximum Dark Knowledge Preservation (MDKP), which selectively refines target-class logits while retaining non-target relations.

This work ground a model in the Sheep Pain Facial Expression Scale (SPFES) by letting each detected facial region attend over text embeddings of the clinical descriptors and then test whether the resulting explanations mean anything, and shows that architectural necessity does not imply semantic validity.

A deep architecture that is able to jointly optimize cost functions and route-ranking model towards any route preference is proposed, and a novel loss function is proposed that optimizes a single-objective variable, with other variables strictly under constraints.

MM-Future, a world-action model that generates multiple paired scene-action hypotheses and models bidirectional interaction within each pair, shows consistent improvements over both single-mode and action-only variants, validating the benefit of multi-mode joint world-action modeling.

DirtyMoCap is introduced, a robust, marker-layout-free framework that consistently outperforms state-of-the-art configuration-specific baselines in both joint and vertex reconstruction accuracy, while the custom CUDA solver achieves up to a 100x speedup over standard PyTorch implementations.

The architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC result in a model that is on the speed-accuracy Pareto frontier of the Open ASR leaderboard for English short-form ASR while being twice as fast as the fastest competitor.

A closed-loop control architecture for a non-deterministic language model that must satisfy a numeric output constraint, that is, a requirement expressible as a number or an interval and checkable by code, is presented and evaluated.

It is found that functional words are located centrally compared to nouns, but are also distinct, which is congruent with their behaviour as place-holders in a wide variety of contexts.

The ANVIL compiler architecture is proposed, where ANVIL performs exceptionally well on leading benchmarks by using language models purely where they are effective, rather than as a catch-all tool.

The experiments show that self-supervised pretraining and mark localization are the two most impactful factors, with learned cropping recovering the majority of the gain from manual cropping without requiring ground-truth annotations at inference time.

This paper presents an optimized, high-frequency Multispeculative PositMAC architecture for 32- and 64-bits Posit arithmetic, which achieves the highest operating frequency and reduces cycle time by up to 79.0\% with respect to 64-bit quire-enabled alternatives.

A unified framework for the LISA 2026 Challenge that performs all three tasks together within one inference pipeline, and addresses the scarcity of dense annotations using diffeomorphic registration from atlas to target for label propagation and to regularize anatomical reconstruction.

The resulting lightweight streaming model, Mamba-FVAD, outperforms commercial VADs and enrollment-based speaker-aware systems in foreground selectivity while staying competitive on conventional VAD, at 1-2 ms per-frame CPU latency.

Experiments demonstrate that MoWAM achieves strong in-distribution performance, improved out-of-distribution robustness, and higher average real-world success than representative WAM baselines, demonstrating that explicit future motion provides an effective and efficient basis for inference-time scaling.

Numeric results demonstrate stable transitions (without abrupt changes) in both the end-effector parameter space and in the active joints parameters, thereby validating the proposed approach.

A framework that fuses the output of Trajectron++, a neural network-based trajectory predictor, with extended Kalman filter (EKF)-based multiple trajectory candidates at a late stage indicates that EKF-based trajectory candidates can effectively complement neural trajectory prediction through learned fusion.

JaxPIP is introduced, a JAX-based implementation that reformulates PIP/FI evaluation into tensorized linear algebra operations that allows efficient execution with just-in-time compilation and enables large-scale batch evaluation of energies and forces.

This work evaluates cross-embodiment transfer on RoboTwin 2.0 in a controlled setup, two bimanual robots demonstrate disjoint task sets, a policy is trained on all the demonstrations, and each robot is evaluated closed-loop on the tasks only the other demonstrated.

CoreSense is presented, a robot-system integration architecture that combines traceable episodic evidence with a conflict-aware belief gate and bounded, auditable recommendations that supports an auditable integration pattern, not autonomous recovery or certified safety.

The results show that FootprintRAG helps users compare retrieval directions, revise candidate evidence, recover potentially overlooked evidence, recover potentially overlooked evidence, and trace generated summaries back to supporting evidence units.

The DeepSeek-V4.1-Flash model, a multimodal Mixture-of-Experts model with 552B backbone parameters and support for contexts of up to one million tokens, is introduced, substantially improving cost efficiency for agentic workloads and pushing the limits of KV cache compression.

JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF), achieves the lowest one-step and 100-step molecular errors among compared methods in all four systems and supports a common factorized predictive principle across heterogeneous worlds.

A Bayesian model fit to each domain's prediction-powered estimate, with an extension that borrows strength across a reporting taxonomy, and a new, approximately unbiased design-based cross-validation score for choosing among direct and smoothed estimators for estimation and validation are derived.

This paper introduces PersianVox, a fully automated pipeline designed to generate high-quality speech corpora from unlabeled web data, using a novel prosody-aware segmentation strategy that utilizes acoustic turn-detection to preserve linguistic completeness and optimize utterance duration for long-context modeling.

This paper proposes a split-control architecture in which an untrusted planner emits typed security intents, a deterministic governor checks each intent against safety, resource, temporal-stability, and proportionality invariants, and only admitted actions are bound to signed receipts and compiled into pre-installed eBPF map updates.

An online multimodal assessment framework that integrates interaction wrench, planar tool-center-point (TCP) kinematics, and skin conductance level (SCL) into four interpretable workload-related factors that keeps the assessment process transparent through explicit workload-related factors and defined weighting rules, while outperforming the corresponding fixed-weight formulation.

Graph-Guided Token Merging (G2TM) is extended, a single module inserted early in a ViT-based network, beyond its original Segmenter setting, and its behavior and accuracy-efficiency trade-off are consistent across every tested architecture for a given backbone size, indicating that its effectiveness is a property of the encoder rather than the decoder.

This work introduces physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction, and presents ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization.

The single mid-run human intervention that pulled the community out of a monoculture is described, what the trace does and does not establish, and the controlled comparison that would settle whether shared research state improves discovery per unit of compute is described.

The proposed Riemannian--Lorentz Parameter Fusion (RLPF) method projects aligned groups to common coordinates, lifts selected coordinates to the Lorentz hyperboloid model of hyperbolic space, computes a regularized geodesic barycenter, and decodes the result into the two branches.

Presley is presented, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting.

Rosetta is presented, a multi-agent LLM pipeline that automatically generates first-principles analytical models from research paper PDFs, and four design decisions address failure modes of na\"ive LLM-based generation.