Architecture Design (Transformers, SSMs, MoE) - Weekly Roundup

Inspired by Mixture-of-Experts (MoE) principles, Evolvable-Substrate HyperNEAT is partitioned into non-overlapping spatial segments, each assigned to a separately evolved specialist network, which forces evolution to discover features across the entire image.

This work presents RelateAnything, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings.

Latent Interface Training (LIT), a framework-agnostic two-stage strategy that first establishes a spatial-goal-conditioned action prior without images, then constrains visual conditioning through a pose-supervised latent interface, improves overall LIBERO-Plus success while preserving or improving average LIBERO success.

Across reasoning, long-context understanding, and agentic tasks, SAS consistently outperforms trainable sparse attention baselines across attention budgets, with especially large gains under tight budgets, demonstrating more effective context ranking for downstream tasks.

This work presents RelateAnything, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings.

Latent Interface Training (LIT), a framework-agnostic two-stage strategy that first establishes a spatial-goal-conditioned action prior without images, then constrains visual conditioning through a pose-supervised latent interface, improves overall LIBERO-Plus success while preserving or improving average LIBERO success.

Across reasoning, long-context understanding, and agentic tasks, SAS consistently outperforms trainable sparse attention baselines across attention budgets, with especially large gains under tight budgets, demonstrating more effective context ranking for downstream tasks.

ESRL is introduced, an architecture-aware framework that explicitly explores the expert-routing space of MoE models, and preserves high-confidence experts as anchors, and restricts stochastic routing to a plausible candidate pool, thereby retaining reliable computation paths.

This study introduces StepAudio 3 Gen, a general-purpose audio generation model that supports zero-shot text-to-speech (TTS), voice design, vocal generation, sound effects, music, vibe speech, and mixtures of multiple audio types within a unified framework.

eConv-TasNet is proposed, an efficient variant of Conv-TasNet that improves both effectiveness and efficiency without relying on resource-intensive modules, and achieves competitive performance compared with state-of-the-art methods while requiring significantly fewer parameters and lower inference cost.

ActMap is introduced, a white-box representation that compresses the generation-time hidden- state trajectory into a fixed tensor of temporal-statistic channels that preserves structure across transformer depth and pooled hidden coordinates that supports abstention, routing, and selective verification from a single generation, making it a practical primitive for scalable oversight of deployed models.

This work proposes a fully analog readout architecture based on voltage-to-time conversion of the VMM output based on sensing the column voltage, which avoids current-mode summing and scaling circuitry, improving area and energy efficiency.

This work proposes MomentUm SpEctral Clipping (Musec), which replaces Muon's spectral flattening with spectral clipping: rather than setting all singular values of the momentum matrix to approximately one, Musec clips singular values that exceed a threshold while preserving the underlying spectral structure of the momentum.

This work presents FreeFlow, a hierarchical transformer built without any flow-specific components, using instead a single feed-forward encoder--decoder, achieving state-of-the-art results on major benchmarks, including Sintel, KITTI-2015, and Spring.

Experimental results demonstrate that DRG-MAPPO achieves a state-of-the-art win rate of 87%, suggesting that the framework effectively balances relational modeling, interpretability, and optimization stability for cooperative air combat.

T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier by executing each task's own verifier.

This work addresses the adverse interactions between sparsity and data repetition, and presents evidence for the core mechanisms of overfitting and its potential remediation, and suggests promising avenues for future methods to reduce over-specialization in model parameters by disrupting memorization patterns.

The Explainability Assistant is introduced, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large Language Models (LLMs) to overcome limitations and achieves 94% intent-parsing accuracy, supports flexible natural language interaction, and adapts to different ML problem types without task-specific fine-tuning.

An amortized neural sampler that combines operator learning with flow methods for sampling, enabling efficient sampling across families of stochastic differential equations, and theoretically establishes the expressivity and resolution invariance of the framework.

This work generates synthetic plankton imagery conditioned on taxonomy: a CLIP encoder is adapted on a large plankton corpus with a ranked contrastive objective extended to deep, ragged taxonomies, then frozen to condition a parameter-efficient diffusion transformer.

A certification layer is developed that wraps any severity model unmodified, with distribution-free guarantees using this structure, and attaches identical validity and certifies, on the vulnerable road users, a model-independent floor on set width that no base model beats.

Inspired by Mixture-of-Experts (MoE) principles, Evolvable-Substrate HyperNEAT is partitioned into non-overlapping spatial segments, each assigned to a separately evolved specialist network, which forces evolution to discover features across the entire image.

E-CONAN benchmarks that are composed of sentences pairs from various sources, derived from a combination of sources, offers a broader and more robust assessment compared to XNLI and ArNLI, and are used to evaluate state-of-the-art multilingual pretrained models using zero-shot classification.

A phase-decoupled, model-calibrated controller that hypothesizes that the optimal power setting is a property of the deployed (model, quantization, engine, hardware) combination rather than of the GPU class, that each lane warrants its own profile, and that converting SLO headroom into energy safely requires latency-gated calibration under a runtime SLO guard rather than a fixed recipe.

Extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.

A modular agentic-AI platform that converts a heterogeneous corpus of process-development documents into a queryable, dual-layer knowledge graph is presented and it is anticipated graph-based architectures will see broader adoption in pharma as a means of transforming fragmented document repositories into structured process intelligence.

AssayLoop is introduced, a sequential experimental design framework combining AssayFormer, a transformer-based amortized acquisition policy trained across historical screens to adapt from experimental feedback, with LLM-derived biological priors through an adaptive handoff to demonstrate the value of learning acquisition policies across historical experiments and combining them with broad biological priors for efficient adaptive hit discovery.

Evaluating pre-trained convolutional neural networks for skin lesion classification using dermatoscopic and histopathological image datasets demonstrates that model performance differs between dermatoscopic and histopathological image modalities, showing that architectures exhibiting similar performance on dermatoscopic images exhibit different performance on histopathological data.

This paper introduces Agent-Integrated Software (AIS) as a software pattern combining a conventional core, direct interaction, and a built-in agent, and Intent-Level Interaction Abstraction (IIA) as the task semantics through which users inspect and control delegated work.

This study investigates the use of LLMs as post-processing tools to analyze and rank evolved expressions according to their interpretability and medical plausibility, and finds that they are better suited to comparative auditing under expert oversight than to autonomous validation.

RAMamba-Net is proposed, a reliability-aware Mamba-based multimodal fusion network for AAD that effectively exploits complementary EEG-EOG information, yielding accuracy gains over unimodal baselines, and is robust to signal perturbation and parameter variation.

It is shown that the appearance of nodules below six millimeters is highly regular, and proposed GRIPNet (Gaussian Radial Intensity Prior Network), a detector in which every module maps to a measurable property of the intensity distribution, is proposed.

This work proposes two approaches of code-mixed generation using parallel sentences of three languages and fine-tune contextual transformer-based models MuRIL and XLM-RoBERTa best suited for Indian languages for language identification in code-mixed settings.

INDRA is an LLM-powered research partner enabling massive multi-archival investigations, a tool whose outputs are designed to be checked rather than trusted, and whose architecture makes the conditions of knowledge production visible and auditable.

This work empirically diagnose the root cause across six per-layer methods and three speech-LLM architectures and proposes a two-pool allocation that normalises encoder and LLM parameters into independent pools, and shows that joint $\ell_2$ sensitivity and the original $(\varepsilon,\delta)$-DP guarantee are unchanged.

A thorough investigation of distance generalization probes performance when inter-token distances are changed between training and inference, while keeping a fixed context length, finding that it is paramount to improve the understanding of the underlying mechanisms.

SoulAuth, an open-source Rust reference implementation for Humans and long-lived AIActors, is presented, arguing that any subject that must persist under its own identity and remain independently attributable should have an ActorIdentity that is not replaced by an Account, Credential, Client, AuthSession, IdentityBinding, or runtime instance.

This paper introduces Financial Safety for Teens Pay (FST Pay) as an architectural and formal specification that is founded on an immutable operational boundary: strict deterministic safety gating on the real-time authorization path coupled with decoupled downstream AI explanation.

DeFiFusion is presented, a dual-modal PMA detection framework that closes this gap by jointly modeling transaction events and smart contract semantics within a unified pipeline, and proposes a Dual-Modal Projection-Fusion Transformer with T5-style relative positional encoding.

RCL (Retrieval-Confidence Layer), a lightweight module inserted between retrieval and generation that combines a call-graph-derived structural coverage score with a novelty score estimating a query's dependence on knowledge outside the model's prior, to detect insufficient retrieval before generation occurs.

An automated bug triage system built upon the pre-trained RoBERTa-base transformer architecture is proposed, which achieves a strong bug identification accuracy within just five training epochs and highlights the efficiency and high performance of fine-tuned transformer models for practical software engineering automation.

The results show that spectral adapters can efficiently adapt SAM for CRLM segmentation while retaining strong accuracy with limited trainable parameters.

Results suggest that axial order provides a useful inductive bias for weakly supervised PNI prediction from MRI, suggesting that axial order provides a useful inductive bias for weakly supervised PNI prediction from MRI.

With convolutional and Transformer operators and single- or multi-resolution latent interfaces, LoopVAE establishes recurrent depth across scales as a parameter-sharing design axis for visual tokenization.

SCINTILLA-SNN, a 3D spiking network composed of a four-stage hierarchical backbone and a Multi-Scale Spike Aggregation (MSSA) module for PNI prediction, is proposed, which achieves an AUROC of 0.748 under 5-fold cross-validation, while reducing the estimated inference energy by 23.18$\times compared with dense MAC-only computation of the same network.

HighPerViT is presented, a compact vision-only architecture that injects an explicit second-order statistical prior into a transformer-based recognition pipeline, and integrates this token with first-order spatial representations through Perceiver-style latent distillation, supporting explicit statistical tokenization as an effective and robust design principle for texture-centric visual recognition.

A novel Dual-Latent Space Reinforcement Learning (DLSRL) framework, which complements initial-noise steering with representation-level control inside the frozen generator and effectively accelerates online robot policy adaptation and achieves competitive performance.

By decoupling haptic control loops from network bottlenecks, CARLAverse enables scalable, cross-institutional HITL experiments without compromising physical immersion.

This work introduces a probabilistic framework for evaluating VAD onset time under noisy temporal labels, and introduces S4VAD, the first state-space-model-based (SSM-based) VAD architecture that achieves the lowest latency while maintaining a competitive AUROC.

Sewer-Transformer-ML is developed, a hierarchical vision Transformer with multi-level feature fusion together with two lightweight architectures, Sewer-MobileNet-ML and Sewer-Mobile-TransNet, for resource-constrained inspection scenarios and provides a computational basis for automated sewer inspection, lightweight model design, and adaptation across civil infrastructure inspection platforms.

CEM-TUDASR, a computationally efficient unsupervised Transformer-based super-resolution framework for WCE image enhancement without paired low-resolution (LR) and high-resolution (HR) training data, is proposed.

Experiments on 4DMatch and 4DLoMatch show that both variants produce more accurate correspondences than the compared methods and improve downstream registration, with larger gains in low-overlap cases.

Experiments show that ZipCodec substantially outperforms existing streaming codecs at comparable bitrates in both reconstruction and downstream tasks, while operating at a significantly lower frame rate.

This work reifies the input graph by reifying the input graph: every fact becomes a node, connected to its subject, object, and relation type through a fixed vocabulary of six meta-relations, with relation types as anonymous shared nodes rather than model parameters.

GenQAS is introduced, a tensor network-guided RL framework that combines a fixed matrix product state warm-start with prioritized generative replay and can mitigate sample starvation in quantum architecture search and support more resource efficient circuit discovery.

This work introduces a model-aware schedule construction based on fiberwise optimal transport, and evaluates DDPMs and flow matching across prediction targets, training configurations, risk-estimation checkpoints, datasets, and architectures.

This work introduces AmazonSWE, a dataset for training and evaluating large-scale spatiotemporal graph imputation methods that integrates processed satellite altimetry measurements from a range of sources, including the recent wide-swath SWOT sensor.

RISE AI provides an architecture for making bounded, evidence-based claims about Responsibility, Inclusivity, Safety, and Empowerment, and develops a rupture test that links institutional baselines to system evaluation.

This work introduces CoRA-NAS (COarse Ranking + Anchor-residual), a two-stage framework combining a static ranking prior with low-cost learning-curve refinement that combines cross-space ranking robustness with low-cost architecture selection.

A four-coefficient parameterization lambda_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions, and which adds multi-channel composition and an explicit bias as further degrees of freedom.

AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes is introduced, which is scalable, model-agnostic, and straightforward to integrate into existing training pipelines.

A novel framework for model-based reinforcement learning which introduces approximate inverse process models within the training of reinforcement policies, and proposes a lightweight feedforward architecture for approximate inverse models and integrate them within the policy network of standard RL algorithms.

A Mobius layer is proposed, with rational functions replacing matrix operations, and the recently discovered activation function eml(x,1/x), which allows exp(x) and ln(x) to be recovered separately, and hence all elementary functions to be evaluated within a rational generalization of the neural network.

HerALD (High-fidelity Exemplar Retrieval with Adaptive Landmark Distillation with Adaptive Landmark Distillation), a gradient-free graph condensation framework that adapts the node scoring and feature selection in the condensation pipeline to the graph's measured heterophily, is proposed.

NovGauge is presented, a human-anchored benchmark for fine-grained novelty assessment diagnosis, and a cascading diagnostic pipeline that verifies per-dimension correctness, evidence grounding, and logical support is proposed.

An information-weighted cross-entropy loss that rescales token-level contributions using TF-IDF statistics, emphasizing semantically informative tokens while down-weighting ubiquitous ones is presented, offering a lightweight and principled way to mitigate memorization without disrupting standard training dynamics.

A data story is understood as a narrative document that integrates explanatory text and images with executable SPARQL queries and their visualized results and serves several purposes: guiding users through an unfamiliar graph, creating reproducible narratives, and surfacing data-quality issues previously hidden in aggregate statistics.

CoMA-DiT is proposed, a bidirectional cross-modal Diffusion Transformer for latent augmentation that treats paired modalities as sources of mutual generative supervision rather than merely as inputs to be fused.

A unified, patch-based framework that processes raw multimodal imaging through training-free registration, automated localization, and mask-filtered patch extraction that culminates in a hierarchical strategy that aggregates patch-level insights into subject-level diagnostics.

AI-ER is developed as a two-dimensional assessment framework that keeps exposure and resilience separate and adds an explicit assessment of evidence quality and confidence, and is a traceable company profile that supports comparison without concealing uncertainty in the underlying evidence.

An initial assessment of the suitability of VLM-based methods for V&V of defence applications is provided and a VLM-based error slice detection (ESD) method that independently groups and labels systematic errors made by a classification model is proposed.

A novel electro-photonic accelerator architecture optimized for TFHE, featuring OPCM-based FFT units, a twiddle-stationary dataflow tailored for OPCM, and a scheduling mechanism to maximize the utilization of the FFT units is introduced.

This study forms continuous recognition of the Group Emotion at a one-second resolution, introduces the Mixed state, which captures the emotional divergence among participants in the group, and proposes a multimodal temporal framework that integrates audio and video information using a sliding-window context.

Xiaomi-CocktailASR-1 achieves state-of-the-art performance, effectively addressing the cocktail party problem through a unified architecture that balances multispeaker and single-speaker recognition accuracy, along with rejection capability.

This paper proposes a model-centric DevOps architecture for deploying DEVS-based digital twin simulations as managed services, treating simulation models as first-class DevOps artefacts defined in a declarative YAML language with a formal mapping to multiPDEVS.

It is shown that structured LLM prompting and scalar supervision collapse rubric dimensions, yielding near-zero correlation with human ratings and strong cross-dimension coupling, are effective for segment-level SI evaluation.

It is argued that mobile OS and app store interventions are urgently needed to close this architectural blind spot before it is further exploited.

Evaluated on recommendation models, Auto-RecSys significantly reduces the human time required per experiment cycle and improves execution reliability as its playbooks mature.

This work found that certain attention blocks naturally form an Intrinsic Spatial Grounding Map that precisely locates reference subjects, and proposes Dual-phase Intrinsic Attention Leveraging (DIAL), a framework that uses these internal signals for both training and inference.

TripleBound is proposed, a hybrid framework for automated monolith-to-microservices decomposition that augments a heterogeneous graph neural network with weakly supervised triplet constraints derived from parser-inferred service groups based on package structure, naming conventions, and code location.

This report demonstrates using a detailed transaction-level model (TLM) of a probabilistic analogue in-memory computing (AIMC) processor that the same energy-based model can execute well over 1000x faster than data-center-grade hardware by eliminating the HBM interface and performing computation directly within on-chip memory.

This paper introduces AccelForge, which improves upon existing accelerator modeling frameworks in capabilities, speed, and ease-of-use and includes fast mappers that enable accurate evaluation in orders of magnitude less (computer and human) time.

F Fengshui, a chiplet ecosystem and accelerator co-design framework that jointly optimizes chiplet pool composition and bespoke application-specific integrated circuit (BASIC) design, is introduced.

A unified post-training framework that equips pretrained text-to-speech models with natural-language control over segment-level emotion and duration is proposed, highlighting post-training as a practical approach to extending existing speech synthesis models.

Qualitative generalization across trajectory, scene, object, embodiment, and viewpoint shifts highlights Pelican-Sim's potential as a general-purpose world model simulator.

All three AI chatbots exhibited a high level of ability in the understanding of radiological cases with abnormal anatomy, indicating their potential use as a supplementary tool in radiological assessment and radiological diagnostic support.

Covariance-based GNNs are fundamentally learnable generalizations of PCA, offering rigorous stability bounds against finite-sample noise that standard graph theory fails to capture.

Removing an architectural prior abruptly tanks Transformer recall from 77% to 9%, but smoothly annealing it away allows the model to permanently consolidate the underlying circuit.

Long-range vessel trajectory prediction error drops by 55% at a 5-hour horizon simply by replacing dense self-attention with sub-quadratic ProbSparse attention augmented by multi-frequency temporal priors.