Search papers, labs, and topics across Lattice.
StrTransformer is introduced as a novel framework for unsupervised blind source recovery, directly optimizing a latent source matrix using an observation-space mixer and source-wise structured Transformer branches instead of relying on an encoder. Each Transformer branch imposes a differentiable structural constraint on a latent source trajectory by converting sources into multi-scale patch tokens, masking them, and processing them with a locality-biased Transformer, using masked patch reconstruction energy as a source-wise structural prior. Experiments demonstrate that the learned branches converge to distinct temporal-scale structures and recover source-aligned latent trajectories.
Unsupervised blind source recovery gets a boost from StrTransformer, which learns disentangled, interpretable latent source trajectories by directly optimizing a latent source matrix with source-wise structured Transformers.
This paper proposes StrTransformer, a source-wise structured Transformer framework for blind source recovery and branch-wise latent modeling. Instead of using an encoder to infer latent variables, StrTransformer directly optimizes the latent source matrix together with an observation-space mixer and source-wise structural Transformer branches. The mixer enforces reconstruction consistency, while each Transformer branch imposes a differentiable structural constraint on one latent source trajectory. Specifically, each source is converted into multi-scale patch tokens, randomly masked, processed by a locality-biased Transformer, and evaluated through a masked patch reconstruction energy. This energy acts as an implicit source-wise structural prior. To encourage different latent branches to specialize into different temporal regimes, StrTransformer further introduces an ordered multi-scale controller that learns branch-specific patch-scale weights, ordered scale centers, and locality attention slopes. The resulting objective combines observation reconstruction, source-wise structural regularization, and modular auxiliary penalties for separation and scale specialization. We analyze the decoupling and coupling structure of the objective, the regularized exact-reconstruction fiber, and the reduction of permutation symmetry induced by ordered branch descriptors. A controlled case study shows that the learned branches converge to distinct temporal-scale structures and recover source-aligned latent trajectories under post-hoc evaluation.
