Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Marformer, a Transformer architecture designed to directly predict conditional marginal distributions of missing variables given observed data. This approach bypasses the need to model the full joint distribution or rely on domain-specific knowledge, enabling efficient prediction of key components for decision-making under uncertainty. The Marformer achieves competitive or superior performance to classical and generative methods across synthetic and real-world datasets, while offering significant speed advantages.
Directly predicting conditional marginals with a single Transformer pass unlocks efficient decision-making under uncertainty, outperforming generative models and classical methods even with full model knowledge.
Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The \textbf{conditional marginals} over the missing variables are the key ingredient for computing Bayes risk and Value of Information (VOI), the expected gain from acquiring one more observation before deciding. We present the Marformer, a Transformer trained to directly predict conditional marginals given any set of observed values. Like BERT, which is trained to predict missing words from context, the Marformer constructs a hidden-vector representation for each distribution $p(X_i)$ and iteratively refines it through attention to other distributions $p(X_j)$. Unlike generative approaches, the Marformer does not model the full joint distribution, requires no domain knowledge of the data-generating process, and makes all predictions in a single forward pass. We evaluate across three synthetic domains with missing data---Bayesian networks, discretized multivariate Gaussians, and structured annotation data. The Marformer can match or outperform classical missing-data methods, even when those methods are given the true model family and prior that generated the synthetic data. We also evaluate on a real annotation dataset, where the Marformer outperforms the evaluated baselines at the largest training size. In both cases, the Marformer is substantially faster than the evaluated generative baselines.