Search papers, labs, and topics across Lattice.
This paper introduces a generalized framework for analyzing multiple heterogeneous datasets using generative AI-based inverse problem solvers, addressing the challenge of extracting unknown quantities from diverse measurements. By extending the Scalable Asynchronous Generative Inverse Problem Solver (SAGIPS) to handle non-identically distributed datasets, the authors ensure that each dataset contributes unique constraints while maintaining global parameter consistency. Validation through a multi-detector scattering experiment demonstrates the framework's robustness to varying data fidelities and its scalability on multi-GPU systems, highlighting its applicability in complex real-world scenarios.
Jointly analyzing heterogeneous datasets with generative AI can yield precise estimates of unknowns that independent analyses miss, even under varying experimental conditions.
Extracting a shared set of unknown, not directly measurable quantities from multiple, heterogeneous datasets is a common challenge across scientific domains. A prominent example is the combination of datasets obtained from different measurements with different settings (e.g. varying detector resolutions). Analyzing such datasets jointly, rather than independently or after naive merging, is essential for obtaining precise and unbiased estimates of the unknowns, but requires careful treatment of dataset heterogeneity and is computationally demanding. We present a generalized framework for simultaneously analyzing multiple heterogeneous datasets in the context of generative AI-based inverse problem solvers. Building on our recent Scalable Asynchronous Generative Inverse Problem Solver (SAGIPS) framework, we extend the well-established distributed data-parallel training paradigm to non-identically distributed datasets, where each dataset is controlled by the same set of unknown inference parameters but covers a different region of the available feature space. Each dataset is processed through its own forward operator and discriminator, providing complementary constraints that collectively guide a shared generator toward global parameter consistency. We validate the approach using a controlled setup inspired by a multi-detector scattering experiment. We provide numerical evidence that our framework is robust to different data fidelities, which arise from unknown detector systematics in the Rutherford experiment, and we show the scaling behavior on multi-GPU leadership computing systems. The results show that our approach is well suited for real-world multi-dataset analyses in which experimental conditions vary across measurements.
