Search papers, labs, and topics across Lattice.
This paper addresses the challenge of calibrating Scanning Transmission Electron Microscopes (STEM) by using variational autoencoders (VAEs) trained on simulated data to learn low-dimensional image representations. They then employ an expectation maximization (EM) approach to jointly estimate the mapping between calibration parameters and the encoded representations, explicitly tackling the simulation-to-reality gap. The method leverages the symmetry properties of the optical system to ensure global identifiability, resulting in a 2x reduction in estimation error and faster calibration compared to existing methods on a real STEM.
By jointly estimating the mapping from calibration parameters to VAE-encoded image representations, this work achieves a 2x reduction in error when calibrating electron microscopes, demonstrating the power of bridging simulation and reality.
Electron microscopy has enabled many scientific breakthroughs across multiple fields. A key challenge is the tuning of microscope parameters based on images to overcome optical aberrations that deteriorate image quality. This calibration problem is challenging due to the high-dimensional and noisy nature of the diagnostic images, and the fact that optimal parameters cannot be identified from a single image. We tackle the calibration problem for Scanning Transmission Electron Microscopes (STEM) by employing variational autoencoders (VAEs), trained on simulated data, to learn low-dimensional representations of images, whereas most existing methods extract only scalar values. We then simultaneously estimate the model that maps calibration parameters to encoded representations and the optimal calibration parameters using an expectation maximization (EM) approach. This joint estimation explicitly addresses the simulation-to-reality gap inherent in data-driven methods that train on simulated data from a digital twin. We leverage the known symmetry property of the optical system to establish global identifiability of the joint estimation problem, ensuring that a unique optimum exists. We demonstrate that our approach is substantially faster and more consistent than existing methods on a real STEM, achieving a 2x reduction in estimation error while requiring fewer observations. This represents a notable advance in automated STEM calibration and demonstrates the potential of VAEs for information compression in images. Beyond microscopy, the VAE-EM framework applies to inverse problems where simulated training data introduces a reality gap and where non-injective mappings would otherwise prevent unique solutions.
