Jonathan Schwartz

Cryo-electron tomography (cryo-ET) is a powerful technique for imaging molecular complexes in their native cellular environments. However, identifying the vast majority of molecular species in cellular tomograms remains prohibitively difficult. Machine learning (ML) methods provide an opportunity to automate the annotation process, but algorithm development has been hindered by the lack of large, standardized datasets. Here we present an experimental phantom dataset with comprehensive ground-truth annotations for six molecular species to spur new algorithm development and benchmark existing tools. This annotated dataset is available on the CryoET Data Portal with infrastructure to streamline access for methods developers across fields.

Measuring the three-dimensional (3D) distribution of chemistry in nanoscale matter is a longstanding challenge for metrological science. The inelastic scattering events required for 3D chemical imaging are too rare, requiring high beam exposure that destroys the specimen before an experiment is completed. Even larger doses are required to achieve high resolution. Thus, chemical mapping in 3D has been unachievable except at lower resolution with the most radiation-hard materials. Here, high-resolution 3D chemical imaging is achieved near or below one-nanometer resolution in an Au-Fe3O4 metamaterial within an organic ligand matrix, Co3O4-Mn3O4 core-shell nanocrystals, and ZnS-Cu0.64S0.36 nanomaterial using fused multi-modal electron tomography. Multi-modal data fusion enables high-resolution chemical tomography often with 99% less dose by linking information encoded within both elastic (HAADF) and inelastic (EDX/EELS) signals. We thus demonstrate that sub-nanometer 3D resolution of chemistry is measurable for a broad class of geometrically and compositionally complex materials.

The demand for high-throughput electron tomography is rapidly increasing in biological and material sciences. However, this 3D imaging technique is computationally bottlenecked by alignment and reconstruction which runs from hours to days. We demonstrate real-time tomography with dynamic 3D tomographic visualization to enable rapid interpretation of specimen structure immediately as data is collected on an electron microscope. Using geometrically complex chiral nanoparticles, we show volumetric interpretation can begin in less than 10 minutes and a high-quality tomogram is available within 30 minutes. Real-time tomography is integrated into tomviz, an open-source and cross-platform 3D data analysis tool that contains intuitive graphical user interfaces (GUI), to enable any scientist to characterize biological and material structure in 3D.

Efforts to map atomic-scale chemistry at low doses with minimal noise using electron microscopes are fundamentally limited by inelastic interactions. Here, fused multi-modal electron microscopy offers high signal-to-noise ratio (SNR) recovery of material chemistry at nano- and atomic-resolution by coupling correlated information encoded within both elastic scattering (high-angle annular dark-field (HAADF)) and inelastic spectroscopic signals (electron energy loss (EELS) or energy-dispersive x-ray (EDX)). By linking these simultaneously acquired signals, or modalities, the chemical distribution within nanomaterials can be imaged at significantly lower doses with existing detector hardware. In many cases, the dose requirements can be reduced by over one order of magnitude. This high SNR recovery of chemistry is tested against simulated and experimental atomic resolution data of heterogeneous nanomaterials.

Highly-directional image artifacts such as ion mill curtaining, mechanical scratches, or image striping from beam instability degrade the interpretability of micrographs. These unwanted, aperiodic features extend the image along a primary direction and occupy a small wedge of information in Fourier space. Deleting this wedge of data replaces stripes, scratches, or curtaining, with more complex streaking and blurring artifacts—known within the tomography community as "missing wedge" artifacts. Here, we overcome this problem by recovering the missing region using total variation minimization, which leverages image sparsity-based reconstruction techniques—colloquially referred to as compressed sensing (CS)—to reliably restore images corrupted by stripe-like features. Our approach removes beam instability, ion mill curtaining, mechanical scratches, or any stripe features and remains robust at low signal-to-noise. The success of this approach is achieved by exploiting CS's inability to recover directional structures that are highly localized and missing in Fourier Space.