Nature Machine Intelligence · 2023 · 43 citations · 36 references
Natural SciencesBioengineeringBiomedical EngineeringBiomedical ImagingComputer VisionNanotechnologyCellular ImagingConfocal MicroscopyFluorescence ImagingFluorescence MicroscopyOptical ImagingSuper-resolution MicroscopyGenerative Adversarial NetworksMicroscopyNanostructure CharacterizationResolution EnhancementStimulated Emission Depletion MicroscopyStructured Illumination MicroscopySuper-resolution Fluorescence ImagingSuper-resolution Fluorescence Microscopy
Super-resolution fluorescence microscopy methods enable the characterization of nanostructures in living and fixed biological tissues. However, they require the adjustment of multiple imaging parameters while attempting to satisfy conflicting objectives, such as maximizing spatial and temporal resolution while minimizing light exposure. To overcome the limitations imposed by these trade-offs, post-acquisition algorithmic approaches have been proposed for resolution enhancement and image-quality improvement. Here we introduce the task-assisted generative adversarial network (TA-GAN), which incorporates an auxiliary task (for example, segmentation, localization) closely related to the observed biological nanostructure characterization. We evaluate how the TA-GAN improves generative accuracy over unassisted methods, using images acquired with different modalities such as confocal, bright-field, stimulated emission depletion and structured illumination microscopy. The TA-GAN is incorporated directly into the acquisition pipeline of the microscope to predict the nanometric content of the field of view without requiring the acquisition of a super-resolved image. This information is used to automatically select the imaging modality and regions of interest, optimizing the acquisition sequence by reducing light exposure. Data-driven microscopy methods like the TA-GAN will enable the observation of dynamic molecular processes with spatial and temporal resolutions that surpass the limits currently imposed by the trade-offs constraining super-resolution microscopy.
36
Fiji: an open-source platform for biological-image analysis
Johannes Schindelin, Ignacio Arganda‐Carreras, Erwin Frise et al. · Nature Methods · 2012 · 70.9K citations
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
Jun-Yan Zhu, Taesung Park, Phillip Isola et al. · 2017 · 21.9K citations · Full text
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa et al. · arXiv (Cornell University) · 2019 · 16.2K citations · Full text
Berthold K. P. Horn, Brian G. Schunck · Artificial Intelligence · 1981 · 9.9K citations · Full text