Enhancing Super-Resolution Microscopy with AI
Deep learning and reinforcement learning can increase the accessibility and applicability of high-end microscopy techniques to neuroscience.
Background
Over the past two decades, super-resolution microscopy (SRM) has had a transformative impact on the field of cell biology, expanding our ability to characterize subcellular structures and molecular dynamics at the nanoscale.
These high-end microscopy techniques enable scientists to monitor and modulate molecular processes in living cells and tissues with unprecedented spatio-temporal resolution and minimal invasiveness.
They hold the promise to revolutionize our understanding of brain function, enabling simultaneous measurements of molecular dynamics and localized activity within a complex brain network.
Objectives
Exploiting the full potential of SRM in brain tissue and living cell imaging environments poses significant challenges, ranging from the complexities of thick tissue imaging to the unification of measurements occurring at different scales, all requiring innovative solutions for practical implementation.
This project aims to further develop AI-assisted SRM microscopy approaches to improve its performance in living cells and tissues, diversify the applications of SRM microscopy in neuroscience, and push the limits of spatio-temporal resolution for the observation of dynamic processes at the nanoscale.
About the Project
We are pursuing two main research ideas: how to extend the achievable multimodal performance of optical nanoscopy with deep learning and how to develop microscopy control strategies based on reinforcement learning to optimize acquisition processes in living neurons and brain tissue.
These aims will be achieved through the following research objectives:
Improve multimodal image quality, taking into account spatial, temporal and spectral aspects;
Propose approaches for assessing the quality, uncertainty and relevance of subregions of the field of view;
Design decision-making procedures to optimize acquisition, based on limited human feedback.
AI-assisted SRM will recognize structures of interest and cellular responses during the imaging process, and modify imaging parameters, treatment and modalities according to observed structural and functional changes.
Microscopes will thus be able to adapt in real time to experimental conditions and the sample under study.
AI offers great potential for accelerating scientific research. By enhancing microscopy techniques using deep neural networks and reinforcement learning, we aim to help scientists conduct their research more efficiently, which could lead to novel breakthroughs in neuroscience.
Audrey Durand, Assistant Professor, Université Laval, Associate Academic Member, Mila
Publications
Resources
TA-GAN for resolution enhancement (from Bouchard et al. 2023)
Repository for "Task-Assisted GAN for Resolution Enhancement and Modality Translation in Fluorescence Microscopy"
Learn more
https://github.com/FLClab/TA-GAN
pySTED simulator (from Turcotte et al. 2022)
STED simulation platform written in Python.
Learn more
https://github.com/FLClab/pySTED
MICRA-Net (from Bilodeau et al. 2022)
Source code of the publication MICRA-Net: MICRoscopy Analysis Neural Network to solve detection, classification, and segmentation from a single simple auxiliary task.
Learn more
https://github.com/FLClab/MICRA-Net
OpenAI gym implementation of the pySTED simulator (used in Bilodeau et al. 2022)
Learn more
https://github.com/FLClab/gym-sted
STEDActinFCN (to reproduce results from Lavoie-Cardinal et al. 2020)
This folder contains the necessary packages to run a STED actin experiment.
Learn more
https://github.com/FLClab/STEDActinFCN
Meet the Team
Mila Members
Other Members
Flavie Lavoie-Cardinal (Associate Professor, Université Laval)