Wadduwage Lab for Differentiable Microscopy (𝜕𝜇)

In Wadduwage Lab, we work on novel computational microscopy solutions that can measure biological systems at their most information rich form, with minimum redundancy.

𝜕𝜇 – Differentiable Microscopy - With applications ranging from, rare cellular event detection to drug screening, high content imaging provides biomedically important morphological features of cells or tissues via high speed acquisition hardware and fast image processing algorithms. Despite significant advances in faster and more multiplexed imaging sensors, the imaging throughput is currently limited by the speed of electronics hardware. In Wadduwage Lab we use an orthogonal approach, termed differential microscopy (𝜕𝜇), to improve the imaging throughput beyond existing electronic hardware bottleneck. The rationale for 𝜕𝜇 is that low-dimensional compressed representations of image signals exists and can be found through learning-based techniques; instruments can thus be designed to perform measurements on the lower dimensional compressed representation, improving the throughput by the factor of compression. To achieve this, 𝜕𝜇 models the front-end optics, the back-end image -reconstruction and -processing algorithms together as a differentiable model of learnable parameters.

selected publications

Chip-Based Resonance Raman Spectroscopy Using Tantalum Pentoxide Waveguides

David A. Coucheron, Dushan N. Wadduwage, G. Senthil Murugan, and 2 more authors

IEEE Photonics Technology Letters 2019

Abs

Blood analysis is an important diagnostic tool, as it provides a wealth of information about the patientâ€™s health. Raman spectroscopy is a promising tool for blood analysis, but widespread clinical application is limited by its low signal strength, as well as complex and costly instrumentation. The growing field of waveguide-based Raman spectroscopy tries to solve these challenges by working toward fully integrated Raman sensors with increased interaction areas. In this letter, we demonstrate resonance Raman measurements of hemoglobin, a crucial component of blood, at 532-nm excitation using a tantalum pentoxide (Ta2O5) waveguide platform. We have also characterized the background signal from Ta2O5 waveguide material when excited at 532 nm. In addition, we demonstrate spontaneous Raman measurements of isopropanol and methanol using the same platform. Our results suggest that Ta2O5 is a promising waveguide platform for resonance Raman spectroscopy at 532 nm and, in particular, for blood analysis.
De-scattering with Excitation Patterning enables rapid wide-field imaging through scattering media

Cheng Zheng, Jong Kang Park, Murat Yildirim, and 5 more authors

Science Advances 2021

Abs

Nonlinear optical microscopy has enabled in vivo deep tissue imaging on the millimeter scale. A key unmet challenge is its limited throughput especially compared to rapid wide-field modalities that are used ubiquitously in thin specimens. Wide-field imaging methods in tissue specimens have found successes in optically cleared tissues and at shallower depths, but the scattering of emission photons in thick turbid samples severely degrades image quality at the camera. To address this challenge, we introduce a novel technique called De-scattering with Excitation Patterning or "DEEP," which uses patterned nonlinear excitation followed by computational imaging-assisted wide-field detection. Multiphoton temporal focusing allows high-resolution excitation patterns to be projected deep inside specimen at multiple scattering lengths due to the use of long wavelength light. Computational reconstruction allows high-resolution structural features to be reconstructed from tens to hundreds of DEEP images instead of millions of point-scanning measurements.
Differentiable microscopy (∂μ) for high-throughput imaging cytometry

Dushan N. Wadduwage, and Udith Haputhanthri

2021

Abs

Imaging cytometry is a vital tool in many bio-imaging studies. Here we introduce, Differentiable Microscopy () a general end-to-end optimizable design framework to improve the throughput of imaging cytometry using content aware sampling and reconstruction.
Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

Kithmini Herath, Udith Haputhanthri, Ramith Hettiarachchi, and 5 more authors

arXiv 2022

Abs arXiv Website

Ever since the first microscope by Zacharias Janssen in the late 16th century, scientists have been inventing new types of microscopes for various tasks. Inventing a novel architecture demands years, if not decades, worth of scientific experience and creativity. In this work, we introduce Differentiable Microscopy (∂μ), a deep learning-based design paradigm, to aid scientists design new interpretable microscope architectures. Differentiable microscopy first models a common physics-based optical system however with trainable optical elements at key locations on the optical path. Using pre-acquired data, we then train the model end-to-end for a task of interest. The learnt design proposal can then be simplified by interpreting the learnt optical elements. As a first demonstration, based on the optical 4-f system, we present an all-optical quantitative phase microscope (QPM) design that requires no computational post-reconstruction. A follow-up literature survey suggested that the learnt architecture is similar to the generalized phase concept developed two decades ago. We then incorporate the generalized phase contrast concept to simplify the learning procedure. Furthermore, this physical optical setup is miniaturized using a diffractive deep neural network (D2NN). We outperform the existing benchmark for all-optical phase-to-intensity conversion on multiple datasets, and ours is the first demonstration of its kind on D2NNs. The proposed differentiable microscopy framework supplements the creative process of designing new optical systems and would perhaps lead to unconventional but better optical designs.
From Hours to Seconds: Towards 100x Faster Quantitative Phase Imaging via Differentiable Microscopy

Udith Haputhanthri, Kithmini Herath, Ramith Hettiarachchi, and 5 more authors

arXiv 2022

Abs

With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the speed of electronic hardware. Complementarily, to improve throughput further, here we propose to acquire images in a compressed form such that more information can be transferred beyond the existing electronic hardware bottleneck. To this end, we present a learnable optical compression-decompression framework that learns content-specific features. The proposed differentiable optical-electronic quantitative phase microscopy (∂μ) first uses learnable optical feature extractors as image compressors. The intensity representation produced by these networks is then captured by the imaging sensor. Finally, a reconstruction network running on electronic hardware decompresses the QPM images. The proposed system achieves compression of \times 64 while maintaining the SSIM of ∼0.90 and PSNR of ∼30 dB. The promising results demonstrated by our experiments open up a new pathway for achieving end-to-end optimized (i.e., optics and electronic) compact QPM systems that provide unprecedented throughput improvements.