Logo image
Back
Novel Multimodal, Multiscale Imaging System with Augmented Reality
Journal article   Open access   Peer reviewed

Novel Multimodal, Multiscale Imaging System with Augmented Reality

Christopher Mela, Francis Papay and Yang Liu
Diagnostics (Basel), Vol.11(3), p.441
03/01/2021
DOI: 10.3390/diagnostics11030441
PMID: 33806547
url
https://doi.org/10.3390/diagnostics11030441View
Published (Version of record) Open Access

Abstract

A novel multimodal, multiscale imaging system with augmented reality capability were developed and characterized. The system offers 3D color reflectance imaging, 3D fluorescence imaging, and augmented reality in real time. Multiscale fluorescence imaging was enabled by developing and integrating an in vivo fiber-optic microscope. Real-time ultrasound-fluorescence multimodal imaging used optically tracked fiducial markers for registration. Tomographical data are also incorporated using optically tracked fiducial markers for registration. Furthermore, we characterized system performance and registration accuracy in a benchtop setting. The multiscale fluorescence imaging facilitated assessing the functional status of tissues, extending the minimal resolution of fluorescence imaging to ~17.5 µm. The system achieved a mean of Target Registration error of less than 2 mm for registering fluorescence images to ultrasound images and MRI-based 3D model, which is within clinically acceptable range. The low latency and high frame rate of the prototype system has shown the promise of applying the reported techniques in clinically relevant settings in the future.
 augmented reality computer vision fluorescence imaging multimodal imaging multiscale imaging

Details

Metrics

13 Record Views
4 Times Cited - Web of Science
Logo image