open access

https://doi.org/10.14943/doctoral.k15545

https://hdl.handle.net/2115/91282

非線形ラマン硬性内視鏡による無標識神経イメージング高速化への深層学習の応用 : 蛍光画像を用いた転移学習法と最大撮像速度の評価法の構築

英語

2023-03-23

Hokkaido University

vi, 128p.

An imaging system for nerve visualization/identification during surgery is expected to improve the quality of life after surgery. This is because postsurgical pain and　body dysfunction are induced by nerve damage during surgery. Although nerve-sparing approaches based on anatomical knowledge and surgeon’s experiments have　been reported, it is unclear how well the peripheral nerves are preserved. The visibility of peripheral nerves is severely poor because peripheral nerves are transparent, colorless, and thin. Therefore, an intraoperative nerve dentification tool is required to improve the nerve-sparing rate and evaluate the efficiency of nerve-sparing approaches. In this thesis, an imaging acceleration system for a coherent anti-Stokes Raman scattering (CARS) rigid endoscope is developed for nerve-sparing surgery. The nerves can be visualized in a label-free manner using CARS, which is sensitive to species of molecular vibration in samples. The imaging rate acceleration of the CARS rigid endoscope developed in this laboratory is needed for clinical application. In this dissertation, the author shows the results of introducing image processing by deep learning to improve the nerve imaging rate of the CARS endoscope. The author developed the critical imaging rate (CIR) as a quantitative metric of imaging speed accelerated by deep learning. CIR is defined as the highest imaging rate to satisfy the image quality needed for medical images and is calculated from a harmonic mean of CIRPSNR and CIRSSIM which are the imaging rates satisfying criteria of PSNR=30 and SSIM=0.8, respectively. It is demonstrated that noise reduction of CARS endoscopic images made the imaging rate about five times faster using CIR (from 1.4 images/min to 7.0 images/min). Because noise reduction enhances the quality of the image, it is possible to improve the imaging rate of CARS rigid endoscopy by enhancing the quality of noisy nerve images captured at high rates. In general, a large dataset is needed for training deep-learning models. However, it is time-consuming and laborious to prepare a large dataset. The author proposed transfer learning using CARS microscopic images before training models on CARS endoscopic images. CARS microscopy made it easier to search peripheral nerves because the CARS images with a high signal-to-noise ratio can be obtained at a high imaging rate using an objective lens with a higher numerical aperture. The author compared three denoising models (WIN5R, DenoiseNet, and Noise2Noise) to determine the optimal model for denoising of CARS nerve images and to conduct pretraining before fine-tuning with CARS endoscopic images. Noise2Noise showed the highest denoising performance between the three models quantitatively and qualitatively. After training on CARS microscopic images, the model was fine-tuned using a few CARS endoscopic images. The author proposed a learning method to utilize images obtained with another modality as pre-training. Fluorescence microscopy was used as another modality to acquire a large dataset for nerve segmentation. Nerve segmentation is needed for displaying nerve areas to surgeons. The author prepared fluorescence images labeling lipids for pre-training nerve segmentation because CARS images have lipid information using CH2 vibration. U-Net, which is famous for semantic segmentation, was used. U-Net consists of an encoder part extracting feature maps from input images and a decoder part reconstructing output images from the feature maps. A VGG16 encoder, which learns to extract feature maps for image classification, was used in the encoder part of U-Net. It is found that both pre-training with fluorescence images and using the VGG16 encoder enhanced the segmentation performance significantly. In particular, pre-training with fluorescence images boosted the performance of the nerve segmentation from CARS images with a low signal-to-noise ratio. Therefore, the CIRsegmentation was improved about 47.5 times using semantic segmentation with deep learning than CIRdenoising. Semantic egmentation of nerves in CARS endoscopic images improved the imaging rate by about 47.5 times factor compared with denoising. These results open the opportunity for incorporating CARS rigid endoscopes into medical settings.

2023-03-23

甲第15545号

博士(情報科学)

(主査) 教授 橋本 守, 教授 平田 拓, 教授 雲林院 宏, 准教授 工藤 信樹

情報科学院(情報科学専攻)

博士論文

Version of Record