Sigger, Neetu, Nguyen, Tuan T. ORCID: 0000-0003-0055-8218 and Tozzi, Gianluca (2024) Brain tissue classification in hyperspectral images using multistage diffusion features and transformer. Journal of Microscopy. ISSN 0022-2720 (Print), 1365-2818 (Online) (doi:https://doi.org/10.1111/jmi.13372)

Preview

PDF (Open Access Article) 48675 NGUYEN_Brain_Tissue_Classification_In_Hyperspectral_Images_Using_Multistage_Diffusion_Features_And_Transformer_(OA)_2024.pdf - Published Version Available under License Creative Commons Attribution. Download (2MB) | Preview

Abstract

Brain surgery is a widely practised and effective treatment for brain tumours, but accurately identifying and classifying tumour boundaries is crucial to maximise resection and avoid neurological complications. This precision in classification is essential for guiding surgical decisions and subsequent treatment planning. Hyperspectral (HS) imaging (HSI) is an emerging multidimensional optical imaging method that captures detailed spectral information across multiple wavelengths, allowing for the identification of nuanced differences in tissue composition, with the potential to enhance intraoperative tissue classification. However, current frameworks often require retraining models for each HSI to extract meaningful features, resulting in long processing times and high computational costs. Additionally, most methods utilise the deep semantic features at the end of the network for classification, ignoring the spatial details contained in the shallow features. To overcome these challenges, we propose a novel approach called MedDiffHSI, which combines diffusion and transformer techniques. Our method involves training an unsupervised learning framework based on the diffusion model to extract high-level and low-level spectral–spatial features from HSI. This approach eliminates the need for retraining of spectral–spatial feature learning model, thereby reducing time complexity. We then extract intermediate multistage features from different timestamps for classification using a pretrained denoising U-Net. To fully explore and exploit the rich contextual semantics and textual information hidden in the extracted diffusion feature, we utilise a spectral–spatial attention module. This module not only learns multistage information about features at different depths, but also extracts and enhances effective information from them. Finally, we employ a supervised transformer-based classifier with weighted majority voting (WMV) to perform the HSI classification. To validate our approach, we conduct comprehensive experiments on in vivo brain database data sets and also extend the analysis to include additional HSI data sets for breast cancer to evaluate the framework performance across different types of tissue. The results demonstrate that our framework outperforms existing approaches by using minimal training samples (5%) while achieving state-of-the-art performance.

Item Type:	Article
Uncontrolled Keywords:	bioinformatics, brain tumour, cancer surgery, deep learning, diffusion model, hyperspectral imaging, precision medicine
Subjects:	Q Science > Q Science (General) Q Science > QM Human anatomy T Technology > T Technology (General)
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS)
Last Modified:	22 Nov 2024 15:27
URI:	http://gala.gre.ac.uk/id/eprint/48675

Actions (login required)

View Item

Downloads