Automatic cancer nuclei segmentation on histological images: comparison study of deep learning methods


Gabdullin M.T. Mukasheva A. Koishiyeva D. Umarov T. Bissembayev A. Kim K.-S. Kang J.W.
December 2024Korean Society for Biotechnology and Bioengineering

Biotechnology and Bioprocess Engineering
2024#29Issue 61034 - 1047 pp.

Cancer is one of the most common health problems affecting individuals worldwide. In the field of biomedical engineering, one of the main methods for cancer diagnosis is the analysis of histological images of tissue structures and cell nuclei using artificial intelligence. Here, we compared the performance of 15 deep learning methods viz: UNet, Deep-UNet, UNet-CBAM, RA-UNet, SA-Unet and Nuclei-SegNet, UNet-VGG2016, UNet-Resnet-101, TransResUNet, Inception-UNet, Att-UNet++, FF-UNet, Att-UNet, Res-UNet and a new model, DanNucNet, in pathological nuclei segmentation on tissue slices from different organs on five open datasets: MoNuSeg, CoNSeP, CryoNuSeg, Data Science Bowl, and NuInsSeg. Before training on the data, the pixel intensity and color distribution were analyzed, and different augmentation techniques were applied. The results showed that the UNet-based model with 34.57 million Deep-UNet parameters performed the best, outperforming all models in terms of the Dice coefficient from 3.13 to 22.91%. The implementation of Deep-UNet in this context provides a valuable tool for accurate extraction of cancer cell nuclei from histological images, which in turn will contribute to further developments in cancer pathology and digital histology.

Augmentation , Cancer , Convolutional neural networks , Histology , Medical segmentation

Text of the article Перейти на текст статьи

School of Materials Science and Green Technologies, Kazakh-British Technical University, Almaty, 050000, Kazakhstan
School of Information Technology and Engineering, Kazakh-British Technical University, Almaty, 050000, Kazakhstan
Department of Chemical and Biological Engineering, Korea National University of Transportation, Chungju, 27469, South Korea
Department of Transportation System Engineering, Korea National University of Transportation, Uiwang, 16106, South Korea

School of Materials Science and Green Technologies
School of Information Technology and Engineering
Department of Chemical and Biological Engineering
Department of Transportation System Engineering

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026