Retinal Lesion Segmentation Using Transfer Learning with an Encoder-Decoder CNN

Authors

  • Rafael Ortiz-Feregrino Universidad Autónoma de Querétaro, México https://orcid.org/0000-0003-0892-9976
  • Saúl Tovar-Arriaga Universidad Autónoma de Querétaro, México
  • Carlos Pedraza Ortega Universidad Autónoma de Querétaro, México https://orcid.org/0000-0001-5125-8907
  • Andras Takacs Universidad Autónoma de Querétaro, México

DOI:

https://doi.org/10.17488/RMIB.43.2.4

Keywords:

Transfer learning, encoder-decoder, retinal images, lesion segmentation, deep learning

Abstract

Deep learning (DL) techniques achieve high performance in the detection of illnesses in retina images, but the majority of models are trained with different databases for solving one specific task. Consequently, there are currently no solutions that can be used for the detection/segmentation of a variety of illnesses in the retina in a single model. This research uses Transfer Learning (TL) to take advantage of previous knowledge generated during model training of illness detection to segment lesions with encoder-decoder Convolutional Neural Networks (CNN), where the encoders are classical models like VGG-16 and ResNet50 or variants with attention modules. This shows that it is possible to use a general methodology using a single fundus image database for the detection/segmentation of a variety of retinal diseases achieving state-of-the-art results. This model could be in practice more valuable since it can be trained with a more realistic database containing a broad spectrum of diseases to detect/segment illnesses without sacrificing performance. TL can help achieve fast convergence if the samples in the main task (Classification) and sub-tasks (Segmentation) are similar. If this requirement is not fulfilled, the parameters start from scratch.

Downloads

Download data is not yet available.

References

Trucco E, MacGillivray T, Xu Y (eds). Computational Retinal Image Analysis [Internet]. Cambridge, United States: Elsevier; 2019. 481p. Available from: https://doi.org/10.1016/B978-0-08-102816-2.09994-9

Prado-Serrano A, Guido-Jiménez MA, Camas-Benítez JT. Prevalencia de retinopatía diabética en población mexicana. Rev Mex Oftalmol [Internet]. 2009; 83(5):261–266. Available: https://www.medigraphic.com/pdfs/revmexoft/rmo-2009/rmo095c.pdf

Ricci E, Perfetti R. Retinal Blood Vessel Segmentation Using Line Operators and Support Vector Classification. IEEE Trans Med Imaging [Internet]. 2007;26(10):1357-1365. Available from: https://doi.org/10.1109/TMI.2007.898551

Sarhan MH, Nasseri MA, Zapp D, Maier M, et al. Machine Learning Techniques for Ophthalmic Data Processing: A Review. IEEE J Biomed Health Inform [Internet]. 2020;24(12):3338–3350. Available from: http://dx.doi.org/10.1109/JBHI.2020.3012134

Ammu R, Sinha N. Small Segment Emphasized Performance Evaluation Metric for Medical Images. 2020 International Conference on Signal Processing and Communications (SPCOM) [Internet]. Bangalore : IEEE;2020; 1–5. Available from: http://dx.doi.org/10.1109/SPCOM50965.2020.9179617

Quellec G, Charrière K, Boudi Y, Cochener B, et al. Deep image mining for diabetic retinopathy screening. Med Image Anal [Internet]. 2017;39:178–193. Available from: https://doi.org/10.1016/j.media.2017.04.012

Zhang X, Thibault G, Decencière E, Marcotegui B, et al. Exudate detection in color retinal images for mass screening of diabetic retinopathy. Med Image Anal [Internet]. 2014;18(7):1026–1043. Available from: https://doi.org/10.1016/j.media.2014.05.004

Feng Z, Yang J, Yao L, Qiao Y, et al. Deep Retinal Image Segmentation: A FCN-Based Architecture with Short and Long Skip Connections for Retinal Image Segmentation. In: Liu D, Xie S, Li Y, Zhao D, et al. (eds). Neural Information Processing ICONIP 2017. Lecture Notes in Computer Science, vol. 10637 [Internet]. Cham: Springer; 2017; 713-722. Available from: https://doi.org/10.1007/978-3-319-70093-9_76

Ye JC, Sung WK. Understanding Geometry of Encoder-Decoder CNNs. 36th International Conference on Machine Learning, ICML 2019 [Internet]. Long Beach: Proceedings of Machine Learning Research; 2019;97:12245–12254. Available from: https://proceedings.mlr.press/v97/ye19a.html

Tan C, Sun F, Kong T, Zhang W, et al. A Survey on Deep Transfer Learning. In: Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I (eds). Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2018. 11041: 270–279. Available from: http://dx.doi.org/10.1007/978-3-030-01424-7_27

Tang S, Qi Z, Granley J, Beyeler M. U-Net with Hierarchical Bottleneck Attention for Landmark Detection in Fundus Images of the Degenerated Retina. In: Fu H, Garvin MK, MacGillivray T, Xu Y, et al (eds). Ophthalmic Medical Image Analysis. OMIA 2021. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2021. 12970:62–71. Available from: https://doi.org/10.1007/978-3-030-87000-3_7

Welikala RA, Foster PJ, Whincup PH, Rudnicka AR, et al. Automated arteriole and venule classification using deep learning for retinal images from the UK Biobank cohort. Comput Biol Med [Internet]. 2017;90:23–32. Available from: https://doi.org/10.1016/j.compbiomed.2017.09.005

Fraz MM, Rudnicka AR, Owen CG, Barman SA. Delineation of blood vessels in pediatric retinal images using decision trees-based ensemble classification. Int J Comput Assist Radiol Surg [Internet]. 2014;9:795–811. Available from: https://doi.org/10.1007/s11548-013-0965-9

Adel A, Soliman MM, Khalifa NEM, Mostafa K. Automatic Classification of Retinal Eye Diseases from Optical Coherence Tomography using Transfer Learning. 2020 16th International Computer Engineering Conference (ICENCO) [Internet]. Cairo: IEEE; 2020: 37–42. Available from: https://doi.org/10.1109/ICENCO49778.2020.9357324

Xiao Z, Adel M, Bourennane S. Bayesian Method with Spatial Constraint for Retinal Vessel Segmentation. Comput Math Methods Med [Internet]. 2013:401413. Available from: https://doi.org/10.1155/2013/401413

Lam C, Yu C, Huang L, Rubin D. Retinal Lesion Detection with Deep Learning Using Image Patches. Invest Ophthalmol Vis Sci [Internet]. 2018;59(1):590–596. Available from: https://doi.org/10.1167/iovs.17-22721

Li Q, Feng B, Xie L, Liang P, et al. A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images. IEEE Trans Med Imaging [Internet]. 2016;35(1):109–118. Available from: https://doi.org/10.1109/TMI.2015.2457891

Long S, Chen J, Hu A, Liu H, et al. Microaneurysms detection in color fundus images using machine learning based on directional local contrast. Biomed Eng Online [Internet]. 2020;19(1):21. Available from: https://doi.org/10.1186/s12938-020-00766-3

Aziz T, Ilesanmi AE, Charoenlarpnopparut C. Efficient and Accurate Hemorrhages Detection in Retinal Fundus Images Using Smart Window Features. Appl Sci [Internet]. 2021;11(14):6391. Available from: https://doi.org/10.3390/app11146391

Liu Q, Liu H, Zhao Y, Liang Y. Dual-Branch Network with Dual-Sampling Modulated Dice Loss for Hard Exudate Segmentation in Colour Fundus Images. IEEE J Biomed Health Inform [Internet]. 2022;26(3):1091-1102. Available from: https://doi.org/10.1109/jbhi.2021.3108169

Zong Y, Chen J, Yang L, Tao S, et al. U-net Based Method for Automatic Hard Exudates Segmentation in Fundus Images Using Inception Module and Residual Connection. IEEE Access [Internet]. 2020;8:167225–35. Available from: http://dx.doi.org/10.1109/ACCESS.2020.3023273

Tan JH, Fujita H, Sivaprasad S, Bhandary SV, et al. Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci [Internet]. 2017;420:66–76. Available from: https://doi.org/10.1016/j.ins.2017.08.050

Kou C, Li W, Yu Z, Yuan L. An Enhanced Residual U-Net for Microaneurysms and Exudates Segmentation in Fundus Images. IEEE Access [Internet]. 2020;8:185514–185525. Available from: http://dx.doi.org/10.1109/ACCESS.2020.3029117

Gondal WM, Köhler JM, Grzeszick R, Fink GA, et al. Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. 2017 IEEE International Conference on Image Processing (ICIP) [Internet]. Beijing: IEEE; 2017; 2069-2073. Available from: https://doi.org/10.1109/ICIP.2017.8296646

Joshi GD, Sivaswamy J, Krishnadas SR. Optic Disk and Cup Segmentation From Monocular Color Retinal Images for Glaucoma Assessment. IEEE Trans Med Imaging [Internet]. 2011;30(6):1192–1205. Available from: https://doi.org/10.1109/TMI.2011.2106509

Harangi B, Antal B, Hajdu A. Automatic exudate detection with improved naïve-Bayes classifier. 2012 25th IEEE International Symposium on Computer-Based Medical Systems (CBMS) [Internet]. Rome: IEEE; 2012; 1-4. Available from: https://doi.org/10.1109/CBMS.2012.6266341

Cheung CY, Xu D, Cheng CY, Sabanayagam C, et al. A deep-learning system for the assessment of cardiovascular disease risk via the measurement of retinal-vessel caliber. Nat Biomed Eng [Internet]. 2021;5(6):498–508. Available from: https://doi.org/10.1038/s41551-020-00626-4

Dai L, Wu L, Li H, Cai C, et al. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nat Commun [Internet]. 2021;12(1):3242. Available from: https://doi.org/10.1038/s41467-021-23458-5

Kou C, Li W, Liang W, Yu Z, et al. Microaneurysms segmentation with a U-Net based on recurrent residual convolutional neural network. J Med Imaging [Internet]. 2019;6(2):025008. Available from: https://dx.doi.org/10.1117%2F1.JMI.6.2.025008

Xu X, Tan T, Xu F. An Improved U-Net Architecture for Simultaneous Arteriole and Venule Segmentation in Fundus Image. In: Nixon M, Mahmoodi S, Zwiggelaar R (eds). Medical Image Understanding and Analysis. MIUA 2018. Communications in Computer and Information Science [Internet]. Cham: Springer; 2018; 333–340. Available from: https://doi.org/10.1007/978-3-319-95921-4_31

Yadav G, Maheshwari S, Agarwal A. Contrast limited adaptive histogram equalization based enhancement for real time video system. 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) [Internet]. Delhi: IEEE; 2014; 2392-2397. Available from: https://doi.org/10.1109/ICACCI.2014.6968381

Decencière E, Zhang X, Cazuguel G, Lay B, et al. Feedback on a publicly distributed image database: The Messidor database. Image Anal Stereol [Internet]. 2014;33(3):231. Available from: https://doi.org/10.5566/ias.1155

Kaggle. Diabetic Retinopathy Detection. [Internet] Kaggle. 2015. Available from: https://www.kaggle.com/c/diabetic-retinopathy-detection

Porwal P, Pachade S, Kamble R, Kokare M, et al. Indian Diabetic Retinopathy Image Dataset (IDRiD) [Internet]. IEEE Dataport; 2018. Available from: https://dx.doi.org/10.21227/H25W98

Decencière E, Cazuguel G, Zhang X, Thibault G, et al. TeleOphta: Machine learning and image processing methods for teleophthalmology. IRBM [Internet]. 2013;34(2):196–203. Available from: https://doi.org/10.1016/j.irbm.2013.01.010

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun Y (eds). 3rd International Conference on Learning Representations, ICLR 2015 [Internet]. San Diego: arXiv;2015; 1–14. Available from: https://doi.org/10.48550/arXiv.1409.1556

He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [Internet]. Las Vegas: IEEE; 2016; 770-778. Available from: https://doi.org/10.1109/CVPR.2016.90

Woo S, Park J, Lee JY, Kweon IS. CBAM: Convolutional Block Attention Module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds). Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2018; 3–19. Available from: https://doi.org/10.1007/978-3-030-01234-2_1

Cortes C, Research G, Mohri M, Rostamizadeh A. L2 Regularization for Learning Kernels. 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009) [Internet]. Montreal: Association for Uncertainty in Artificial Intelligence (AUAI); 2009; 109-116. Available from: https://dl.acm.org/doi/pdf/10.5555/1795114.1795128

Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (ICML'15) [Internet]. Lille: JMLR; 2015; 448-456. Available from: http://proceedings.mlr.press/v37/ioffe15.pdf

Srivastava N, Hinton G, Krizhevsky A, Sutskever I, et al. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res [Internet]. 2014; 15(56):1929–1958. Available from: http://jmlr.org/papers/v15/srivastava14a.html

Wisaeng K, Sa-Ngiamvibool W. Exudates Detection Using Morphology Mean Shift Algorithm in Retinal Images. IEEE Access [Internet]. 2019;7:11946–11958. Available from: http://dx.doi.org/10.1109/ACCESS.2018.2890426

van Grinsven MJJP, van Ginneken B, Hoyng CB, Theelen T, et al. Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images. IEEE Trans Med Imaging [Internet]. 2016;35(5):1273–1284Available from: https://doi.org/10.1109/TMI.2016.2526689

Downloads

Published

2022-07-28

How to Cite

Ortiz-Feregrino, R., Tovar-Arriaga, S., Pedraza Ortega, C., & Takacs, A. (2022). Retinal Lesion Segmentation Using Transfer Learning with an Encoder-Decoder CNN . Revista Mexicana De Ingenieria Biomedica, 43(2), 53–63. https://doi.org/10.17488/RMIB.43.2.4

Issue

Section

Research Articles

Share on:

Dimensions Citation

Most read articles by the same author(s)