thesis medical imaging

Head Start Your Radiology Residency [Online] ↗️

• Radiology Thesis – More than 400 Research Topics (2022)!

Username or Email Address

Remember Me

Introduction

A thesis or dissertation, as some people would like to call it, is an integral part of the Radiology curriculum, be it MD, DNB, or DMRD. We have tried to aggregate radiology thesis topics from various sources for reference.

Not everyone is interested in research, and writing a Radiology thesis can be daunting. But there is no escape from preparing, so it is better that you accept this bitter truth and start working on it instead of cribbing about it (like other things in life. #PhilosophyGyan!)

Start working on your thesis as early as possible and finish your thesis well before your exams, so you do not have that stress at the back of your mind. Also, your thesis may need multiple revisions, so be prepared and allocate time accordingly.

Tips for Choosing Radiology Thesis and Research Topics

Keep it simple silly (kiss).

Retrospective > Prospective

Retrospective studies are better than prospective ones, as you already have the data you need when choosing to do a retrospective study. Prospective studies are better quality, but as a resident, you may not have time (, energy and enthusiasm) to complete these.

Choose a simple topic that answers a single/few questions

Original research is challenging, especially if you do not have prior experience. I would suggest you choose a topic that answers a single or few questions. Most topics that I have listed are along those lines. Alternatively, you can choose a broad topic such as “Role of MRI in evaluation of perianal fistulas.”

You can choose a novel topic if you are genuinely interested in research AND have a good mentor who will guide you. Once you have done that, make sure that you publish your study once you are done with it.

Get it done ASAP.

In most cases, it makes sense to stick to a thesis topic that will not take much time. That does not mean you should ignore your thesis and ‘Ctrl C + Ctrl V’ from a friend from another university. Thesis writing is your first step toward research methodology so do it as sincerely as possible. Do not procrastinate in preparing the thesis. As soon as you have been allotted a guide, start researching topics and writing a review of the literature.

At the same time, do not invest a lot of time in writing/collecting data for your thesis. You should not be busy finishing your thesis a few months before the exam. Some people could not appear for the exam because they could not submit their thesis in time. So DO NOT TAKE thesis lightly.

Do NOT Copy-Paste

Reiterating once again, do not simply choose someone else’s thesis topic. Find out what are kind of cases that your Hospital caters to. It is better to do a good thesis on a common topic than a crappy one on a rare one.

Books to help you write a Radiology Thesis

Event country/university has a different format for thesis; hence these book recommendations may not work for everyone.

• Amazon Kindle Edition
• Gupta, Piyush (Author)
• English (Publication Language)
• 206 Pages - 10/12/2020 (Publication Date) - Jaypee Brothers Medical Publishers (P) Ltd. (Publisher)

In A Hurry? Download a PDF list of Radiology Research Topics!

Sign up below to get this PDF directly to your email address.

100% Privacy Guaranteed. Your information will not be shared. Unsubscribe anytime with a single click.

List of Radiology Research /Thesis / Dissertation Topics

• State of the art of MRI in the diagnosis of hepatic focal lesions
• Multimodality imaging evaluation of sacroiliitis in newly diagnosed patients of spondyloarthropathy
• Multidetector computed tomography in oesophageal varices
• Role of positron emission tomography with computed tomography in the diagnosis of cancer Thyroid
• Evaluation of focal breast lesions using ultrasound elastography
• Role of MRI diffusion tensor imaging in the assessment of traumatic spinal cord injuries
• Sonographic imaging in male infertility
• Comparison of color Doppler and digital subtraction angiography in occlusive arterial disease in patients with lower limb ischemia
• The role of CT urography in Haematuria
• Role of functional magnetic resonance imaging in making brain tumor surgery safer
• Prediction of pre-eclampsia and fetal growth restriction by uterine artery Doppler
• Role of grayscale and color Doppler ultrasonography in the evaluation of neonatal cholestasis
• Validity of MRI in the diagnosis of congenital anorectal anomalies
• Role of sonography in assessment of clubfoot
• Role of diffusion MRI in preoperative evaluation of brain neoplasms
• Imaging of upper airways for pre-anaesthetic evaluation purposes and for laryngeal afflictions.
• A study of multivessel (arterial and venous) Doppler velocimetry in intrauterine growth restriction
• Multiparametric 3tesla MRI of suspected prostatic malignancy.
• Role of Sonography in Characterization of Thyroid Nodules for differentiating benign from
• Role of advances magnetic resonance imaging sequences in multiple sclerosis
• Role of multidetector computed tomography in evaluation of jaw lesions
• Role of Ultrasound and MR Imaging in the Evaluation of Musculotendinous Pathologies of Shoulder Joint
• Role of perfusion computed tomography in the evaluation of cerebral blood flow, blood volume and vascular permeability of cerebral neoplasms
• MRI flow quantification in the assessment of the commonest csf flow abnormalities
• Role of diffusion-weighted MRI in evaluation of prostate lesions and its histopathological correlation
• CT enterography in evaluation of small bowel disorders
• Comparison of perfusion magnetic resonance imaging (PMRI), magnetic resonance spectroscopy (MRS) in and positron emission tomography-computed tomography (PET/CT) in post radiotherapy treated gliomas to detect recurrence
• Role of multidetector computed tomography in evaluation of paediatric retroperitoneal masses
• Role of Multidetector computed tomography in neck lesions
• Estimation of standard liver volume in Indian population
• Role of MRI in evaluation of spinal trauma
• Role of modified sonohysterography in female factor infertility: a pilot study.
• The role of pet-CT in the evaluation of hepatic tumors
• Role of 3D magnetic resonance imaging tractography in assessment of white matter tracts compromise in supratentorial tumors
• Role of dual phase multidetector computed tomography in gallbladder lesions
• Role of multidetector computed tomography in assessing anatomical variants of nasal cavity and paranasal sinuses in patients of chronic rhinosinusitis.
• magnetic resonance spectroscopy in multiple sclerosis
• Evaluation of thyroid nodules by ultrasound elastography using acoustic radiation force impulse (ARFI) imaging
• Role of Magnetic Resonance Imaging in Intractable Epilepsy
• Evaluation of suspected and known coronary artery disease by 128 slice multidetector CT.
• Role of regional diffusion tensor imaging in the evaluation of intracranial gliomas and its histopathological correlation
• Role of chest sonography in diagnosing pneumothorax
• Role of CT virtual cystoscopy in diagnosis of urinary bladder neoplasia
• Role of MRI in assessment of valvular heart diseases
• High resolution computed tomography of temporal bone in unsafe chronic suppurative otitis media
• Multidetector CT urography in the evaluation of hematuria
• Contrast-induced nephropathy in diagnostic imaging investigations with intravenous iodinated contrast media
• Comparison of dynamic susceptibility contrast-enhanced perfusion magnetic resonance imaging and single photon emission computed tomography in patients with little’s disease
• Role of Multidetector Computed Tomography in Bowel Lesions.
• Role of diagnostic imaging modalities in evaluation of post liver transplantation recipient complications.
• Role of multislice CT scan and barium swallow in the estimation of oesophageal tumour length
• Malignant Lesions-A Prospective Study.
• Value of ultrasonography in assessment of acute abdominal diseases in pediatric age group
• Role of three dimensional multidetector CT hysterosalpingography in female factor infertility
• Comparative evaluation of multi-detector computed tomography (MDCT) virtual tracheo-bronchoscopy and fiberoptic tracheo-bronchoscopy in airway diseases
• Role of Multidetector CT in the evaluation of small bowel obstruction
• Sonographic evaluation in adhesive capsulitis of shoulder
• Utility of MR Urography Versus Conventional Techniques in Obstructive Uropathy
• MRI of the postoperative knee
• Role of 64 slice-multi detector computed tomography in diagnosis of bowel and mesenteric injury in blunt abdominal trauma.
• Sonoelastography and triphasic computed tomography in the evaluation of focal liver lesions
• Evaluation of Role of Transperineal Ultrasound and Magnetic Resonance Imaging in Urinary Stress incontinence in Women
• Multidetector computed tomographic features of abdominal hernias
• Evaluation of lesions of major salivary glands using ultrasound elastography
• Transvaginal ultrasound and magnetic resonance imaging in female urinary incontinence
• MDCT colonography and double-contrast barium enema in evaluation of colonic lesions
• Role of MRI in diagnosis and staging of urinary bladder carcinoma
• Spectrum of imaging findings in children with febrile neutropenia.
• Spectrum of radiographic appearances in children with chest tuberculosis.
• Role of computerized tomography in evaluation of mediastinal masses in pediatric
• Diagnosing renal artery stenosis: Comparison of multimodality imaging in diabetic patients
• Role of multidetector CT virtual hysteroscopy in the detection of the uterine & tubal causes of female infertility
• Role of multislice computed tomography in evaluation of crohn’s disease
• CT quantification of parenchymal and airway parameters on 64 slice MDCT in patients of chronic obstructive pulmonary disease
• Comparative evaluation of MDCT  and 3t MRI in radiographically detected jaw lesions.
• Evaluation of diagnostic accuracy of ultrasonography, colour Doppler sonography and low dose computed tomography in acute appendicitis
• Ultrasonography , magnetic resonance cholangio-pancreatography (MRCP) in assessment of pediatric biliary lesions
• Multidetector computed tomography in hepatobiliary lesions.
• Evaluation of peripheral nerve lesions with high resolution ultrasonography and colour Doppler
• Multidetector computed tomography in pancreatic lesions
• Multidetector Computed Tomography in Paediatric abdominal masses.
• Evaluation of focal liver lesions by colour Doppler and MDCT perfusion imaging
• Sonographic evaluation of clubfoot correction during Ponseti treatment
• Role of multidetector CT in characterization of renal masses
• Study to assess the role of Doppler ultrasound in evaluation of arteriovenous (av) hemodialysis fistula and the complications of hemodialysis vasular access
• Comparative study of multiphasic contrast-enhanced CT and contrast-enhanced MRI in the evaluation of hepatic mass lesions
• Sonographic spectrum of rheumatoid arthritis
• Diagnosis & staging of liver fibrosis by ultrasound elastography in patients with chronic liver diseases
• Role of multidetector computed tomography in assessment of jaw lesions.
• Role of high-resolution ultrasonography in the differentiation of benign and malignant thyroid lesions
• Radiological evaluation of aortic aneurysms in patients selected for endovascular repair
• Role of conventional MRI, and diffusion tensor imaging tractography in evaluation of congenital brain malformations
• To evaluate the status of coronary arteries in patients with non-valvular atrial fibrillation using 256 multirow detector CT scan
• A comparative study of ultrasonography and CT – arthrography in diagnosis of chronic ligamentous and meniscal injuries of knee
• Multi detector computed tomography evaluation in chronic obstructive pulmonary disease and correlation with severity of disease
• Diffusion weighted and dynamic contrast enhanced magnetic resonance imaging in chemoradiotherapeutic response evaluation in cervical cancer.
• High resolution sonography in the evaluation of non-traumatic painful wrist
• The role of trans-vaginal ultrasound versus magnetic resonance imaging in diagnosis & evaluation of cancer cervix
• Role of multidetector row computed tomography in assessment of maxillofacial trauma
• Imaging of vascular complication after liver transplantation.
• Role of magnetic resonance perfusion weighted imaging & spectroscopy for grading of glioma by correlating perfusion parameter of the lesion with the final histopathological grade
• Magnetic resonance evaluation of abdominal tuberculosis.
• Diagnostic usefulness of low dose spiral HRCT in diffuse lung diseases
• Role of dynamic contrast enhanced and diffusion weighted magnetic resonance imaging in evaluation of endometrial lesions
• Contrast enhanced digital mammography anddigital breast tomosynthesis in early diagnosis of breast lesion
• Evaluation of Portal Hypertension with Colour Doppler flow imaging and magnetic resonance imaging
• Evaluation of musculoskeletal lesions by magnetic resonance imaging
• Role of diffusion magnetic resonance imaging in assessment of neoplastic and inflammatory brain lesions
• Radiological spectrum of chest diseases in HIV infected children High resolution ultrasonography in neck masses in children
• with surgical findings
• Sonographic evaluation of peripheral nerves in type 2 diabetes mellitus.
• Role of perfusion computed tomography in the evaluation of neck masses and correlation
• Role of ultrasonography in the diagnosis of knee joint lesions
• Role of ultrasonography in evaluation of various causes of pelvic pain in first trimester of pregnancy.
• Role of Magnetic Resonance Angiography in the Evaluation of Diseases of Aorta and its Branches
• MDCT fistulography in evaluation of fistula in Ano
• Role of multislice CT in diagnosis of small intestine tumors
• Role of high resolution CT in differentiation between benign and malignant pulmonary nodules in children
• A study of multidetector computed tomography urography in urinary tract abnormalities
• Role of high resolution sonography in assessment of ulnar nerve in patients with leprosy.
• Pre-operative radiological evaluation of locally aggressive and malignant musculoskeletal tumours by computed tomography and magnetic resonance imaging.
• The role of ultrasound & MRI in acute pelvic inflammatory disease
• Ultrasonography compared to computed tomographic arthrography in the evaluation of shoulder pain
• Role of Multidetector Computed Tomography in patients with blunt abdominal trauma.
• The Role of Extended field-of-view Sonography and compound imaging in Evaluation of Breast Lesions
• Evaluation of focal pancreatic lesions by Multidetector CT and perfusion CT
• Evaluation of breast masses on sono-mammography and colour Doppler imaging
• Role of CT virtual laryngoscopy in evaluation of laryngeal masses
• Triple phase multi detector computed tomography in hepatic masses
• Role of transvaginal ultrasound in diagnosis and treatment of female infertility
• Role of ultrasound and color Doppler imaging in assessment of acute abdomen due to female genetal causes
• High resolution ultrasonography and color Doppler ultrasonography in scrotal lesion
• Evaluation of diagnostic accuracy of ultrasonography with colour Doppler vs low dose computed tomography in salivary gland disease
• Role of multidetector CT in diagnosis of salivary gland lesions
• Comparison of diagnostic efficacy of ultrasonography and magnetic resonance cholangiopancreatography in obstructive jaundice: A prospective study
• Evaluation of varicose veins-comparative assessment of low dose CT venogram with sonography: pilot study
• Role of mammotome in breast lesions
• The role of interventional imaging procedures in the treatment of selected gynecological disorders
• Role of transcranial ultrasound in diagnosis of neonatal brain insults
• Role of multidetector CT virtual laryngoscopy in evaluation of laryngeal mass lesions
• Evaluation of adnexal masses on sonomorphology and color Doppler imaginig
• Role of radiological imaging in diagnosis of endometrial carcinoma
• Comprehensive imaging of renal masses by magnetic resonance imaging
• The role of 3D & 4D ultrasonography in abnormalities of fetal abdomen
• Diffusion weighted magnetic resonance imaging in diagnosis and characterization of brain tumors in correlation with conventional MRI
• Role of diffusion weighted MRI imaging in evaluation of cancer prostate
• Role of multidetector CT in diagnosis of urinary bladder cancer
• Role of multidetector computed tomography in the evaluation of paediatric retroperitoneal masses.
• Comparative evaluation of gastric lesions by double contrast barium upper G.I. and multi detector computed tomography
• Evaluation of hepatic fibrosis in chronic liver disease using ultrasound elastography
• Role of MRI in assessment of hydrocephalus in pediatric patients
• The role of sonoelastography in characterization of breast lesions
• The influence of volumetric tumor doubling time on survival of patients with intracranial tumours
• Role of perfusion computed tomography in characterization of colonic lesions
• Role of proton MRI spectroscopy in the evaluation of temporal lobe epilepsy
• Role of Doppler ultrasound and multidetector CT angiography in evaluation of peripheral arterial diseases.
• Role of multidetector computed tomography in paranasal sinus pathologies
• Role of virtual endoscopy using MDCT in detection & evaluation of gastric pathologies
• High resolution 3 Tesla MRI in the evaluation of ankle and hindfoot pain.
• Transperineal ultrasonography in infants with anorectal malformation
• CT portography using MDCT versus color Doppler in detection of varices in cirrhotic patients
• Role of CT urography in the evaluation of a dilated ureter
• Characterization of pulmonary nodules by dynamic contrast-enhanced multidetector CT
• Comprehensive imaging of acute ischemic stroke on multidetector CT
• The role of fetal MRI in the diagnosis of intrauterine neurological congenital anomalies
• Role of Multidetector computed tomography in pediatric chest masses
• Multimodality imaging in the evaluation of palpable & non-palpable breast lesion.
• Sonographic Assessment Of Fetal Nasal Bone Length At 11-28 Gestational Weeks And Its Correlation With Fetal Outcome.
• Role Of Sonoelastography And Contrast-Enhanced Computed Tomography In Evaluation Of Lymph Node Metastasis In Head And Neck Cancers
• Role Of Renal Doppler And Shear Wave Elastography In Diabetic Nephropathy
• Evaluation Of Relationship Between Various Grades Of Fatty Liver And Shear Wave Elastography Values
• Evaluation and characterization of pelvic masses of gynecological origin by USG, color Doppler and MRI in females of reproductive age group
• Radiological evaluation of small bowel diseases using computed tomographic enterography
• Role of coronary CT angiography in patients of coronary artery disease
• Role of multimodality imaging in the evaluation of pediatric neck masses
• Role of CT in the evaluation of craniocerebral trauma
• Role of magnetic resonance imaging (MRI) in the evaluation of spinal dysraphism
• Comparative evaluation of triple phase CT and dynamic contrast-enhanced MRI in patients with liver cirrhosis
• Evaluation of the relationship between carotid intima-media thickness and coronary artery disease in patients evaluated by coronary angiography for suspected CAD
• Assessment of hepatic fat content in fatty liver disease by unenhanced computed tomography
• Correlation of vertebral marrow fat on spectroscopy and diffusion-weighted MRI imaging with bone mineral density in postmenopausal women.
• Comparative evaluation of CT coronary angiography with conventional catheter coronary angiography
• Ultrasound evaluation of kidney length & descending colon diameter in normal and intrauterine growth-restricted fetuses
• A prospective study of hepatic vein waveform and splenoportal index in liver cirrhosis: correlation with child Pugh’s classification and presence of esophageal varices.
• CT angiography to evaluate coronary artery by-pass graft patency in symptomatic patient’s functional assessment of myocardium by cardiac MRI in patients with myocardial infarction
• MRI evaluation of HIV positive patients with central nervous system manifestations
• MDCT evaluation of mediastinal and hilar masses
• Evaluation of rotator cuff & labro-ligamentous complex lesions by MRI & MRI arthrography of shoulder joint
• Role of imaging in the evaluation of soft tissue vascular malformation
• Role of MRI and ultrasonography in the evaluation of multifidus muscle pathology in chronic low back pain patients
• Role of ultrasound elastography in the differential diagnosis of breast lesions
• Role of magnetic resonance cholangiopancreatography in evaluating dilated common bile duct in patients with symptomatic gallstone disease.
• Comparative study of CT urography & hybrid CT urography in patients with haematuria.
• Role of MRI in the evaluation of anorectal malformations
• Comparison of ultrasound-Doppler and magnetic resonance imaging findings in rheumatoid arthritis of hand and wrist
• Role of Doppler sonography in the evaluation of renal artery stenosis in hypertensive patients undergoing coronary angiography for coronary artery disease.
• Comparison of radiography, computed tomography and magnetic resonance imaging in the detection of sacroiliitis in ankylosing spondylitis.
• Mr evaluation of painful hip
• Role of MRI imaging in pretherapeutic assessment of oral and oropharyngeal malignancy
• Evaluation of diffuse lung diseases by high resolution computed tomography of the chest
• Mr evaluation of brain parenchyma in patients with craniosynostosis.
• Diagnostic and prognostic value of cardiovascular magnetic resonance imaging in dilated cardiomyopathy
• Role of multiparametric magnetic resonance imaging in the detection of early carcinoma prostate
• Role of magnetic resonance imaging in white matter diseases
• Role of sonoelastography in assessing the response to neoadjuvant chemotherapy in patients with locally advanced breast cancer.
• Role of ultrasonography in the evaluation of carotid and femoral intima-media thickness in predialysis patients with chronic kidney disease
• Role of H1 MRI spectroscopy in focal bone lesions of peripheral skeleton choline detection by MRI spectroscopy in breast cancer and its correlation with biomarkers and histological grade.
• Ultrasound and MRI evaluation of axillary lymph node status in breast cancer.
• Role of sonography and magnetic resonance imaging in evaluating chronic lateral epicondylitis.
• Comparative of sonography including Doppler and sonoelastography in cervical lymphadenopathy.
• Evaluation of Umbilical Coiling Index as Predictor of Pregnancy Outcome.
• Computerized Tomographic Evaluation of Azygoesophageal Recess in Adults.
• Lumbar Facet Arthropathy in Low Backache.
• “Urethral Injuries After Pelvic Trauma: Evaluation with Uretrography
• Role Of Ct In Diagnosis Of Inflammatory Renal Diseases
• Role Of Ct Virtual Laryngoscopy In Evaluation Of Laryngeal Masses
• “Ct Portography Using Mdct Versus Color Doppler In Detection Of Varices In
• Cirrhotic Patients”
• Role Of Multidetector Ct In Characterization Of Renal Masses
• Role Of Ct Virtual Cystoscopy In Diagnosis Of Urinary Bladder Neoplasia
• Role Of Multislice Ct In Diagnosis Of Small Intestine Tumors
• “Mri Flow Quantification In The Assessment Of The Commonest CSF Flow Abnormalities”
• “The Role Of Fetal Mri In Diagnosis Of Intrauterine Neurological CongenitalAnomalies”
• Role Of Transcranial Ultrasound In Diagnosis Of Neonatal Brain Insults
• “The Role Of Interventional Imaging Procedures In The Treatment Of Selected Gynecological Disorders”
• Role Of Radiological Imaging In Diagnosis Of Endometrial Carcinoma
• “Role Of High-Resolution Ct In Differentiation Between Benign And Malignant Pulmonary Nodules In Children”
• Role Of Ultrasonography In The Diagnosis Of Knee Joint Lesions
• “Role Of Diagnostic Imaging Modalities In Evaluation Of Post Liver Transplantation Recipient Complications”
• “Diffusion-Weighted Magnetic Resonance Imaging In Diagnosis And
• Characterization Of Brain Tumors In Correlation With Conventional Mri”
• The Role Of PET-CT In The Evaluation Of Hepatic Tumors
• “Role Of Computerized Tomography In Evaluation Of Mediastinal Masses In Pediatric patients”
• “Trans Vaginal Ultrasound And Magnetic Resonance Imaging In Female Urinary Incontinence”
• Role Of Multidetector Ct In Diagnosis Of Urinary Bladder Cancer
• “Role Of Transvaginal Ultrasound In Diagnosis And Treatment Of Female Infertility”
• Role Of Diffusion-Weighted Mri Imaging In Evaluation Of Cancer Prostate
• “Role Of Positron Emission Tomography With Computed Tomography In Diagnosis Of Cancer Thyroid”
• The Role Of CT Urography In Case Of Haematuria
• “Value Of Ultrasonography In Assessment Of Acute Abdominal Diseases In Pediatric Age Group”
• “Role Of Functional Magnetic Resonance Imaging In Making Brain Tumor Surgery Safer”
• The Role Of Sonoelastography In Characterization Of Breast Lesions
• “Ultrasonography, Magnetic Resonance Cholangiopancreatography (MRCP) In Assessment Of Pediatric Biliary Lesions”
• “Role Of Ultrasound And Color Doppler Imaging In Assessment Of Acute Abdomen Due To Female Genital Causes”
• “Role Of Multidetector Ct Virtual Laryngoscopy In Evaluation Of Laryngeal Mass Lesions”
• MRI Of The Postoperative Knee
• Role Of Mri In Assessment Of Valvular Heart Diseases
• The Role Of 3D & 4D Ultrasonography In Abnormalities Of Fetal Abdomen
• State Of The Art Of Mri In Diagnosis Of Hepatic Focal Lesions
• Role Of Multidetector Ct In Diagnosis Of Salivary Gland Lesions
• “Role Of Virtual Endoscopy Using Mdct In Detection & Evaluation Of Gastric Pathologies”
• The Role Of Ultrasound & Mri In Acute Pelvic Inflammatory Disease
• “Diagnosis & Staging Of Liver Fibrosis By Ultraso Und Elastography In
• Patients With Chronic Liver Diseases”
• Role Of Mri In Evaluation Of Spinal Trauma
• Validity Of Mri In Diagnosis Of Congenital Anorectal Anomalies
• Imaging Of Vascular Complication After Liver Transplantation
• “Contrast-Enhanced Digital Mammography And Digital Breast Tomosynthesis In Early Diagnosis Of Breast Lesion”
• Role Of Mammotome In Breast Lesions
• “Role Of MRI Diffusion Tensor Imaging (DTI) In Assessment Of Traumatic Spinal Cord Injuries”
• “Prediction Of Pre-eclampsia And Fetal Growth Restriction By Uterine Artery Doppler”
• “Role Of Multidetector Row Computed Tomography In Assessment Of Maxillofacial Trauma”
• “Role Of Diffusion Magnetic Resonance Imaging In Assessment Of Neoplastic And Inflammatory Brain Lesions”
• Role Of Diffusion Mri In Preoperative Evaluation Of Brain Neoplasms
• “Role Of Multidetector Ct Virtual Hysteroscopy In The Detection Of The
• Uterine & Tubal Causes Of Female Infertility”
• Role Of Advances Magnetic Resonance Imaging Sequences In Multiple Sclerosis Magnetic Resonance Spectroscopy In Multiple Sclerosis
• “Role Of Conventional Mri, And Diffusion Tensor Imaging Tractography In Evaluation Of Congenital Brain Malformations”
• Role Of MRI In Evaluation Of Spinal Trauma
• Diagnostic Role Of Diffusion-weighted MR Imaging In Neck Masses
• “The Role Of Transvaginal Ultrasound Versus Magnetic Resonance Imaging In Diagnosis & Evaluation Of Cancer Cervix”
• “Role Of 3d Magnetic Resonance Imaging Tractography In Assessment Of White Matter Tracts Compromise In Supra Tentorial Tumors”
• Role Of Proton MR Spectroscopy In The Evaluation Of Temporal Lobe Epilepsy
• Role Of Multislice Computed Tomography In Evaluation Of Crohn’s Disease
• Role Of MRI In Assessment Of Hydrocephalus In Pediatric Patients
• The Role Of MRI In Diagnosis And Staging Of Urinary Bladder Carcinoma
• USG and MRI correlation of congenital CNS anomalies
• HRCT in interstitial lung disease
• X-Ray, CT and MRI correlation of bone tumors
• “Study on the diagnostic and prognostic utility of X-Rays for cases of pulmonary tuberculosis under RNTCP”
• “Role of magnetic resonance imaging in the characterization of female adnexal  pathology”
• “CT angiography of carotid atherosclerosis and NECT brain in cerebral ischemia, a correlative analysis”
• Role of CT scan in the evaluation of paranasal sinus pathology
• USG and MRI correlation on shoulder joint pathology
• “Radiological evaluation of a patient presenting with extrapulmonary tuberculosis”
• CT and MRI correlation in focal liver lesions”
• Comparison of MDCT virtual cystoscopy with conventional cystoscopy in bladder tumors”
• “Bleeding vessels in life-threatening hemoptysis: Comparison of 64 detector row CT angiography with conventional angiography prior to endovascular management”
• “Role of transarterial chemoembolization in unresectable hepatocellular carcinoma”
• “Comparison of color flow duplex study with digital subtraction angiography in the evaluation of peripheral vascular disease”
• “A Study to assess the efficacy of magnetization transfer ratio in differentiating tuberculoma from neurocysticercosis”
• “MR evaluation of uterine mass lesions in correlation with transabdominal, transvaginal ultrasound using HPE as a gold standard”
• “The Role of power Doppler imaging with trans rectal ultrasonogram guided prostate biopsy in the detection of prostate cancer”
• “Lower limb arteries assessed with doppler angiography – A prospective comparative study with multidetector CT angiography”
• “Comparison of sildenafil with papaverine in penile doppler by assessing hemodynamic changes”
• “Evaluation of efficacy of sonosalphingogram for assessing tubal patency in infertile patients with hysterosalpingogram as the gold standard”
• Role of CT enteroclysis in the evaluation of small bowel diseases
• “MRI colonography versus conventional colonoscopy in the detection of colonic polyposis”
• “Magnetic Resonance Imaging of anteroposterior diameter of the midbrain – differentiation of progressive supranuclear palsy from Parkinson disease”
• “MRI Evaluation of anterior cruciate ligament tears with arthroscopic correlation”
• “The Clinicoradiological profile of cerebral venous sinus thrombosis with prognostic evaluation using MR sequences”
• “Role of MRI in the evaluation of pelvic floor integrity in stress incontinent patients” “Doppler ultrasound evaluation of hepatic venous waveform in portal hypertension before and after propranolol”
• “Role of transrectal sonography with colour doppler and MRI in evaluation of prostatic lesions with TRUS guided biopsy correlation”
• “Ultrasonographic evaluation of painful shoulders and correlation of rotator cuff pathologies and clinical examination”
• “Colour Doppler Evaluation of Common Adult Hepatic tumors More Than 2 Cm  with HPE and CECT Correlation”
• “Clinical Relevance of MR Urethrography in Obliterative Posterior Urethral Stricture”
• “Prediction of Adverse Perinatal Outcome in Growth Restricted Fetuses with Antenatal Doppler Study”
• Radiological evaluation of spinal dysraphism using CT and MRI
• “Evaluation of temporal bone in cholesteatoma patients by high resolution computed tomography”
• “Radiological evaluation of primary brain tumours using computed tomography and magnetic resonance imaging”
• “Three dimensional colour doppler sonographic assessment of changes in  volume and vascularity of fibroids – before and after uterine artery embolization”
• “In phase opposed phase imaging of bone marrow differentiating neoplastic lesions”
• “Role of dynamic MRI in replacing the isotope renogram in the functional evaluation of PUJ obstruction”
• Characterization of adrenal masses with contrast-enhanced CT – washout study
• A study on accuracy of magnetic resonance cholangiopancreatography
• “Evaluation of median nerve in carpal tunnel syndrome by high-frequency ultrasound & color doppler in comparison with nerve conduction studies”
• “Correlation of Agatston score in patients with obstructive and nonobstructive coronary artery disease following STEMI”
• “Doppler ultrasound assessment of tumor vascularity in locally advanced breast cancer at diagnosis and following primary systemic chemotherapy.”
• “Validation of two-dimensional perineal ultrasound and dynamic magnetic resonance imaging in pelvic floor dysfunction.”
• “Role of MR urethrography compared to conventional urethrography in the surgical management of obliterative urethral stricture.”

Search Diagnostic Imaging Research Topics

You can also search research-related resources and direct download PDFs for radiology articles on our custom radiology search engine .

Free Resources for Preparing Radiology Thesis

• Radiology thesis topics- Benha University – Free to download thesis
• Radiology thesis topics – Faculty of Medical Science Delhi
• Radiology thesis topics – IPGMER
• Fetal Radiology thesis Protocols
• Radiology thesis and dissertation topics
• Radiographics

Proofreading Your Thesis:

Make sure you use Grammarly to correct your spelling ,  grammar , and plagiarism for your thesis. Grammarly has affordable paid subscriptions, windows/macOS apps, and FREE browser extensions. It is an excellent tool to avoid inadvertent spelling mistakes in your research projects. It has an extensive built-in vocabulary, but you should make an account and add your own medical glossary to it.

Guidelines for Writing a Radiology Thesis:

These are general guidelines and not about radiology specifically. You can share these with colleagues from other departments as well. Special thanks to Dr. Sanjay Yadav sir for these. This section is best seen on a desktop. Here are a couple of handy presentations to start writing a thesis:

Read the general guidelines for writing a thesis (the page will take some time to load- more than 70 pages!

A format for thesis protocol with a sample patient information sheet, sample patient consent form, sample application letter for thesis, and sample certificate.

Resources and References:

• Guidelines for thesis writing.
• Format for thesis protocol
• Thesis protocol writing guidelines DNB
• Informed consent form for Research studies from AIIMS 
• Radiology Informed consent forms in local Indian languages.
• Sample Informed Consent form for Research in Hindi
• Guide to write a thesis by Dr. P R Sharma
• Guidelines for thesis writing by Dr. Pulin Gupta.
• Preparing MD/DNB thesis by A Indrayan
• Another good thesis reference protocol

Hopefully, this post will make the tedious task of writing a Radiology thesis a little bit easier for you. Best of luck with writing your thesis and your residency too!

More guides for residents :

• Guide for the MD/DMRD/DNB radiology exam!

Guide for First-Year Radiology Residents

• FRCR Exam: THE Most Comprehensive Guide (2022)!
• Radiology Practical Exams Questions compilation for MD/DNB/DMRD !
• Radiology Exam Resources (Oral Recalls, Instruments, etc )!
• Tips and Tricks for DNB/MD Radiology Practical Exam
• FRCR 2B exam- Tips and Tricks !
• FRCR exam preparation – An alternative take!
• Why did I take up Radiology?
• Radiology Conferences – A comprehensive guide!
• ECR (European Congress Of Radiology)

European Diploma in Radiology (EDiR) – The Complete Guide!

• Radiology NEET PG guide – How to select THE best college for post-graduation in Radiology (includes personal insights)!
• Interventional Radiology – All Your Questions Answered!
• What It Means To Be A Radiologist: A Guide For Medical Students!
• Radiology Mentors for Medical Students (Post NEET-PG)
• MD vs DNB Radiology: Which Path is Right for Your Career?
• DNB Radiology OSCE – Tips and Tricks

More radiology resources here: Radiology resources This page will be updated regularly. Kindly leave your feedback in the comments or send us a message here . Also, you can comment below regarding your department’s thesis topics.

Note: All topics have been compiled from available online resources. If anyone has an issue with any radiology thesis topics displayed here, you can message us here , and we can delete them. These are only sample guidelines. Thesis guidelines differ from institution to institution.

Image source: Thesis complete! (2018). Flickr. Retrieved 12 August 2018, from https://www.flickr.com/photos/cowlet/354911838 by Victoria Catterson

About The Author

Dr. amar udare, md, related posts ↓.

9 thoughts on “Radiology Thesis – More than 400 Research Topics (2022)!”

Amazing & The most helpful site for Radiology residents…

Thank you for your kind comments 🙂

Dr. I saw your Tips is very amazing and referable. But Dr. Can you help me with the thesis of Evaluation of Diagnostic accuracy of X-ray radiograph in knee joint lesion.

Wow! These are excellent stuff. You are indeed a teacher. God bless

Glad you liked these!

happy to see this

Glad I could help :).

Greetings Dr, thanks for your constant guides. pls Dr, I need a thesis research material on “Retrieving information from scattered photons in medical imaging”

Hey! Unfortunately I do not have anything relevant to that thesis topic.

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Get Radiology Updates to Your Inbox!

This site is for use by medical professionals. To continue, you must accept our use of cookies and the site's Terms of Use. Learn more Accept!

Wish to be a BETTER Radiologist? Join 14000 Radiology Colleagues !

Enter your email address below to access HIGH YIELD radiology content, updates, and resources.

No spam, only VALUE! Unsubscribe anytime with a single click.

Deep learning for medical image interpretation

Abstract/contents, description.

Creators/Contributors

Bibliographic information

Access conditions

Each version has a distinct URL, but you can use this PURL to access the latest version. https://purl.stanford.edu/jc097kx0188

Also listed in

Loading usage metrics...

• DSpace@MIT Home
• MIT Libraries
• Doctoral Theses

Effective Modeling in Medical Imaging with Constrained Data

Terms of use

Date issued, collections.

Show Statistical Information

Advertisement

A holistic overview of deep learning approach in medical imaging

• Regular Paper
• Published: 21 January 2022
• Volume 28 , pages 881–914, ( 2022 )

Cite this article

• Rammah Yousef 1 ,
• Gaurav Gupta 1 ,
• Nabhan Yousef 2 &
• Manju Khari   ORCID: orcid.org/0000-0001-5395-5335 3  

9858 Accesses

50 Citations

20 Altmetric

Explore all metrics

Medical images are a rich source of invaluable necessary information used by clinicians. Recent technologies have introduced many advancements for exploiting the most of this information and use it to generate better analysis. Deep learning (DL) techniques have been empowered in medical images analysis using computer-assisted imaging contexts and presenting a lot of solutions and improvements while analyzing these images by radiologists and other specialists. In this paper, we present a survey of DL techniques used for variety of tasks along with the different medical image’s modalities to provide critical review of the recent developments in this direction. We have organized our paper to provide significant contribution of deep leaning traits and learn its concepts, which is in turn helpful for non-expert in medical society. Then, we present several applications of deep learning (e.g., segmentation, classification, detection, etc.) which are commonly used for clinical purposes for different anatomical site, and we also present the main key terms for DL attributes like basic architecture, data augmentation, transfer learning, and feature selection methods. Medical images as inputs to deep learning architectures will be the mainstream in the coming years, and novel DL techniques are predicted to be the core of medical images analysis. We conclude our paper by addressing some research challenges and the suggested solutions for them found in literature, and also future promises and directions for further developments.

Similar content being viewed by others

Medical Image Analysis Through Deep Learning Techniques: A Comprehensive Survey

Survey on Deep Learning Techniques for Medical Imaging Application Area

Enhancing Medical Diagnosis Through Deep Learning and Machine Learning Approaches in Image Analysis

Explore related subjects.

• Artificial Intelligence
• Medical Imaging

Avoid common mistakes on your manuscript.

1 Introduction

Health no doubt is on the top of concerns hierarchy in our life. Through the lifetime, human has struggled of diseases which cause death; in our life scope, we are fighting against enormous number of diseases, moreover, improving life expectancy and health status significantly. Historically medicine could not find the cure of numerous diseases due to a lot of reasons starting from clinical equipment and sensors to the analytical tools of the collected medical data. The fields of big data, AI, and cloud computing have played a missive role at each aspect of handling these data. Across the worldwide, Artificial Intelligence (AI) has been widely common and well known enough to most of the people due to the rapid progress achieved in almost every domain in our life. The importance of AI comes from the remarkable progress within the last 2 decades only, and it is still growing and specialists from different fields are investing. AI’s algorithms were attributed to the availability of big data and the efficiency of modern computing criteria that is provided lately.

This paper aims to give a holistic overview in the field of healthcare as an application of AI and deep learning particularly. The paper starts by giving an overview of medical imaging as an application of deep learning and then moving to why do we need AI in healthcare; in this section, we will give the key terms of how AI is used in both the main medical data types which are medical imaging and medical signals. To provide a moderate and rich general perspective, we will mention the well-known data which are widely used for generalization and the main pathologies, as well. Starting from classification and detection of a disease to segmentation and treatment and finally survival rate and prognostics. We will talk in detail about each pathology with the relevant key features and the significant results found in literature. In the last section, we will discuss about the challenges of deep learning and the future scope of AI in healthcare. Generally, AI is being a fundamental path in nowadays medicine which is in short a software that can learn from data like human being and it can develop an experience systematically and finally deliver a solution or diagnostic even faster than humans. AI has become an assistive tool in medicine with benefits like error reduction, improving accuracy, fast computing, and better diagnosis were introduced to help doctors efficiently. From clinical perspective, AI is used now to help the doctors in decision-making due to faster pattern recognition from the medical data which also in turn are registered more precisely in computers than humans; moreover, AI has the ability to manage and monitor the patients’ data and creating a personalized medical plan for future treatments. Ultimately, AI has proved to be helpful in medical field with different levels, such as telemedicine diagnosis diseases, decision-making assistant, and drug discovery and development. Machine learning (ML) and deep learning (DL) have tremendous usages in healthcare such as clinical decision support (CDS) system which incorporate human’s knowledge or large datasets to provide clinical recommendations. Another application is to analyze large historical data and get the insights which can predict the future cases of a patient using pattern identification. In this paper, we will highlight the top deep learning advancement and applications in medical imaging. Figure  1 shows the workflow chart of paper highlights.

Deep learning implementation and traits for medical imaging application

2 Background concepts

2.1 medical imaging.

Deep learning in medical imaging [ 1 ] is the contemporary scope of AI which has the top breakthroughs in numerous scientific domains including computer vision [ 2 ], Natural Language Processing (NLP) [ 3 ] and chemical structure analysis, where deep learning is specialized with highly complicated processes. Lately due to deep learning robustness while dealing with images, it has attracted big interest in medical imaging, and it holds big promising future for this field. The main idea that DL is preferable is that medical data are large and it has different varieties such as medical images, medical signals, and medical logs’ data of patients monitoring of body sensed information. Analyzing these data especially historical data by learning very complex mathematical models and extracting meaningful information is the key feature where DL scheme outperformed humans. In other words, DL framework will not replace the doctors, but it will assist them in decision-making and it will enhance the accuracy of the final diagnosis analysis. Our workflow procedure is shown in Fig.  1 .

2.1.1 Types of medical imaging

There are plenty of medical image types, and selecting the type depends on the usage, in a study which was held in US [ 4 ], it was found that there are some basic and widely used modalities of these medical images which also have increased, and these modalities are Magnetic Resonance Images (MRI), Computed Tomography (CT) scans, and Positron Emission Tomography (PET) to be on the top and some other common modalities like, X-ray, Ultrasound, and histology slides. Medical images are known to be so complicated, and in some cases, acquisition of these images is considered to be long process and it needs specific technical implications, e.g., an MRI which may need over 100 Mega Byte of memory storage.

Because of a lack of standardization while image acquisition and diversity in the scanning devices’ settings, a phenomenon called “distribution drift” might arise and cause non-standard acquisition. From a clinical need perspective, medical images are the key part of diagnosis of a disease and then the treatment too. In traditional diagnosis, a radiologist reviews the image, and then, he provides the doctors with a report of his findings. Images are an important part of the invasive process to be used in further treatment, e.g., surgical operations or radiology therapies for example [ 5 , 6 ].

2.2 DL frameworks

Conceptually, Artificial Neural Networks (ANN) are a mimic of the human neuro system in the structure and work. Medical imaging [ 7 ] is a field by which is specialized in observing and analyzing the physical status of the human body by generating visual representations like images of internal tissues or some organs of the body through either invasive or non-invasive procedure.

2.2.1 Key technologies and deep learning

Historically, AI scheme has been proposed in 1970s and it has mainly the two major subcategories, such as Machine Learning (ML) and Deep Learning (DL). The earlier AI used heuristics-based techniques for extracting features from data, and further developments started using handcrafted features’ extraction and finally to supervised learning. Where basically Convolutional Neural Networks (CNN) [ 8 ] is used in images and specifically in medical images. CNN is known to be hungry for data, so it is the most suitable methodology for images, and the recent developments in hardware specifications and GPUs have helped a lot in performing CNN algorithms for medical image analysis. The generalized formulation of how CNN work was proposed by Lecun et al. [ 9 ], where they have used the error backpropagation for the first example of digits hand written recognition. Ultimately, CNNs have been the predominant architecture among all other algorithms which belong to AI, and the number of research of CNN has increased especially in medical images analysis and many new modalities have been proposed. In this section, we explain the fundamentals of DL and its algorithmic path in medical imaging. The commonly known categories of deep learning and their subcategories are discussed in this section and are shown in Fig.  2 .

DL basic categories as per paper organization

2.2.2 Supervised learning

Convolutional neural networks: CNN [ 10 ] have taken the major role in many aspects and have lead the work in image-based tasks, including image reconstruction, enhancement, classification, segmentation, registration, and localization. CNNs are considered to be the most deep learning algorithm regarding images and visual processing because of its robustness in image dimensionality reduction without losing image’s important features; in this way, CNN algorithm deals with less parameters which mean increasing the computational efficiency. Another key term about CNN is that this architecture is suitable for hospitals use, because it can handle both 2D and 3D images, because some of medical images modalities like X-ray images are 2D-based images, while MRI and CT scan images are 3-dimensional images. In this section, we will explain the framework of CNN architecture as the heart of deep learning in medical imaging.

Convolutional layer: Before deep learning and CNN, in image processing, convolution terminology was used for extracting specific features from an image, such as corners, edges (e.g., sobel filter), and noise by applying a particular filters or kernels on the image. This operation is done by sliding the filter all over the image in a sliding window form until all the image is covered. In CNN, usually, the startup layers are designed to extract low-level features, such as lines and edges, and the progressive layers are built up for extracting higher features like full objects within an image. The goodness of using modern CNNs is that the filters could be 2D or 3D filters using multiple filters to form a volume and this depends on the application. The main discrimination in CNN is that this architecture obliges the elements in a filter to be the network weights. The idea behind CNN architecture is the convolution operation which is denoted by the symbol *. Equation ( 1 ) represents the convolution operation

where s ( t ) is the output feature map and I ( t ) is the original image to be convolved with the filter K ( a ).

Activation function : Activation functions are the enable button of a neuron; in CNN, there are multiple popular activation functions which are widely used such as, sigmoid, tanh, ReLU, Leaky ReLU, and Randomized ReLU. Especially, in medical imaging, most papers found in literature uses ReLU activation function which is defined using the formula

where x represents the input of a neuron.

There are other used activation functions used in CNN, such as sigmoid, tanh, and leaky-ReLu

Pooling layer: Mainly, this layer is used to reduce the parameters needed to be computed and it reduces the size of the image but not the number of channels. There are few pooling layers, such as Max-pooling, average- pooling, and L2-normalization pooling, where Max-pooling is the widely used pooling layer. Max-pooling means taking the maximum value of a position of the feature map after convolution operation.

Fully connecter layer: This layer is the same layer that is used in a casual ANN where usually in such network each neuron is connected to all other neurons in both the previous and next layer’s neurons; this makes the computation very expensive. A CNN model can get the help of the stochastic gradient descent to learn significant associations from the existing examples used for training. Thus, the benefit of a CNN usage is that it gradually reduces the feature map size before finally is get flatten to feed the fully connected layer which in turn computes the probability scores of the targeted classes for the classification. Fc-connected layer is the last layer in a CNN model, Furthermore, this layer processes the strongly extracted features from an image due to the convolutional a pooling layer before and finally fc-layer indicate to which class is an image belong to.

Recurrent neural networks: RNN is a major part from supervised deep learning models, and this model is specific with analyzing sequential data and time series. We can imagine an RNN as a casual neural network, while each layer of it represents the observations at a particular time (t). In [ 11 ], RNN was used for text generating which further connected to speech recognition and text prediction and other applications too. RNN are recurrent, because same work is done for every element in a sequence and the output depends on the previous output computation of the previous element in that sequence general, the output of a layer is fed as an input to the new input of the same layer as it is shown in Fig.  3 . Moreover, since the backpropagation of the output will suffer of vanishing gradient with time, so commonly a network is evolved which is Long Short-Term Memory (LSTM).

Basic common deep learning architectures. A Restricted Boltzmann machine. B Recurrent Neural Network (RNN). C Autoencoders. D GANs

In network and three bidirectional gated recurrent units is (BGRU) to help the RNN to hold long-term dependencies.

There were few papers found in the literature of RNN in medical imaging and particularly in segmentation, in [ 12 ], Chen et al. have used RNN along with CNN for segmenting fungal and neuronal structures from 3D images. Another application of RNN is in image caption generation [ 13 ], where these models can be used for annotating medical images like X-ray with text captions extracted and trained from radiologists’ reports [ 14 ]. RuoxuanCui et al. [ 15 ] have used a combination of CNN and RNN for diagnosing Alzheimer disease where their CNN model was used for classification task, after that the CNN model’s output is fed to an RNN model with cascaded bidirectional gated recurrent units (BGRU) layers to extract the longitudinal features of the disease. In summary, RNN is commonly used with a CNN model in medical imaging. In [ 16 ], authors have developed a novel RNN for speeding up an iterative MAP estimation algorithm.

2.2.3 Unsupervised deep learning

Beside the CNN as a supervised machine leaning algorithm in medical imaging, there are a few unsupervised learning algorithms for this purpose as well, such as Deep Belief Networks (DBNs), Autoencoders, and Generative Adversarial Networks (GANs), where the last has been used for not only performing the image-based tasks but as a data synthesis and augmentation too. Unsupervised learning models have been used for different medical imaging applications, such as motion tracking [ 17 ] general modeling, classification improvement [ 18 ], artifact reduction [ 19 ], and medical image registration [ 20 ]. In this section, we will list the mostly used unsupervised learning structures.

2.2.3.1 Autoencoders

Autoencoders [ 21 , 22 ] are an unsupervised deep learning algorithm by which this model refers to the important features of an input data and dismisses the other data. These important representations of features are called ‘codings’ where it is commonly called representation learning. The basic architecture is shown in Fig.  3 . The robustness of autoencoders stems from the ability to reconstruct output data, which is similar to the input data, because it has cost function which applies penalties to the model when the output and input data are different. Moreover, autoencoders are considered as an automatic features detector, because they do not need labeled data to learn from due to the unsupervised manner. Autoencoders architecture is similar to a formal CNN model, but with the feature is that the number of input neurons must be equal to the number in the output layer. Reducing dimensionality of the raw input data is one of the features of autoencoders, and in some cases, autoencoders are used for denoising purpose [ 23 ], where this autoencoders are called denoising autoencoders. In general, there are few kinds of autoencoders used for different purposes, we mention here the common autoencoders, for example, Sparse Autoencoders [ 24 ] where the neurons in the hidden layer are deactivated through a threshold which means limiting the activated neurons to get a representation in the output similar to the input where for extracting most of the features from the input, most of the hidden layer neurons should be set to zero. Variational autoencoders (VAEs) [ 25 ] are generative model with two networks (Encoder and Decoder) where the encoder network projects the input into latent representation using Gaussian distribution approximation, and the decoder network maps the latent representations into the output data. Contractive autoencoders [ 26 ] and adversarial autoencoders are mostly similar to a Generative Adversarial Network (GAN).

2.2.3.2 Generative Adversarial Networks

GANs [ 27 ] 28 were first introduced by Ian Goodfellow in 2014; it consists basically on a combination of two CNN networks: the first one is called Generative model and another is the discriminator model. For better understanding how GANs work, scientists describe the two networks as a two players who competing against each other, where the generator network tries to fool the discriminator network by generating near authentic data (e.g., artificial images), while the discriminator network tries to distinguish between the generator output and the real data, Fig.  3 . The name of the network is inspired from the objective of the generator to overcome the discriminator. After the training process, both the generator and discriminator networks get better, where the first generates more real data, and the second learns how to differentiate between both previously mentioned data better until the end-point of the whole process where the discriminator network is unable to distinguish between real and artificial data (images). In fact, the criteria by which both networks learn from each other are using the backpropagation for the both, Markov chains, and dropout too. Recently, we have seen tremendous usage of GANs for different applications in medical imaging such as, synthetic images for generating new images and enhance the deep learning models efficiency by increasing the number of training images in the dataset [ 29 ], classification [ 30 , 31 ], detection [ 32 ], segmentation [ 33 , 34 ], image-to-image translation [ 35 ], and other application too. In a study by Kazeminia et al. [ 36 ], they have listed all the applications of GANs in medical imaging and the most two used applications of this unsupervised models are image synthesis and segmentation.

2.2.3.3 Restricted Boltzmann machines

Axkley et al. were the first to introduce the Boltzmann machines in 1985 [ 37 ], Fig.  3 , also known as Gibbs distribution, and further Smolensky has modified it to be known as Restricted Boltzmann Machines (RBMs) [ 38 ] . RBMs consist on two layers of neural networks with stochastic, generative, and probabilistic capabilities, and they can learn probability distributions and internal representations from the dataset. RBMs work using the backpropagation path of input data for generating and estimating the probability distribution of the original input data using gradient descent loss. These unsupervised models are used mostly for dimensionality reduction, filtering, classification, and features representation learning. In medical imaging, Tulder et al. [ 39 ] have modified the RBMs and introduced a novel convolutional RBMs for lung tissue classification using CT scan images; they have extracted the features using different methodologies (generative, discriminative, or mixed) to construct the filters; after that, Random Forest (RF) classifier was used for the classification objective. Ultimately, a stacked version of RBMs is called Deep Belief Networks (DBNs) [ 40 ]. Each RBM model performs non-linear transformation which will again be the input for the next RBM model; performing this process progressively gives the network a lot of flexibility while expansion.

DBNs are generative models, which allow them to be used as a supervised or unsupervised settings. The feature learning is done through an unsupervised manner by doing the layer-by-layer pre-training. For the classification task, a backpropagation (gradient descent) through the RBM stacks is done for fine-tuning on the labeled dataset. In medical imaging applications, DBNs were used widely; for example, Khatami et al. [ 41 ] used this model for classification of X-ray images of anatomic regions and orientations; in [ 42 ], AVN Reddy et al. have proposed a hybrid deep belief networks (DBN) for glioblastoma tumor classification from MRI images. Another significant application of DBNs was reported in [ 43 ] where they have used a novel DBNs’ framework for medical images’ fusion.

2.2.4 Self-supervised learning

Self-supervised learning is basically a subtype of unsupervised Learning, by which it learns features’ representations using a proxy task where the data contain supervisory signals. After representation learning, it is fine-tuned using annotated data. The benefit of self-supervised learning is that it eliminates the need of humans to label the data, where this system extracts the visibly natural relevant context from the data and assign metadata with the representations as supervisory signals. This system matches with unsupervised learning, because both systems learn representations without using explicitly provided labels, but the difference is that self-supervised learning does not learn inherent structure of data and it is not centered around clustering, anomaly detection, dimensionality reduction, and density estimation. The genesis model of this system can retrieve the original image from a distorted image (e.g., non-linear gray-value transformation, image inpainting, image out-painting, and pixels shuffle) using proxy task [ 44 ]. Zhu et al. [ 45 ] have used self-supervised learning and its proxy task to solve Rubik’s cube which mainly contain three operations (rotating, masking, and ordering) the robustness of this model comes from that the network is robust to noise and it learns features that are invariant to rotation and translation. Shekoofeh et al. [ 46 ] have exploited the effectiveness of self-supervised learning in pre-training strategy used to classify medical images for tow tasks (dermatology skin condition classification, and multi-label chest X-ray classification). Their study has improved the classification accuracy after using two self-supervised learning systems: the first one is trained on ImageNet dataset and the second one is trained on unlabeled domain specific medical images.

2.2.5 Semi-supervised learning

Semi-supervised learning is a system by which it stands in between supervised learning and unsupervised learning systems, because for example, it is used for classification task (supervised learning) but without having all the data labeled (Unsupervised learning). Thus, this system is trained on small, labeled dataset, and then generates pseudo-labels to get larger dataset with labels, and the final model is trained by mixing up both the original dataset and the generated one of images. Nie et al. [ 47 ] have proposed semi-supervised learning-based deep network for image segmentation, the proposed method trains adversarially a segmentation model, from the confidence map is computed, and the semi-supervised learning strategy is used to generate labeled data. Another application of semi-supervised learning is used for cardiac MRI segmentation, [ 48 ]. Liu et al. [ 49 ] have presented a novel relation-driven semi-supervised model to classify medical images, they have introduced a novel Sample Relation Consistency (SRC) paradigm to use unlabeled data by generalizing and modeling the relationship information between different samples; in their experiment, they have applied the novel method on two benchmark medical images for classification, skin lesion diagnosis from ISIC 2018 challenge, and thorax disease classification from the publicly dataset ChestX-ray14, and the results have achieved the state-of-the-art criteria.

2.2.6 Weakly (partially) supervised learning

Weak supervision is basically a branch of machine learning used to label unlabeled data by exploiting noisy, limited sources to provide supervision signal that is responsible of labeling large amount of training data using supervised manner. In general, the new labeled data in “weakly-supervised learning” are imperfect, but it can be used to create a robust predictive model. The weakly supervised method uses image-level annotations and weak annotations (e.g., dots and scribbles) [ 50 ]. Weakly supervised multi-label disease system was used for classification task of chest X-ray [ 51 ], Also, it is used for multi-organ segmentation, [ 52 ] by learning single multi-class network from a combination of multiple datasets, where each one of these datasets contains partially organ labeled data and low sample size. Roth et al. [ 53 ] have used weakly supervised learning system for medical image segmentation and their results has speeded up the process of generating new training dataset used for the development purpose of deep learning in medical images analysis. Schleg et al. [ 54 ] have used this type of deep learning approach to detect abnormal regions from test images. Hu et al. [ 55 ] proposed an end-to-end CNN approach for displacement field prediction to align multiple labeled corresponding structures, and the proposed work was used for medical image registration of prostate cancer from T2-weighted MRI and 3D transrectal ultrasound images; the results reached 0.87 of Mean Dice score. Another application is applied in diabetic retinopathy detection in a retinal image dataset [ 56 ].

2.2.7 Reinforcement learning

Reinforcement learning (RL) is subtype of deep learning by which it takes the beneficial action toward maximizing the rewards of specific situation. The main difference between supervised learning and reinforcement learning is that in the first one, the training data have the answer within it, but in case of reinforcement learning, the agent decides how to act with the task where in the absence of the training dataset the model learn from its experience. Al Walid et al. [ 57 ] have used reinforcement learning for landmark localization in 3D medical images; they have introduced the partial policy-based RL, by learning optimal policy of smaller partial domains; in this paper, the proposed method was used on three different localization task in 3D-CT scans and MR images and proved that learning the optimal behavior requires significantly smaller number of trials. Also in [ 58 ], RL was used for object detection PET images. RL was also used for color image classification on neuromorphic system [ 59 ].

2.2.7.1 Transfer learning

Transfer learning is one of the powerful enablers of deep learning [ 60 ], which involves training a deep leaning model by re-using of a an already trained model with related or un-related large dataset. It is known that medical data face the problem of lacking and insufficient for training deep learning models perfectly, so Transfer learning can provide the CNN models with large learned features from non-medical images which in turn can be useful for this case [ 61 ]. Furthermore, Transfer Learning is a key feature for time-consuming problem while training a deep neural network, because it uses the freeze weights and hyperparameters of another model. In usual using transfer learning the weights which is already trained on different data (images) are freezed to be used for another CNN model, and only in the few last layers, modifications are done and these few last layers are trained on the real data for tuning the hyperparameters and weights. For these reasons, transfer learning was widely used in medical imaging, for example a classification of the interstitial lung disease [ 61 ] and detecting the thoraco-abdominal lymph nodes from CT scans; it was found that transfer learning is efficient, even though the disparity between the medical images and natural images. Transfer learning as well could be used for different CNN models (e.g., VGG-16, Resnet-50, and Inception-V3), Xue et al. [ 62 ], have developed transfer learning-based model for these models, and furthermore, they have proposed an Ensembled Transfer Learning (ETL) framework for classification enhancement of cervical histopathological images. Overall, in many computer vision tasks, tuning the last classification layers (fully connected layers) which is called “shallow tuning” is probably efficient, but in medical imaging, a deep tuning for more layers is needed [ 63 ], where they have studied the benefit of using transfer learning in four applications within three imaging modalities (polyp detection from colonoscopy videos, segmentation of the layers of carotid artery wall from ultrasound scans, and colonoscopy video frame classification), their study results found that training more CNN layers on the medical images is efficient more than training from the scratch.

2.3 Best deep learning models and practices

Convolutional Neural Networks (CNNs) based models are usually used in different ways with keeping in minds that CNNs remains the heart of any model; in general, CNN could be trained on the available dataset from the scratch when the available dataset is very large to perform a specific task (e.g., segmentation, classification, detection, etc.), or a pre-trained model with a large dataset (e.g., ImageNet) where this model could be used to train new datasets (e.g., CT scans) with fine-tuning some layers only; this approach is called transfer learning (TL) [ 60 ]. Moreover, CNN models could be used for feature extraction only from the input images with more representation power before proceeding to the next stage of processing these features. In the literature, there were commonly used CNN models which has proven their effectiveness, and based on these models, some developments have arisen; we will mention the most efficient and used models of deep learning in medical images analysis. First, it was AlexNet which was introduced by Alex Krizhevsky [ 64 ] and Siyuan Lu et al. [ 65 ], have used transfer learning with a pre-trained AlexNet with replacing the parameters of the last three layers with a random parameters for pathological brain detection. Another frequently used model is Visual Geometry Group (VGG-16) [ 66 ] where 16 refers to the number of layers; later on, some developments were proposed for VGG-16 like VGG-19; in [ 67 ], they have listed medical imaging applications using different VGGNet architectures. Inception Network [ 68 ] is one of the most common CNN architectures which aim to limit the resources consumption. And further modifications on this basic network were reported with new versions of it [ 69 ]. Gao et al. [ 70 ] have proposed a new architecture of Residual Inception Encoder–Decoder Neural Network (RIEDNet) for medical images synthesis. Later on, Inception network was called Google Net [ 71 ]. ResNet [ 72 ] is a powerful architecture for very deep architectures sometimes over than 100 layers, and it helps in limiting the loss of gradient in the deeper layers, because it adds residual connections between some convolutional layers Fig.  4 . Some of ResNet models in medical imaging are mostly used for robust classification [ 73 , 74 ], for pulmonary nodes and intracranial hemorrhage.

The basic models used in medical imaging: A ResNet architecture, B U-Net architecture [ 75 ], C CNN AlexNet architecture for breast cancer [ 76 ], and D Dense Net architecture [ 77 ]

DenseNet exploits same aspect of residual CNN (ResNet) but in a compact mode for achieving good representations and feature extraction. Each layer of the network has in its input outputs from the previous layers, so comparing to a traditional CNN, DenseNet contains more connections (L) than CNN (L connections) where DenseNet has [ L ( L  − 1)]/2 connections. DenseNet is widely used with medical images, Mahmood et al . [ 78 ] have proposed a Multimodal DenseNet for fusing multimodal data to give the model the flexibility of combining information from multiple resources, and they have used this novel model for polyp characterization and landmark identification in endoscopy. Another application used transfer learning with DenseNet for fundus medical images [ 79 ].

U-net [ 80 ] is one of the most popular network architectures used mostly for segmentation, Fig.  4 . The reason behind it is mostly used in medical images is that because it is able to localize and highlight the borders between classes (e.g., brain normal tissues and malignant tissues) by doing the classification for each pixel. It is called U-net, because the network architecture takes the shape of U alphabet and it contains concatenation connections; Fig.  4 shows the basic structure of the U-Net. Some developments of U-Net were U-Net +  + [ 75 ], have proposed a new architecture U-Net +  + for medical image segmentation, and in their experiments, U_Net +  + has outperformed both U-Net and wide U-Net architectures for multiple medical image segmentation tasks, such as liver segmentation from CT scans, polyp segmentation in colonoscopy videos, and nuclei segmentation from microscopy images. From these popular and basic DL models, some other models were inspired and even some of these models were inspired and rely on the insights from others (e.g., inception and ResNet); Fig.  5 shows the timeline of the mentioned models and other popular models too.

Timeline of mostly used DL models in medical imaging

3 Deep learning applications in medical imaging

For the purpose of studying the most applications of deep learning in medical imaging, we have organized a study based on the most-cited papers found in literature from 2015 to 2021; the number of surveyed literatures for segmentation, detection, classification, registration, and characterization are: 30, 20, 30, 10, and 10, respectively. Figure  6 shows the pie chart of these applications.

Surveyed DL applications in medical imaging

3.1 Image segmentation

Deep learning is used to segment different body structures from different imaging modalities such as, MRI, CT scans, PET, and ultrasound images. Segmentation means portioning an image into different segments where usually these segments belongs to specific classes (tissue classes, organ, or biological structure) [ 81 ]. In general overview, for CNN models, there are two main approaches for segmenting a medical image; the first is using the entire image as an input and the second is using patches from the image. Segmentation process of Liver tumor using CNN architecture is shown in Fig.  7 according to Li et al., and both the methods work well in generating an output map which provides the segmented output image. Segmentation is potential for surgical planning and determining the exact boundaries of sub-regions (e.g., tumor tissues) for better guidance during the direct surgery resection. Most likely segmentation is common in neuroimaging field and with brain segmentation more than other organs in the body. Akkus et al. [ 82 ] have reviewed different DL models for segmentation of different organs with their datasets. Since CNN architecture can handle both 2-dimensional and 3-dimensional images, it is considered suitable for MRI which is in 3D scheme; Milleteria et al. [ 83 ] have used 3D MRI images and applier 3D-CNN for segmenting prostate images. They have proposed new CNN architecture which is V-Net which relies on the insights of U-Net [ 80 ] and their output results have achieved 0.869 dice similarity coefficient score; this is considered as efficient model regarding to the small dataset (50 MRI for training and 30 MRI for testing). Havaei et al. [ 84 ] have worked on Glioma segmentation from BRATS-2013 with 2D-CNN model and this model took only 3 min to run. From clinical point of view, segmentation of organs is used for calculating clinical parameters (e.g., volume) and improving the performance of Computer-Aided Detection (CAD) to define the regions accurately. Taghanaki et al. [ 85 ] have listed the segmentation challenges from 2007 to 2020 with different imaging modalities; Fig.  8 shows the number of these challenges. We have summarized Deep Learning models for segmentation for different organs in the body, based on the highly cited paper and variations in deep learning models shown in Table 1

Liver tumor segmentation using CNN architecture [ 86 ]

The number of challenges related to segmentation in medical imaging from 2007 to 2020 listed on Grand Challenges regarding the imaging modalities

3.2 Image detection/localization

Detection simply means is to identify a specific region of interest in an image and finally to draw a bounding box around it. Localization is just another terminology of detection which means to determine the location of a particular structure in images. In deep learning for medical images, analysis detection is referred as Computer-Aided Detection (CAD), Fig.  9 . CAD is divided commonly for anatomical structure detection or for lesions (abnormalities) detection. Anatomical structure detection is a crucial task in medical images analysis due to determining the locations of organs substructures and landmarks which in turn guide for better organ segmentation and radiotherapy planning for analysis and further surgical purposes. Deep learning for organ or lesion detection can be either classification-based or regression-based methods; the first one is used for discriminating body parts, while the second method is used for determining more detailed locations information. In fact, most of the deep learning pathologies are connected; for example, Yang et al. [ 114 ] have proposed a custom CNN classifier for locating landmarks which is the initialization steps for the femur bone segmentation. In case of lesion detection which is considered to be clinically time-consuming for the radiologists and physicians and it may lead to errors due to the lack of data needed to find the abnormalities and also to the visual similarity of the normal and abnormal tissues in some cases (e.g., low contrast lesions in mammography). Thus, the potential of CAD systems comes from overcoming these cons, where it reduces the times needed, computational cost, providing alternative way for the people who live in areas that lacks specialists and improve the efficiency of thereby streamlining in the clinical workflow. Some CNN custom models were developed specifically for lesion detection [ 115 , 116 ]. Both organ anatomical structures and lesion detection are applicable for mostly all body’s organs (e.g., Brain, Eye, Chest, Abdominal, etc.), and CNN architectures are used for both 2D and 3D medical images. When using 3D volumes like MRI, it is better to use patches fashion, because it is more efficient than sliding window fashion, so in this way, the whole CNN architecture will be trained using patches before the fully connected layer, [ 117 ]. Table 2 shows top-cited papers with different deep learning models for both structure and lesion detection within different organs.

Lesion detection algorithm flowchart [ 118 ]

3.3 Image classification

This task is the fundamental task for the computer-aided diagnosis (CAD), and it aims to discover the presence of disease indicators. Commonly in medical images, the deep learning classification model’s output is a number that represents the disease presence or absence. A subtype of classification is called lesion classification and is used in a segmented images from the body [ 136 ]. Traditionally, classification used to rely on the color, shape, and texture, etc. but in medical images, features are more complicated to be categorized as these low-level features which lead to poor model generalization due to the high-level features for medical image. Recently, deep learning has provided an efficient way of building an n end-to-end model which produce classification labels-based from different medical images’ modalities. Because of the high resolution of medical images, expensive computational costs arise and limitations in the number of deep model layers and channels; Lai Zhifei et al. [ 137 ] have proposed the Coding Network with Multilayer Perceptron (CNMP) to overcome these problems by combining high-level features extracted by CNN and other manually selected common features. Xiao et al. [ 138 ] have used parallel attention module (PAM-DenseNet) for COVID-10 diagnosis, and their model can learn strong features automatically from channel-wise and spatial-wise which help in making the network to automatically detect the infected areas in CT scans of lungs without the need of manual delineation. As any deep learning application, classification task is performed on different body organs for detecting diseases’ patterns. Back in 1995, a CNN model was developed for detecting lung nodules from X-ray of chest [ 139 ]; classifying medical images is essential part for clinical aiding and further treatments, for example detecting and classifying pneumonia presence from chest X-ray scans [ 140 ]; CNN-based models have introduced various stratifies to better the classification performance especially when using small datasets, for example data augmentation [ 141 , 142 ]. GANs’ network was widely used for data augmentation and image synthesis [ 143 ]. Another robust strategy is transfer learning [ 61 ]. Rajpurkar et al. have used custom DenseNet for classifying 14 different diseases using chest X-ray from the chestXray14 dataset [ 129 ]. Li et al. have used 3D-CNN for interpolating the missing pixels data between MRI and PET modalities, where they have reconstructed PET images from MRI images from the (ADNI) dataset of Alzheimer disease which contain MRI and PET images [ 144 ]. Xiuli Bi et al. [ 31 ] have also worked on Alzheimer disease diagnosing using a CNN architecture for feature extraction and unsupervised predictor for the final diagnosis results on (ADNI-1 1.5 T) dataset and achieved accuracy of 97.01% for AD vs. MCI, and 92.6% for MCI vs. NC. Another 3D-CNN architecture employed in an autoencoder architecture is also used to classify Alzheimer disease using transfer learning on a pre-trained CAD Dementia dataset, they have reported accuracy of 99% on the publicly dataset ADNI, and the fine-tuning process is done in a supervised manner [ 145 ]. Diabetic Retinopathy (DR) could be diagnosed using fundus photographs of the eye, Abramoff et al . [ 146 ] have used custom CNN inspired from Alexnet and VGGNet to train a device (IDx-DR) version X2.1 on a dataset of 1.2 million DR images to record 0.98 AUC score. Figure  10 shows the classification of medical images. A few notable results found in literature are summarized in Table 3 .

Classification of brain tumor using general CNN architecture

3.4 Image registration

Image registration means to allow images’ spatial alignment to a common anatomical field. Previously, image registration was done manually by clinical experts, but after deep learning, image registration has changed [ 176 , 177 , 178 ]. Practically, this task is considered main scheme in medical images, and it relies on aligning and establishing accurate anatomical correspondences between a source image and target image using transformations. In the main theme of image registration, both handcrafted and selected features are employed in a supervised manner. Wu et al. [ 179 , 180 ] have employed unsupervised deep learning approach for learning the basis filters which in turn represent image’s patches and detect the correspondence detection for image registration. Yang et al. [ 177 ] have used an autoencoder architecture for predicting of deformation diffeomorphic metrics mapping (LDDMM) to get fast deformable image registration and the results shows improvements in computational time. Commonly, image registration is employed for spinal surgery or neurosurgery in form of localization of spinal bony or tumor landmarks to facilitate the spinal screw implant or tumor removal operation. Miao et al. [ 181 ] have trained a customized CNN on X-ray images to register 3D models of hand implant and knee implant onto 2D X-ray images for pose estimation. An overview of registration operation is shown in Table 4 , which shows a summary of medical images registration as an application of deep learning.

3.5 Image characterization

Characterization of a disease within deep learning is a stage of computer-aided diagnosis (CADx) systems. For example, radiomics is an expansion of CAD systems for other tasks such as prognosis, staging, and cancer subtypes’ determination. In fact, characterization of a disease will rely on the disease type in the first place and on the clinical questions related to it. There is two ways used for features extraction, either handcrafted features extraction or deep learned features, in the first, radiomic features is similar to radiologist’s way of interpretation and analysis of medical images. These features might include tumor size, texture, and shape. In literature, the handcrafted features are used for many purposes, such as tumor aggressiveness, the probability of having cancer in the future, and the malignancy probability [ 190 , 191 ]. There are two main categories for characterization, lesion characterization and tissue characterization. In deep learning applications of medical imaging, each computerized medical image requires some normalization plus customization to be handled and suited to the task and image modality. Conventional CAD is used for lesion characterization. For example, to track the growth of lung nodules, the characterization task is needed for the nodules and the change of lung nodules over time, and this will help of reducing the false-positive of lung cancer diagnosis. Another example of tumor characterization is found in imaging genomics, where the radiomic features are used as phenotypes for associative analysis with genomics and histopathology. A good report which was done with multi-institutes’ collaboration about breast phenotype group through TGCA/TCIA [ 192 , 193 , 194 ]. Tissue characterization is to examine when particular tumor areas are not relevant. The main focus in this type of characterization is on the healthy tissues that are susceptible for future disease; also focusing on the diffuse disease such as interstitial lung disease and liver disease [ 195 ]. Deep learning has used conventional texture analysis for lung tissue. The characterization of lung pattern using patches can be informative of the disease which commonly is interpreted by radiologists. Many researchers have employed DL models with different CNN architectures for interstitial lung disease classification characterized by lung tissue sores [ 149 , 196 ]. CADx is not only a detection/localization task only, but it is classification and characterization task as well. Finding the likelihood of disease subtyping is the output of a DL model and characteristic features’ presentation of a disease too. For the characterization task, especially with limited dataset, CNN models are not trained from scratch in general, data augmentation is an essential tool for this application, and performing CNN on dynamic contrast-enhanced MRI is important too. For example, while using VGG-19-Net, researchers have used DCE-MRI temporal images with pre-contrast, first post-contrast, and the second post-contrast MR images as an input to the RGB channels. Antropova et al. [ 197 ] have used the maximum intensity projections (MIP) as an input to their CNN model. Table 5 shows some highlighted literature of characterization which includes diagnosis and prognosis.

3.6 Prognosis and staging

Prognosis and staging refer to the future prediction of a disease status for example after cancer identification, further treatment process through biopsies which give a track on the stage, molecular type, and genomics which finally provides information about prognosis and the further treatment process and options. Since most of the cancers are spatially heterogeneous, specialists and radiologists are interested about the information on spatial variations that medical imaging can provide. Mostly, many imaging biomarkers include only the size and another simple enhancement procedures; therefore, the current investigators are more interested in including radiomic features and extending the knowledge from medical images. Some deep learning analysis have been investigated in cancerous tumors for prognosis and staging [ 192 , 206 ]. The goal of prognosis is to analyze the medical images (MRI or ultrasound) of cancer and get the better presentation of it by gaining the prognostic biomarkers from the phenotypes of the image (e.g., size, margin morphology, texture, shape, kinetics, and variance kinetics). For example, Li et al. [ 192 ] found that texture phenotype enhancement can characterize the tumor pattern from MRI, which lead to prediction of the molecular classification of the breast cancers; in other words, the computer-extracted phenotypes provide promises regarding the quality of the breast cancer subtypes’ discrimination which leads to distinct quantitative prediction in terms of the precise medicine. Moreover, with the enhancement of the texture entropy, the vascular uptake pattern related to the tumor became heterogeneous which in turn reflects the heterogeneous temperament of the angiogenesis and the treatment process applicability and this is termed as the virtual digital biopsy location based. Gonzalez et al. [ 204 ] have applied DL on thoracic CT scans for prediction of staging of chronic obstructive pulmonary disease (COPD). Hidenori et al. [ 207 ] have used CNN model for grading diabetic retinopathy and determining the treatment and prognosis which involves a non-typically visualized on fundoscopy of retinal area; their novel AI system suggests treatment and determines prognoses.

Another term related to staging and prognosis is survival prediction and disease outcome, Skrede et al. [ 208 ] have performed DL using a large dataset over 12 million pathology images to predict the survival outcome for colorectal cancer in its early stages, a common evaluation metric is Hazard function which indicate the risk measures of a patient after treatment, and their results yield a hazard ration of 3.84 for poor against good prognosis in the validation set cohort of 1122 patients, and a hazard ratio of 3.04 after adjusting for prognostic markers which contain T and N stages. Sillard et al. [ 209 ] used deep learning for predicting survival outcomes after hepatocellular carcinoma resection.

3.7 Medical imaging in COVID-19

Basically, after COVID-19 has been identified in 31 December 2019 [ 210 ] and it is based on polymerase chain reaction (PCR) test. However, it was found that it can be analyzed and diagnosed through medical imaging, even though most radiologists’ societies do not recommend it, because it has similar features of various pneumonia diseases. Simpson et al. [ 211 ], have prospected a potential use of CT scans for clinical managing, and eventually, they have proposed four standard categories for reporting COVID-19 languages. Mahmood et al. [ 212 ] have studied 12,270 patients and recommend to be subjected for CT screening for early detection of COVID-19 to limit the speedy spread of the disease. Another approach for classification of COVID-19 is using portable (PCXR) which uses chest X-ray scans instead of the expensive CT scans; furthermore, this has the potential of minimizing the chances of spreading the virus. For the identification of COVID-19, Pereira et al. [ 152 ] have flowed using chest X-ray scans using the portable manner. For the comparison of different screening methods, it was suggested by Soldati et al. [ 213 ], which stated the Lung Ultrasound (LUS) is needed to be compared with chest X-ray and CT scans to help designing better diagnostic system to be suitable for the technological resources. COVID-19 has gained the attention of deep learning researchers who have employed different DL models for the main pathologies for diagnosing this disease using different medical imaging modalities from different datasets. Starting with segmentation, a new proposed system for screening coronavirus disease was done by Butt et al. [ 214 ], who have employed 3D-CNN architecture for segmenting multiple volumes of CT scans; a classification step is included to categorize patches into COVID-19 from other pneumonia diseases, such as influenza and a viral pneumonia. After that, Bayesian function is used to calculate the final analysis report. Wang et al. [ 215 ] have performed their CNN model on chest X-ray images, for extracting the feature map, classification, regression, and finally the needed mask for segmentation. Another DL model using chest X-ray scans was introduced by Murphy et al. [ 108 ], using U-Net architecture for detecting of tuberculosis and finally classifying images, with AUC of 0.81. For the detection of COVID-19, Li et al. [ 124 ] have developed a new tool of deep learning to detect COVID-19 from CT scans; the main work consists of few steps starting from extracting the lungs as ROI using U-Net, then generating features using ResNet-50, and finally using fully connected layer for generating the probability score of COVID-19 and the final results have reported AUC of 0.96. Another COVID-19 detection system from X-rays and CT scans was proposed by Kassani et al. [ 126 ], who have used multiple models for their strategy, DenseNet 121 have achieved accuracy of 99%, and REsNet achieved accuracy of 98% after being trained by LightGBM, and also, they have used other backbone models such as MobileNet, Xception, and Inception-ResNet-V2,NASNe, and VGG-Net. For classification of COVID-19, Wu et al. [ 150 ] have used the fusion of DL networks, starting from segmenting lung regions using threshold-based method using CT scans, next using ResNet-50 to extract the features map which further is fed to fully connected layer to record AUC of 0.732 and 70% accuracy. Ardakani et al. [ 216 ] have compared ten DL models for classification of COVID-19, including AlexNet, VGG-16, VGG-19, GoogleNet, MobileNet, Xception, ResNet-101, ResNet-18, ResNet-50, and SqueezNet. Where ResNet-101 has recorded the best results regarding sensitivity. A few used deep learning themes that have been used for different applications of COVID-19 are listed in Tables 1 , 2 , 3 .

4 Deep learning schemes

4.1 data augmentation.

It was clearly that deep learning approach performs better than the traditional machine learning and shallow learning methods and other handcrafted feature extraction from images, because deep learning models learn image descriptors automatically for analysis. It is commonly possible to combine deep learning approach with the knowledge learned from the handcrafted features for analyzing medical images [ 153 , 200 , 217 ]. The main key feature of deep learning is the large-scale datasets which contain images from thousands of patients. Although some vast data of clinical images, reports, and annotations are recorded and stored digitally in many hospitals for example, Picture Archiving and Communication systems (PACS) and Oncology Information System (OIS), in practice, these kinds of large-scale datasets with semantic labels are an efficiency measure for deep learning models used in medical imaging analysis. As it is known that medical images face the lack of dataset, data augmentation has been used to create new samples either depending on the existing samples or using generative models to generate new images. The new augmented samples are emerged with the original samples; thus, the size of the dataset is increased with the variation in the data points. Data augmentation is used by default with deep learning due to its added efficiency, since it reduces the chance of overfitting and it eliminates the imbalanced issue while using multi-class datasets, because it increases the number of the training samples and this also helps in generalizing the models and enhance the testing results. The basic data augmentation techniques are simple and it was widely adopted in medical imaging, such as cropping, rotating, flipping, shearing, scaling, and translation of images [ 80 , 218 , 219 ]. Pezeshk et al. [ 220 ] have proposed mixing tool which can seamlessly merge a lesion patch into a CT scan or mammography modality, so the merged lesion patches can be augmented using the basic transformations and inserted to the lesion shape and characteristics.

Zhang et al. [ 221 ] have used DCNN for extracting features and obtaining image representations and similarity matrix too, their proposed data augmentation method is called unified learning of features representation, their model was trained on seed-labeled dataset, and authors intended to classify colonoscopy and upper endoscopy medical images. The second method to tackle limited datasets is to synthesize medical data using an object model or physics principles of image formation and using generative models schemes to serve as applicable medical examples and therefore increase the performance of any deep learning task at hand. The most used model for synthesizing medical data is Generative Adversarial Networks (GANs); for example [ 143 ], GANs were used to generate lesion samples which increase CNN performance while the classification task of liver lesions. Yang et al. [ 222 ] used Radon Transform for objects with different modeled conditions by adding noise to the data for synthesizing CT dataset and the trained CNN model does the estimation of high-dose projection from low-dose. Synthesizing medical images is used for different purposes; for example, Chen et al. [ 223 ] have generating training data for noise reduction for reconstructed CT scans by applying deep learning algorithm by synthesizing noisy projections from patient images. While, CUI et al. [ 224 ] have used simulated dynamic PET data and used stacked sparse autoencoders for dynamic PET reconstruction framework.

4.2 Datasets

Deep learning models are famous to be dataset hungry, and the good quality of dataset has been always the key-parameter for deep learning for learning computational models and provide trusted results. The task of deep learning models is more potential when handling medical data because the accuracy is highly needed, recently many publicly available datasets have been released online for evaluating the new developed DL models. Commonly, there are different repositories which provide useful compilations of the public datasets (e.g., Github, Kaggle, and other webpages). Comparing to the datasets for general computer vision tasks (thousands to million annotated images), medical imaging datasets are considered to be too small. According to the Conference on Machine Intelligence in Medical Imaging (C-MIMI) that was held in 2016 [ 225 ], ML and DL are starving for large-scale annotated datasets, and the most common regularities and specifications (e.g., sample size, cataloging and discovery, pixel data, metadata, and post-processing) related to medical images datasets are mentioned in this white paper. Therefore, different trends in medical imaging community have started to adopt different approaches for generating and increasing the number of samples in dataset, such as generative models, data augmentation, and weakly supervised learning, to avoid overfitting on the small dataset and finally provide an end-to-end fashion reliable deep learning model. Martin et al. [ 226 ] have described the fundamental steps for preparing the medical imaging datasets for the usage of AI applications. Fig.  11 shows the flowchart of the process; moreover, they have listed the current limitations and problem of data availability of such datasets. Examples of popular used databases for medical images analysis which exploit deep learning were listed in [ 227 ]. In this paper, we provide the typically mostly used datasets in the literature of medical imaging which are exploited by deep learning approaches in Table 6 .

Flowchart of medical images data handling

4.3 Feature’s extraction and selection

Feature extraction is the tool of converting training data and trying to establish as maximum features as possible to make deep learning algorithms much efficient and adequate. There are some common algorithms used for medical image features’ extractors, such as Gray-Level-Run-Length-Matrix (GLRM), Local Binary Patterns (LBP), Local Tetra Patterns (LTrP), Completed Local Binary Patterns (CLBP), and Gray-Level-Co-Occurrence Matrix (GLCM); these techniques are used in the first place before applying the main DL algorithm for different medical imaging tasks.

GLCM : is a common used feature extractor by which it searches for the textural patterns and their nature within gray-level gradients [ 234 ]. The main extracted features through this technique are autocorrelation, contrast, Dissimilarity, correlation, cluster prominence, energy, homogeneity, variance, entropy, difference variance, sum variance, cluster shade, sum entropy, information measure of correlation.

LBP : is another famous feature extractor which uses the locally regional statistical features [ 235 ]. The main theme of this technique is to select a central pixel and the rest pixels along a circle are taken to be binary encoded as 0 if their values are less than the central pixel, and 1 for the pixels which have values greater than the central pixel. In histogram statistics, these binary codes are encoded to decimal numbers.

Gray-Level Run Length Matrix (GLRLM ): this method removes the higher order statistical texture data. In case of the maximum gray dimensions G, the image is repeatedly re-quantizing to aggregate the network. The mathematical formula of GLRLM is given as follows:

where ( u , v ) refers to the sizes of the array values, N r refers to the maximum gray-level values, and K max is the more length.

Raj et al. [ 236 ] have used both GLCM and GLRLM as the main features’ extraction techniques for extracting the optimal features from the pre-processed medical images, which further the optimal features have improved the final results of classification task. Figure  12 shows the features extraction and selection types used for dimensionality reduction.

4.3.1 Feature selection techniques

Analysis of Variance—ANOVA: is a statistical model by which it evaluates and compares two or more experiments averages. The idea behind this model is that the difference between means are substantial to evaluate the performance of two estimates [ 237 ]. Surendiran et al. [ 238 ] have used the stepwise ANOVA Discriminant Analysis (DA) for mammogram masses’ classification.

The basic steps of performing ANOVA on data distribution are

Defining Hypothesis:

Calculating the sum of squares

It is used to determine the dispersion from datapoints and it can be written as

ANOVA performs F test to compare the variance difference between groups and within groups. And this can be done using total sum of squares which is defined as the distance between each point from the grand mean x -bar.

Determining the degree of freedom

Calculating F value

Acceptance or rejection of null hypothesis

Principal Component Analysis (PCA): is considered as the most used tool for extracting structural features from potentially high-dimensional datasets. It extracts the eigenvectors ( q ) which are connected to ( q ) largest eigenvalues from an input distribution. PCA results develop new features that are independent of another. The main goal of PCA is to apply linear transformation for obtaining a new set of samples, so that the components of y are un-correlated, [ 239 ]. The linear transform is given as follows:

where x is the input element vector ∈  R I , after that the PCA algorithm will choose the most significant components ( y ), and the main steps to do this are summarized as follows:

Standardize and normalization of the datapoints: after calculating the mean and standard deviation of the input distribution

Calculating the covariance matrix from the input datapoints:

From the covariance matrix extract the eigenvalues:

Choosing k eigenvectors with the highest eigenvalues by sorting the eigenvalues and eigenvectors, k refers to the number of dimensions in the dataset

Another major feature of PCA algorithm is used for feature dimensionality reduction.

In medical imaging, PCA was used mostly for dimensionality reduction, Wu et al. [ 240 ] have used PCA-based nearest neighbor for estimation of local structure distribution and extracted the entire connected tree, and in their results over retinal fundus data, they have achieved state-of-the-art results by producing more information regarding the tree structure.

PCA was also used as a data augmentation process before training the discriminative CNN for different medical imaging tasks; for capturing the important characteristics of natural images, different algorithms were compared to perform data augmentation [ 241 ] (Fig.  12 ).

Features’ extraction and selection types used for dimensionality reduction

4.4 Evaluation metrics

For the purpose of evaluating and measuring the performance of deep learning models while validating medical images, different evaluation metrics are used according to some specific regularities and criteria. For example, some particular evaluation metrics are used with specific tasks like Dice score and F 1-score are mostly used for segmentation, while accuracy and sensitivity are mostly used for classification task. Here, we will focus on the most used performance measurement metrics in the literature and will cover the metrics mentioned in our tables of comparison.

The Dice coefficient is the most used metric for segmentation task for validating the medical images. It is common also to use dice score to measure reproducibility [ 242 ]. The general formula to calculate the Dice coefficient is

Jaccard-index (similarity coefficient) [JAC]:

Jaccard-index is a statistic metric used for finding the similarities between sample-sets. It is defined as the ratio between the size of intersection and the size of union of the sample-set, and the mathematical formula is given by

From the formula above, we note that the JAC-index is always greater than the dice score and the relation between the two metrics is defined by

True-Positive Rate (TPR):

Also is called as Sensitivity and Recall, is used to maximize the prediction of a particular class and it measures the portion of the positive voxels from the ground truth which also were identified as positive when performing segmentation process. It is given by the formula

True-Negative Rate (TNR):

Also called specificity, it measures the number of negative voxels (background) from the ground truth which are also identified to be negative after the segmentation process, and it is given by the formula

However, both TNR and TPR metrics are not used commonly for medical images’ segmentation due to their sensibility to the segments size.

Accuracy [ACC]

Accuracy means exactly how good the DL model at guessing the right labels (ground truth). Accuracy is commonly used to validate the classification task and it is given using the formula

It is used to get the best precision and recall together; thus, the F 1-score is called the harmonic mean of precision and recall values; it is given by the formula

The predictive accuracy of a classification model is related to the F 1-score; when F 1-score is higher means, we have better classification accuracy.

F-beta score:

It is a combination of advantages of precision and recall metrics when both the False-Negative (FN) and False-Positive (FP) have equal importance. F -beta-score is given using the same formula for F 1-score by altering its formula a bit by including an adjustable parameter (beta), and the formula became

This evaluation metric measures the effectiveness of a DL model according to a user who attaches beta times.

Receiver-Operating Characteristics Curve (ROC) is a graph between True-Positive Rate (TPR) (sensitivity) and False-Positive Rate (FPR) (1- specificity) by which it shows the performance of classification model, and the plot is characterized at different classification thresholds. The biggest advantage of ROC curve is that its independency of the change in number of responders and response rate.

AUC is the area under curve of ROC, and it measures the 2D area under the ROC curve which in turn means the integral of the ROC curve from 0 to the AUC measures the aggregate performance of classification at all the possible thresholds. One way to understand the AUC is as the probability that a model classifies random positive samples more than random negative samples. The ROC curve is shown in Fig.  13 .

ROC and AUC graph

5 Discussion and conclusion

5.1 technical challenges.

In this overview paper, we have presented a review of the previous literature of deep learning applications in medical imaging. It contributed three main sections: first, we have presented the core of deep learning concepts considering the main highlights of understanding of basic frameworks in medical images analysis. The second section contains the main applications of deep learning in medical imaging (e.g., segmentation, detection, classification, and registration) and we have presented a comprehensive review of the literature. The criteria that we have built our overview consists of the mostly cited papers, the mostly recent (from 2015 to 2021), and the papers with better results. The third major part of this paper is focused on the deep learning themes regarding some challenges and the future directions of addressing those challenges. Besides focusing on the quality of the mostly recent works, we have highlighted the suitable solutions for different challenges in this field and the future directions that have been concluded from different scientific perspective. Medical imaging can get the benefit from other fields of deep learning, that have been encouraged from collaborative research works from computer vision communities, and furthermore, this collaboration is used to overcome the lack of medical dataset using transfer learning. Cho et al. [ 243 ] have answered the question of how much is the size of medical dataset needed to train a deep learning model. Creating synthetic medical images is another solution presented in deep learning using Variational Autoencoders (VAEs) and GANs for tackling the limited labeled medical data. For instance, Guibas et al. [ 244 ] have used 2 GANs for segmenting and then generating new retinal fundus images successfully. Another applications of GANs for segmentation and synthetic data generation were found [ 132 , 245 ].

Data or class imbalance [ 246 ] is considered a critical problem in medical imaging, and it refers to that medical images that are used for training are skewed toward non-pathological images; rare diseases have less number of training examples which cause the problem of imbalanced data which lead to incorrect results. Data augmentation represents good solution for this, because it increases the number of samples of the small classes. Away from dataset challenges strategies, there are algorithmic modification strategies which are used to improve DL models’ performance for data imbalance issue [ 247 ].

Another important non-technical challenge is the public reception of humans that the results are being analyzed using DL models (not human). In some papers in our report, DL models have outperformed specialists in medicine (e.g., dermatologists and radiologists) and mostly in image recognition tasks. Yet, a moral culpability may arise whenever a patient is mistakenly diagnosed or morbidity cases may arise too when using DL-based diagnostic, since the work of a DL algorithms is considered a black box. However, the continued development and evolving of DL models might take a major role in the medicine as it is involving in various facets of our life.

AI systems have started to emerge in hospitals from a clinical perspective. Bruijne [ 248 ] have presented five challenges facing the broader family of deep learning which is machine learning in medical imaging field including the data preprocessing of different modalities, interpretation of results to clinical practice, improving the access of medical data, and training the models with little training data. These challenges further have addressed the future directions of DL models improvement. Another solutions of small datasets were reported in [ 8 , 249 ].

DL models’ architectures were found not to be the only factor that provides quality results, where data augmentation and preprocessing techniques are also substantial tools for a robust and efficient system. The big question is that how to benefit from the results of DL models for the best of medical images analysis in the community.

Considering the historical developments of ML techniques in medical imaging gives us a future perspective how DL models will continue to improve in the same field. Accordingly, medical images quality and data annotations are crucial for proper analysis. A significant concept is the relevance between statistical sense and clinical sense, even though the statistical analysis are quiet important in research, but in this field, researchers should not lose the sight from clinical perspective; in other words, even when a good CNN models provides good answers from the statistical perspective, it does not mean that it will replace a radiologist even after using all the helping techniques like data augmentation and adding more layers to get better accuracies.

5.2 Future promises

After reviewing literature and the most competitive challenges that face deep leaning in medical imaging, we concluded that three aspects will probably carry the revolution of DL according to most of researchers, which are the availability of large-scale datasets, advances in deep learning algorithms, and the computational power for data processing and evaluation of DL models. Thus, most of the DL techniques are directed into the above aspects for alleviating the DL performance more; moreover, the need for investigations to improve data harmonization, standards developments which is needed for reporting and evaluation, and accessibility of larger annotated data such as the public datasets which lead to better independent benchmarks’ services. Some of the interesting applications in medical imaging was proposed by Nie et al. [ 250 ], by which they have used GANs to generate or CT scans from MRI images for brain; the benefit of such work will reduce the risk of patients being exposed to ionizing radiation from CT scanners, which also reserve patients’ safety. Another significant perspective relies on increasing the resolution and quality of medical images and also reduces the blurriness from CT scans and MRI images which means getting higher resolution with lower costs and better results, because it has lower field strength [ 251 ].

The new trends’ technology of deep learning approach is concerned about medical data collection. Wearable technologies are getting the interest of the new research which provide the benefits of flexibility, real-time monitoring of patients, and the immediate communication of the collected data. Whenever the data become available, Deep learning and AI will start to use the unsupervised data exploration, which in turn will provide better analysis power plus suggesting better treatments’ methodologies in healthcare. In summary, the new trends of AI in healthcare pass through stages; the quality of performance (QoP) related to deep learning will lead to standardization in terms of wearable technology which represent the next stage of healthcare applications and personalized treatment. Diagnosing and treatment depend on specialists, but with deep learning enabled, some small changes and signs in human body can be seen and early detection becomes possible which in turn will launch the treatment process of pre-stage of diseases. DL model optimization mainly focuses on the network architecture, while the standard term of optimization means the distribution and standardization with respect to other parts of DL (e.g., optimizers, loss functions, preprocessing and post-processing, etc.). In many cases to achieve better diagnosis, medical images are not sufficient, where another data are required to be combined (e.g., historical medical reports, genetic information, lab values, and other non-image data), though by linking and normalizing these data with medical images will lead to better diagnosis of diseases, more accurately through analyzing these data in higher dimensions.

LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521 (7553), 436–444 (2015). https://doi.org/10.1038/nature14539

Article   Google Scholar  

Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. IEEE Access 6 , 14410–14430 (2018). https://doi.org/10.1109/ACCESS.2018.2807385

Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Adv. Neural Inf. Process. Syst. 4 (January), 3104–3112 (2014)

Google Scholar  

Smith-Bindman, R., et al.: Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996–2010. JAMA 307 (22), 2400–2409 (2012). https://doi.org/10.1001/jama.2012.5960

Rubin, D.L.: Measuring and improving quality in radiology: meeting the challenge with informatics. Radiographics 31 (6), 1511–1527 (2011). https://doi.org/10.1148/rg.316105207

Recht, M.P., et al.: Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur. Radiol. 30 (6), 3576–3584 (2020). https://doi.org/10.1007/s00330-020-06672-5

Bosma, M., van Beuzekom, M., Vähänen, S., Visser, J., Koffeman, E.: The influence of edge effects on the detection properties of Cadmium Telluride. In: 2011 IEEE Nuclear Science Symposium Conference Record IEEE, pp. 4812–4817 (2011)

Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42 (December 2012), 60–88 (2017). https://doi.org/10.1016/j.media.2017.07.005

LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1 (4), 541–551 (1989). https://doi.org/10.1162/neco.1989.1.4.541

Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86 (11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

Sutskever, I., Martens, J. and Hinton, G.: Generating text with recurrent neural networks. In: Proc. 28th Int. Conf. Mach. Learn. ICML 2011, pp. 1017–1024 (2011)

Balkanski, E., Rubinstein, A. and Singer, Y.: The power of optimization from samples. In: Advances in Neural Information Processing Systems, 2016, vol. 29. Available: https://proceedings.neurips.cc/paper/2016/file/c8758b517083196f05ac29810b924aca-Paper.pdf

Karpathy, A and Fei-Fei, L.: Deep Visual-Semantic Alignments for Generating Image Descriptions - Karpathy_Deep_Visual-Semantic_Alignments_2015_CVPR_paper.pdf. Cvpr (2015)

Shin, H.C., Roberts, K., Lu, L., Demner-Fushman, D., Yao, J., Summers, R.M.: Learning to read chest x-rays: recurrent neural cascade model for automated image annotation. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016 , 2497–2506 (2016). https://doi.org/10.1109/CVPR.2016.274

Cui, R., Liu, M.: RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Comput. Med. Imaging Graph. 73 , 1–10 (2019). https://doi.org/10.1016/j.compmedimag.2019.01.005

Zhang, J., Zuo, H.: A deep RNN for CT image reconstruction. Proc. SPIE (2020). https://doi.org/10.1117/12.2549809

Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences, vol. 11071. Springer International Publishing, LNCS (2018)

Ben-Cohen, A., Mechrez, R., Yedidia, N. and Greenspan, H.: Improving CNN training using disentanglement for liver lesion classification in CT. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019, pp. 886–889, https://doi.org/10.1109/EMBC.2019.8857465

Liao, H., Lin, W.-A., Zhou, S.K., Luo, J.: ADN: artifact disentanglement network for unsupervised metal artifact reduction. IEEE Trans. Med. Imaging 39 (3), 634–643 (2020). https://doi.org/10.1109/TMI.2019.2933425

Qin, C., Shi, B., Liao, R., Mansi, T., Rueckert, D., Kamen, A.: Unsupervised deformable registration for multi-modal images via disentangled representations, vol. 11492. Springer International Publishing, LNCS (2019)

Creswell, A., Bharath, A. A.: Denoising adversarial autoencoders. IEEE transactions on neural networks and learning systems, 30 (4), 968–984 (2018)

Lopez Pinaya, W.H., Vieira, S., Garcia-Dias, R., Mechelli, A.: Autoencoders. Elsevier Inc. (2019)

Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11 , 3371–3408 (2010)

MathSciNet   MATH   Google Scholar  

Ranzato, M.A., Poultney, C., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy-based model. Adv. Neural Inf. Process. Syst. (2007). https://doi.org/10.7551/mitpress/7503.003.0147

Kingma, D.P. and Welling, M.: Auto-encoding variational bayes. In: 2nd International Conference of Learning Representation. ICLR 2014 - Conf. Track Proc., no. Ml, pp. 1–14 (2014)

Rifai, S., Vincent, P., Muller, X., Glorot, X. and Bengio, Y.: Contractive auto-encoders: explicit invariance during feature extraction. In: Proceeding 28th International Conference of Machine Learning. ICML 2011, no. 1, pp. 833–840 (2011)

Li, C., Xu, K., Zhu, J. and Zhang, B.: Triple generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 2017-Decem, pp. 4089–4099 (2017)

Goodfellow, I. et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, 2014, vol. 27. Available: https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W. and Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: Proceeding - 30th IEEE Conference of Computer Vission Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2242–2251, (2017) https://doi.org/10.1109/CVPR.2017.241

Hacihaliloglu, J.R.B.I., Singer, E.A., Foran, D.J.: For Classification of Prostate Histopathology Whole-Slide Images, vol. 1. Springer International Publishing, Berlin (2018)

Bi, X., Li, S., Xiao, B., Li, Y., Wang, G., Ma, X.: Computer aided Alzheimer’s disease diagnosis by an unsupervised deep learning technology. Neurocomputing 392 , 296–304 (2020). https://doi.org/10.1016/j.neucom.2018.11.111

Baumgartner, C.F., Koch, L.M., Tezcan, K.C., Ang, J.X., Konukoglu, E.: Visual feature attribution using wasserstein GANs. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (2018). https://doi.org/10.1109/CVPR.2018.00867

Son, J., Park, S.J. and Jung, K.-H.: Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks (2017). Available: http://arxiv.org/abs/1706.09318

Dou, Q., et al.: PnP-AdaNet: plug-and-play adversarial domain adaptation network with a benchmark at cross-modality cardiac segmentation. CoRR, vol. abs/1812.0, (2018). Available: http://arxiv.org/abs/1812.07907

Welander, P., Karlsson, S. and Eklund, A.: Generative adversarial networks for image-to-image translation on multi-contrast {MR} images - {A} comparison of CycleGAN and {UNIT}. CoRR vol. abs/1806.0, (2018). Available: http://arxiv.org/abs/1806.07777

Kazeminia, S., et al.: GANs for medical image analysis. Artif. Intell. Med. 109 , 101938 (2020). https://doi.org/10.1016/j.artmed.2020.101938

Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for boltzmann machines. Cogn. Sci. 9 (1), 147–169 (1985). https://doi.org/10.1016/S0364-0213(85)80012-4

Paul, S.: Information processing in dynamical systems: foundations of harmony theory. J. Jpn. Soc. Fuzzy Theory Syst. 4 (2), 220–228 (1986)

van Tulder, G., de Bruijne, M.: Combining generative and discriminative representation learning for lung CT analysis with convolutional restricted boltzmann machines. IEEE Trans. Med. Imaging 35 (5), 1262–1272 (2016). https://doi.org/10.1109/TMI.2016.2526687

Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18 (7), 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527

Article   MathSciNet   MATH   Google Scholar  

Khatami, A., Khosravi, A., Nguyen, T., Lim, C.P., Nahavandi, S.: Medical image analysis using wavelet transform and deep belief networks. Expert Syst. Appl. 86 , 190–198 (2017). https://doi.org/10.1016/j.eswa.2017.05.073

Reddy, A.V.N., et al.: Analyzing MRI scans to detect glioblastoma tumor using hybrid deep belief networks. J. Big Data. (2020). https://doi.org/10.1186/s40537-020-00311-y

Kaur, M., Singh, D.: Fusion of medical images using deep belief networks. Cluster Comput. 23 (2), 1439–1453 (2020). https://doi.org/10.1007/s10586-019-02999-x

Zhou, Z., et al.: Models genesis: generic autodidactic models for 3D medical image analysis. In: Medical Image Computing and Computer Assisted Intervention -- MICCAI 2019, pp. 384–393 (2019)

Zhu, J., Li, Y., Hu, Y., Ma, K., Zhou, S.K., Zheng, Y.: Rubik’s Cube+: a self-supervised feature learning framework for 3D medical image analysis. Med. Image Anal. 64 , 101746 (2020). https://doi.org/10.1016/j.media.2020.101746

Azizi, S., et al.: Big self-supervised models advance medical image classification. no. 1, (2021). Available: http://arxiv.org/abs/2101.05224

Nie, D., Gao, Y., Wang, L. and Shen, D.: ASDNet: attention based semi-supervised deep networks for medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention -- MICCAI 2018, pp. 370–378 (2018)

Bai, W., et al.: Semi-supervised learning for network-based cardiac MR image segmentation. In: Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017, pp. 253–260 (2017)

Liu, Q., Yu, L., Luo, L., Dou, Q., Heng, P.A.: Semi-supervised medical image classification with relation-driven self-ensembling model. IEEE Trans. Med. Imaging 39 (11), 3429–3440 (2020). https://doi.org/10.1109/TMI.2020.2995518

Kervadec, H., Dolz, J., Tang, M., Granger, E., Boykov, Y., Ben Ayed, I.: Constrained-CNN losses for weakly supervised segmentation. Med. Image Anal. 54 , 88–99 (2019). https://doi.org/10.1016/j.media.2019.02.009

Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray: hospital-scale chest x-ray database and benchmarks on weakly supervised classification and localization of common thorax diseases. Adv. Comput. Vis. Pattern Recognit. (2019). https://doi.org/10.1007/978-3-030-13969-8_18

Shi, G., Xiao, L., Chen, Y., Zhou, S.K.: Marginal loss and exclusion loss for partially supervised multi-organ segmentation. Med. Image Anal. 70 , 101979 (2021). https://doi.org/10.1016/j.media.2021.101979

Roth, H.R., Yang, D., Xu, Z., Wang, X., Xu, D.: Going to extremes: weakly supervised medical image segmentation. Mach. Learn. Knowl. Extr. 3 (2), 507–524 (2021). https://doi.org/10.3390/make3020026

Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54 , 30–44 (2019). https://doi.org/10.1016/j.media.2019.01.010

Hu, Y., et al.: Weakly-supervised convolutional neural networks for multimodal image registration. Med. Image Anal. 49 , 1–13 (2018). https://doi.org/10.1016/j.media.2018.07.002

Quellec, G., Laniard, M., Cazuguel, G., Abràmoff, M.D., Cochener, B. and Roux, C.: Weakly supervised classification of medical images. In: 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), pp. 110–113 (2012) doi: https://doi.org/10.1109/ISBI.2012.6235496

Abdullah Al, W., Yun, I.D.: Partial policy-based reinforcement learning for anatomical landmark localization in 3D medical images. IEEE Trans. Med. Imaging 39 (4), 1245–1255 (2020). https://doi.org/10.1109/TMI.2019.2946345

Smith, R.L., Ackerley, I.M., Wells, K., Bartley, L., Paisey, S. and Marshall, C.: Reinforcement learning for object detection in PET imaging. In: 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pp. 1–4 (2019). doi: https://doi.org/10.1109/NSS/MIC42101.2019.9060031

Park, J., Jo, S., Lee, J., and Sun, W.: Color image classification on neuromorphic system using reinforcement learning. In: 2020 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–2 (2020). doi: https://doi.org/10.1109/ICEIC49074.2020.9051310

Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22 (10), 1345–1359 (2010). https://doi.org/10.1109/TKDE.2009.191

Shin, H.C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35 (5), 1285–1298 (2016). https://doi.org/10.1109/TMI.2016.2528162

Xue, D., et al.: An application of transfer learning and ensemble learning techniques for cervical histopathology image classification. IEEE Access 8 , 104603–104618 (2020). https://doi.org/10.1109/ACCESS.2020.2999816

Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35 (5), 1299–1312 (2016). https://doi.org/10.1109/TMI.2016.2535302

Krizhevsky, B.A., Sutskever, I., Hinton, G.E.: Cnn实际训练的. Commun. ACM 60 (6), 84–90 (2012)

Lu, S., Lu, Z., Zhang, Y.D.: Pathological brain detection based on AlexNet and transfer learning. J. Comput. Sci. 30 , 41–47 (2019). https://doi.org/10.1016/j.jocs.2018.11.008

Simonyan, K. and Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference of Learning Representation. ICLR 2015 - Conf. Track Proc., pp. 1–14 (2015)

Sahiner, B., et al.: Deep learning in medical imaging and radiation therapy. Med. Phys. 46 (1), e1–e36 (2019). https://doi.org/10.1002/mp.13264

Article   MathSciNet   Google Scholar  

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 2818–2826 (2016). https://doi.org/10.1109/CVPR.2016.308

Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A.A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: 31st AAAI Conference of Artificial Intelligence. AAAI 2017, pp. 4278–4284 (2017)

Gao, F., Wu, T., Chu, X., Yoon, H., Xu, Y., Patel, B.: Deep residual inception encoder–decoder network for medical imaging synthesis. IEEE J. Biomed. Heal. Informatics 24 (1), 39–49 (2020). https://doi.org/10.1109/JBHI.2019.2912659

A. {Szegedy, Christian and Liu, Wei and Jia, Yangqing and Sermanet, Pierre and Reed, Scott and Anguelov, Dragomir and Erhan, Dumitru and Vanhoucke, Vincent and Rabinovich, “{Going Deeper With Convolutions}e,” 2015, [Online]. Available: Szegedy, Christian, et al. %22Going deeper with convolutions.%22 Proceedings of the IEEE conference on computer vision and pattern recognition. 2015

He, K., Zhang, X.,S. Ren, S. and Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition., vol. 2016-Decem, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

Wang, Q., Shen, F., Shen, L., Huang, J., Sheng, W.: Lung nodule detection in CT images using a raw patch-based convolutional neural network. J. Digit. Imaging 32 (6), 971–979 (2019). https://doi.org/10.1007/s10278-019-00221-3

Mantas, J., Hasman, A., Househ, M.S., Gallos, P., Zoulias, E.: Preface. Stud. Health Technol. Inform. 272 , v (2020). https://doi.org/10.3233/SHTI272

Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N. and Liang, J.: UNet++: A nested U-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. pp. 3–11 (2018)

Mohammed Senan, E., Waselallah Alsaade, F., Ibrahim Ahmed Al-mashhadani, M., Aldhyani, T.H.H., Hmoudal-Adhaileh, M.: Classification of histopathological images for early detection of breast cancer using deep learning. J. Appl. Sci. Eng. 24 (3), 323–329 (2021). https://doi.org/10.6180/jase.202106_24(3).0007

Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q.: Densely connected convolutional networks. In: Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua. pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243

Mahmood, F., Yang, Z., Ashley, T. and Durr, N.J.: Multimodal Densenet (2018). Available: http://arxiv.org/abs/1811.07407

Xu, X., Lin, J., Tao, Y. and Wang, X.: An improved DenseNet method based on transfer learning for fundus medical images. In: 2018 7th International Conference on Digital Home (ICDH). pp. 137–140 (2018). https://doi.org/10.1109/ICDH.2018.00033

Ronneberger, O., Fischer, P. and Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015. pp. 234–241 (2015)

Forouzanfar, M., Forghani, N., Teshnehlab, M.: Parameter optimization of improved fuzzy c-means clustering algorithm for brain MR image segmentation. Eng. Appl. Artif. Intell. 23 (2), 160–168 (2010). https://doi.org/10.1016/j.engappai.2009.10.002

Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D.L., Erickson, B.J.: Deep learning for brain MRI segmentation: state of the art and future directions. J. Digit. Imaging 30 (4), 449–459 (2017). https://doi.org/10.1007/s10278-017-9983-4

Milletari, F., Navab, N. and Ahmadi, S.-A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). pp. 565–571 (2016). https://doi.org/10.1109/3DV.2016.79

Havaei, M., et al.: Brain tumor segmentation with Deep Neural Networks. Med. Image Anal. 35 , 18–31 (2017). https://doi.org/10.1016/j.media.2016.05.004

Asgari Taghanaki, S., Abhishek, K., Cohen, J.P., Cohen-Adad, J., Hamarneh, G.: Deep semantic segmentation of natural and medical images: a review. Artif. Intell. Rev. 54 (1), 137–178 (2021). ( Springer Netherlands )

Li, W., Jia, F., Hu, Q.: Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. J. Comput. Commun. 03 (11), 146–151 (2015). https://doi.org/10.4236/jcc.2015.311023

Dong, H., Yang, G., Liu, F., Mo, Y. and Guo Y.: Automatic brain tumor detection and segmentation using U-net based fully convolutional networks. In: Medical Image Understanding and Analysis, pp. 506–517 (2017)

Soltaninejad, M., Zhang, L., Lambrou, T., Allinson, N. and Ye, X.: Multimodal MRI brain tumor segmentation using random forests with features learned from fully convolutional neural network. (2017). Available: http://arxiv.org/abs/1704.08134

Ker, J., Wang, L., Rao, J., Lim, T.: Deep learning applications in medical image analysis. IEEE Access 6 , 9375–9379 (2017). https://doi.org/10.1109/ACCESS.2017.2788044

Chen, L., Bentley, P., Rueckert, D.: Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks. NeuroImage Clin. 15 (January), 633–643 (2017). https://doi.org/10.1016/j.nicl.2017.06.016

Li, Z., Wang, Y. and Yu, J.: Brain tumor segmentation using an adversarial network. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. pp. 123–132 (2018)

Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Jambawalikar, S. R.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. (2018). arXiv:1811.02629

Isensee, F., Kickingereder, P., Wick, W., Bendszus, M. and Maier-Hein, K. H.: Brain tumor segmentation and radiomics survival prediction: contribution to the BRATS 2017 challenge. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pp. 287–297 (2018)

Korfiatis, P., Kline, T.L., Erickson, B.J.: Automated segmentation of hyperintense regions in FLAIR MRI using deep learning. Tomography 2 (4), 334–340 (2016)

Liu, M., Zhang, J., Adeli, E., Shen, D.: Landmark-based deep multi-instance learning for brain disease diagnosis. Med. Image Anal. 43 , 157–168 (2018). https://doi.org/10.1016/j.media.2017.10.005

Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imaging 35 (11), 2369–2380 (2016). https://doi.org/10.1109/TMI.2016.2546227

Fang, L., Cunefare, D., Wang, C., Guymer, R.H., Li, S., Farsiu, S.: Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed. Opt. Express 8 (5), 2732–2744 (2017). https://doi.org/10.1364/BOE.8.002732

Shankaranarayana, S.M., Ram, K., Mitra, K. and Sivaprakasam, M.: Joint optic disc and cup segmentation using fully convolutional and adversarial networks. In: Fetal, Infant and Ophthalmic Medical Image Analysis. pp. 168–176 (2017)

Fu, H., Cheng, J., Xu, Y., Wong, D.W.K., Liu, J., Cao, X.: Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Trans. Med. Imaging 37 (7), 1597–1605 (2018). https://doi.org/10.1109/TMI.2018.2791488

Hu, P., Wu, F., Peng, J., Liang, P., Kong, D.: Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution. Phys. Med. Biol. 61 (24), 8676–8698 (2016). https://doi.org/10.1088/1361-6560/61/24/8676

Ben-Cohen, A., Diamant, I., Klang, E., Amitai, M. and Greenspan, H.: Fully convolutional network for liver segmentation and lesions detection. In: Deep Learning and Data Labeling for Medical Applications, pp. 77–85 (2016)

Yang, D., et al.: Automatic liver segmentation using an adversarial image-to-image network. In: Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. pp. 507–515 (2017)

Cheng, J.Z., et al.: Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci. Rep. 6 (March), 1–13 (2016). https://doi.org/10.1038/srep24454

Al-antari, M.A., Al-masni, M.A., Choi, M.T., Han, S.M., Kim, T.S.: A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int. J. Med. Inform. 117 (April), 44–54 (2018). https://doi.org/10.1016/j.ijmedinf.2018.06.003

Ait Skourt, B., El Hassani, A., Majda, A.: Lung CT image segmentation using deep neural networks. Procedia Comput. Sci. 127 , 109–113 (2018). https://doi.org/10.1016/j.procs.2018.01.104

Kalinovsky, A. and Kovalev, V.: Lung image segmentation using deep learning methods and convolutional neural networks. Int. Conf. Pattern Recognit. Inf. Process., no. July 2017, 21–24 (2016)

Roy, S., et al.: Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound. IEEE Trans. Med. Imaging 39 (8), 2676–2687 (2020). https://doi.org/10.1109/TMI.2020.2994459

Murphy, K., et al.: COVID-19 on chest radiographs: a multireader evaluation of an artificial intelligence system. Radiology 296 (3), E166–E172 (2020). https://doi.org/10.1148/radiol.2020201874

Kline, T.L., et al.: Performance of an artificial multi-observer deep neural network for fully automated segmentation of polycystic kidneys. J. Digit. Imaging 30 (4), 442–448 (2017). https://doi.org/10.1007/s10278-017-9978-1

Ma, J., Wu, F., Jiang, T., Zhao, Q., Kong, D.: Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. 12 (11), 1895–1910 (2017). https://doi.org/10.1007/s11548-017-1649-7

Zhang, R., Huang, L., Xia, W., Zhang, B., Qiu, B., Gao, X.: Multiple supervised residual network for osteosarcoma segmentation in CT images. Comput. Med. Imaging Graph. 63 (January), 1–8 (2018). https://doi.org/10.1016/j.compmedimag.2018.01.006

Yu, L., Guo, Y., Wang, Y., Yu, J., Chen, P.: Segmentation of fetal left ventricle in echocardiographic sequences based on dynamic convolutional neural networks. IEEE Trans. Biomed. Eng. 64 (8), 1886–1895 (2017). https://doi.org/10.1109/TBME.2016.2628401

Jafari, M.H., Nasr-Esfahani, E., Karimi, N., Soroushmehr, S.M.R., Samavi, S., Najarian, K.: Extraction of skin lesions from non-dermoscopic images for surgical excision of melanoma. Int. J. Comput. Assist. Radiol. Surg. 12 (6), 1021–1030 (2017). https://doi.org/10.1007/s11548-017-1567-8

Yang, D., Zhang, S., Yan, Z., Tan, C., Li, K. and Metaxas, D.: Automated anatomical landmark detection ondistal femur surface using convolutional neural network. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). pp. 17–21 (2015). doi: https://doi.org/10.1109/ISBI.2015.7163806

Orlando, J.I., Prokofyeva, E., del Fresno, M., Blaschko, M.B.: An ensemble deep learning based approach for red lesion detection in fundus images. Comput. Methods Programs Biomed. 153 , 115–127 (2018). https://doi.org/10.1016/j.cmpb.2017.10.017

Yang, X., et al.: Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med. Image Anal. 42 , 212–227 (2017). https://doi.org/10.1016/j.media.2017.08.006

Dou, Q., et al.: Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans. Med. Imaging 35 (5), 1182–1195 (2016). https://doi.org/10.1109/TMI.2016.2528129

Yan, K., Wang, X., Lu, L., Summers, R.M.: DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5 (3), 1–11 (2018). https://doi.org/10.1117/1.JMI.5.3.036501

Zhang, J., Liu, M., Shen, D.: Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans. Image Process. 26 (10), 4753–4764 (2017). https://doi.org/10.1109/TIP.2017.2721106

Nakao, T., et al.: Deep neural network-based computer-assisted detection of cerebral aneurysms in MR angiography. J. Magn. Reson. Imaging 47 (4), 948–953 (2018). https://doi.org/10.1002/jmri.25842

Tsehay, Y., et al.: Biopsy-guided learning with deep convolutional neural networks for prostate cancer detection on multiparametric MRI Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Department of Radiology and Imaging Science, National Institute of Health, C. In: 2017 IEEE 14th Int. Symp. Biomed. Imaging (ISBI 2017), pp. 642–645 (2017)

Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R.J., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35 (5), 1196–1206 (2016). https://doi.org/10.1109/TMI.2016.2525803

Setio, A.A.A., et al.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imaging 35 (5), 1160–1169 (2016). https://doi.org/10.1109/TMI.2016.2536809

Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., Cao, K.: Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology 296 (2), E65–E71 (2020)

Luz, E., et al.: Towards an effective and efficient deep learning model for COVID-19 patterns detection in X-ray images. Res. Biomed. Eng. (2021). https://doi.org/10.1007/s42600-021-00151-6

Kassania, S.H., Kassanib, P.H., Wesolowskic, M.J., Schneidera, K.A., Detersa, R.: Automatic detection of coronavirus disease (COVID-19) in X-ray and CT images: a machine learning based approach. Biocybern. Biomed. Eng. 41 (3), 867–879 (2021). https://doi.org/10.1016/j.bbe.2021.05.013

Wang, D., Khosla, A., Gargeya, R., Irshad, H. and Beck, A.H.: Deep learning for identifying metastatic breast cancer. pp. 1–6 (2016). Available: http://arxiv.org/abs/1606.05718

Dou, Q., Chen, H., Yu, L., Qin, J., Heng, P.A.: Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans. Biomed. Eng. 64 (7), 1558–1567 (2017). https://doi.org/10.1109/TBME.2016.2613502

Rajpurkar, P., et al.: CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. pp. 3–9, (2017). Available: http://arxiv.org/abs/1711.05225

Ma, J., Wu, F., Jiang, T., Zhu, J., Kong, D.: Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med. Phys. 44 (5), 1678–1691 (2017). https://doi.org/10.1002/mp.12134

Baka, N., Leenstra, S., Van Walsum, T.: Ultrasound aided vertebral level localization for lumbar surgery. IEEE Trans. Med. Imaging 36 (10), 2138–2147 (2017). https://doi.org/10.1109/TMI.2017.2738612

Alex, V., Safwan, P.K.M., Chennamsetty, S.S., Krishnamurthi, G.: Generative adversarial networks for brain lesion detection. Med. Imaging 2017 Image Process. (2017). https://doi.org/10.1117/12.2254487

Bogunovic, H., et al.: RETOUCH: the retinal OCT fluid detection and segmentation benchmark and challenge. IEEE Trans. Med. Imaging 38 (8), 1858–1874 (2019). https://doi.org/10.1109/TMI.2019.2901398

Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. J. Am. Med. Assoc. 316 (22), 2402–2410 (2016). https://doi.org/10.1001/jama.2016.17216

Nasrullah, N., Sang, J., Alam, M.S., Mateen, M., Cai, B., Hu, H.: Automated lung nodule detection and classification using deep learning combined with multiple strategies. Sensors (2019). https://doi.org/10.3390/s19173722

Domingues, I., Cardoso, J. S.: Mass detection on mammogram images: a first assessment of deep learning techniques. In: 19th Portuguese Conference on Pattern Recognition (RECPAD) (2013)

Lai, Z., Deng, H.: Medical image classification based on deep features extracted by deep model and statistic feature fusion with multilayer perceptron. Comput. Intell. Neurosci. (2018). https://doi.org/10.1155/2018/2061516

Xiao, B., et al.: PAM-DenseNet: a deep convolutional neural network for computer-aided COVID-19 diagnosis. IEEE Trans. Cybern. (2021). https://doi.org/10.1109/TCYB.2020.3042837

Lo, S.-C.B., Lou, S.-L.A., Lin, J.-S., Freedman, M.T., Chien, M.V., Mun, S.K.: Artificial convolution neural network techniques and applications for lung nodule detection. IEEE Trans. Med. Imaging 14 (4), 711–718 (1995). https://doi.org/10.1109/42.476112

World Health Organization: Standardization of interpretation of chest radiographs for the diagnosis of pneumonia in children / World Health Organization Pneumonia Vaccine Trial Investigators’ Group (2001). Available: http://www.who.int/iris/handle/10665/66956

Ding, J., Chen, B., Liu, H., Huang, M.: Convolutional neural network with data augmentation for SAR target recognition. IEEE Geosci. Remote Sens. Lett. 13 (3), 364–368 (2016). https://doi.org/10.1109/LGRS.2015.2513754

Perez, L. and Wang, J.: The effectiveness of data augmentation in image classification using deep learning. CoRR, vol. abs/1712.0 (2017). Available: http://arxiv.org/abs/1712.04621

Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321 , 321–331 (2018). https://doi.org/10.1016/j.neucom.2018.09.013

Li, R., et al.: Deep learning based imaging data completion for improved brain disease diagnosis. In: Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2014. pp. 305–312 (2014)

Hosseini-Asl, E., Gimel’farb, G.L. and El-Baz, A.: Alzheimer’s disease diagnostics by a deeply supervised adaptable 3D convolutional network. CoRR vol. abs/1607.0 (2016). Available: http://arxiv.org/abs/1607.00556

Abràmoff, M.D., et al.: Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest. Ophthalmol. Vis. Sci. 57 (13), 5200–5206 (2016). https://doi.org/10.1167/iovs.16-19964

Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., Mougiakakou, S.: Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans. Med. Imaging 35 (5), 1207–1216 (2016). https://doi.org/10.1109/TMI.2016.2535865

Nibali, A., He, Z., Wollersheim, D.: Pulmonary nodule classification with deep residual networks. Int. J. Comput. Assist. Radiol. Surg. 12 (10), 1799–1808 (2017). https://doi.org/10.1007/s11548-017-1605-6

Christodoulidis, S., Anthimopoulos, M., Ebner, L., Christe, A., Mougiakakou, S.: Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J. Biomed. Heal. Informatics 21 (1), 76–84 (2017). https://doi.org/10.1109/JBHI.2016.2636929

Wu, X., et al.: Deep learning-based multi-view fusion model for screening 2019 novel coronavirus pneumonia: a multicentre study. Eur. J. Radiol. 128 (March), 1–9 (2020). https://doi.org/10.1016/j.ejrad.2020.109041

Farid, A.A., Selim, G.I., Khater, H.A.A.: A novel approach of CT images feature analysis and prediction to screen for corona virus disease (COVID-19). Int. J. Sci. Eng. Res. 11 (03), 1141–1149 (2020). https://doi.org/10.14299/ijser.2020.03.02

Pereira, R.M., Bertolini, D., Teixeira, L.O., Silla, C.N., Costa, Y.M.G.: COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios. Comput. Methods Programs Biomed. 194 , 105532 (2020). https://doi.org/10.1016/j.cmpb.2020.105532

Huynh, B.Q., Li, H., Giger, M.L.: Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J. Med. Imaging 3 (3), 034501 (2016). https://doi.org/10.1117/1.jmi.3.3.034501

Sun, W., Tseng, T.L.B., Zhang, J., Qian, W.: Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput. Med. Imaging Graph. 57 , 4–9 (2017). https://doi.org/10.1016/j.compmedimag.2016.07.004

Swati, Z.N.K., et al.: Brain tumor classification for MR images using transfer learning and fine-tuning. Comput. Med. Imaging Graph. 75 , 34–46 (2019). https://doi.org/10.1016/j.compmedimag.2019.05.001

Sajjad, M., Khan, S., Muhammad, K., Wu, W., Ullah, A., Baik, S.W.: Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J. Comput. Sci. 30 , 174–182 (2019). https://doi.org/10.1016/j.jocs.2018.12.003

Deepak, S., Ameer, P.M.: Brain tumor classification using deep CNN features via transfer learning. Comput. Biol. Med. 111 (June), 103345 (2019). https://doi.org/10.1016/j.compbiomed.2019.103345

Afshar, P., Mohammadi, A. and Plataniotis, K. N.: Brain tumor type classification via capsule networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3129–3133 (2018). https://doi.org/10.1109/ICIP.2018.8451379

Gao, X.W., Hui, R., Tian, Z.: Classification of CT brain images based on deep learning networks. Comput. Methods Programs Biomed. 138 , 49–56 (2017). https://doi.org/10.1016/j.cmpb.2016.10.007

Bharati, S., Podder, P., Mondal, M.R.H.: Hybrid deep learning for detecting lung diseases from X-ray images. Inform. Med. Unlocked 20 , 100391 (2020). https://doi.org/10.1016/j.imu.2020.100391

Zhou, J., et al.: Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images. J. Magn. Reson. Imaging 50 (4), 1144–1151 (2019). https://doi.org/10.1002/jmri.26721

Zhang, Q., et al.: Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics 72 , 150–157 (2016). https://doi.org/10.1016/j.ultras.2016.08.004

Yang, C., Rangarajan, A., Ranka, S.: Visual explanations from deep 3D Convolutional Neural Networks for Alzheimer’s disease classification. AMIA … Annu. Symp. proceedings. AMIA Symp. 2018 , 1571–1580 (2018)

Schwyzer, M., et al.: Automated detection of lung cancer at ultralow dose PET/CT by deep neural networks—initial results. Lung Cancer 126 (November), 170–173 (2018). https://doi.org/10.1016/j.lungcan.2018.11.001

de Carvalho Filho, A.O., Silva, A.C., de Paiva, A.C., Nunes, R.A., Gattass, M.: Classification of patterns of benignity and malignancy based on CT using topology-based phylogenetic diversity index and convolutional neural network. Pattern Recognit. 81 , 200–212 (2018). https://doi.org/10.1016/j.patcog.2018.03.032

Shen, W., et al.: Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification. Pattern Recognit. 61 , 663–673 (2017). https://doi.org/10.1016/j.patcog.2016.05.029

Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., Rajendra Acharya, U.: Automated detection of COVID-19 cases using deep neural networks with X-ray images. Comput. Biol. Med. 121 (April), 103792 (2020). https://doi.org/10.1016/j.compbiomed.2020.103792

Ucar, F., Korkmaz, D.: COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray images. Med. Hypotheses 140 (April), 109761 (2020). https://doi.org/10.1016/j.mehy.2020.109761

Wang, L., Lin, Z.Q., Wong, A.: COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. Sci. Rep. 10 (1), 1–12 (2020). https://doi.org/10.1038/s41598-020-76550-z

Rezvantalab, A., Safigholi, H. and Karimijeshni, S.: Dermatologist level dermoscopy skin cancer classification using different deep learning convolutional neural networks algorithms. (2018). Available: http://arxiv.org/abs/1810.10348

Dorj, U.O., Lee, K.K., Choi, J.Y., Lee, M.: The skin cancer classification using deep convolutional neural network. Multimed. Tools Appl. 77 (8), 9909–9924 (2018). https://doi.org/10.1007/s11042-018-5714-1

Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542 (7639), 115–118 (2017). https://doi.org/10.1038/nature21056

Awais, M., Muller, H., Tang, T. B. and Meriaudeau, F.: Classification of SD-OCT images using a deep learning approach. In: Proc. 2017 IEEE Int. Conf. Signal Image Process. Appl. ICSIPA 2017, vol. c, pp. 489–492 (2017). https://doi.org/10.1109/ICSIPA.2017.8120661

Ting, D.S.W., et al.: Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. J. Am. Med. Assoc. 318 (22), 2211–2223 (2017). https://doi.org/10.1001/jama.2017.18152

Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J. and Greenspan, H.: Synthetic data augmentation using GAN for improved liver lesion classification. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 289–293 (2018). https://doi.org/10.1109/ISBI.2018.8363576

Krebs, J., et al.: Robust non-rigid registration through agent-based action learning. In: Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. pp. 344–352 (2017)

Yang, X., Kwitt, R., Styner, M., Niethammer, M.: Quicksilver: fast predictive image registration—a deep learning approach. Neuroimage 158 , 378–396 (2017). https://doi.org/10.1016/j.neuroimage.2017.07.008

Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B. P. F., Išgum, I. and Staring, M.: Nonrigid image registration using multi-scale 3D Convolutional Neural Networks. In: Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, pp. 232–239 (2017)

Wu, G., Kim, M., Wang, Q., Gao, Y., Liao, S. and Shen, D.: Unsupervised deep feature learning for deformable registration of MR brain images. In: Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2013, pp. 649–656 (2013)

Wu, G., Kim, M., Wang, Q., Munsell, B.C., Shen, D.: Scalable high-performance image registration framework by unsupervised deep feature representations learning. IEEE Trans. Biomed. Eng. 63 (7), 1505–1516 (2016). https://doi.org/10.1109/TBME.2015.2496253

Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35 (5), 1352–1363 (2016). https://doi.org/10.1109/TMI.2016.2521800

de Vos, B.D., Berendsen, F. F., Viergever, M. A., Staring, M., Išgum, I.: End-to-end unsupervised deformable image registration with a convolutional neural network. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 10553 LNCS, pp. 204–212 (2017). doi: https://doi.org/10.1007/978-3-319-67558-9_24

Sun, L., Zhang, S.: Deformable MRI-ultrasound registration using 3D convolutional neural network, vol. 11042. Springer International Publishing, LNCS (2018)

Chen, Y., He, F., Li, H., Zhang, D., Wu, Y.: A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration. Appl. Soft Comput. J. 93 , 106335 (2020). https://doi.org/10.1016/j.asoc.2020.106335

Niethammer, M., Kwitt, R., Vialard, F.X.: Metric learning for image registration. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2019 , 8455–8464 (2019). https://doi.org/10.1109/CVPR.2019.00866

Wang, S., Kim, M., Wu, G., Shen, D.: Scalable high performance image registration framework by unsupervised deep feature representations learning. Deep Learn. Med. Image Anal. 63 (7), 245–269 (2017). https://doi.org/10.1016/B978-0-12-810408-8.00015-8

Kang, E., Min, J., Ye, J.C.: A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Med. Phys. 44 (10), e360–e375 (2017). https://doi.org/10.1002/mp.12344

Abanoviè, E., Stankevièius, G. and Matuzevièius, D.: Deep Neural Network-based feature descriptor for retinal image registration. In: 2018 IEEE 6th Workshop on Advances in Information, Electronic and Electrical Engineering (AIEEE), pp. 1–4 (2018) https://doi.org/10.1109/AIEEE.2018.8592033

Haskins, G., et al.: Learning deep similarity metric for 3D MR–TRUS image registration. Int. J. Comput. Assist. Radiol. Surg. 14 (3), 417–425 (2019). https://doi.org/10.1007/s11548-018-1875-7

Giger, M.L., Chan, H.-P., Boone, J.: Anniversary paper: history and status of CAD and quantitative image analysis: the role of medical physics and AAPM. Med. Phys. 35 (12), 5799–5820 (2008). https://doi.org/10.1118/1.3013555

Giger, M.L., Karssemeijer, N., Schnabel, J.A.: Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. Annu. Rev. Biomed. Eng. 15 (1), 327–357 (2013). https://doi.org/10.1146/annurev-bioeng-071812-152416

Li, H., et al.: Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set. NPJ Breast Cancer 2 (1), 16012 (2016). https://doi.org/10.1038/npjbcancer.2016.12

Guo, W., et al.: Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data. J. Med. Imaging 2 (4), 1–12 (2015). https://doi.org/10.1117/1.JMI.2.4.041007

Li, H., et al.: MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of MammaPrint, Oncotype DX, and PAM50 Gene assays. Radiology 281 (2), 382–391 (2016). https://doi.org/10.1148/radiol.2016152110

Katsuragawa, S., Doi, K., MacMahon, H., Monnier-Cholley, L., Ishida, T., Kobayashi, T.: Classification of normal and abnormal lungs with interstitial diseases by rule-based method and artificial neural networks. J. Digit. Imaging 10 (3), 108–114 (1997). https://doi.org/10.1007/BF03168597

Kim, G.B., et al.: Comparison of shallow and deep learning methods on classifying the regional pattern of diffuse lung disease. J. Digit. Imaging 31 (4), 415–424 (2018). https://doi.org/10.1007/s10278-017-0028-9

Antropova, N.O., Abe, H., Giger, M.L.: Use of clinical MRI maximum intensity projections for improved breast lesion classification with deep convolutional neural networks. J. Med. Imaging 5 (1), 1–6 (2018). https://doi.org/10.1117/1.JMI.5.1.014503

Mohamed, A.A., Berg, W.A., Peng, H., Luo, Y., Jankowitz, R.C., Wu, S.: A deep learning method for classifying mammographic breast density categories. Med. Phys. 45 (1), 314–321 (2018). https://doi.org/10.1002/mp.12683

Lee, J., Nishikawa, R.M.: Automated mammographic breast density estimation using a fully convolutional network. Med. Phys. 45 (3), 1178–1190 (2018). https://doi.org/10.1002/mp.12763

Antropova, N., Huynh, B.Q., Giger, M.L.: A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med. Phys. 44 (10), 5162–5171 (2017). https://doi.org/10.1002/mp.12453

Samala, R.K., Chan, H.-P., Hadjiiski, L.M., Helvie, M.A., Cha, K.H., Richter, C.D.: Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms. Phys. Med. Biol. 62 (23), 8894–8908 (2017). https://doi.org/10.1088/1361-6560/aa93d4

Kooi, T., van Ginneken, B., Karssemeijer, N., den Heeten, A.: Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network. Med. Phys. 44 (3), 1017–1027 (2017). https://doi.org/10.1002/mp.12110

Masood, A., et al.: Computer-Assisted Decision Support System in Pulmonary Cancer detection and stage classification on CT images. J. Biomed. Inform. 79 (January), 117–128 (2018). https://doi.org/10.1016/j.jbi.2018.01.005

Gonzalez, G., et al.: Disease staging and prognosis in smokers using deep learning in chest computed tomography. Am. J. Respir. Crit. Care Med. 197 (2), 193–203 (2018). https://doi.org/10.1164/rccm.201705-0860OC

Lao, J., et al.: A deep learning-based radiomics model for prediction of survival in Glioblastoma Multiforme. Sci. Rep. 7 (1), 1–8 (2017). https://doi.org/10.1038/s41598-017-10649-8

Garapati, S.S., et al.: Urinary bladder cancer staging in CT urography using machine learning. Med. Phys. 44 (11), 5814–5823 (2017). https://doi.org/10.1002/mp.12510

Takahashi, H., Tampo, H., Arai, Y., Inoue, Y., Kawashima, H.: Applying artificial intelligence to disease staging: deep learning for improved staging of diabetic retinopathy. PLoS One 12 (6), 1–11 (2017). https://doi.org/10.1371/journal.pone.0179790

Skrede, O.-J., et al.: Deep learning for prediction of colorectal cancer outcome: a discovery and validation study. Lancet 395 (10221), 350–360 (2020). https://doi.org/10.1016/S0140-6736(19)32998-8

Saillard, C., et al.: Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatology 72 (6), 2000–2013 (2020). https://doi.org/10.1002/hep.31207

“No Title,” https://www.who.int/news-room/fact-sheets/detail . 2019, [Online]. Available: https://www.who.int/news-room/fact-sheets/detail

Simpson, S., et al.: Radiological society of North America expert consensus document on reporting chest CT findings related to COVID-19: endorsed by the society of thoracic radiology, the American College of Radiology, and RSNA. Radiol. Cardiothorac. Imaging 2 (2), e200152 (2020). https://doi.org/10.1148/ryct.2020200152

Mahmood, A., Gajula, C., Gajula, P.: COVID 19 diagnostic tests: a study of 12,270 patients to determine which test offers the most beneficial results. Surg. Sci. 11 (04), 82–88 (2020). https://doi.org/10.4236/ss.2020.114011

Soldati, G., et al.: Is there a role for lung ultrasound during the COVID-19 pandemic? J. Ultrasound Med. 39 (7), 1459–1462 (2020). https://doi.org/10.1002/jum.15284

Butt, C., Gill, J., Chun, D. et al.: RETRACTED ARTICLE: Deep learning system to screen coronavirus disease 2019 pneumonia. Appl Intell (2020). https://doi.org/10.1007/s10489-020-01714-3

Wang, B., Wu, Z., Khan, Z. U., Liu, C. and Zhu, M.: Deep convolutional neural network with segmentation techniques for chest x-ray analysis. In: 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA). pp. 1212–1216 (2019). https://doi.org/10.1109/ICIEA.2019.8834117

Ardakani, A.A., Kanafi, A.R., Acharya, U.R., Khadem, N., Mohammadi, A.: Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: results of 10 convolutional neural networks. Comput. Biol. Med. 121 , 103795 (2020). https://doi.org/10.1016/j.compbiomed.2020.103795

Li, H., Giger, M.L., Huynh, B.Q., Antropova, N.O.: Deep learning in breast cancer risk assessment: evaluation of convolutional neural networks on a clinical dataset of full-field digital mammograms. J. Med. Imaging 4 (4), 1–6 (2017). https://doi.org/10.1117/1.JMI.4.4.041304

Roth, H.R., et al.: DeepOrgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015. pp. 556–564 (2015)

Asperti, A. and Mastronardo, C.: The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images. CoRR, vol. abs/1712.0 (2017). Available: http://arxiv.org/abs/1712.03689

Pezeshk, A., Petrick, N., Chen, W., Sahiner, B.: Seamless lesion insertion for data augmentation in CAD training. IEEE Trans. Med. Imaging 36 (4), 1005–1015 (2017). https://doi.org/10.1109/TMI.2016.2640180

Zhang, C., Tavanapong, W., Wong, J., de Groen, P.C. and Oh, J.: Real data augmentation for medical image classification. In: Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. pp. 67–76 (2017)

Yang, X., et al.: Low-dose x-ray tomography through a deep convolutional neural network. Sci. Rep. 8 (1), 2575 (2018). https://doi.org/10.1038/s41598-018-19426-7

Chen, H., et al.: Low-dose CT via convolutional neural network. Biomed. Opt. Express 8 (2), 679–694 (2017). https://doi.org/10.1364/BOE.8.000679

Cui, J., Liu, X., Wang, Y., Liu, H.: Deep reconstruction model for dynamic PET images. PLoS One 12 (9), 1–21 (2017). https://doi.org/10.1371/journal.pone.0184667

Kohli, M.D., Summers, R.M., Geis, J.R.: Medical image data and datasets in the era of machine learning—whitepaper from the 2016 C-MIMI meeting dataset session. J. Digit. Imaging 30 (4), 392–399 (2017). https://doi.org/10.1007/s10278-017-9976-3

Willemink, M.J., et al.: Preparing medical imaging data for machine learning. Radiology 295 (1), 4–15 (2020). https://doi.org/10.1148/radiol.2020192224

Altaf, F., Islam, S.M.S., Akhtar, N., Janjua, N.K.: Going deep in medical image analysis: concepts, methods, challenges, and future directions. IEEE Access 7 (3), 99540–99572 (2019). https://doi.org/10.1109/ACCESS.2019.2929365

Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34 (10), 1993–2024 (2015). https://doi.org/10.1109/TMI.2014.2377694

Petersen, R.C., et al.: Alzheimer’s disease neuroimaging initiative (ADNI). Neurology 74 (3), 201–209 (2010). https://doi.org/10.1212/WNL.0b013e3181cb3e25

Armato, S.G., et al.: Lung image database consortium: developing a resource for the medical imaging research community. Radiology 232 (3), 739–748 (2004). https://doi.org/10.1148/radiol.2323032035

Depeursinge, A., Vargas, A., Platon, A., Geissbuhler, A., Poletti, P.-A., Müller, H.: Building a reference multimedia database for interstitial lung diseases. Comput. Med. Imaging Graph. 36 (3), 227–238 (2012). https://doi.org/10.1016/j.compmedimag.2011.07.003

Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23 (4), 501–509 (2004). https://doi.org/10.1109/TMI.2004.825627

Hoover, A.D., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19 (3), 203–210 (2000). https://doi.org/10.1109/42.845178

Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man. Cybern. SMC-3 (6), 610–621 (1973). https://doi.org/10.1109/TSMC.1973.4309314

Çamlica, Z., Tizhoosh, H. R. and Khalvati, F.: Medical image classification via SVM using LBP features from saliency-based folded data. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA). pp. 128–132 (2015). https://doi.org/10.1109/ICMLA.2015.131

Raj, R.J.S., Shobana, S.J., Pustokhina, I.V., Pustokhin, D.A., Gupta, D., Shankar, K.: Optimal feature selection-based medical image classification using deep learning model in internet of medical things. IEEE Access 8 , 58006–58017 (2020). https://doi.org/10.1109/ACCESS.2020.2981337

Miller Jr, R. G.: Beyond ANOVA: basics of applied statistics. CRC press (1997)

Surendiran, A., Vadivel, B.: Feature selection using stepwise ANOVA discriminant analysis for mammogram mass classification. Int. J. Signal Image Process. 2 (1), 17 (2011). https://www.researchgate.net/profile/Surendiran_Balasubramanian/publication/258052973_Feature_selection_using_stepwise_ANOVA_discriminant_analysis_for_mammogram_mass_classification/links/0c96052942c3e97cda000000.pdf

Theodoridis, D.: Sergios and Pikrakis, Aggelos and Koutroumbas, Konstantinos and Cavouras, Introduction to pattern recognition: a matlab approach . 2010

Wu, A., Xu, Z., Gao, M., Mollura, D.J.: Deep vessel tracking: a generalized probabilistic approach via deep learning Aaron Wu, Ziyue Xu?, Mingchen Gao, Mario Buty, Daniel J. Mollura Department of Radiology and Imaging Sciences, National Institutes of Health, Bethesda, MD 20892. Isbi 2016 , 1363–1367 (2016)

Hussain, Z., Gimenez, F., Yi, D., Rubin, D.: Differential data augmentation techniques for medical imaging classification tasks. AMIA … Annu. Symp. Proc. AMIA Symp. 2017 , 979–984 (2017)

Zou, K.H., et al.: Statistical validation of image segmentation quality based on a spatial overlap index1: scientific reports. Acad. Radiol. 11 (2), 178–189 (2004). https://doi.org/10.1016/S1076-6332(03)00671-8

Cho, J., Lee, K., Shin, E., Choy, G. and Do, S.: Medical image deep learning with hospital {PACS} dataset. CoRR vol. abs/1511.0, (2015). Available: http://arxiv.org/abs/1511.06348

Guibas, J. T., Virdi, T. S. and Li, P. S.: Synthetic medical images from dual generative adversarial networks. CoRR vol. abs/1709.0, (2017). Available: http://arxiv.org/abs/1709.01872

Moeskops, P., Veta, M., Lafarge, M. W., Eppenhof, K. A. J. and Pluim, J. P. W.: Adversarial training and dilated convolutions for brain MRI segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. pp. 56–64 (2017)

Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw. 21 (2), 427–436 (2008). https://doi.org/10.1016/j.neunet.2007.12.031

Galar, M., Fernandez, A., Barrenechea, E., Bustince, H. and Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42 (4), 463–484 (2012). https://doi.org/10.1109/TSMCC.2011.2161285

de Bruijne, M.: Machine learning approaches in medical image analysis: from detection to diagnosis. Med. Image Anal. 33 , 94–97 (2016). https://doi.org/10.1016/j.media.2016.06.032

Lundervold, A.S., Lundervold, A.: An overview of deep learning in medical imaging focusing on MRI. Z. Med. Phys. 29 (2), 102–127 (2019). https://doi.org/10.1016/j.zemedi.2018.11.002

Nie, D., Cao, X., Gao, Y., Wang, L. and Shen, D.: Estimating CT image from MRI data using 3D fully convolutional networks. In: Deep Learning and Data Labeling for Medical Applications, pp. 170–178 (2016)

Ledig, C., et al.: “\href{ https://ieeexplore.ieee.org/abstract/document/8099502 }{Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network},” Cvpr , vol. 2, no. 3, p. 4, 2017. Available: http://openaccess.thecvf.com/content_cvpr_2017/papers/Ledig_Photo-Realistic_Single_Image_CVPR_2017_paper.pdf

Download references

Author information

Authors and affiliations.

Yogananda School of AI Computer and Data Sciences, Shoolini University, Solan, 173229, Himachal Pradesh, India

Rammah Yousef & Gaurav Gupta

Electronics and Communication Engineering, Marwadi University, Rajkot, Gujrat, India

Nabhan Yousef

Jawaharlal Nehru University, New Delhi, India

Manju Khari

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Manju Khari .

Ethics declarations

Conflict of interest.

Conflict of interest on the behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Communicated by B. Xiao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Yousef, R., Gupta, G., Yousef, N. et al. A holistic overview of deep learning approach in medical imaging. Multimedia Systems 28 , 881–914 (2022). https://doi.org/10.1007/s00530-021-00884-5

Download citation

Received : 09 August 2021

Accepted : 23 December 2021

Published : 21 January 2022

Issue Date : June 2022

DOI : https://doi.org/10.1007/s00530-021-00884-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical imaging
• Deep learning (DL)
• Medical data augmentation
• Transfer learning
• Find a journal
• Publish with us
• Track your research

The Institute for Computational Imaging and AI in Medicine (CompAI) at TUM and the Institute of Machine Learning in Biomedical Imaging (IML) at Helmholtz Center Munich focus on research to leverage machine learning for the grand challenges in biomedical imaging in areas of unmet clinical need. Novel and affordable solutions should empower clinics to make more accurate, fast and reliable decisions for early detection, treatment planning and improved patient outcome. We are looking for team members , please contact us .

Meet the Team

Julia A. Schnabel

Technical university munich, helmholtz center munich, king's college london, professor for computational imaging and ai in medicine, director of the institute of machine learning in biomedical imaging.

Biomedical Imaging, Artificial Intelligence in Medicine

Team Support

Sandra Mayer

Administrative assistant.

Project Management and Administration, Team Management and Support, Communication and Relations

Sabine Franke

Principal investigators.

Georgios Kaissis

Principal investigator.

Reliable artificial intelligence, Medical image computing, Probabilistic methods

Lina Felsner

Technical university of munich.

Medical Image Computing, Inverse Problems, Machine Learning

Senior Researchers

Cosmin I. Bercea

Tum school of medicine and health, research scientist.

Vision & Multimodal Learning, Generative AI, Foundation Models, Anomaly Detection

Medical image computing, Transfer learning, Multi-modal learning

Medical image segmentation, Multi-modal Learning, Foundation models

Maxime Di Folco

Representation learning, Cardiac imaging

Daniel M. Lang

Self-supervised Learning, Transfer Learning, Survival Analysis

Researchers

Sameer Ambekar

Doctoral researcher.

Domain Generalization, Meta Learning, Variational Inference

Hannah Eichhorn

Brain Magnetic Resonance Imaging, Image reconstruction & Motion Correction, Deep learning

Stefan Fischer

Mri tum munich.

Segmentation, Radiooncology, Transfer Learning/Curriculum Learning

Marta Hasny

Foundation Models, Generative AI, Cardiology

Johannes Kiechle

Shape Analysis, Representation Learning, Magnetic Resonance Imaging

Ha Young Kim

Ge healthcare.

Magnetic Resonance Imaging, Alzheimer’s disease

Fryderyk Kögl

Deep Learning-Based Image Registration, Data Curation & Visualisation, Neuronavigation

Munich Center for Machine Learning (MCML)

Vision and Language, Multi-Modal Learning, Cross-Modality Generation

Natascha Niessen

Ge healthcare.

MRI reconstruction, Early Alzheimer’s Disease prediction

Anna Reithmeir

Image Registration, Physics-Inspired Regularization, Manifold-Valued Data

Anneliese Riess

Mathematical Foundations of Privacy Preserving AI, Probability Theory

Veronika Spieker

MRI Reconstruction, Motion Detection & Correction, Neural Implicit Representations

Visiting Researchers

Richard Osuala

Bcn-aim lab, university of barcelona, doctoral researcher (visiting).

Generative AI, Image Synthesis, Medical Image Analysis

Chun Kit Wong

Section for visual computing, technical university of denmark, pioneer centre for artificial intelligence.

Medical Image Analysis

Simona Bottani

Deep learning, Big data sets, Brain imaging

Veronika Zimmer

Medical Image Computing, Ultrasound Image Analysis, Fetal Image Analysis

Publications

PDF Cite DOI

Open Positions

If you are interested in our projects please contact us and attach a motivation letter, transcript of academic records and CV.

Master Thesis.

Master Thesis. I’m interested

Multimodal Machine Learning for Medical Imaging

Humans comprehend the world through images, and use words to communicate. Similarly, radiologists interpret medical images and describe their findings and interpretation in the form of radiology reports. Research shows that up to 30% of imaging studies miss subtle findings, leading to multiple scans, and in the worst case death of patients. These diagnostic errors are mainly due to increasing patient volume, fatigue, inability to locate subtle findings, and the subjective nature of human perception. A recent estimate shows that one billion radiology examinations are performed worldwide annually. Taking 4% of error rate, it equates to 40 million diagnostic errors per year. In order to reduce these errors, there is a need to develop automated clinical decision support systems that can interpret medical images and generate reports to augment radiologists’ work. A trained radiologist can interpret medical images to find abnormalities and generate radiology reports with competence, but having these capabilities in an intelligent system is a significant challenge due to the differences in structure and characteristics of different types of medical images and their radiology reports.

Despite considerable research at the intersection of language and vision technology for generic applications, its adaptation to the medical domain is not fully explored. In this thesis, we develop multimodal machine learning models that can reason jointly on medical images and radiology reports for automatic generation of radiology reports from medical images. Specifically, we propose a unified approach where we first identify the correct modality of medical images and predict relevant clinical concepts present in them. We also propose a self-attention guided convolutional neural network for identification of common thoracic diseases including the COVID-19 disease. Armed with these contributions, we propose two multimodal machine learning models for automatically generating radiology reports from chest X-rays. We propose an encoder-decoder framework, build on the convolutional neural network and multi-stage recurrent neural network, with separate generation of normal and abnormal radiology reports. Inspired from radiology practice, we propose the “show, tell, and summarise" model for radiology report generation, which first generates findings text from medical images and then summarises the findings to output an impression section concluding the study. We perform extensive experiments by varying parameters for both the encoder (medical images side) and the decoder (radiology reports side), and find that these state-of-the-art vision and language models improve radiology report generation. To provide robust measures of model performance in generating coherent, factually complete, and clinically accurate radiology reports, we highlight the need to use both language generation metrics and classification metrics, given their complementary nature in evaluating radiology reports. Finally, we bring together these sub-system and incorporate best practices to design an integrated and robust radiology report generation system.

The work in this thesis offers the potential to augment radiologists by automating the repetitive process of radiology report drafting, detecting possible medical conditions, accelerating clinical workflow by triaging patients depending upon the level of urgency, and reducing diagnostic errors, in turn saving human life.

Table of Contents

Awarding institution, degree type, department, centre or school, year of award, principal supervisor, usage metrics.

• Other education not elsewhere classified

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Machine learning and deep learning approach for medical image analysis: diagnosis to detection

Meghavi rana.

School of Computing, DIT University, Dehradun, India

Megha Bhushan

Associated data.

Data sharing is applicable to this article as MRI datasets was analyzed during the current study.

Computer-aided detection using Deep Learning (DL) and Machine Learning (ML) shows tremendous growth in the medical field. Medical images are considered as the actual origin of appropriate information required for diagnosis of disease. Detection of disease at the initial stage, using various modalities, is one of the most important factors to decrease mortality rate occurring due to cancer and tumors. Modalities help radiologists and doctors to study the internal structure of the detected disease for retrieving the required features. ML has limitations with the present modalities due to large amounts of data, whereas DL works efficiently with any amount of data. Hence, DL is considered as the enhanced technique of ML where ML uses the learning techniques and DL acquires details on how machines should react around people. DL uses a multilayered neural network to get more information about the used datasets. This study aims to present a systematic literature review related to applications of ML and DL for the detection along with classification of multiple diseases. A detailed analysis of 40 primary studies acquired from the well-known journals and conferences between Jan 2014–2022 was done. It provides an overview of different approaches based on ML and DL for the detection along with the classification of multiple diseases, modalities for medical imaging, tools and techniques used for the evaluation, description of datasets. Further, experiments are performed using MRI dataset to provide a comparative analysis of ML classifiers and DL models. This study will assist the healthcare community by enabling medical practitioners and researchers to choose an appropriate diagnosis technique for a given disease with reduced time and high accuracy.

Introduction

The significance of disease classification and prediction can be observed from the previous years. The important properties and features given in a dataset should be well-known to identify the exact cause along with the symptom of the disease. Artificial Intelligence (AI) has shown promising results by classifying and assisting in decision making. Machine Learning (ML), a subset of AI, has accelerated many research related to the medical field. Whereas, Deep Learning (DL) is a subset of ML that deals with neural network layers, analyzing the exact features required for disease detection [ 34 , 71 , 94 ]. The existing studies from 2014 to present, discusses many applications and algorithms developed for enhancing the medical field by providing accurate results for a patient. Using data, ML has driven advanced technologies in many areas including natural language processing, automatic speech recognition, and computer vision to deliver robust systems such as driverless cars, automated translation, etc. Despite all advances, the application of ML in medical care remained affected with hazards. Many of these issues were raised from medical care stating the goal of making accurate predictions using the collected data and managed by the medical system.

AI examines a given dataset using various techniques to get the required features or highlights from a huge amount of data resulting in difficulty for tracking down an ideal arrangement of significant features and excluding repetitive ones. Considering such features is inconvenient and accuracy metrics becomes erroneous. Hence, choosing a small subset from a wide scope of features will upgrade the efficiency of the model. Subsequently, the exclusion of inconvenient and repetitive features will decline the dimensionality of the information, speed up the learned model similar to boosting [ 37 ]. From the existing features, the significant features are extracted using practical approaches such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). Particularly, choosing a feature has two essential clashing objectives, first, boosting the presentation of arrangement and second, limiting the count of features to conquer the issue of dimensionality. Hence, selection of features is considered as an essential task for aforementioned objectives. Later, research related to the features improvement was enhanced by using choice-based multi-target strategies. Thus, in this review, strategies to choose efficient features will be focused.

Cancer disease was identified using multiple techniques of image segmentation, feature selection, and regression using Root Mean Square Error (RMSE), with the parameters such as recognizing patterns, detecting objects, and classifying the image [ 7 ]. Brain tumor was detected using six classifiers and Transfer Learning (TL) techniques for image segmentation with Magnetic Resonance Imaging (MRI) of the brain [ 28 ]. Also, a TL approach was implemented to identify lung cancer and brain disease in [ 55 ]. It analyzed MRI and Computer-Tomography (CT) scan images by using supervised learning Support Vector Machine (SVM) classifiers. The image analysis process has been well understood in the existing studies. However, the techniques using ML and DL are continuously being updated. Therefore, it is a complex task for researchers to identify an accurate method for analyzing images and feature selection techniques varying with every method. The key contributions of this study include:

• (i) Classification of diseases after reviewing primary studies,
• (ii) Recognition of various image modalities provided by existing articles,
• (iii) Description of tools along with reliable ML and DL techniques for disease prediction,
• (iv) Dataset description to provide awareness of available sources,
• (v) Experimental results using MRI dataset to compare different ML and DL methods,
• (vi) Selection of suitable features and classifiers to get better accuracy, and.
• (vii) Insights on classification as well as review of the techniques to infer future research.

The significance of this review is to enable physicians or clinicians to use ML or DL techniques for precise and reliable detection, classification and diagnosis of the disease. Also, it will assist clinicians and researchers to avoid misinterpretation of datasets and derive efficient algorithms for disease diagnosis along with information on the multiple modern medical imaging modalities of ML and DL.

The study presented consists of 11 sections. The organization of the section is described as follows: Section 2  discusses the background of study, Section  3  discusses the review techniques, search criteria, source material and the quality assessment. Section 4  summarizes the current techniques and important parameters to acquire good accuracy. Section 5  gives an insight of medical image modalities. Section 6  sums up the tools and techniques being used in ML and DL models. Section 7  discusses the datasets used by the authors previously and gives an insight of data. Section 8  represents the experimental section using ML classifiers and DL models over brain MRI dataset. Section 9  recaps the analytic discussion about the techniques, datasets being used, tools in ML and DL, journals studied for the given article. Discussion, conclusion and future scope is discussed in Sections  10 and 11 , respectively.

This section discusses the preliminary terms which are required to comprehend this review. Further, it also presents the statistical analysis of ML and DL techniques used for medical image diagnosis.

Machine learning

ML is a branch of AI where a machine learns from the data by identifying patterns and automates decision-making with minimum human intervention [ 96 , 24 , 12 ]. The most important characteristic of a ML model is to adapt independently, learn from previous calculations and produce reliable results when new datasets are exposed to models repeatedly. The two main aspects include (i) ML techniques help the physicians to interpret medical images using Computer Aided Design (CAD) in a small period of time, and (ii) algorithms used for challenging tasks like segmentation with CT scan [ 81 ], breast cancer and mammography, segmenting brain tumors with MRI. Traditional ML models worked on structured datasets where the techniques were predefined for every step, the applied technique fails if any of the steps were missed. The process of evaluating the data quality used by ML and DL algorithms is essential [ 16 – 22 , 61 ]. Whereas, new algorithms adapt the omission of data based on the requirement for robustness of the algorithm. Figure ​ Figure1 1 illustrates the process used by ML algorithms for the prediction and diagnosis of disease.

ML process [ 10 ]

Deep learning

DL models enable machines to achieve the accuracy by advancements in techniques to analyze medical images. In [ 58 ], the heart disease was diagnosed using the labelled chest X-Rays, cardiologist reviewed and relabelled all the data while discarding the data other than heart failure and normal images. To extract the exact features from the images, data augmentation and TL were used with 82% accuracy, 74% specificity and 95% sensitivity for heart failure. In [ 14 ], an automatic feature selection, using histopathology images with the labelling of positive and negative cancer images, was developed with minimum manual work. Two networks named Deep Neural Network (DNN) 2-F and DNN1-F were used with PCA to reduce features in DNN whereas for unsupervised feature learning a single-layer network of K-means centroids was used. Later, the results of unsupervised (93.56%) and supervised (94.52%) learning were compared. The DL model automates the feature extraction procedure to handle data efficiently [ 14 , 74 ]. Figure ​ Figure2 2 depicts the process used by DL algorithms for the prediction and diagnosis of various diseases.

To process the medical images for better prediction and accuracy, ML and DL techniques were used as shown in Figs.  1 and ​ and2, 2 , respectively. As input, medical images from various modalities are taken into consideration, and then algorithms are applied to these images. Further, the input image is segmented based on various factors, these segments were used to extract the essential and maximum features using feature extraction techniques. After the extraction of the required features, they are further refined to obtain actual features used for the identification of diseases [ 60 ]. Also, ML approaches were used to denoise the medical images for better prediction and accuracy in [ 46 ]. Once the feature selection and noise removal from the data are achieved, the classification of the images according to the disease using classifiers like SVM, Decision Tree (DT), etc. was attained.

ML is the process where computers learn from data and use algorithms to carry out a task without being explicitly programmed. It uses pattern recognition to make predictions with new dataset. Alternatively, DL is modeled according to the human brain including a complex structure of algorithms enabling machines to process images, text and documents. It uses layered-structure algorithms such as Convolutional Neural Network (CNN), Artificial Neural Network (ANN), etc., to analyze the data with logics. Comparatively, DL is more capable of processing huge amount of data than ML models.

Review technique

In this section, an overview of the technique used to conduct this review systematically is discussed. It provides the details of the electronic databases used to search, retrieve information, and discuss the research questions framed to execute the review successfully. The systematic review guidelines implemented by [ 49 , 50 ] were followed for this literature review.

Research questions

In this review, following review questions will be discussed:

• 1.1 What are the considered parameters while selecting the classifiers?
• 1.2 What are the evaluation metrics used to evaluate classification models?
• What are various medical image modalities for classifying the diseases?
• What are the tools and techniques used for medical imaging?
• What are various datasets used by several researchers in the domain of healthcare?
• What are the results of comparative analysis of ML classifiers and DL models based on experiments using MRI dataset?

Source material

The guidelines given in [ 49 , 50 ] are followed for searching the existing literature related to the area of ML and DL in medical imaging. Following electronic database sources are used for searching:

• ScienceDirect ( https://www.sciencedirect.com/ ) .
• IEEE Xplore ( https://ieeexplore.ieee.org/Xplore/home.jsp ) .
• Springer ( https://www.springer.com/in ) .
• PubMed ( https://pubmed.ncbi.nlm.nih.gov/16495534/ ) .
• Wiley Interscience ( https://onlinelibrary.wiley.com/ ) .
• Google Scholar ( https://scholar.google.co.in/ ) .
• IOP ( https://www.iop.org/#gref ) .
• Oxford Publications ( https://india.oup.com/ ) .
• Elsevier ( Elsevier Books and Journals - Elsevier ).
• Hindawi ( https://www.hindawi.com ) .
• Bentham science ( Bentham Science - International Publisher of Journals and Books ).

Search criteria

This review consists of the articles written in English language between the years 2014–2022. The review process can be considered as the filtering process for attaining the quality research articles with the inclusion and exclusion criteria at various stages. The search was based on the keywords as shown in Table ​ Table1 1 to retrieve research articles from various journals, conferences, book chapters, and other sources.

Keywords used

ML  Machine Learning, DL  Deep Learning, ANFC  Adaptive Neuro-Fuzzy Classifier, BI-RADS  Breast Imaging Reporting and Data System, CNN  Convolutional Neural Network, AI  Artificial Intelligence, ANN  Artificial Neural Network, DNN  Deep Neural Network, TL  Transfer Learning

The journals and conferences included were taken from IEEE, Science Direct, Springer, Oxford Publication, etc. The article selection method is depicted in Fig.  3 . As depicted in Fig.  3 , the initial search consisted of 16,900 articles which were refined to 250 based on the specific keywords used as shown in Table ​ Table1. 1 . Then 100 articles were retrieved based on their titles and were reduced to 75 articles based on their abstract and introduction. Finally, 40 articles were selected as primary studies based on the criteria of exclusion and inclusion.

Article selection method

Quality assessment

The quality of this review was assured after inclusion and exclusion criteria discussed in sub-section 3.3 . These primary studies were from various journals, conferences, workshops, and others (manuscripts, online records, and society publications). To retrieve the quality articles, analysis of each article was done to maintain fairness and validation (external and internal) of the results based on the CRD guidelines [ 50 ].

Table ​ Table2 2 presents the top 20 highly influential and cited articles related to the classification of diseases, identification of tools and techniques, explanation for the cause of disease, and solutions to the diagnosed disease (source: https://scholar.google.co.in ).

Top 20 cited articles

J  Journal, BC  Book Chapter, C  Conference, O  Other (Manuscripts, online records, Society publications, Proceedings)

Data extraction

Initially, many challenges were faced to extract the relevant data for this review, therefore, some researchers were approached to acquire the necessary information. The method for extracting the required data in this review is as follow:

• One of the authors extracted the data after a thorough review of 40 articles.
• The acquired results of the review were cross checked by another author to maintain consistency.
• During the process of cross checking (in case of a conflict), issues were resolved by meetings between the authors.

ML and DL techniques for medical imaging

Research question 1 is answered in this section to provide an overview of the current techniques of ML and DL for medical imaging. Further, followed by various parameters considered for selecting the classifiers and the evaluation metrics used to evaluate classification models. The existing literature review is divided according to the diseases such as breast cancer, brain tumor, lung disease, diabetes, multiple disease detection, etc.

Breast disease

In this subsection, articles related to breast disease symptoms, detection, classification, prediction and diagnosis using ML and DL methods are discussed. In [ 33 ], significant features were identified using BI-RADS (Breast Imaging Reporting and Data System) to develop a CAD system for obtaining breast ultrasound. Also, 10-fold cross validation technique was used upon the benign and malignant lesions. As a result, 77% accuracy was achieved using the SVM classifier. However, some methods with a few algorithms handling the vast variety of data need to be understood and analyzed precisely [ 84 ]. CNN was used to train the system with the available clinical data and to comprehend the complex structure. Moreover, it was suggested to study radiomics and expansion of CADx to get the tumor signs using a CAD system. Breast cancer disease was classified using the parameters like Area Under Curve (AUC), sensitivity, and specificity [ 100 ]. A CAD system was developed using CNN where a large number of features were required, using multiview features. These features provide the maximum details of the image data to be extracted for the accuracy of detection and classification.

DL was used for analyzing medical images and also, the limitations along with success of DL techniques for medical imaging were discussed in [ 86 ]. Recent ML and DL technologies were reviewed for the classification and detection of medical imaging modalities [ 39 ]. It provided an insight on the progress of the technology used in the medical field. Various ML techniques used for image processing and DL techniques with the architecture of the algorithm were discussed. To study the technologies, the evaluation of various images such as histological images, thermography images, mammography, ultrasound and MRI using the CAD system was explored. Moreover, the system included ML techniques like SVM, ANN, DT, Naive bayes, K-Nearest Neighbor (KNN), etc.

Brain disease

The concept of TL was used for image segmentation where the MRI scan of the brain was segmented using voxel wise classification [ 7 ]. ML classifiers were applied for the classification of multiple diseases. Later, the results obtained were compared with the existing results to detect the disease.

A brief introduction of DNN in medical image analysis to diagnose the brain tumor using brain tissues is provided in [ 56 ]. It indicated the ways for applying DL to the entire process of MRI scanning, image retrieval, segmentation and disease prediction. It also focused on image acquisition to image retrieval, and from feature segmentation to prediction of disease. The entire process was divided into two parts: (i) the signal processing of MRI including the image restoration and image registration, and (ii) usage of DL for disease detection and prediction-based reports in the form of text and images. Also, the influence of DL in medical imaging was discussed in [ 82 ]. Image segmentation approaches using DL included tumor segmentation, brain and lung’s structure with bone tissues or cells. Patches were taken as input and 2-Dimensional Convolutional Neural Network (2D-CNN) was used to preprocess these at a later stage.

Lung disease

DL has the ability to automate the process of image interpretation which enhances the clinical decision making, identifying the disease and predicting the best treatment for the patient by reviewing the pros and cons of the DL techniques [ 51 ]. These techniques were used for the cardiovascular medication, following are the steps for implementing DL model: (i) problem identification, (ii) data selection, (iii) hardware and software selection, (iv) data preparation, (v) feature selection, and (vi) splitting of data for training as well as validation process. In [ 13 ], a disease was analyzed automatically using labeled data and achieved the accuracy by processing medical images using DL models. The automatic prediction of the disease using ML techniques and the concept of big data was summarized to detect the patterns [ 23 ]. The advantages and disadvantages for each algorithm were also discussed.

A comparative analysis of the classification algorithms based on iris images, using an iridology chart, was done for the diagnosis of diabetes [ 76 ]. Type-2 diabetes was detected by identifying the center of the pupil of an eye at the early stage using the I-Scan-2. Also, a filter-based feature selection method was used with the combination of five classifiers namely binary tree, SVM, neural network model, Random Forest (RF) and adaptive boosting model. Later, in [ 77 ] a study was compiled using the textural, statistical and various features (62 features of iris) to detect the same disease, however, an iridology chart was not used. ML and DL techniques were used to diagnose the errors in existing diagnostic systems [ 81 ]. These techniques were used to analyze the medical images and extract the features which are required for the diagnosis of errors in existing diagnostic systems. Both supervised and unsupervised algorithms were used for the prediction of the disease in specific datasets.

It was observed that DL technique is a way more powerful to investigate medical images [ 65 ]. Various techniques such as image classification, object detection, pattern recognition, etc. were used for the proper decision-making. It improved medical treatments by predicting the early symptoms of a disease. Moreover, an overview of ML and DL techniques used in the medical field was given for providing knowledge to the future researchers. In [ 78 ], techniques such as rubber sheet normalization, ML classifiers, PCA, etc. were used with self-created data and computed six parameters (i) accuracy, (ii) sensitivity, (iii) specificity, (iv)AUC, (v) precision, and (vi) F-score for accurate prediction of Type-2 diabetes.

Multiple disease detection

Multiple diseases were identified with different radiology techniques like MRI imaging for breast cancer along with brain tumor, CAD for breast cancer along with skin lesions, and X-Rays for chest analysis [ 46 ]. Also, ML techniques were used to attain better accuracy with denoising techniques including homographic wavelet, soft thresholding, non-homomorphic and wavelet thresholding. A CAD system using CNN was proposed to diagnose breast lesions as benign and malignant to assist the radiologists [ 100 ]. It was implemented using Inception-v3 architecture to extract the multiview features from Automated Breast Ultrasound (ABUS) images. For the implementation of the model, 316 breast lesions data were trained and evaluated. ML feature extraction scheme was compared with the given method, resulting in 10% increase in AUC value.

A review on image fusion was presented in [ 42 ], it reduced the randomness and improved the quality of available images. Various methods and challenges related to image fusion were also summarized. In [ 44 ], ML and DL techniques focusing on small labeled dataset were discussed as it was considered one of the important factors in decision making. Further, noisy data in medical images was analyzed with pros and cons of various ML algorithms.

In [ 4 ], data augmentation techniques were used to evaluate the dermatology diseases such as acne, atopic dermatitis, impetigo, psoriasis, and rosacea. To diagnose the mentioned diseases, the model was retrained in two phases: (i) with data augmentation, and (ii) without data augmentation using TensorFlow Inception V3. For statistical analysis, both the models were then compared and six parameters namely: (i) Positive Predictive Value (PPV), (ii) Negative Predictive Value (NPV), (iii) Matthew’s Correlation Coefficient (MCC), (iv) sensitivity, (v) specificity, and (vi) F1 score were calculated resulting in an increase of 7.7% average correlation coefficient.

Multiple diseases like diabetes, heart disease, liver disease, dengue and hepatitis were identified by recognizing the pattern in the available data and classifying them further using ML classifiers [ 29 , 27 , 47 ]. It used high-dimensional and multimodal dataset to predict the diseases accurately. The deteriorating condition of a patient was predicted using ML techniques like ML pipelines, classifiers (SVM and 5-fold cross-validation) with the baseline variables from MRI imaging [ 79 ]. AI applications in medical imaging, DL tools for the prediction and pattern recognition were described in [ 87 ]. In addition, apart from AI techniques, ANN and CNN were also useful for predicting the disease by analyzing the image pattern and classification of the disease can be carried out with the help of classifiers [ 62 , 63 ].

Various algorithms were reviewed to detect the error in the diagnosis system implying the importance of ML and DL for early diagnosis of the disease [ 81 ]. Whereas, [ 104 ] discussed the three main challenges: (i) to cope up with image variations, (ii) learning from weak labels, and (iii) interpreting the results with accuracy for the diagnosis of cancer through given medical images. It concluded that TL was used to cope up with image variations. The concept of Multiple Instance Learning (MIL) and weighted TL were used to overcome the weakly labeled data and improve the accuracy of the disease classification for better medical results, respectively. It was suggested to comprehend the relation between image label and image collection instead of learning about the individual instance. The main advantage of the used technique is that it does not require the local manual annotations.

Table ​ Table3 3 represents the current ML and DL techniques for medical imaging, various parameters considered while selecting the classifiers, identified disease and evaluation metrics. Also, early tumor detection can assist clinicians to treat patients timely.

Summary of existing works related to ML and DL techniques for medical imaging

CNN  Convolutional Neural Network, SVM  Support Vector Machine, ML  Machine Learning, DL  Deep Learning, MRI  Magnetic Resonance Imaging, PCA  Principal Component Analysis, BT  Binary Tree, RF  Random Forest, NN  Neural Network, AB  Adaptive Boosting, CAD  Computer-aided Diagnosis System, ANN  Artificial Neural Network, AUC  Area Under Curve, RMSE  Root Mean Square Error, 2D-DWT  Two-Dimensional Discrete Wavelet Transform, MAE  Mean Absolute Error, QSSIM  Quaternion Structure Similarity Index Metric, SSIM  Structure Similarity Index Metric, PCC  Pearson Correlation Coefficient, MoE  Mixture of Experts, MIL  Multiple Instance Learning, PPV  Positive Predictive Value, NPV  Negative Predictive Value, MCC  Matthew’s Correlation Coefficient, TL  Transfer Learning.

Modalities for medical image

Research question 2 (refer subsection 3.1 ) is addressed in this section, various medical image modalities (I-Scan-2, CT-Scan, MRI, X-Ray, Mammogram and Electrocardiogram (ECG)) used for classifying the diseases in the primary studies are shown in Table ​ Table4. 4 . As observed, following modalities were used for the evaluation of medical data using ML and DL techniques.

Modalities for medical imaging and digital signal

CT-SCAN  Computed Tomography Scan, MRI  Magnetic Resonance Imaging, X-Ray  X Radiation, ECG  Electrocardiogram. “+” and “-” signify that the article does and does not support the corresponding parameter, respectively

• MRI : It uses magnetic resonance for obtaining electromagnetic signals. These signals are generated from human organs, which further reconstructs information about human organ structure [ 91 ]. MRIs with high resolution have more structural details which are required to locate lesions and disease diagnosis.
• CT-Scan : It is a technology which generates 3-D images from 2-D X-Ray images using digital geometry [ 88 ].
• Mammogram : For the effective breast cancer screening and early detection of abnormalities in the body, mammograms are used. Calcifications and masses are considered as the most common abnormalities resulting in breast cancer [ 5 ].
• ECG : It is used to measure the heart activity electrically and to detect the cardiac problems in humans [ 8 , 9 , 105 ].

Tools and techniques

This section addresses research question 3 (refer subsection 3.1 ). After a thorough analysis of primary studies, various techniques (refer Table ​ Table6) 6 ) and tools (refer Fig.  4 ) related to ML and DL techniques for healthcare were identified [ 67 , 89 ]. It was observed that techniques have used scanned images with the help of image modalities such as MRI, CT-Scan, X-Rays, and so on. Also, in order to automate the process of image segmentation and classification, programming languages like R, MATLAB and Python were used to obtain accurate results. The subsections 6.1  and 6.2  precisely explain the tools and techniques used in primary studies for medical images, respectively.

Tools used for medical image analysis

ML and DL techniques used for medical imaging

DT  Decision Tree, AB  Adaptive Boosting, ANN  Artificial Neural Network, CNN  Convolutional Neural Network, SVM  Support Vector Machine, RF  Random Forest, ML  Machine Learning, TL  Transfer Learning, PCA  Principal Component Analysis, ICA  Independent Component Analysis.

Tools used for medical images

Figure ​ Figure4 4 depicts the percentage of various tools (Table ​ (Table5) 5 ) used in the primary studies for the implementation of ML and DL models where MATLAB and NumPy have the percentage of 38 and 37, respectively, which signify the popularity of these tools among researchers. R and TensorFlow are the second most used tools with a percentage of 13 and 12, respectively.

Tool description

Techniques used for medical images

This subsection includes the description and identification of the most common ML and DL techniques (i) used for disease classification, detection and diagnosis, (ii) based on type of disease, and (iii) used for EEG and MEG data processing.

Description of techniques

• Convolutional Layer: It is responsible to apply the filters systematically to create feature maps for summarizing features present in the input image.
• Pooling Layer: It is used for ordering the repeated layers in a model. It operates on each feature map, received from the convolutional layer, to produce a new set of feature maps pooled together. Pooling operation is used to reduce the feature map size with required pixels or values in each feature map, hence, reducing the overfitting problem. It consists of two main functions namely, average pooling and maximum pooling.
• Fully-Connected Layer: It is simply the feed-forward neural network where input is received from the final pooling layer. Based on the extracted features, a fully connected layer predicts the image class.

CNN architecture

ANN architecture

TL architecture

RF architecture

DT architecture

SVM architecture

ML and DL techniques

Table ​ Table6 6 summarizes ML and DL techniques such as Naïve bayes [ 43 , 69 ], KNN [ 6 ], DTs [ 36 , 48 ], neural networks, and SVM [ 59 , 73 , 90 ] which are used for medical imaging in primary studies. Here, column 1 represents articles and row 1 represents various techniques. Further, “+” and “-” signify that the article does and does not support the corresponding technique, respectively. The most reliable ML and DL techniques based on the type of disease are shown in Table ​ Table7. 7 . The most significant ML and DL techniques for EEG and MEG data processing are shown in Table ​ Table8 8 .

ML and DL techniques based on the type of disease

ML  Machine Learning, DL  Deep Learning, RF  Random Forest, CNN  Convolutional Neural Network, SVM  Support Vector Machine, DNN  Deep Neural Network, LR  Linear Regression, TL  Transfer Learning

ML and DL techniques for EEG and MEG data processing

SVM  Support Vector Machine, LS-SVM  Least Square-SVM, FD  Fractal Dimensions, RF  Random Forest, KNN  K-Nearest Neighbor, SCNN  Spatial Summary Convolutional Neural Network, Ra-SCNN  SCNN model augmented with attention focused Recurrence, ANN  Artificial Neural Network, DNN  Deep Neural Network, LSTM  Long Short-Term Memory Networks, CNN-RNN  Convolutional Neural Network-Recurrent Neural Network, MLP  Multi-Layer Perceptron, EEG  Electroencephalogram, MEG  Magnetoencephalography

Dataset description

Following section addresses the research question 4 (refer subsection 3.1 ) by providing the details of the datasets used in primary studies for implementing ML and DL algorithms. Table ​ Table9 9 summarizes the description of dataset(s) such as MRI, X-Rays, lesion data, infra-red images and CT-Scan. The accessibility to a dataset is divided as (i) public (available at online repositories), and (ii) own created (created by the authors).

MRI  Magnetic Resonance Imaging, CT  Computer Tomography, DCE  Dynamic Contrast-Enhanced, PET  Positron Emission Tomography. Dermatology dataset (32): 32 features from dermatology dataset, Heart-C dataset (15): 15 features from Heart-C dataset, Lung cancer dataset (55): 55 features from lung cancer dataset, Pima Indian dataset (9): 9 features from Pima Indian dataset, Hepatitis dataset (18): 18 features from hepatitis dataset, Iris dataset (5): 5 features from iris dataset, Wisconsin cancer dataset (10): 10 features from Wisconsin cancer dataset, Lympho dataset (17): 17 features from Lympho dataset, Diabetes dataset (8): 8 features from diabetes dataset, Stalog disease dataset (12): 12 features from stalog disease dataset

Experimental description

Research question 5 (refer subsection 3.1 ) is addressed in this section. MRI dataset is used for the experiments to show the comparative analysis of ML classifiers and DL models. Dataset¹ description and experimental setup are discussed in subsections 8.1  and 8.2 , respectively. Similarly, the methodology and results are discussed in subsections 8.3  and 8.4 , respectively.

The experiments to classify the brain tumor include the publicly available tumor dataset. ( https://www.kaggle.com/datasets/masoudnickparvar/brain-tumor-mri-dataset ). The MRI dataset contains the 711 images of meningioma tumor and no tumor. Dataset is divided into two parts: testing and training with different image resolutions.

Experimental setup

The whole series of experiments were performed on a 64-bit computer with an Intel(R) 221 Core(TM) i3-10110U CPU @ 2.10 GHz 2.59 GHz, 8GB RAM. To train and validate the model, code was implemented in python language in Google colab platform.

Methodology

Figure ​ Figure11 11 depicts the methodology used in the experiments for disease classification. It is described as follows:

Methodology used for disease prediction

• Import dataset : Dataset¹ is retrieved from the public website which is divided into two categories namely: no tumor and meningioma tumor. The dimensions of images given in the dataset were different from one another, which was further resized to 200 × 200.
• Label dataset : Dataset is labeled in the form of 0 and 1, where 0 and 1 indicate the data having no tumor and data having meningioma tumor, respectively.
• Split dataset : Further, the dataset is splitted in the ratio of 80:20 for training (80%) and testing (20%) dataset.
• Feature scaling and feature selection : ML algorithms work on numbers without knowing what the number represents. Feature scaling helps to resolve the given problem by scaling the features into a specific defined range, so that one feature does not dominate the other one. In this experiment, PCA technique is used to reduce the feature count and select the required features.
• Apply ML classifiers : For this experiment, ML classifiers (SVM, RF, DT, LR) and DL models (CNN, ResNet50V2) are used, which further classified the dataset into two categories i.e., 0 and 1.
• Prediction and testing the model : The model was tested with testing data (20% of the dataset) and predicted the disease accurately for the given dataset.

Publisher by number of citations for a  ML articles b  DL articles

This subsection discusses the results obtained by ML classifiers as shown in Fig.  12 ; Table ​ Table10. 10 . In Fig.  12a , ​ ,b, b , ​ ,c, c , and ​ andd d illustrate the confusion matrix obtained from SVM, LR, RF, and DT, respectively. Table ​ Table10 10 shows the values of accuracy obtained after implementing the considered ML classifiers and DL models for the MRI dataset. The results show that CNN and RF have better accuracy with 97.6% and 96.93%, respectively.

Confusion matrix for a  SVM, b  LR, c  RF, and d  DT

Accuracy results for MRI images using ML classifier/DL model

SVM  Support Vector Machine, LR  Logistic Regression, RF  Random Forest, DT  Decision Tree, CNN  Convolutional Neural Network.

Analytical discussion

The primary studies were analyzed based on the publisher citation count, year wise publications, keywords, various diseases, techniques, imaging modalities and type of publication.

Publisher by citations

A schematic view of the influential publishers in the concerned domain is presented by the citations of the articles published in it. Figure ​ Figure13 13 shows all the publishers considered for this review in between 2014 and 2022. Moreover, it depicts the number of citations of ML and DL articles with respect to the publishers in Fig.  14a and ​ andb, b , respectively. Due to many types of indexing procedures along with time, there is a variation in the count of citations in Google Scholar. It was observed that most of the articles for ML and DL were published in ScienceDirect and IEEE publishers with the maximum citation 2425 and 42,866, respectively.

Year wise publication of ML and DL in healthcare

Scholarly articles published between 2014 and 2022

In this subsection, Fig.  14 depicts that out of 40 primary studies, the most published articles for ML were from the year 2020 with a count of 10, which is equivalent to 25% of the total. Followed by the year 2021 with 8 (20%), 2019 with 6 (15%), and 2022 with 4 (10%). Other years like 2017 and 2018 have the same count of 3 with 7.5%, 2014 and 2016 have the same count of 2 with 5%, and 2015 has the count of 1 with 2.5%. Thus, it can be observed that the maximum number of articles for primary study was considered from the year 2020 and minimum from 2014 to 2017.

Most commonly used keywords in the primary studies

Word cloud is the simple way to identify the relevant terms and themes utilized in the referenced research articles. Figure ​ Figure15 15 depicts the word cloud which represents larger font for the most often used keywords and smaller font for less frequent keywords.

Word cloud for frequently used keywords

Disease types

Figure ​ Figure16 16 depicts the percentage of multiple diseases diagnosed in the primary studies. As observed, breast disease is the most common disease with the highest percentage (21%) among all. Brain tumor took the second place (18%) followed by diabetes (16%) and lung disease (16%). Also, other diseases such as eye, liver, skin, hepatitis and cancer were diagnosed using various techniques.

Percentage of diseases

Publication by techniques and statistical analysis of techniques

It was observed that researchers have used multiple techniques to attain better results as shown in Table ​ Table5. 5 . For classification, ML classifiers like SVM, RF and Naïve bayes were combinedly used for the same. Detection was performed using neural networks such as ANN or CNN, and TL was performed frequently due to its capability of breaking down the large datasets. Figure ​ Figure17 17 depicts the percentage of various techniques used in primary studies. It summarizes that SVM (20%) is the most widely used technique for medical image classification.

Percentage of ML and DL techniques in healthcare

The statistical analysis of ML and DL techniques for medical diagnosis is represented in Fig.  18 .

Statistical analysis of ML and DL techniques for medical diagnosis

Imaging modalities

Figure ​ Figure19 19 demonstrates the multiple image modalities used for the evaluation of medical images. However, MRI/X-Ray dominates the subject area with 45%. The second most used modality is CT-Scan (30%), followed by mammogram (10%) and I-Scan-2 (10%). Moreover, to automate the process of retrieving and analyzing the features, computer modalities such as CAD was included for the detection of hepatitis and cancer [ 55 , 60 ].

Percentage of modalities used in medical imaging

Type of publication

Figure ​ Figure20 20 illustrates the distribution of articles according to the type of publications considered for this review. Majority of the articles were considered from journals with 70%, book Chaps. (8%), conference proceeding papers (7%), workshop articles (2%) and others (13%) including the society articles, online database articles, articles from publications like Bentham Science, springer archives and the transcript.

Percentage of type of publication

From this study, it was observed that the variability in the literature occurred due to uncertainty of the evaluated data and models (refer Fig.  21 ). Data uncertainty was caused due to the multiple sources such as transmission noise, missing values and measurement noise. Whereas, model uncertainty was observed due to the less understanding of architecture and prediction of future data with parameters. The observed uncertainty was helpful to attain different results with various methods. Recently, many advanced technologies were introduced to attain enormous amounts of raw data in different scenarios.

Data and model uncertainties

Further, while reviewing the literature, it has been observed that focusing on every aspect of data (noisy or clear) is important as it impacts the results. The utilization of an appropriate algorithm to analyze images can be used for increasing the success ratio. Thus, variation in expected standard results is due to the use of raw data which may incorporate a certain amount of noise (refer Fig.  22 ). CNN is not much sensitive to the noise due to which it can extract information from noisy data [ 44 ]. Moreover, Hermitian basis functions were used for extracting the accumulated data from the ECG data which reduce the effects of Gaussian noise.

Noisy data [ 11 ]

Therefore, dealing with the uncertainty of data and models with ML along with DL techniques is the most important issue to be handled by researchers. These techniques are useful for obtaining accurate and better results for decision making in every respective domain [ 2 , 3 ,  45 , 64 , 75 , 93 ]. Therefore, there is a need to deal with the variance in ML and DL algorithms such as RF, Rubber Sheet Normalization, DT, bagging-boosting, ANN, CNN, SVM, TL, Bayes Net, and GLCM. Further, such strategies can be used to deal with ambiguity in medical data for achieving high performance. Based on this review, it has been observed that medical professionals may be able to treat tumors promptly if they are identified early.

Conclusions and future work

This study provides an overview of various ML and DL approaches for the disease diagnosis along with classification, imaging modalities, tools, techniques, datasets and challenges in the medical domain. MRI and X-Ray scans are the most commonly used modalities for the disease diagnosis. Further, among all the tools and techniques studied, MATLAB and SVM dominated, respectively. It was observed that MRI dataset is widely used by researchers. Also, a series of experiments using MRI dataset has provided a comparative analysis of ML classifiers and DL models where CNN (97.6%) and RF (96.93%) have outperformed other algorithms. This study indicates that there is a need to include denoising techniques with DL models in the healthcare domain. It also concludes that various classical ML and DL techniques are extensively applied to deal with data uncertainty. Due to the superior performance, DL approaches have recently become quite popular among researchers. This review will assist healthcare community, physicians, clinicians and medical practitioners to choose an appropriate ML and DL technique for the diagnosis of disease with reduced time and high accuracy.

Future work will incorporate DL approaches for the diagnosis of all diseases considering noise removal from any given dataset. The additional aspects and properties of DL models for medical images can be explored. To increase the accuracy, enormous amount of data is required, therefore, the potential of the model should be improved to deal with large datasets. Also, different data augmentation techniques along with required features of the dataset can be explored to attain better accuracy.

Data availability

Declarations.

The authors declare that they have no conflict of interest.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Meghavi Rana, Email: [email protected] .

Megha Bhushan, Email: [email protected] .

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Journal Proposal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Estimation of deformation intensity above a flooded potash mine near berezniki (perm krai, russia) with sar interferometry.

Graphical Abstract

1. Introduction

2. data and methodology, 4. discussion and conclusions, author contributions, acknowledgments, conflicts of interest.

• Raucoules, D.; Maisons, C.; Carnec, C.; Le Mouelic, S.; King, C.; Hosford, S. Monitoring of slow ground deformation by ERS radar interferometry on the Vauvert salt mine (France): Comparison with ground-based measurement. Remote Sens. Environ. 2003 , 88 , 468–478. [ Google Scholar ] [ CrossRef ]
• Gourmelen, N.; Amelung, F.; Casu, F.; Manzo, M.; Lanari, R. Mining-related ground deformation in Crescent Valley, Nevada: Implications for sparse GPS networks. Geophys. Res. Lett. 2007 , 34 . [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Ng, A.H.M.; Ge, L.; Yan, Y.; Li, X.; Chang, H.C.; Zhang, K.; Rizos, C. Mapping accumulated mine subsidence using small stack of SAR differential interferograms in the Southern coalfield of New South Wales, Australia. Eng. Geol. 2010 , 115 , 1–15. [ Google Scholar ] [ CrossRef ]
• Samsonov, S.; d’Oreye, N.; Smets, B. Ground deformation associated with post-mining activity at the French-German border revealed by novel InSAR time series method. Int. J. Appl. Earth Obs. Geoinf. 2013 , 23 , 142–154. [ Google Scholar ] [ CrossRef ]
• Samsonov, S.; Gonzalez, P.; Tiampo, K.; d’Oreye, N. Methodology for spatio-temporal analysis of ground deformation occurring near Rice Lake (Saskatchewan) observed by RADARSAT-2 DInSAR during 2008–2011. Can. J. Remote Sens. 2013 , 39 , 27–33. [ Google Scholar ] [ CrossRef ]
• Samsonov, S.; Gonzalez, P.; Tiampo, K.; d’Oreye, N. Modeling of fast ground subsidence observed in southern Saskatchewan (Canada) during 2008–2011. Nat. Hazards Earth Syst. Sci. 2014 , 14 , 247–257. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Alejano, L.; Ramírez-Oyanguren, P.; Taboada, J. FDM predictive methodology for subsidence due to flat and inclined coal seam mining. Int. J. Rock Mech. Min. Sci. 1999 , 36 , 475–491. [ Google Scholar ] [ CrossRef ]
• Baryakh, A.; Telegina, E.; Samodelkina, N.; Devyatkov, S. Prediction of the Intensive Surface Subsidences in Mining Potash Series. J. Min. Sci. Vol. 2005 , 41 , 312–319. [ Google Scholar ] [ CrossRef ]
• Sepehri, M.; Apel, D.B.; Hall, R.A. Prediction of mining-induced surface subsidence and ground movements at a Canadian diamond mine using an elastoplastic finite element model. Int. J. Rock Mech. Min. Sci. 2017 , 100 , 73–82. [ Google Scholar ] [ CrossRef ]
• Darmody, R.; Hetzler, R.; Simmons, F. Coal mine subsidence: Effects of mitigation on crop yields. Int. J. Surf. Min. Reclam. Environ. 1992 , 6 , 187–190. [ Google Scholar ] [ CrossRef ]
• Chatterjee, R.S.; Thapa, S.; Singh, K.; Varunakumar, G.; Raju, E. Detecting, mapping and monitoring of land subsidence in Jharia Coalfield, Jharkhand, India by spaceborne differential interferometric SAR, GPS and precision leveling techniques. J. Earth Syst. Sci. 2015 , 124 , 1359–1376. [ Google Scholar ] [ CrossRef ]
• Sanmiquel, L.; Bascompta, M.; Vintro, C.; Yubero, T. Subsidence Management System for Underground Mining. Minerals 2018 , 8 , 243. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Garrett, D. Origin of Potash Deposits. In Potash ; Springer: Dordrecht, The Netherlands, 1996. [ Google Scholar ] [ CrossRef ]
• Baryakh, A.; Samodelkina, N. Geomechanical Estimation of Deformation Intensity above the Flooded Potash Mine. J. Min. Sci. 2017 , 35 , 630–642. [ Google Scholar ] [ CrossRef ]
• Steding, S.; Zirkler, A.; Kühn, M. Geochemical reaction models quantify the composition of transition zones between brine occurrence and unaffected salt rock. Chem. Geol. 2020 , 532 , 119349. [ Google Scholar ] [ CrossRef ]
• Rodriguez, E.; Martin, J. Theory and Design of interferometric synthetic aperture radars. Proc. IEEE 1992 , 139 , 147–159. [ Google Scholar ] [ CrossRef ]
• Massonnet, D.; Feigl, K. Radar interferometry and its application to changes in the Earth’s surface. Rev. Geophys. 1998 , 36 , 441–500. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Bush, V.; Hebel, H.P.; Schaffer, M.; Walter, D.; Baryakh, A. Control of underworked areas subsidence using the radar interferometry methods. Mine Surv. Subsurf. Use 2009 , 2 , 52–57. [ Google Scholar ]
• Ashrafianfar, N.; Hebel, H.P.; Busch, W. Monitoring of mining induced land subsidence—Differential SAR Interferometry (DInSAR) and Persistent Scatterer Interferometry (PSI) using TerraSAR-X data in comparison with ENVISAT data. In Proceedings of the TerraSAR-X Science Team Meeting, Oberpfaffenhofen, Germany, 14–16 February 2011. [ Google Scholar ]
• Ferretti, A.; Prati, C.; Rocca, F. Permanent scatterers in SAR interferometry. IEEE Trans. Geosci. Remote Sens. 2001 , 39 , 8–20. [ Google Scholar ] [ CrossRef ]
• Samsonov, S.; d’Oreye, N. Multidimensional time series analysis of ground deformation from multiple InSAR data sets applied to Virunga Volcanic Province. Geophys. J. Int. 2012 , 191 , 1095–1108. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Samsonov, S.; d’Oreye, N. Multidimensional Small Baseline Subset (MSBAS) for Two-Dimensional Deformation Analysis: Case Study Mexico City. Can. J. Remote Sens. 2017 , 43 , 318–329. [ Google Scholar ] [ CrossRef ]
• Wegmuller, U.; Werner, C. GAMMA SAR processor and interferometry software. In Proceedings of the 3rd ERS Symposium on Space at the Service of Our Environment, Florence, Italy, 14–21 March 1997. [ Google Scholar ]
• Farr, T.; Kobrick, M. Shuttle Radar Topography Mission produces a wealth of data. EOS Trans. AGU 2000 , 81 , 583–585. [ Google Scholar ] [ CrossRef ]
• Goldstein, R.; Werner, C. Radar interferogram filtering for geophysical applications. Geophys. Res. Lett. 1998 , 25 , 4035–4038. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Costantini, M. A novel phase unwrapping method based on network programming. IEEE Trans. Geosci. Remote Sens. 1998 , 36 , 813–821. [ Google Scholar ] [ CrossRef ]
• Samsonov, S.; Czarnogorska, M.; Charbonneau, F. Selecting optimal RADARSAT Constellation Mission beams for monitoring ground deformation in Alberta’s oil sands. Can. J. Remote Sens. 2015 , 41 , 390–400. [ Google Scholar ] [ CrossRef ]
• Hu, B.; Chen, J.; Zhang, X. Monitoring the land subsidence area in a coastal urban area with InSAR and GNSS. Sensors 2019 , 19 , 3181. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Argyrakis, P.; Ganas, A.; Valkaniotis, S.; Tsioumas, V.; Sagias, N.; Psiloglou, B. Anthropogenically induced subsidence in Thessaly, central Greece: New evidence from GNSS data. Nat. Hazards 2020 , 102 , 179–200. [ Google Scholar ] [ CrossRef ]
• Yalvac, S. Validating InSAR-SBAS results by means of different GNSS analysis techniques in medium- and high-grade deformation areas. Environ. Monit. Assess. 2020 , 192 , 120. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Samsonov, S.; Baryakh, A. Estimation of Deformation Intensity above a Flooded Potash Mine Near Berezniki (Perm Krai, Russia) with SAR Interferometry. Remote Sens. 2020 , 12 , 3215. https://doi.org/10.3390/rs12193215

Samsonov S, Baryakh A. Estimation of Deformation Intensity above a Flooded Potash Mine Near Berezniki (Perm Krai, Russia) with SAR Interferometry. Remote Sensing . 2020; 12(19):3215. https://doi.org/10.3390/rs12193215

Samsonov, Sergey, and Alexandr Baryakh. 2020. "Estimation of Deformation Intensity above a Flooded Potash Mine Near Berezniki (Perm Krai, Russia) with SAR Interferometry" Remote Sensing 12, no. 19: 3215. https://doi.org/10.3390/rs12193215

Article Metrics

Article access statistics, supplementary material.

• Externally hosted supplementary file 1 Doi: http://dx.doi.org/10.17632/ckc3hgz2dh.1 Link: http://dx.doi.org/10.17632/ckc3hgz2dh.1 Description: RADARSAT-2 and Sentinel-1 differential interferograms and deformation time series and linear rates for Berezniki-1 potash mine (Perm Krai, Russia) during November 2011 – April 2014 and July 2016 – June 2020.

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

LANDSCAPE AND BIOLOGICAL DIVERSITY OF PROTECTED AREAS NETWORK IN PERM KRAI

• 1 Perm State National Research University, Russia

The problem of creating the systems of specially Protected natural Areas (PA) adequately representing the geographical diversity of different territories has been acute. Creation of the territory nature protection systems always requires comprehensive assessment of the representativeness of the existing PA network. In Perm region the only such research was carried out at the end of the last century. Since then, the region borders, structure and PA network size, as well as the structure of natural resource use have significantly changed. In this study we assess the representativeness of the PA network of Perm region. For this purpose the representation of the PA network on landscape and biodiversity was analyzed. The study identified the endowment of natural areas and the representation of wetlands in the PA network in the region. Protected species of plants and animals which need development of measures for the territorial protection were identified. The size of the PAs necessary to develop the nature protection network was calculated.

How to Cite: Buzmakov, S. A. & Sannikov, P. Y. (2014). LANDSCAPE AND BIOLOGICAL DIVERSITY OF PROTECTED AREAS NETWORK IN PERM KRAI. American Journal of Environmental Sciences , 10 (5), 516-522. https://doi.org/10.3844/ajessp.2014.516.522

• 3,506 Views
• 3,071 Downloads
• 1 Citations
• Protected Areas (PAs)
• Landscape Diversity
• Biodiversity
• Protected Species

• DOI: 10.17072/2218-1067-2022-1-49-57
• Corpus ID: 249915637

2 Citations

Managing an “unfinished project”: analyzing the elites of perm krai through the lens of expert opinion, related papers.

Showing 1 through 3 of 0 Related Papers