Deep learning - cancer genetics and application of deep learning to cancer oncology

Doan Hoang, Simon Hoang
Author affiliations


  • Doan Hoang School of Electrical and Data Engineering, University of Technology Sydney, 15 Broadway,Ultimo, NSW2007, Australia
  • Simon Hoang Sydney Local Health District, Sydney, NSW 2137, Australia



deep learning, cancer genetics, cancer oncology, drug response prediction, deep learning applications


Arguably the human body has been one of the most sophisticated systems we encounter but until now we are still far from understanding its complexity. We have been trying to replicate human intelligence by way of artificial intelligence but with limited success. We have discovered the molecular structure in terms of genetics, performed gene editing to change an organism’s DNA and much more, but their translatability into the field of oncology has remained limited. Conventional machine learning methods achieved some degree of success in solving problems that we do not have an explicit algorithm. However, they are basically shallow learning methods, not rich enough to discover and extract intricate features that represent patterns in the real environment. Deep learning has exceeded human performance in pattern recognition as well as strategic games and are powerful for dealing with many complex problems. High-throughput sequencing and microarray techniques have generated vast amounts of data and allowed the comprehensive study of gene expression in tumor cells. The application of deep learning with molecular data enables applications in oncology with information not available from clinical diagnosis. This paper provides fundamental concepts of deep learning, an essential knowledge of cancer genetics, and a review of applications of deep learning to cancer oncology. Importantly, it provides an insightful knowledge of deep learning and an extensive discussion on its challenges. The ultimate purpose is to germinate ideas and facilitate collaborations between cancer biologists and deep learning researchers to address challenging oncological problems using advanced deep learning technologies.


Download data is not yet available.


Adjiri A. - Mutations May Not Be the Cause of Cancer, Oncology and Therapy 5 (1) (2017) 85-101. DOI:

Motofei I. G. - Biology of cancer; from cellular and molecular mechanisms to developmental processes and adaptation, Seminars in Cancer Biology, 2021, doi: DOI:">

DeepMind - "AlphaGo." alphago (accessed June 13, 2022). alphago (accessed June 13, 2022).">

Krizhevsky A., Sutskever I., and Hinton G. - ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25 (2012) 1097-1105.

Russakovsky O., et al. - ImageNet Large Scale Visual Recognition Challenge, Int. J. Comput Vis 115 (2015) 211-252. doi: DOI:">

Alpaydin E. - Introduction to Machine Learning, 3 Ed. MIT Press, 2014.

Burkov A. - The Hundred-Page Machine Learning Book, Kindle Ed. 2019.

McCulloch W. S. and Pitts W. - A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics 5 (1943) 115-133. doi: 10.1007/BF02478259 DOI: 10.1007/BF02478259">

Morin P. J., Trent J. M., Collins F. S., and Vogelstein B. - Cancer Genetics, in Harrisson’s Principles of Internal Medicine, D. L. Kasper, A. S. Fauci, S. L. Hauser, D. L. Longo, and J. L. Jameson Eds., 19 edS.: McGrawHill Education, 2015.

Campbell M. A., et al. - Biology, Pearson Education Australia, 2009.

Kolmogorov A. N. - On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition, Dokl. Akad. Nauk SSSR.115 (5) (1957) 953-956.

Cybenko G. - Approximation by superpositions of a sigmoidal function, Math. Control Signal Systems 2 (1989) 303-314. doi: DOI:">

Gershenfeld N. - The nature of Mathematical Modeling, Cambridge University Press, 2002.

He K., Zhang X., Ren S., and Sun J. - Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, doi: arXiv.1502.01852. arXiv.1502.01852.">

Xu J., Li Z., Du B., Zhang M., and Liu J. - Reluplex made more practical: Leaky ReLU, presented at the 2020 IEEE Symposium on Computers and Communications (ISCC), 2020. DOI:

Clevert A., Unterthiner T., and Hochreiter S. - Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs), 2016. [Online]. Available: arXiv:1511.07289v5.

Sutton R. S. and Barto A. G. - Reinforcement learning: An introduction, MIT press Cambridge, 1998. DOI:

LeCun Y., Bengio Y., and Hinton G. - Deep learning, Nature 521 (2015) 436-444. doi: DOI:">

Graves A., Mohamed A. R., and Hinton G. - Speech recognition with deep recurrent neural networks, presented at the 2013 IEEE international conference on acoustics, speech and signal processing, 2013. DOI:

Murugan P. - Facial information recovery from heavily damaged images using generative adversarial network-part 1, [Online]. Available: arXiv preprint arXiv:180808867

Schulz W. A. - Molecular Biology of Human Cancers - An Advanced Student’s Textbook, Springer, 2007.

Fior R. and Zilhão R. (Eds.) - Molecular and Cell Biology of Cancer - When Cells Break the Rules and Hijack Their Own Planet, Springer, 2019. DOI:

Jameson J. L. and Kopp P. - Principles of Human Genetics, in Harrisson’s Principles of Internal Medicine, D. L. Kasper, A. S. Fauci, S. L. Hauser, D. L. Longo, and J. L. Jameson (Eds.): McGrawHill Education, 2015, ch. 82.

The Human Genome Completed [Online] Available: 060515/full/news060515-12.html 060515/full/news060515-12.html">

Hanahan D. and Weinberg R. A. - Hallmarks of cancer: the next generation, Cell 144 (5) (2011) 646-674. doi: DOI:">

Silver D. et al. - Mastering the game of Go with deep neural networks and tree search, Nature 529 (2016) 484-489. doi: DOI:">

Danaee P., Ghaeini R., and Hendrix D. - A deep learning approach for cancer detection and relevant gene identification, Pac Symp Biocomput 22 (2017) 219-229. doi:10.1142/ 9789813207813_0022.

Bychkov D., et al. - Deep learning based tissue analysis predicts outcome in colorectal cancer, Scientific Reports 8 (2018) Art no. 3395, doi: DOI:">

Chang Y., et al. - Cancer Drug Response Profile scan (CDRscan): A Deep Learning Model That Predicts Drug Effectiveness from Cancer Genomic Signature, Sci. Rep. 8 (2018) Art no. 8857, doi: DOI:">

Yap C. W. - PaDEL-descriptor: an open source software to calculate molecular descriptors and fngerprints, J. Comput. Chem. 32 (2011) 1466-1474. DOI:

Menden M. P., et al. - Machine Learning Prediction of Cancer Cell Sensitivity to Drugs Based on Genomic and Chemical Properties, PLoS ONE 8 (4) (2013). doi: 10.1371/journal.pone.0061318. DOI: 10.1371/journal.pone.0061318.">

Zou J., Huss M., A. Abid, P. Mohammadi, A. Torkamani, and A. Telenti - A primer on deep learning in genomics, Nature Genetics 51 (2019) 12-18. doi: s41588-018-0295-5. DOI: s41588-018-0295-5.">

Goodfellow I., Bengio Y., and Courville A. - Deep Learning, Cambridge, MA, USA: The MIT Press, 2016.

Burkov A. - Machine Learning Engineering, True Positive Inc., 2020.

Géron A. - Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2 Ed. O'Reilly Media, 2019.

Weinstein J., Collisson E., et al. - The Cancer Genome Atlas Pan-Cancer analysis project, Nature Genetics 45 (2013) 1113-1120. doi: DOI:">

Tomczak K., Czerwińska P., and Wiznerowicz M. - Cancer Genome Atlas (TCGA): an immeasurable source of knowledge," Contemp Oncol (Pozn) 19 (1A) (2015) A68-77. doi: 10.5114/wo.2014.47136. DOI:

Barretina J., Caponigro G., Stransky N., et al. - The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity, Nature 483 (2012) 603-607. doi: DOI:">

Clough E. and Barrett T. - The Gene Expression Omnibus database, Methods in Molecular Biology 1418 (2016) 93-110. doi: doi:10.1007/978-1-4939-3578-9_5. DOI:

Edgar R., Domrachev M., Lash A. E. - Gene Expression Omnibus: NCBI gene expression and hybridization array data repository, Nucleic Acids Research 30 (2002) 207-210. doi: DOI:">

Lonsdale J., Thomas J., Salvatore M., et al. - The Genotype-Tissue Expression (GTEx) project, Nature Genetics 45 (2013) 580-585. doi: DOI:">

Hu Z., Tang J., Wang Z., Zhang K., Zhang L., and Sun Q. - Deep learning for image-based cancer detection and diagnosis - A survey, Pattern Recognition 83 (2018) 134-149. doi: DOI:">

Khanam N. and Kumar R. - Recent Applications of Artificial Intelligence in Early Cancer Detection, Curr. Med. Chem. (2022). doi:10.2174/0929867329666220222154733. DOI:

Chiu Y. C., et al. - Predicting and characterizing a cancer dependency map of tumors with deep learning, Sci. Adv. 7 (34) (2021). doi:10.1126/sciadv.abh1275. DOI:

CTD Data Portal. (accessed. (accessed.">

Newton Y., et al. - TumorMap: exploring the molecular similarities of Cancer samples in an interactive portal, Cancer Res. 77 (2) (2017) 111-114. DOI:

Iorio F., et al. - A landscape of Pharmacogenomic interactions in Cancer, Cell 166 (3) (2016) 740-754. DOI:

Li M., et al. - DeepDSC: A Deep Learning Method to Predict Drug Sensitivity of Cancer Cell Lines, IEEE/ACM Trans Comput Biol Bioinform 18 (2) (2021) 575-582. doi:10.1109/TCBB.2019.2919581. DOI:

Bolton E. E., Wang Y., Thiessen P. A., and Bryant S. H. - PubChem: Integrated Platform of Small Molecules and Biological Activities, Annual Reports in Computational Chemistry 4 (2008) 217-241. DOI:

Fakoor R., Ladhak F., Nazi A., and Huber M. - Using deep learning to enhance cancer diagnosis and classication, Presented at the Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, 2013.

Hosny K. M., Kassem M. A., and Foaud M. M. - Skin Cancer Classification using Deep Learning and Transfer Learning, Presented at the 9th Cairo International Biomedical Engineering Conference (CIBEC), Cairo, 2018. DOI:

Yuan Y., et al. - DeepGene: an advanced cancer type classifier based on deep learning and somatic point mutations, BMC Bioinformatics 17 (2016) Art no. 476, doi: /10.1186/s12859-016-1334-9. DOI: /10.1186/s12859-016-1334-9.">

Lyu B. and Haque A. - Deep Learning Based Tumor Type Classification Using Gene Expression Data, Presented at the BCB '18: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, 2018. DOI:

Shen L., Margolies L. R., Rothstein J. H., Fluder E., McBride R., and Sieh W. - Deep Learning to Improve Breast Cancer Detection on Screening Mammography, Sci. Rep. 9 (2019). doi: DOI:">

Khan A., Sohail A., Zahoora U., and Qureshi A. S. - A survey of the recent architectures of deep convolutional neural networks, Artif Intell Rev. 53 (2020) 5455-5516. doi: DOI:">

Li Y., Li X., Xie X., and Shen L. - Deep learning based gastric cancer identification, Presented at the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018. DOI:

Cheerla A. and Gevaert O. - Deep learning with multimodal representation for pancancer prognosis prediction, Bioinformatics 35 (14) (2019) 446-454. doi: bioinformatics/btz342 DOI: bioinformatics/btz342">

Hu J., Shen L., Albanie S., Sun G., and W. E. - Squeeze-and-Excitation Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (8) (2020) 2011-2023. doi:10.1109/TPAMI.2019.2913372. DOI:

Xu Y., et al. - Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging, Clin Cancer Res. 25 (11) (2019) 3266-3275. doi:10.1158/1078-0432.CCR-18-2495. DOI:

Cha K. H., et al. - Bladder Cancer Treatment Response Assessment in CT using Radiomics with Deep-Learning, Scientific Reports 7 (2017) Art no. 8738. doi:https:// DOI:

Yala A., Lehman C., Schuster T., Portnoi T., and Barzilay R. A. -A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction, Radiology 292 (1) (2019) 60-66. doi: 10.1148/radiol.2019182716. DOI:

He K., Zhang X., Ren S., and Sun J. - Deep residual learning for image recognition, Presented at the The IEEE Conference on Computer Vision and Pattern Recognition, 2016. DOI:

Maxwell A., et al. - Deep learning architectures for multi-label classification of intelligent health risk prediction, BMC Bioinformatics 18 (2017) Art no. 523. doi: 10.1186/s12859-017-1898-z. DOI: 10.1186/s12859-017-1898-z.">

Polya G. - How to solve it - A new aspect of Mathematical method, 2 Ed. Princeton University Press, 1973.

Liu H., Simonyan K., Vinyals O., Fernando C., and Kavukcuoglu K. - Hierarchical Representations for Efficient Architecture Search, 2018, doi: 10.48550/arXiv.1711.00436 10.48550/arXiv.1711.00436">

Hoang D. B. and James M. R. - Stability and discriminative properties of the AMI model, Presented at the Proceedings of International Conference on Neural Networks (ICNN'97), 1997.

Hoang D. B. and James M. R. (Eds.) - AMI: A model of intelligence (PRICAI'96: Topics in Artificial Intelligence. Lecture Notes in Computer Science. Berlin: Springer, 1996. DOI:

Bienenstock E. L., Cooper L. N., and Munro P. W. - Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex, The Journal of Neuroscience 2 (1) (1982) 32-48. DOI:

Von der Malsburg C. - Self-organization of orientation sensitive cells in the striate cortex, Kybernetic 14 (1973) 85-420. DOI:

Liu Y., Chen P. H. C., Krause J., and Peng L. - How to read articles that use machine learning: users’ guides to the medical literature, JAMA 322 (2019) 1806-1816. DOI:

Ransohoff D. F. - Bias as a threat to the validity of cancer molecular-marker research, Nat. Rev. Cancer 5 (2005) 142-149. DOI:

Tran K. A., Kondrashova O., Bradley A., Williams E. D., Pearson J. V., and Waddell N. -Deep learning in cancer diagnosis, prognosis and treatment selection, Genome Med. 13 (2021) Art no. 152. doi: DOI:">

Grossberg S. - The resonant brain: How attentive conscious seeing regulates action sequences that interact with attentive cognitive learning, recognition, and prediction, Atten Percept Psychophys 81 (2019) 2237-2264. doi: DOI:">

Zhang C., Bengio S., Hardt M., Recht B., and Vinyals O. - Understanding deep learning requires rethinking generalization, Presented at the Proc. Int. Conf. Learn. Represent, 2017. [Online]. Available:">

Kleppe A., Skrede O. J., De Raedt S., Liestol K., Kerr D. J., and Danielsen H. E. - Designing deep learning studies in cancer diagnostics, Nat. Rev. Cancer 21 (2021) 199-211. doi: DOI:">




How to Cite

D. Hoang and S. Hoang, “Deep learning - cancer genetics and application of deep learning to cancer oncology”, Vietnam J. Sci. Technol., vol. 60, no. 6, pp. 885–928, Dec. 2022.