A Systematic Review of Intrusion Detection using Hidden Markov Models: Approaches, Applications, and Challenges

Document Type : Computer Article

Authors

Computer Engineering Department, Engineering Faculty, Ferdowsi University of Mashhad, Mashhad, Iran.

Abstract

Nowadays, due to the increasing use of the Internet, security of computer systems and networks has become one of the main quality of service (QoS) criteria in ICT-based services. Apart from using traditional security solutions in software systems such as cryptography, firewalls and access control mechanisms, utilizing intrusion detection systems are also necessary. Intrusion detection is a process in which a set of methods are used to detect malicious activities against the victims. Many techniques for detecting potential intrusions in software systems have already been introduced. One of the most important techniques for intrusion detection based on machine learning is using Hidden Markov Models (HMM). Three main advantages of these techniques are high degree of precision, detecting unseen intrusion activities, and visual representation of intrusion models. Hence, in recent decades, many research communities have been working in HMM-based intrusion detection. Therefore, a large volume of research works has been published and hence, various research areas have emerged in this field. However, until now, there has been no systematic and up-to-date review of research works within the field. This paper aims to survey the research in this field and provide open problems and challenges based on the analysis of advantages, limitations, types of architectural models, and applications of current techniques.

Keywords

Main Subjects

References

 
   [1]      Mitchell, R., & Chen, I. R. (2014). A survey of intrusion detection techniques for cyber-physical systems. ACM Computing Surveys (CSUR), 46(4), 55.
   [2]      Axelsson, S. (2000). Intrusion detection systems: A survey and taxonomy (Vol. 99). Technical report.
   [3]      Debar, H., Dacier, M., & Wespi, A. (1999). Towards a taxonomy of intrusion-detection systems. Computer Networks, 31(8), 805-822.
   [4]      Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey.ACM Computing Surveys (CSUR), 41(3), 15.
   [5]      Lazarevic, A., Ertöz, L., Kumar, V., Ozgur, A., & Srivastava, J. (2003, May). A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection. In SDM (pp. 25-36).
   [6]      Sukhwani, H., Sharma, V., & Sharma, S. (2014). A Survey of Anomaly Detection Techniques and Hidden Markov Model. International Journal of Computer Applications, 93(18), 26-31.
   [7]      Hu, J. (2010). Host-based anomaly intrusion detection. In Handbook of Information and Communication Security (pp. 235-255). Springer Berlin Heidelberg.
   [8]      Shimpi, S., & Patil, V. (2013). Hidden Markov Model as Classifier: A survey. 56(1),  pp. 13530--13533.
   [9]      Jha, S., Tan, K., & Maxion, R. A. (2001, June). Markov chains, classifiers, and intrusion detection. In Computer Security Foundations Workshop, IEEE (pp. 0206-0206). IEEE Computer Society.
[10]      Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering–a systematic literature review. Information and software technology, 51(1), 7-15.
[11]      Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80(4), 571-583.
[12]      The Results of the SLR of this Paper, (Available Online: http://sqlab.um.ac.ir/images/219/files/SQLab-RequestFarsi.pdf).
[13]      Abdel-Aziz, A. S., Hassanien, A. E., Azar, A. T., & Hanafi, S. E. O. (2013). Machine Learning Techniques for Anomalies Detection and Classification. InAdvances in Security of Information and Communication Networks (pp. 219-229). Springer Berlin Heidelberg.
[14]      Verwoerd, T., & Hunt, R. (2002). Intrusion detection techniques and approaches. Computer communications, 25(15), 1356-1365.
[15]      Tsai, C. F., Hsu, Y. F., Lin, C. Y., & Lin, W. Y. (2009). Intrusion detection by machine learning: A review. Expert Systems with Applications, 36(10), 11994-12000.
[16]      Blunsom, P. (2004). Hidden markov models. Lecture notes, August, 15, 18-19.
[17]      Rabiner, L., & Juang, B. H. (1986). An introduction to hidden Markov models.ASSP Magazine, IEEE, 3(1), 4-16.
[18]      Lunt, T. F. (1993). A survey of intrusion detection techniques. Computers & Security, 12(4), 405-418.
[19]      Lee, W., & Stolfo, S. J. (1998, January). Data mining approaches for intrusion detection. In Usenix Security.
[20]      Zhang, Y., & Lee, W. (2000, August). Intrusion detection in wireless ad-hoc networks. In Proceedings of the 6th annual international conference on Mobile computing and networking (pp. 275-283). ACM.
[21]      Srivastava, A., Kundu, A., Sural, S., & Majumdar, A. K. (2008). Credit card fraud detection using hidden Markov model. Dependable and Secure Computing, IEEE Transactions on, 5(1), 37-48.
[22]      Luktarhan, N., Jia, X., Hu, L., & Xie, N. (2012). Multi-stage attack detection algorithm based on hidden markov model. In Web Information Systems and Mining (pp. 275-282). Springer Berlin Heidelberg.
[23]      DARPA Intrusion Detection Datasets, (Available Online: https://www.ll.mit.edu/mission/communications/cyber/CSTcorpora/ideval/data/2000data.html).
[24]      Hoang, X. A., & Hu, J. (2004, November). An efficient hidden Markov model training scheme for anomaly intrusion detection of server applications based on system calls. In Networks, 2004.(ICON 2004). Proceedings. 12th IEEE International Conference on (Vol. 2, pp. 470-474). IEEE.
[25]      Cho, S. B., & Park, H. J. (2003). Efficient anomaly detection by modeling privilege flows using hidden Markov model. Computers & Security, 22(1), 45-55.
[26]      Li, W., & Guo, Z. (2009, April). Hidden Markov Model Based Real Time Network Security Quantification Method. In Networks Security, Wireless Communications and Trusted Computing, 2009. NSWCTC'09. International Conference on (Vol. 2, pp. 94-100). IEEE.
[27]      Vartak, A., Patil, C. D., & Patil, C. K. Hidden Markov Model for Credit Card Fraud Detection, International Journal of Computer Science and Information Technologies, Vol. 5 (6) , 2014, 7446-7451.
[28]      Dhok, S. S. (2012). Credit Card Fraud Detection Using Hidden Markov Model.International Journal of Soft Computing and Engineering (IJSCE), 2.
[29]      Frossi, A., Maggi, F., Rizzo, G. L., & Zanero, S. (2009). Selecting and improving system call models for anomaly detection. In Detection of Intrusions and Malware, and Vulnerability Assessment (pp. 206-223). Springer Berlin Heidelberg.
[30]      Yolacan, E. N., Dy, J. G., & Kaeli, D. R. (2014, June). System Call Anomaly Detection Using Multi-HMMs. In Software Security and Reliability-Companion (SERE-C), 2014 IEEE Eighth International Conference on (pp. 25-30). IEEE.
[31]      Pathak, M., Rane, S., Sun, W., & Raj, B. (2011, May). Privacy preserving probabilistic inference with hidden Markov models. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on (pp. 5868-5871). IEEE.
[32]      Guo, S., Zhong, S., & Zhang, A. (2013, September). A Privacy Preserving Markov Model for Sequence Classification. In Proceedings of the International Conference on Bioinformatics, Computational Biology and Biomedical Informatics (p. 561). ACM.
[33]      Gobel, W. Detecting Botnets Using Hidden Markov Models on Network Traces, white paper.
[34]      Austin, T. H., Filiol, E., Josse, S., & Stamp, M. (2013, January). Exploring hidden Markov models for virus analysis: A semantic approach. In System Sciences (HICSS), 2013 46th Hawaii International Conference on (pp. 5039-5048). IEEE.
[35]      Årnes, A., Valeur, F., Vigna, G., & Kemmerer, R. A. (2006, January). Using hidden markov models to evaluate the risks of intrusions. In Recent Advances in Intrusion Detection (pp. 145-164). Springer Berlin Heidelberg.
[36]      Li, H., Liu, Y., & He, D. (2006, November). Research on Technology for IT System Security Assessment Based on Markov Chain. In Signal Processing, 2006 8th International Conference on (Vol. 4). IEEE.
[37]      Lai-Cheng, C. (2007, December). A high-efficiency intrusion prediction technology based on markov chain. In Computational Intelligence and Security Workshops, 2007. CISW 2007. International Conference on (pp. 518-521). IEEE.
[38]      Valeur, F., Vigna, G., Kruegel, C., & Kemmerer, R. A. (2004). Comprehensive approach to intrusion detection alert correlation. Dependable and Secure Computing, IEEE Transactions on, 1(3), 146-169.
[39]      Farhadi, H., AmirHaeri, M., & Khansari, M. (2011). Alert correlation and prediction using data mining and HMM. ISeCure-The ISC International Journal of Information Security, 3(2), 77-102.
[40]      Zan, X., Gao, F., Han, J., & Sun, Y. (2009, November). A Hidden Markov Model based framework for tracking and predicting of attack intention. In Multimedia Information Networking and Security, 2009. MINES'09. International Conference on (Vol. 2, pp. 498-501). IEEE.
[41]      Zhicai, S., & Yongxiang, X. (2010, June). A novel hidden Markov model for detecting complicate network attacks. In Wireless Communications, Networking and Information Security (WCNIS), 2010 IEEE International Conference on (pp. 312-315). IEEE.
[42]      Hu, J., Yu, X., Qiu, D., & Chen, H. H. (2009). A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection. Network, IEEE, 23(1), 42-47.
[43]      Wang, W., Guan, X. H., & Zhang, X. L. (2004, August). Modeling program behaviors by hidden markov models for intrusion detection. In Machine Learning and Cybernetics, 2004. Proceedings of 2004 International Conference on (Vol. 5, pp. 2830-2835). IEEE.
[44]      Shin, S., Lee, S., Kim, H., & Kim, S. (2013). Advanced probabilistic approach for network intrusion forecasting and detection. Expert Systems with Applications, 40(1), 315-322.
[45]      Salazar-Hernández, R., & Díaz-Verdejo, J. E. (2010). Hybrid detection of application layer attacks using Markov models for normality and attacks. InInformation and Communications Security (pp. 416-429). Springer Berlin Heidelberg.
[46]      Kumar, P., Nitin, N., Sehgal, V., Shah, K., Shukla, S. S. P., & Chauhan, D. S. (2011, December). A novel approach for security in Cloud Computing using Hidden Markov Model and clustering. In Information and Communication Technologies (WICT), 2011 World Congress on (pp. 810-815). IEEE.
[47]      Banafar, H., & Sharma, S. (2014). Intrusion Detection and Prevention System for Cloud Simulation Environment using Hidden Markov Model and MD5.International Journal of Computer Applications, 90(19), 6-11.
[48]      Khreich, W., Granger, E., Sabourin, R., & Miri, A. (2009, June). Combining hidden markov models for improved anomaly detection. In Communications, 2009. ICC'09. IEEE International Conference on (pp. 1-6). IEEE.
[49]      Zhou, X., PENG, Q. K., & WANG, J. B. (2008). Intrusion detection method based on two-layer HMM [J]. Application Research of Computers, 3, 075.
[50]      Hoang, X. D., Hu, J., & Bertok, P. (2003, September). A multi-layer model for anomaly intrusion detection using program sequences of system calls. In Proc. 11th IEEE Int’l Conf. Networks (pp. 531-536).
[51]      Zhang, J., Liu, Y., & Liu, X. (2006). Anomalous Detection Based on Adaboost-HMM. In Intelligent Control and Automation, 2006. WCICA 2006. The Sixth World Congress on (Vol. 1, pp. 4360-4363). IEEE.
[52]      Chen, Y. S., & Chen, Y. M. (2009, June). Combining incremental Hidden Markov Model and Adaboost algorithm for anomaly intrusion detection. InProceedings of the ACM SIGKDD Workshop on Cybersecurity and intelligence informatics (pp. 3-9). ACM.
[53]      Zeng, F., Yin, K., Chen, M., & Wang, X. (2009, June). A new anomaly detection method based on rough set reduction and HMM. In Computer and Information Science, 2009. ICIS 2009. Eighth IEEE/ACIS International Conference on (pp. 285-289). IEEE.
[54]      Jiang, W., Xu, Y., & Xu, Y. (2005). A novel intrusions detection method based on HMM embedded neural network. In Advances in Natural Computation (pp. 139-148). Springer Berlin Heidelberg.
[55]      Hoang, X. D., Hu, J., & Bertok, P. (2009). A program-based anomaly intrusion detection scheme using multiple detection engines and fuzzy inference. Journal of Network and Computer Applications, 32(6), 1219-1228.
[56]      Coronado, A. S. (2013). Computer Security: Principles and Practice. Journal of Information Privacy and Security, 9(2), 62-65.
[57]      Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., & Vázquez, E. (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. computers & security, 28(1), 18-28.
[58]      Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., & Vázquez, E. (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. computers & security, 28(1), 18-28.
[59]      Wang, P., Shi, L., Wang, B., Wu, Y., & Liu, Y. (2010). Survey on HMM based anomaly intrusion detection using system calls. In 2010 5th International Conference on Computer Science&Education (pp. 102-105).
[60]      Shameli Sendi, A., Dagenais, M., Jabbarifar, M., & Couture, M. (2012). Real time intrusion prediction based on optimized alerts with hidden markov model.Journal of Networks, 7(2), 311-321.
[61]      Xiang, S., Lv, Y., Xia, C., Li, Y., & Wang, Z. (2015, November). A Method of Network Security Situation Assessment Based on Hidden Markov Model. In International Symposium on Intelligence Computation and Applications (pp. 631-639). Springer Singapore.
[62]      Liu, S. C., & Liu, Y. (2016, May). Network security risk assessment method based on HMM and attack graph model. In Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), 2016 17th IEEE/ACIS International Conference on (pp. 517-522). IEEE.
[63]      Pao, H. K., Lee, Y. J., & Huang, C. Y. (2015). Statistical learning methods for information security: fundamentals and case studies. Applied Stochastic Models in Business and Industry, 31(2), 97-113.
[64]      Patil, S. J., & Dange, A. S. (2016). Credit Card Fraud Detection Using Hidden Markov Model. International Journal of Engineering Science, 3715.
[65]      Buczak, A. L., & Guven, E. (2015). A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials, 18(2), 1153-1176.
[66]      Bharati, M. M. (2016). A Survey on Hidden Markov Model (HMM) Based Intention Prediction Techniques. International Journal of engineering Research and Applications, 1(6), 167-172.
Journal of Modeling in Engineering
Volume 16, Issue 53
July 2018
Pages 183-206

Files

History

  • Receive Date: 09 July 2016
  • Revise Date: 30 May 2017
  • Accept Date: 09 July 2017

Share

How to cite

Statistics