Anomaly Detection using LSTM AutoEncoder

Document Type : Computer Article

Authors

1 Faculty of Computer & IT Engineering, Shahrood University of Technology, Shahrood, Iran,

2 Faculty of Computer & IT Engineering, Shahrood University of Technology, Shahrood, Iran

Abstract

Anomaly detection means detecting samples that are different from the normal samples in the dataset. One of the great challenges in this area is finding labeled data, especially for the abnormal categories. In this paper, we propose a method that uses normal data to detect anomalies. This method is based on established neural networks which are called automated encoder and are considered in deep learning studies. An automated encoder reproduces its input as output and reconstruction deviation to rate anomalies. We have used LSTM blocks to construct encoder instead of using ordinary neurons. In fact, these blocks are a category of recurring neural networks that are specialized in discovering and fetching time and proximity dependencies. The result of employing an automated encoder using LSTM blocks to detect point anomalies shows that this approach has been promising and successful in extracting the normal data’s internal model and also detecting anomalous data. The AUC factor of the model, in almost all cases, is better than the AUC of an ordinary automated encoder and One Class Support Vector Machine (OC-SVM).

Keywords

Main Subjects

References

[1] F.E. Grubbs, "Procedures for Detecting Outlying Observations in Samples", Technometrics, Vol. 11, No. 1, 1969, pp. 1–21.
[2] W. Rechenberg, "Identification of outliers", Fresenius' Zeitschrift für analytische Chemie, Vol. 311, No. 6, 1982, pp. 590–597.
[3] Y. Ma, P. Zhang, Y. Cao, and L. Guo, "Parallel auto-encoder for efficient outlier detection", 2013 IEEE International Conference on Big Data, Vol. 2, No. 3, 2013, pp. 15–17.
[4] C. Zhou and R.C. Paffenroth, "Anomaly Detection with Robust Deep Autoencoders", Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. - KDD ’17, 2017, pp. 665–674.
[5] M.S. Aldosari and E. Blaisten-Barojas, "Unsupervised Anomaly Detection in Sequences Using Long Short Term Memory Recurrent Neural Networks", George Mason University, 2016, pp. 1-25.
[6] S.P. Singh, A. Kumar, H. Darbari, L. Singh, A. Rastogi, and S. Jain, "Machine translation using deep learning: An overview", 2017 International Conference on Computer, Communications and Electronics (Comptelix), 2017, pp. 162–167.
[7] S. Chauhan and L. Vig, "Anomaly detection in ECG time signals via deep long short-term memory networks", in Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015, 2015, pp. 1–7.
[8] M. Markou and S. Singh, "Novelty detection: a review—part 1: statistical approaches", Signal Processing, Vol. 83, No. 12, 2003, pp. 2481–2497.
[9] M. Markou and S. Singh, "Novelty detection: a review—part 2: neural network based approaches", Signal Processing, Vol. 83, No. 12, 2003, pp. 2499–2521.
[10] E.R. de Faria, I.R. Goncalves, J. ao Gama, and A.C.P. de L.F. Carvalho, "Evaluation of Multiclass Novelty Detection Algorithms for Data Streams", IEEE Trans. Knowl. Data Eng., Vol. 27, No. 11, 2015, pp. 2961–2973.
[11] C. Satheesh Chandran, S. Kamal, A. Mujeeb, and M.H. Supriya, "Novel class detection of underwater targets using Self-Organizing neural networks", in 2015 IEEE Underwater Technology (UT), 2015, pp. 1–5.
[12] L. Tarassenko, "Novelty detection for the identification of masses in mammograms", in 4th International Conference on Artificial Neural Networks, Vol. 1995, 1995, pp. 442–447.
[13] K. WORDEN, G. MANSON, and D. ALLMAN, "Experimental Validation of a Structural Health Monitoring Methodology: Part I. Novelty Detection on a Laboratory Structure", Journal of Sound and Vibration, Vol. 259, No. 2, 2003, pp. 323–343.
[14] J. Foote, "Automatic audio segmentation using a measure of audio novelty", in 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532), Vol. 1, 2000, pp. 452–455.
[ 15 [ غ. شفابخش، ح. نادر پور، ف. فصیحی، “ انتخاب الگوریتم بهینه شبکه عصبی در تحلیل روسازیهای انعطافپذیر راهها”، مدلسازی در مهندسی، دوره 8 ، شماره 01 ، 1389 ، صفحه 56 - 45 .
[ 16 [ ع. مرتضایی، ع. خیرالدین، “مدلسازی و تخمین طول مفصل پلاستیک ستونهای بتن آرمه به کمک شبکه های عصبی مصنوعی”، مدلسازی در مهندسی، دوره 10 ، شماره 09 ، 1391 ، صفحه 17 - 1 .
[ 17 [ ز. مروج، ج. آذرخش، “شبیهسازی و طبقهبندی وقایع کیفیت توان با استفاده از شبکه عصبی”، مدلسازی در مهندسی, دوره 13، شماره 41 ، 1394 ، صفحه 146 - 137 .
[ 18 [ س.ع. سلیمانی ایوری، م. فدوی امیری، ح. مروی، “ تولید سیگنال مصنوعی زلزله به کمک مدلی جدید در فشرده سازی و آموزش شبکه های عصبی مصنوعی”، مدلسازی در مهندسی, دوره 14 , شماره 46 , 1395 ، صفحه 85 - 75 .
[19] E.W. Tavares Ferreira, G. Arantes Carrijo, R. de Oliveira, and N. Virgilio de Souza Araujo, "Intrusion Detection System with Wavelet and Neural Artifical Network Approach for Networks Computers", IEEE Latin America Transactions, Vol. 9, No. 5, 2011, pp. 832–837.
[20] M.A.F. Pimentel, D.A. Clifton, L. Clifton, and L. Tarassenko, "A review of novelty detection", Signal Processing, Vol. 99, 2014, pp. 215–249.
[21] B.B. Thompson, R.J. Marks, J.J. Choi, M.A. El-Sharkawi, and C. Bunje, "Implicit learning in autoencoder novelty assessment", Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290), 2002, pp. 2878–2883.
[22] M. Sabokrou, M. Fathy, M. Hoseini, and R. Klette, "Real-time anomaly detection and localization in crowded scenes", 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),2015, pp. 56–62.
[23] W. Yan and L. Yu, "On Accurate and Reliable Anomaly Detection for Gas Turbine Combustors: A Deep Learning Approach", Annual Conference of the Prognostics and Health Management Society, 2015, pp. 1–8.
[24] Y. Xiong and R. Zuo, "Recognition of geochemical anomalies using a deep autoencoder network", Computers & Geosciences, Vol. 86, 2016, pp. 75–82.
[25] P. Malhotra, L. Vig, G. Shroff, and P. Agarwal, "Long Short Term Memory Networks for Anomaly Detection in Time Series", in European Symposium on Artificial Neural Networks, No. April, 2015, pp. 22–24.
[26] P. Malhotra, A. Ramakrishnan, G. Anand, L. Vig, P. Agarwal, and G. Shroff, "LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection", in Anomaly Detection Workshop at 33rd International Conference on Machine Learning (ICML 2016), 2016, pp. 25–30.
[27] M. Cheng, Q. Xu, J. Lv, W. Liu, Q. Li, and J. Wang, "MS-LSTM: A multi-scale LSTM model for BGP anomaly detection", Proc. - Int. Conf. Netw. Protoc. ICNP, Vol. 2016–Decem, no. NetworkML, 2016, pp. 1–6.
[28] B. Schölkopf, R. Williamson, A. Smola, J. Shawe-Taylor, and J. Platt, "Support vector method for novelty detection", Proceedings of the 12th International Conference on Neural Information Processing Systems. MIT Press, 1999, pp. 582–588.
[29] J. Ma and S. Perkins, "Time-series novelty detection using one-class support vector machines", in Proceedings of the International Joint Conference on Neural Networks, 2003. Vol. 3, 2003, pp. 1741–1745.
[30] P. Hayton, B. Schölkopf, L. Tarassenko, and P. Anuzis, "Support vector novelty detection applied to jet engine vibration spectra", Proceedings of the 13th International Conference on Neural Information Processing Systems. MIT Press, 2000, pp. 907–913.
[31] L. Tarassenko, A. Nairac, N. Townsend, and P. Cowley, "Novelty detection in jet engines", in IEE Colloquium on Condition Monitoring: Machinery, External Structures and Health (Ref. No. 1999/034), 1999, pp. 1–5.
[32] L. Clifton, D. A. Clifton, Y. Zhang, P. Watkinson, L. Tarassenko, and H. Yin, "Probabilistic Novelty Detection With Support Vector Machines", IEEE Transactions on Reliability, Vol. 63, No. 2, 2014, pp. 455–467.
[33] D. R. Hardoon and L. M. Manevitz, "One-class machine learning approach for fMRI analysis", in Proceedings of Postgraduate Research Conference in Electronics, Photonics, Communications and Networks, and Computer Science (PREP), Lancaster, UK, 2005b, 2000, pp. 1–2.
[34] M. Davy, F. Desobry, A. Gretton, and C. Doncarli, "An online support vector machine for abnormal events detection", Signal Processing, Vol. 86, No. 8, 2006, pp. 2009–2025.
[35] J. Elman, "Finding structure in time* 1", Cognitive Science, Vol. 14, No. 1, 1990, pp. 179–211.
[36] M. Jordan, "Serial order: A parallel distributed processing approach", Advances in Psychology, Vol. 121, 1997, pp. 471–495.
[37] Z.C. Lipton, J. Berkowitz, and C. Elkan, "A Critical Review of Recurrent Neural Networks for Sequence Learning", 2015, pp. 1–38.
[38] S. Hochreiter and J. Urgen Schmidhuber, "Long Short-Term Memory", Neural Computation, Vol. 9, No. 8, 1997, pp. 1735–1780.
[39] F.A. Gers and J. Schmidhuber, "Recurrent nets that time and count", in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, Vol. 3, 2000, pp. 189–194.
[40] K. Greff, R.K. Srivastava, J. Koutník, B.R. Steunebrink, and J. Schmidhuber, "LSTM: A Search Space Odyssey", IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, No. 10. 2017, pp. 2222–2232.
[41] T. Fawcett, "An introduction to ROC analysis", Pattern Recognition Letters, Vol. 27, No. 8, 2006, pp. 861–874.
[42] M. Goldstein, and S. Uchida, "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data", PLoS One, Vol. 11, No. 4, 2016, pp. 1–31.
[43] R.C. Staudemeyer, "Applying long short-term memory recurrent neural networks to intrusion detection", Sacj, Vol. 56, No. 56, 2015, pp. 136–154.
[44] G.O. Campos, A. Zimek, J. Sander, R.J.G.B. Campello, B. Micenková, E. Schubert, I. Assent, and M.E. Houle, "On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study", Data Mining and Knowledge Discovery, Vol. 30, No. 4, 2016, pp. 891–927.
[45] A. Emmott, S. Das, T. Dietterich, A. Fern, and W.-K. Wong, "A Meta-Analysis of the Anomaly Detection Problem", Oregon State University Libraries & Press, 2015, pp. 12-23.
[46] O.L. Mangasarian, W.N. Street, and W.H. Wolberg, "Breast Cancer Diagnosis and Prognosis Via Linear Programming", Oper. Res., Vol. 43, No. 4, 1995, pp. 570–577.
[47] H.-P. Kriegel, P. Kröger, E. Schubert, and A. Zimek, "LoOP: local outlier probabilities", Proceedings of the 18th ACM Conference on Information and Knowledge Management, 2009, pp. 1649–1652.
[48] B. Micenková, B. McWilliams, and I. Assent, "Learning Outlier Ensembles: The Best of Both Worlds – Supervised and Unsupervised", Proc. ACM SIGKDD Work. Outlier Detect. Descr. ODD., 2014, pp. 1–4.
[49] W. Schi, M. Joost, R. Werner, and D.- Koblenz, "Synthesis and Performance Analysis of Multilayer Neural Network Architectures", Koblenz, 1992, pp. 100-130.
[50] N. Abe, B. Zadrozny, and J. Langford, "Outlier detection by active learning", in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’06, 2006, pp. 504–509.
[51] M. Reif, M. Goldstein, A. Stahl, and T.M. Breuel, "Anomaly detection by combining decision trees and parametric densities", in 2008 19th International Conference on Pattern Recognition, 2008, pp. 1–4.
[52] J.-M. Geusebroek, G.J. Burghouts, and A.W.M. Smeulders, "The Amsterdam Library of Object Images", International Journal of Computer Vision, Vol. 61, No. 1, 2005, pp. 103–112.
[53] E. Schubert, R. Wojdanowski, A. Zimek, and H.-P. Kriegel, "On Evaluation of Outlier Rankings and Outlier Scores", Proceedings of the 2012 SIAM International Conference on Data Mining, 2012, pp. 1047–1058.
[54] U. Carrasquilla, "Benchmarking Algorithms for Detecting Anomalies in Large Datasets", Rev. Lit. Arts Am., 2010, pp. 1–16.
[55] K. Leung and C. Leckie, "Unsupervised anomaly detection in network intrusion detection using clusters", Proceedings of the Twenty-eighth Australasian Conference on Computer Science, Vol. 38, 2005, pp. 333–342.
Journal of Modeling in Engineering
Volume 17, Issue 56
April 2019
Pages 191-211

Files

History

  • Receive Date: 09 November 2017
  • Revise Date: 29 March 2018
  • Accept Date: 25 April 2018

Share

How to cite

Statistics