An ensemble deep learning model to enhance feature representation for entity detection

Document Type : Computer Article

Authors

1 Department of Computer Engineering, َArak Branch, Islamic Azad University, Arak, Iran

2 Department of Computer Engineering, Arak Branch, Islamic Azad University, Arak, Iran

Abstract

One of the main processes in most natural language processing (NLP), is named entity recognition (NER). In this regard, some machine learning techniques have been presented that traditionally use manual features. Also, in recent years, deep neural network-based models have been proposed that achieve higher accuracy without relying on huge computations for feature engineering. Thus, in this article, we employ a combination of two deep learning models to capture the properties of the input sentence, including: long short term memory (LSTM) and convolutional neural network (CNN). In this architecture, extracting local features along with global features, more information is acquired for more accurate classification. We evaluate the performance of this architecture on two datasets CoNLL2003 and ACE05; and demonstrate that by adding a word level CNN, useful local properties are extracted that enhance the accuracy of the performance. Finally, we compare the performance of our system with competitors and our superiority is reported.

Keywords

Main Subjects

References

[1] A. Akkasi, and E. Varoglu, “Improvement of Chemical Named Entity Recognition through Sentence-based Random Under-sampling and Classifier Combination”, Journal of AI and Data Mining, Vol. 7, NO. 2, 2019, pp. 311-319.
[2] E. Parsaeimehr, M. Fartash, and J. Akbari Torkestani, “An Enhanced Deep Neural Network-Based Architecture for Joint Extraction of Entity Mentions and Relations”, Journal of Fuzzy logic and Intelligent Systems, Vol. 20, NO. 1, 2020, pp. 69-76.
[3] علی سلیمانی ایوری، محمد رضا فدوی امیری و حسین مروی، "تولید سیگنال مصنوعی زلزله به کمک مدلی جدید در فشرده سازی و آموزش شبکه های عصبی مصنوعی"، مجله مدل سازی در مهندسی، دانشگاه سمنان، دوره14 ، شماره 46، سال 1395، صفحه 75-85.
[4] علی نظری، "مدلسازی انرژی ضربه ی فولادهای مرتبه ای با استفاده از شبکه های عصبی مصنوعی"، مجله مدل سازی درمهندسی، دانشگاه سمنان، دوره14 ، شماره45 ، تابستان1395 ، صفحه 145-162.
[5] زهرا مروج و جواد آذرخش، "شبیه سازی و طبقه بندی وقایع کیفیت توان با استفاده از شبکه عصبی"، مجله مدل سازی در مهندسی، دانشگاه سمنان، دوره13 ، شماره41 ، تابستان1394 ، صفحه 137-146.
[6] M.A. Spalenza, L. Lusquino-Filho, F. M. G. Franca, P. M. V. Lima, and E. Oliveira1, “LCAD - UFES at FakeDeS 2021:Fake News Detection Using Named Entity Recognition and Part-of-Speech Sequences”, IberLEF@SEPLN 2021, 2021.
[7] B. Song, F. Li, Y. Liu, and X. Zeng, “Deep learning methods for biomedical named entity recognition: a survey and qualitative comparison”, Briefings in Bioinformatics, 2021.
[8] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, “Natural language processing (almost) from scratch,” The Journal of Machine Learning Research, Vol. 12, 2011, pp. 2493– 2537.
[8] X. Ma, and E. Hovy, “End-to-end sequence labeling via bi-directional lstm-cnns-crf”, 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016.
[10] O. Etzioni, M. Cafarella, D. Downey, A.M. Popescu, T. Shaked, S. Soderland, D.S. Weld, and A. Yates, “Unsupervised named entity extraction from the web: An experimental study”, Artificial intelligence, Vol. 165, NO. 1, 2005, pp. 91–134.
[11] S. Zhang, and N. Elhadad, “Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts,” Journal of Biomedical Informatics, Vol. 46, NO. 6, 2013, pp. 1088–1098.
[12] A.P. Quimbaya, A.S. Múnera, R.A.G. Rivera, J.C.D. Rodríguez, O.M.M. Velandia, A.A.G. Peña, and C. Labbé, “Named entity recognition over electronic health records through a combined dictionary-based approach”, Procedia Computer Science, Vl. 100, 2016, pp. 55–61.
[13] D. Hanisch, K. Fundel, H.T. Mevissen, R. Zimmer, and J. Fluck, “Prominer: rule-based protein and gene entity recognition”, BMC bioinformatics, Vol. 6, NO. 1:S14, 2005.
[14] G. Zhou, and J. Su, “Named entity recognition using an hmm based chunk tagger”, 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 2002.
[15] W. Liao, and S. Veeramachaneni, “A simple semi-supervised algorithm for named entity recognition”, NAACL HLT 2009 Workshop on Semi-supervised Learning for Natural Language Processing. Boulder, Colorado, 2009.
[16] J. Hoffart, M.A. Yosef, I. Bordino, H. Fürstenau, M. Pinkal, M. Spaniol, B. Taneva, S. Thater, and G. Weikum, “Robust disambiguation of named entities in text”, 2011 Conference on Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK, 2011.
[17] Z. Ji, A. Sun, G. Cong, and J. Han, “Joint recognition and linking of fine-grained locations from tweets”, International World Wide Web Conference Committee, Montréal, Québec, Canada, 2016.
[18] L. Jafar Tafreshi, and F. Soltanzadeh, “A Novel Approach to Conditional Random Field-based Named Entity Recognition using Persian Specific Features”, Journal of AI and Data Mining, Vol. 8, NO. 2, 2020, pp. 227-236.
[19] D. Nadeau, and S. Sekine, “A survey of named entity recognition and classification”, Lingvisticae Investigationes, Vol. 30, NO. 1, 2007, pp. 3-26.
[20] G. Petasis, A. Cucchiarelli, P. Velardi, G. Paliouras, V. Karkaletsis, and C.D. Spyropoulos, “Automatic adaptation of proper noun dictionaries through cooperation of machine learning and probabilistic methods”, 23rd annual international ACM SIGIR conference on Research and development in information retrieval, Athens, Greece, 2000.
[21] J. Hammerton, “Named entity recognition with long short-term memory”, Seventh Conference on Natural Language Learning at HLT-NAACL 2003, 2003, pp. 172–175.
[22] Z. Huang, W. Xu, and K. Yu, “Bidirectional lstm-crf models for sequence tagging”, arXiv preprint, 2015. arXiv:1508.01991.
[23] R. Chalapathy, E. Zare Borzeshi, and M. Piccardi, “An investigation of recurrent neural architectures for drug name recognition”, Seventh International Workshop on Health Text Mining and Information Analysis, Austin, TX, 2016.
[24] K. Xu, Z. Zhou, T. Hao, and W. Liu, “A bidirectional lstm and conditional random fields approach to medical named entity recognition”, International Conference on Advanced Intelligent Systems and Informatics, 2017.
[25] S. Yan, C. Hardmeier, and J. Nivre, “Multilingual named entity recognition using hybrid neural networks”, Sixth Swedish Language Technology Conference (SLTC), 2016.
[26] J. PC Chiu, and E. Nichols, “Named entity recognition with bidirectional lstm-cnns”, Transactions of the Association for Computational Linguistics, Vol. 4, 2016, pp. 357-370.
[27] S. Misawa, M. Taniguchi, Y. Miura, and T. Ohkuma, “Character-based bidirectional lstm-crf with words and characters for Japanese named entity recognition”, First Workshop on Subword and Character Level Models in NLP, Copenhagen, Denmark, 2017.
[28] C.N. dos Santos, and V. Guimaraes, “Boosting named entity recognition with neural character embeddings”, Proceedings of the Fifth Named Entity Workshop of the Association for Computational Linguistics, 2015, pp. 25-33.
[29] G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer. “Neural architectures for named entity recognition”, Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, pp. 260-270.
[30] M. Habibi, L. Weber, M. Neves, D.L. Wiegandt, and U. Leser, “Deep learning with word embeddings improves biomedical named entity recognition”, Bioinformatics, Vol. 33, No. 14, 2017, pp. 137–148.
[31] M Miwa, and M. Bansal. “End-to-end relation extraction using LSTMs on sequences and tree structures”, Association for Computational Linguistics, 2016, pp. 1105-1116.
[32] S. Zheng, J. Xu, P. Zhou, H. Bao,Q. Zhenyu, and B. Xu, “A neural network framework for relation extraction: learning entity semantic and relation pattern”, Knowledge-Based Systems, Vol. 114, 2016, pp. 12-23.
[33] A. Graves, and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM networks”, Journal of Neural Networks, Vol. 18, NO. 5-6, 2005, pp. 602-610.
[34] J. Lafferty, A. McCallum and F. CN Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data”, In: Proceedings of the 18th International Conference on Machine Learning 2001, San Francisco, US, 2001.
[35] S. Zheng, Y. Hao, D. Lu, H. Bao, J. Xu, H. Hao, and B. Xu. “Joint entity and relation extraction based on a hybrid neural network”, Journal of Neurocomputing, Vol. 257, 2017, pp. 59-66.
[36] T. Liu, J. Yao, and C. Lin, “Towards improving neural named entity recognition with gazetteers”, in ACL, 2019, pp. 5301-5307.
[37] C. Xia, C. Zhang, T. Yang, Y. Li, N. Du, X. Wu, W. Fan, F. Ma, and P. S. Yu, “Multi-grained named entity recognition”, in ACL, 2019, pp. 1430-1440.
Journal of Modeling in Engineering
Volume 20, Issue 69
May 2022
Pages 103-112

Files

History

  • Receive Date: 10 July 2021
  • Revise Date: 19 December 2021
  • Accept Date: 19 February 2022

Share

How to cite

Statistics