Journal of Modeling in Engineering

Journal of Modeling in Engineering

Topic Extraction from Persian Texts Using BERTopic Framework, Language Embedding Models, and Text Clustering

Document Type : Computer Article

Authors
Faculty of Electrical and Computer Engineering, Semnan University, Semnan, Iran
Abstract
With the growth of information, extracting knowledge from textual collections has become essential. Topic modeling is an unsupervised machine learning technique that uncovers the hidden themes in documents. In this paper, inspired by BERTopic, we present an unsupervised method for topic modeling on Persian texts. The proposed approach employs the LaBSE language embedding model to convert texts into embedding vectors, then reduces their dimensions using UMAP, and finally groups similar texts into clusters using the K-Means algorithm. Next, by forming a cluster-token matrix and applying a topic representation technique, various topics are extracted from each cluster. We compared LaBSE model with other language embedding models including XLM-R, ParsBERT, Paraphrase-multilingual-MiniLM-L12-v2, Shiraz, and HooshvareLab (RoBERTa). We also compared the K-Means and HDBSCAN clustering algorithms. For evaluation, the AsreIran dataset was used, and both the coherence evaluation metric (NPMI) and human evaluation confirmed the proposed method’s performance. In HDBSCAN, Hooshvare (RoBERTa) yielded the best coherence, while ParsBERT excelled in human evaluation. In K-Means, Paraphrase-multilingual-MiniLM-L12-v2 performed best in terms of coherence and LaBSE in human evaluation. The superiority of K-Means over HDBSCAN was also verified. Furthermore, using the AsreIran and Tasnim datasets separately, the proposed method was compared with non-negative matrix factorization, latent Dirichlet allocation, and latent semantic analysis, with results demonstrating its outstanding performance.
Keywords
Subjects

Volume 23, Issue 83
Autumn 2025
Pages 217-235

  • Receive Date 05 January 2025
  • Revise Date 28 March 2025
  • Accept Date 06 April 2025