A Comparative Analysis of LDA and BERTopic for Identifying Research Topic Trends in Computer Science Using Journal Article Titles

M. Zam Zami; Andreo Yudertha

doi:10.69693/ijmst.v4i3.14485

Authors

M. Zam Zami Universitas Islam Negeri Sulthan Thaha Saifuddin Jambi
Andreo Yudertha Universitas Islam Negeri Sulthan Thaha Saifuddin Jambi

DOI:

https://doi.org/10.69693/ijmst.v4i3.14485

Keywords:

Topic Modeling, Latent Dirichlet Allocation, BERTopic, Research Trends, Short Texts, Computer Science, Bibliometrics, Indonesia

Abstract

Topic modeling from journal article titles is challenging because titles are short and provide limited word co occurrence and contextual information. This study compares Latent Dirichlet Allocation (LDA) and BERTopic for identifying computer science research topics and examining their temporal trends. The corpus comprises 39,532 titles published between 2015 and 2025 across 283 Indonesian journals indexed in the Science and Technology Index (SINTA). Of these, 46.2% were originally written in English and translated into Indonesian before modeling. Both approaches were evaluated using the same corpus, vocabulary, topic coherence (C_v), topic diversity, and three random seeds. LDA was tested across topic counts from 5 to 30, while BERTopic used multilingual sentence embeddings, UMAP, and HDBSCAN and was reduced to 25 topics for comparison. Temporal trends for 2015–2024 were assessed using three statistical tests and robustness checks for journal composition, SINTA ranking, and original title language. At 25 topics, BERTopic achieved higher coherence (0.671, SD = 0.017) and diversity (0.841) than LDA (0.580, SD = 0.039; diversity = 0.693), but left 37.9% of titles unassigned. LDA assigned all titles and produced a more balanced topic-size distribution. Agreement between the partitions was limited (NMI = 0.31), with 11 of 25 LDA topics having one-to-one counterparts. Seven method–topic combinations passed all reported trend checks, including sentiment analysis, deep learning, and IoT monitoring, whereas some steep declines reflected changes in journal composition. The findings reveal a trade-off between topic quality and document coverage and indicate that research trends should be interpreted only when they remain robust across methods and corpus-composition checks.

References

[1] Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences, 101(suppl. 1), 5228–5235. https://doi.org/10.1073/pnas.0307752101

[2] Yan, X., Guo, J., Lan, Y., & Cheng, X. (2013). A biterm topic model for short texts. In Proceedings of the 22nd International Conference on World Wide Web (WWW 2013) (pp. 1445–1456). ACM. https://doi.org/10.1145/2488388.2488514

[3] Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.

[4] Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv:2203.05794. https://doi.org/10.48550/arXiv.2203.05794

[5] Egger, R., & Yu, J. (2022). A topic modeling comparison between LDA, NMF, Top2Vec, and BERTopic to demystify Twitter posts. Frontiers in Sociology, 7, 886498. https://doi.org/10.3389/fsoc.2022.886498

[6] Dieng, A. B., Ruiz, F. J. R., & Blei, D. M. (2020). Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics, 8, 439–453. https://doi.org/10.1162/tacl_a_00325

[7] Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT networks. In Proceedings of EMNLP-IJCNLP 2019 (pp. 3982–3992). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1410

[8] McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426. https://doi.org/10.48550/arXiv.1802.03426

[9] Campello, R. J. G. B., Moulavi, D., & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining (PAKDD 2013), LNCS 7819 (pp. 160–172). Springer. https://doi.org/10.1007/978-3-642-37456-2_14

[10] McInnes, L., Healy, J., Astels, S. (2017). hdbscan: Hierarchical density based clustering. Journal of Open Source Software, 205. https://doi.org/10.21105/joss.00205 [11] Reimers, N., & Gurevych, I. (2020). Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of EMNLP 2020 (pp. 4512–4525). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.365

[12] Röder, M., Both, A., & Hinneburg, A. (2015). Exploring the space of topic coherence measures. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM 2015) (pp. 399–408). ACM. https://doi.org/10.1145/2684822.2685324

[13] Pedregosa, F., Varoquaux, G., Gramfort, A., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

[14] Řehůřek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks (pp. 45–50). ELRA.

[15] Hubert, L., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2(1), 193–218. https://doi.org/10.1007/BF01908075

[16] Vinh, N. X., Epps, J., & Bailey, J. (2010). Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11, 2837–2854.

[17] Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1–2), 81–93. https://doi.org/10.1093/biomet/30.1-2.81

[18] Sen, P. K. (1968). Estimates of the regression coefficient based on Kendall’s tau. Journal of the American Statistical Association, 63(324), 1379–1389. https://doi.org/10.1080/01621459.1968.10480934

[19] Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

[20] Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series B, 13(2), 238–241. https://doi.org/10.1111/j.2517-6161.1951.tb00088.x

[21] Cramér, H. (1946). Mathematical Methods of Statistics. Princeton University Press.

[22] Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1), 145–151. https://doi.org/10.1109/18.61115

[23] Virtanen, P., Gommers, R., Oliphant, T. E., et al. (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261–272. https://doi.org/10.1038/s41592-019-0686 2 [24] Seabold, S., & Perktold, J. (2010). Statsmodels: Econometric and statistical modeling with Python. In Proceedings of the 9th Python in Science Conference (pp. 92–96).