A new sentence-based interpretative topic modeling and automatic topic labeling
Kozbagarov O. Mussabayev R. Mladenovic N.
May 2021MDPI AG
Symmetry
2021#13Issue 5
This article presents a new conceptual approach for the interpretative topic modeling problem. It uses sentences as basic units of analysis, instead of words or n-grams, which are commonly used in the standard approaches.The proposed approach’s specifics are using sentence probability evaluations within the text corpus and clustering of sentence embeddings. The topic model estimates discrete distributions of sentence occurrences within topics and discrete distributions of topic occurrence within the text. Our approach provides the possibility of explicit interpretation of topics since sentences, unlike words, are more informative and have complete grammatical and semantic constructions inside. The method for automatic topic labeling is also provided. Contextual embeddings based on the BERT model are used to obtain corresponding sentence embeddings for their subsequent analysis. Moreover, our approach allows big data processing and shows the possibility of utilizing the combination of internal and external knowledge sources in the process of topic modeling. The internal knowledge source is represented by the text corpus itself and often it is a single knowledge source in the traditional topic modeling approaches. The external knowledge source is represented by the BERT, a machine learning model which was preliminarily trained on a huge amount of textual data and is used for generating the context-dependent sentence embeddings.
Automatic topic labeling , BERT , Big data , Machine learning , Minimum sum-of-squares clustering (MSSC) , Natural language processing , Topic modeling , Transfer learning
Text of the article Перейти на текст статьи
Institute of Information and Computational Technologies, Pushkin Str., 125, Almaty, 050010, Kazakhstan
Khalifa University, Abu Dhabi, 41009, United Arab Emirates
Institute of Information and Computational Technologies
Khalifa University
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026