TIPS: A text interaction evaluation metric for learning model interpretation
Nie Z. Xiao Z. Wang T. Chronopoulos A.T. Andonie R. Mosavi A.
25 August 2025Elsevier Ltd
Expert Systems with Applications
2025#287
Explaining the decision-making behavior of deep neural networks (DNNs) can increase their trustworthiness in real-world applications. For natural language processing (NLP) tasks, many existing interpretation methods split the text according to the interactions between words. Also, the evaluation of explanation capability focuses on justifying the importance of the divided text spans from the perspective of interaction contribution. However, the prior evaluations are misled by extra interactions, making the evaluation unable to acquire accurate interactions within the text spans. Besides, existing research considers only absolute interaction contribution, which causes the evaluation to underestimate the important text spans with lower absolute interaction contribution and to overestimate the unimportant text spans with higher absolute interaction contribution. In this work, we propose a metric called Text Interaction Proportional Score (TIPS) to evaluate faithful interpretation methods. More specifically, we use a pick scheme to acquire the interactions within the divided text span and eliminate the influence of the extra interactions. Meanwhile, we utilize the relative interaction contribution between the divided text span and whole text to measure the importance of the acquired interactions. The proposed metric is validated using two interpretation methods in explaining three neural text classifiers (LSTM, CNN and BERT) on six benchmark datasets. Experiments show that TIPS outperforms a baseline method in three ways consistently and significantly (i.e., acquiring interactions within the text span, measuring importance of interaction, and distinguishing the important and unimportant text spans).
Interaction contribution , Interpretation method , Model interpretability , Quantitative evaluation
Text of the article Перейти на текст статьи
College of Information Science and Engineering, Hunan University, Changsha, China
Department of Computer Science, University of Texas, San Antonio, 78249, TX, United States
Department of Computer Science, Central Washington University, Ellensburg, 98926, WA, United States
Department of Electronics and Computers, Transilvania University of Braşov, Braşov, 500036, Romania
John von Neumann Faculty of Informatics, Obuda University, Budapest, 1034, Hungary
Institute of the Information Society, Ludovika University of Public Service, Budapest, Hungary
Department of Information and Computing Systems, Abylkas Saginov Karaganda Technical University, Karaganda, Kazakhstan
College of Information Science and Engineering
Department of Computer Science
Department of Computer Science
Department of Electronics and Computers
John von Neumann Faculty of Informatics
Institute of the Information Society
Department of Information and Computing Systems
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026