Assessing AI’s problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory
Aldazharova S. Issayeva G. Maxutov S. Balta N.
October 2024Bastas
Contemporary Educational Technology
2024#16Issue 4
This study investigates the performance of GPT-4, an advanced AI model developed by OpenAI, on the force concept inventory (FCI) to evaluate its accuracy, reasoning patterns, and the occurrence of false positives and false negatives. GPT-4 was tasked with answering the FCI questions across multiple sessions. Key findings include GPT-4’s proficiency in several FCI items, particularly those related to Newton’s third law, achieving perfect scores on many items. However, it struggled significantly with questions involving the interpretation of figures and spatial reasoning, resulting in a higher occurrence of false negatives where the reasoning was correct, but the answers were incorrect. Additionally, GPT-4 displayed several conceptual errors, such as misunderstanding the effect of friction and retaining the outdated impetus theory of motion. The study’s findings emphasize the importance of refining AI-driven tools to make them more effective in educational settings. Addressing both AI limitations and common misconceptions in physics can lead to improved educational outcomes.
AI assisted learning , force concept inventory , GPT-4 , physics education
Text of the article Перейти на текст статьи
Abai Kazakh National Pedagogical University, Almaty, Kazakhstan
SDU University, Almaty, Kazakhstan
Abai Kazakh National Pedagogical University
SDU University
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026