Evaluating the Accuracy of selected AI Tools in Identifying Methodological and Theoretical Gaps in Research Literature

Abstract

This study evaluates the accuracy and effectiveness of various AI tools in identifying methodological and theoretical research literature gaps in educational management. The comparative analysis involves six AI tools: Chat GPT4o, Chat GPT4, Chat GPT 3.5, Perplexity AI, SCI Space, and Julius AI. Each tool was tasked with identifying valid and relevant research gaps and providing accurate sources in APA 7th edition format. The validation process included cross-referencing AI-generated results with existing literature and evaluating the accessibility and relevance of the provided sources. The study's secondary data processing procedure ensured consistency and reliability by using standardized commands, cross-referencing sources through reputable academic databases, and conducting independent reviews. Gaps and sources made by AI were put together into a full file as part of the data analysis. Also looked at were how important and true the gaps were, and a scoring system was created to see how well each AI tool did. The results show that the AI tools are not all the same in how accurate and useful they are. The sources on Chat GPT4o and SCI Space were true and easy to find, so you could trust them. But Chat GPT4 and Chat GPT 3.5 were limited because they didn't have any sources or sources that worked with them. Perplexity AI had trouble with references that weren't very clear, but Julius AI did a good job of filling in the holes with good sources that were easy to find. People need to check the data that AI makes, says the study, to make sure that academic honesty and rigor are maintained. Researchers can learn useful things from this study about how to use AI tools in their work. It also shows how important it is to think critically about AI results and compare them to what has already been written. University ethics review boards might have to change a lot because of this study. This is because the study looks at how to make sure that research papers written by students, teachers, and staff are honest and of high quality.