Investigating the Effectiveness and Benefits and Challenges of ChatGPT as a Learning Tool
A Quantitative Case Study
DOI:
https://doi.org/10.58963/qausrj.v28i28.301Keywords:
ChatGPT , Benefits , Effectiveness , Challenges , Learning ToolAbstract
Artificial Intelligence tools like ChatGPT have significantly impacted higher education, serving as an effective tool for enhancing learning experiences. ChatGPT facilitates personalized and collaborative learning, fostering creativity, motivation, and student engagement. It aids in analyzing and summarizing learning materials while providing accurate responses to students' inquiries. However, its use presents challenges such as academic integrity concerns, over-reliance, and masking students’ learning deficiencies.
This study aims to investigate the effectiveness, benefits, and challenges of ChatGPT as a learning tool from students' perspectives. A quantitative approach was adopted, utilizing an online questionnaire distributed to 231 undergraduate students from five universities. The survey focused on three key areas: students’ attitudes toward ChatGPT, its effectiveness, and the challenges associated with its use.
The findings revealed that students hold positive attitudes toward ChatGPT, considering it a valuable tool that enhances interactive learning and improves learning outcomes. However, challenges such as the potential masking of students' academic weaknesses and threats to educational integrity were identified. This study provides valuable insights into the role of ChatGPT in education and contributes to the development of strategies for its effective utilization.
Metrics
Downloads
References
Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-024-00444-7
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152
AlAfnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses. Journal of Artificial Intelligence and Technology. https://doi.org/10.37965/jait.2023.0184
Alenezi, M. A. K., Mohamed, A. M., & Shaaban, T. S. (2023). Revolutionizing EFL special education: how ChatGPT is transforming the way teachers approach language learning. Innoeduca. International Journal of Technology and Educational Innovation, 9(2), 5–23. https://doi.org/10.24310/innoeduca.2023.v9i2.16774
Anders, B. A. (2023). Is using ChatGPT cheating, plagiarism, both, neither, or forward thinking? Patterns (New York, N.Y.), 4(3), 100694. https://doi.org/10.1016/j.patter.2023.100694
Castillo, A. G. R., Silva, G. J. S., Arocutipa, J. P. F., Berrios, H. Q., Rodriguez, M. A. M., Reyes, G. Y., Lopez, H. R. P., Teves, R. M. V., Rivera, H. V. H., & Arias-Gonzáles, J. L. (2023). Effect of Chat GPT on the digitized learning process of university students. Journal of Namibian Studies : History Politics Culture, 33, 1–15. https://doi.org/10.59670/jns.v33i.411
Chan, M. M. K., Wong, I. S. F., Yau, S. Y., & Lam, V. S. F. (2023). Critical Reflection on Using ChatGPT in Student Learning. Nurse Educator, 48(6), E200–E201. https://doi.org/10.1097/nne.0000000000001476
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? (2023). Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9
ChatGPT in higher education: Considerations for academic integrity and student learning. (2023). Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.17
Chen, X., Zou, D., Xie, H., Chen, G., Lin, J., & Cheng, G. (2022). Exploring contributors, collaborations, and research topics in educational technology: A joint analysis of mainstream conferences. Education and Information Technologies, 28(2), 1323–1358. https://doi.org/10.1007/s10639-022-11209-y
Chinonso, O. E., Theresa, A. M.-E., & Aduke, T. C. (2023). ChatGPT for Teaching, Learning and Research: Prospects and Challenges. Global Academic Journal of Humanities and Social Sciences, 5(02), 33–40. https://doi.org/10.36348/gajhss.2023.v05i02.001
Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023). ChatGPT Goes to Law School. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4335905
de Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023). ChatGPT and the Rise of Large Language Models: The New AI-Driven Infodemic Threat in Public Health. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4352931
Deng, J., & Lin, Y. (2023). The Benefits and Challenges of ChatGPT: An Overview. Frontiers in Computing and Intelligent Systems, 2(2), 81–83. https://doi.org/10.54097/fcis.v2i2.4465
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/J.IJINFOMGT.2023.102642
Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060
Elkhatat, A. M. (2023). Evaluating the authenticity of ChatGPT responses: a study on text-matching capabilities. International Journal for Educational Integrity, 19(1). https://doi.org/10.1007/s40979-023-00137-0
Farhi, F., Jeljeli, R., Aburezeq, I., Dweikat, F. F., Al-shami, S. A., & Slamene, R. (2023). Analyzing the students’ views, concerns, and perceived ethics about chat GPT usage. Computers and Education: Artificial Intelligence, 5, 100180. https://doi.org/10.1016/j.caeai.2023.100180
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 61(3), 460–474. https://doi.org/10.1080/14703297.2023.2195846
Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A. M. A., & Hatta, H. R. (2023). Analysing the Role of ChatGPT in Improving Student Productivity in Higher Education. Journal on Education, 5(4), 14886–14891. https://doi.org/10.31004/joe.v5i4.2563
García-Peñalvo, F. J. (2023). La percepción de la Inteligencia Artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Education in the Knowledge Society (EKS), 24, e31279. https://doi.org/10.14201/eks.31279
Gašević, D., Siemens, G., & Sadiq, S. (2023). Empowering learners for the age of artificial intelligence. Computers and Education: Artificial Intelligence, 4, 100130. https://doi.org/10.1016/j.caeai.2023.100130
Guleria, A., Krishan, K., Sharma, V., & Kanchan, T. (2023). ChatGPT: ethical concerns and challenges in academics and research. The Journal of Infection in Developing Countries, 17(09), 1292–1299. https://doi.org/10.3855/jidc.18738
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089
How to harness the potential of ChatGPT in education? (2023). Knowledge Management & E-Learning: An International Journal, 133–152. https://doi.org/10.34105/j.kmel.2023.15.008
Jauhiainen, J. S., & Guerra, A. G. (2023). Generative AI and ChatGPT in School Children’s Education: Evidence from a School Lesson. Sustainability, 15(18), 14025. https://doi.org/10.3390/su151814025
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(2), 100115. https://doi.org/10.1016/j.tbench.2023.100115
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., de Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198–e0000198. https://doi.org/10.1371/journal.pdig.0000198
Li, L., Ma, Z., Fan, L., Lee, S., Yu, H., & Hemphill, L. (2023). ChatGPT in education: a discourse analysis of worries and concerns on social media. Education and Information Technologies, 29(9), 10729–10762. https://doi.org/10.1007/s10639-023-12256-9
Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023). Generating scholarly content with ChatGPT: ethical challenges for medical publishing. The Lancet Digital Health, 5(3), e105–e106. https://doi.org/10.1016/s2589-7500(23)00019-5
Luo, Y., Weng, H., Yang, L., Ding, Z., & Wang, Q. (2023). College Students’ Employability, Cognition, and Demands for ChatGPT in the AI Era Among Chinese Nursing Students: Web-Based Survey. JMIR Formative Research, 7, e50413–e50413. https://doi.org/10.2196/50413
Megahed, F. M., Chen, Y.-J., Ferris, J. A., Knoth, S., & Jones-Farmer, L. A. (2023). How generative AI models such as ChatGPT can be (mis)used in SPC practice, education, and research? An exploratory study. Quality Engineering, 36(2), 287–315. https://doi.org/10.1080/08982112.2023.2206479
Montenegro-Rueda, M., Fernández-Cerero, J., Fernández-Batanero, J. M., & López-Meneses, E. (2023). Impact of the Implementation of ChatGPT in Education: A Systematic Review. Computers, 12(8), 153. https://doi.org/10.3390/computers12080153
O. Ajlouni, A., Abd-Alkareem Wahba, F., & Salem Almahaireh, A. (2023). Students’ Attitudes Towards Using ChatGPT as a Learning Tool: The Case of the University of Jordan. International Journal of Interactive Mobile Technologies (IJIM), 17(18), 99–117. https://doi.org/10.3991/ijim.v17i18.41753
Rahman, Md. M., & Watanobe, Y. (2023). ChatGPT for Education and Research: Opportunities, Threats, and Strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003
Rozado, D. (2023). The Political Biases of ChatGPT. Social Sciences, 12(3), 148. https://doi.org/10.3390/socsci12030148
Sabzalieva, E., & Valentini, A. (2023). ChatGPT and artificial intelligence in higher education: quick start guide. IESALC. https://unesdoc.unesco.org/notice?id=p::usmarcdef_0000385146
Sallam, M. (2023). ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel, Switzerland), 11(6), 887. https://doi.org/10.3390/healthcare11060887
Silva, C. A. G. da, Ramos, F. N., de Moraes, R. V., & Santos, E. L. dos. (2024). ChatGPT: Challenges and Benefits in Software Programming for Higher Education. Sustainability, 16(3), 1245. https://doi.org/10.3390/su16031245
Singh, S., & Ramakrishnan, N. (2023). Is ChatGPT Biased? A Review. https://doi.org/10.31219/OSF.IO/9XKBU
Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments, 32(9), 5142–5155. https://doi.org/10.1080/10494820.2023.2209881
Susnjak, T., & Mcintosh, T. R. (2024). ChatGPT: The End of Online Exam Integrity? https://doi.org/10.3390/educsci14060656
Syed, A. J. A., & Wahas, Y. M. A. (2020). Challenges and Solutions in Teaching English through Poetry to EFL Students at Hajjah University: A Case Study of William Wordsworth’s Lucy and John Donne’s Death Poems. REFLections, 27(2), 189–198. https://doi.org/10.61508/refl.v27i2.248043
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1). https://doi.org/10.1186/s40561-023-00237-x
Twycross, A. (2004). Research design: qualitative, quantitative and mixed methods approachesResearch design: qualitative, quantitative and mixed methods approaches Creswell John W Sage 320 £29 0761924426 0761924426. Nurse Researcher, 12(1), 82–83. https://doi.org/10.7748/nr.12.1.82.s2
Wahas, Y. (2023). Challenges of E-Learning Faced by ESL Learners during the Covid-19 Pandemic: A Case Study. 3, 41–58. https://doi.org/10.22034/IJLTR.2023.169033
Wahas, Y. M. A., & Syed, A. J. A. (2024). E-assessment challenges during e-learning in higher education: A case study. Education and Information Technologies, 29(11), 14431–14450. https://doi.org/10.1007/s10639-023-12421-0
What ChatGPT means for universities: Perceptions of scholars and students. (2023). Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.22
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2024 by the Author(s) mentioned in this article.
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
