The Influence of AI-Generated Feedback on Students’ Debate Skills in University
DOI:
https://doi.org/10.21070/jees.v10i2.1941Keywords:
AI, ChatGPT, debate, argumentAbstract
The use of Artificial Intelligence (AI) tools like ChatGPT has become more common in education, especially at the university level. When it comes to learning English through debating, ChatGPT has the potential to support students by providing feedback not only on language use, but also on the content of their arguments. The purpose of this study is to investigate how using ChatGPT might impact university students’ performance in debate. The method used a mixed-method exploratory approach, combining a pre-experimental design with interviews to get a fuller picture. Eight students took part in a simulated British Parliamentary debate. They completed assessments before and after the intervention, debating two different motions which had the same topic to collect data. The finding showed that there was no improvement in students’ debate scores after using ChatGPT. The average scores were the same, 75.75, both before and after the intervention. Although there was a moderate positive correlation between pretest and posttest scores (r=0.826), the identical averages suggest that the ChatGPT tool did not have a significant effect. Interviews revealed that the students often copied the AI’s responses instead of critically engaging with the material which weakened the quality of their debate.
Keywords: Artificial Intelligence, ChatGPT, debate, argument
Downloads
References
Aisyah, Yulianti, P.D., Yandhini, S., Sari, A.D.P., Herawani, I., & Oktarini, Ira. (2024). The influence of ai on students' mind patterns. BICC_Proceedings, 2, 183-186. https://doi.org/10.30983/bicc.v1i1.125
Akopiants, N. (2023). USING CHATGPT IN THE PROCESS OF LEARNING ENGLISH: ADVANTAGES AND OPPORTUNITIES. Bulletin of the National Technical University “KhPI”. Series: Actual Problems of Ukrainian Society Development, 1, 69–72. https://doi.org/10.20998/2227-6890.2023.1.13
Alanzi, T., Alzahrani, W., Albalawi, N., Allahyani, T., Alghamdi, A., Al-Zahrani, H. … & Farhah, N. (2023). Public awareness of obesity as a risk factor for cancer in central saudi arabia: feasibility of chatgpt as an educational intervention. Cureus. https://doi.org/10.7759/cureus.50781
Alasmari, A. & Ahmed, S. (2012). Using debate in efl classes. English Language Teaching, 6(1). https://doi.org/10.5539/elt.v6n1p147
Ali, M., Faridi, A., & Sakhiyya, Z. (2020). Performance of critical thinking and existence of logical fallacies in indonesian varsity english debate 2020 in jakarta. International Journal of Research and Studies Publishing, 11(1), 655-659. https://doi.org/10.29322/ijsrp.11.01.2021.p10980
Almazyad, M., Aljofan, F., Abouammoh, N., Muaygil, R., Malki, K., Aljamaan, F., … & Temsah, M. (2023). Enhancing expert panel discussions in pediatric palliative care: innovative scenario development and summarization with chatgpt-4. Cureus. https://doi.org/10.7759/cureus.38249
Astuti, A., Suryani, F., & Kurniati, D. (2022). The efl student teachers' debate practice: problems and solutions. Prominent, 5(1), 72-78. https://doi.org/10.24176/pro.v5i1.7028
Ban, B., Pang, S., & Em, S. (2023). Debate: one of the key factors to improving students’ english language speaking skills. Journal of General Education and Humanities, 2(2), 107-120. https://doi.org/10.58421/gehu.v2i2.69
Baziganya, D. (2024). Investigating teachers and students’ views on debate activities as a tool to improve english speaking skills: a case of rulindo district, rwanda. African Journal of Empirical Research, 5(2), 869-880. https://doi.org/10.51867/ajernet.5.2.77
Chen, X., Wang, L., Zhai, X., & Li, Y. (2022). Exploring the effects of argument map-supported online group debate activities on college students’ critical thinking. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.856462
Chew, Q., Seet, X., & Sim, K. (2021). Use of debate as a pedagogical tool in psychiatry residency teaching: a cross-sectional study. Advances in Medical Education and Practice, Volume 12, 871-877. https://doi.org/10.2147/amep.s320614
Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches (Fifth edition). Los Angeles: SAGE.
Daniswara, L. D., & Cahyono, B.Y. (2023). Investigation of university students’ critical thinking in debate: Justification for the “AREL” argumentation process. JEES (Journal of English Educators Society), 8(2). https://doi.org/10.21070/jees.v8i2.1766
Didonna, N., Shetty, P. N., Khan, K., & Damitz, L. (2024). Unveiling the Potential of AI in Plastic Surgery Education: A Comparative Study of Leading AI Platforms’ Performance on In-training Examinations. Plastic and Reconstructive Surgery. Global Open, 12(6). https://doi.org/10.1097/gox.0000000000005929
Đoàn, T.N. (2023). Motivation from Technology Innovation in Teaching Foreign Languages: A Case Study in Vietnam. Vietnam Academy of Social Sciences's VSSR, 214(2), 92-104. https://doi.org/10.56794/VSSR.2(214).92-104
Esmaeil, A., Dzulkifli, D., Maakip, I., Mantaluk, O., & Marshall, S. (2023). Understanding student perception regarding the use of chatgpt in their argumentative writing: a qualitative inquiry. Jurnal Komunikasi Malaysian Journal of Communication, 39(4), 150-165. https://doi.org/10.17576/jkmjc-2023-3904-08
Ghaleb Barabad, M., & Bilal Anwar, M. (2024). Exploring ChatGPT’s Role in English Learning for EFL Students: Insights and Experiences. International Journal of Innovative Science and Research Technology (IJISRT), 755–766. https://doi.org/10.38124/ijisrt/ijisrt24sep347
Guleria, A., Krishan, K., Sharma, V., & Kanchan, T. (2023). Chatgpt: ethical concerns and challenges in academics and research. The Journal of Infection in Developing Countries, 17(09), 1292-1299. https://doi.org/10.3855/jidc.18738
Gupta, D., Sreelatha, D., Latha, A., Raj, D., & Singh, D. (2024). Navigating the future of education: the impact of artificial intelligence on teacher-student dynamics. EATP, 6006-6013. https://doi.org/10.53555/kuey.v30i4.2332
Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
Han, J., Park, J., & Lee, H. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. BMC Medical Education, 22(1). https://doi.org/10.1186/s12909-022-03898-3
Hernawan, A. & Yusuf, F. (2021). English competitive debating challenges. Atlantis Press, Proceedings of the Thirteenth Conference on Applied Linguistics (CONAPLIN 2020). https://doi.org/10.2991/assehr.k.210427.007
Idroes, G. M., Noviandy, T. R., Maulana, A., Irvanizam, I., Jalil, Z., Lensoni, L., ... & Idroes, R. (2023). Student perspectives on the role of artificial intelligence in education: A survey-based analysis. Journal of Educational Management and Learning, 1(1), 8-15. https://doi.org/10.60084/jeml.v1i1.58
Ifraheem, S., Rasheed, M., & Siddiqui, A. (2024). Transforming education through artificial intelligence: personalization, engagement and predictive analytics. J. Asian Dev. Studies, 13(2), 250-266. https://doi.org/10.62345/jads.2024.13.2.22
Ivanova, M. & Petrova, T. (2023). Towards independent students’ activities, online environment and learning performance: an investigation through synthetic data and artificial neural networks. MDPI, 10(2), 37. https://doi.org/10.3390/informatics10020037
Jiang, F. & Hyland, K. (2024). Does ChatGPT Argue Like Students? Bundles in Argumentative Essays, Applied Linguistics, amae052, https://doi.org/10.1093/applin/amae052
Karakose, T. & Tülübaş, T. (2023). How can chatgpt facilitate teaching and learning: implications for contemporary education. Educational Process International Journal, 12(4). https://doi.org/10.22521/edupij.2023.124.1
Ko, T. & Mezuk, B. (2021). Debate participation and academic achievement among high school students in the houston independent school district: 2012 - 2015. Educational Research and Reviews, 16(6), 219-225. https://doi.org/10.5897/err2021.4137
Lestari, Y. (2018). Students’ perceptions toward speaking achievement and critical thinking on the use of british parliamentary debating system. Els Journal on Interdisciplinary Studies in Humanities, 1(4), 441-447. https://doi.org/10.34050/els-jish.v1i4.5187
Liu, C.-L., Ho, C.-T., & Wu, T.-C. (2024). Custom GPTs Enhancing Performance and Evidence Compared with GPT-3.5, GPT-4, and GPT-4o? A Study on the Emergency Medicine Specialist Examination. Healthcare (Basel, Switzerland), 12(17). https://doi.org/10.3390/healthcare12171726
Liu, H., Azam, M., Naeem, S., & Faiola, A. (2023). An overview of the capabilities of chatgpt for medical writing and its implications for academic integrity. Health Information & Libraries Journal, 40(4), 440-446. https://doi.org/10.1111/hir.12509
Liu, Z. (2023). Research on the impact of chatgpt on the accounting industry and countermeasures. Frontiers in Business, Economics and Management, 11(2), 49-53. https://doi.org/10.54097/fbem.v11i2.12162
Minh, A. (2024). Leveraging chatgpt for enhancing english writing skills and critical thinking in university freshmen. Journal of Knowledge Learning and Science Technology Issn 2959-6386 (Online), 3(2), 51-62. https://doi.org/10.60087/jklst.vol3.n2.p62
Miyazaki, Y. (2024). Performance and errors of chatgpt-4o on the japanese medical licensing examination: solving all questions including images with over 90% accuracy. JMIR Medical Education. https://doi.org/10.2196/preprints.63129
Munir, H., Vogel, B., & Jacobsson, A. (2022). Artificial intelligence and machine learning approaches in digital education: a systematic revision. MDPI, 13(4), 203. https://doi.org/10.3390/info13040203
Nguyen, H, Ngoc, H., & Dan, T. (2024). Efl students’ perceptions and practices of using chatgpt for developing english argumentative essay writing skills. European Journal of Alternative Education Studies, 9(1). https://doi.org/10.46827/ejae.v9i1.5341
Nugroho, A., Syamsi, K., & Putro, N. H. P. S. (2023). The Potentials of ChatGPT for Language Learning: Unpacking its Benefits and Limitations. Register Journal, 16(2), 224–247. https://doi.org/10.18326/register.v16i2.224-247
Nurakhir, A., Palupi, F., Langeveld, C., & Nurmalia, D. (2020). Students’ views of classroom debates as a strategy to enhance critical thinking and oral communication skills. Nurse Media Journal of Nursing, 10(2), 130-145. https://doi.org/10.14710/nmjn.v10i2.29864
Onesi-Ozigagun, O. (2024). Revolutionizing education through ai: a comprehensive review of enhancing learning experiences. International Journal of Applied Research in Social Sciences, 6(4), 589-607. https://doi.org/10.51594/ijarss.v6i4.1011
Oon, M., Syn, N., Tan, N., Tan, K., & Ng, S. (2023). Bridging bytes and biopsies: a comparative analysis of chatgpt and histopathologists in pathology diagnosis and collaborative potential. Histopathology, 84(4), 601-613. https://doi.org/10.1111/his.15100
Pahmi, P. & Niah, S. (2021). Students’ voices towards online debate through whatsapp as alternative media to enhance critical thinking skills during covid-19 pandemic. International Journal of Interactive Mobile Technologies (Ijim), 15(05), 205. https://doi.org/10.3991/ijim.v15i05.18759
Pang, S., Nol, E., & Heng, K. (2024) ChatGPT-4o for English language teaching and learning: Features, applications, and future prospects. Cambodian Education Forum, (4)1, 35-56. http://dx.doi.org/10.2139/ssrn.4837988
Petrič, G. (2024). Everyone talks everything with chatgpt: Students’ uses of ChatGPT and their impact on learning performance. International Journal of Technology and Human Interaction, 20(1), 1-22. https://doi.org/10.4018/ijthi.349225
Rahmi, A. (2021). High school debate: elevating students critical thinking into hoax prevention. Atlantis Press, Proceedings of the 2nd Southeast Asian Acadenic Forum on Sustainable Development (SEA-AFSID 2018). https://doi.org/10.2991/aebmr.k.210305.069
Rashid, A., Skelly, R., Valdes, C., Patel, P., Solberg, L., Giordano, C., Modave, F. (2024). Evaluating chatgpt’s moral competence in health care-related ethical problems. Jamia Open, 7(3). https://doi.org/10.1093/jamiaopen/ooae065
Ruff, E., Engen, M., Franz, J., Mauser, J., West, J., & Zemke, J. (2024). Chatgpt writing assistance and evaluation assignments across the chemistry curriculum. Journal of Chemical Education, 101(6), 2483-2492. https://doi.org/10.1021/acs.jchemed.4c00248
Ruff, E. F., Franz, J. L., & West, J. K. (2024). Using chatgpt for method development and green chemistry education in upper-level laboratory courses. Journal of Chemical Education, 101(8), 3224-3232. https://doi.org/10.1021/acs.jchemed.4c00193
Sanonguthai, S. (2011). Teaching ielts writing module through english debate: a case study in thailand. Language Testing in Asia, 1(4). https://doi.org/10.1186/2229-0443-1-4-39
Sawyer, D. (2015). Step your game up. Humanity & Society, 39(2), 224-235. https://doi.org/10.1177/0160597615574552
Sen, T., Naven, G., Gerstner, L., Bagley, D., Baten, R., Rahman, W., … & Hoque, E. (2023). Dbates: dataset for discerning benefits of audio, textual, and facial expression features in competitive debate speeches. Ieee Transactions on Affective Computing, 14(2), 1028-1043. https://doi.org/10.1109/taffc.2021.3103442
Seniwati, S., Lumumba, P., & Culla, A. S. (2024). The effect of gpt chat on indonesian and italian national security. International Journal of Social Science Humanity &Amp; Management Research, 3(01). https://doi.org/10.58806/ijsshmr.2024.v3i1n08
Shehri, F. A., Maham, R., Malik, A., & Saif, O. B. (2023). Effects of chatgpt on students academic performance:mediating role of prompt engineering. The Asian Bulletin of Big Data Management, 3(2), 137-147. https://doi.org/10.62019/abbdm.v3i2.58
Spaska, A., Savishchenko, V., Komar, O., Нritchenko, T., & Maidanyk, O. (2021). Enhancing analytical thinking in tertiary students using debates. European Journal of Educational Research, volume-10-2021(volume-10-issue-2-april-2021), 879-889. https://doi.org/10.12973/eu-jer.10.2.879
Sugianti & Rosidah, I. (2024). The use of artificial intelligence (ai) in learning results for scientific indonesian language courses at pgri wiranegara university. IJARSS, 2(1), 1-10. https://doi.org/10.59890/ijarss.v2i1.1174
Tjahyaningtijas, H. (2023). Machine learning on academic education: bibliometric studies. E3s Web of Conferences, 450, 02010. https://doi.org/10.1051/e3sconf/202345002010
Trisnawati, W. (2023). The impact of artificial intelligent in education toward 21st century skills: a literature review. PIJED, 2(2), 501-513. https://doi.org/10.59175/pijed.v2i2.152
Wang, B., Sun, H., & Yue, X. (2023). Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate. cornell university. https://doi.org/10.48550/arxiv.2305.13160
Wang, Y. (2024). Reviewing the usage of chatgpt on l2 students' english academic writing learning. Journal of Education Humanities and Social Sciences, 30, 173-178. https://doi.org/10.54097/dvjkj706
Wariyati, W. (2023). Overcoming speaking barrier. Journey (Journal of English Language and Pedagogy), 6(3), 713-720. https://doi.org/10.33503/journey.v6i3.3775
Xiao, Y. & Zhi, Y. (2023). An exploratory study of efl learners’ use of chatgpt for language learning tasks: experience and perceptions. Languages, 8(3), 212. https://doi.org/10.3390/languages8030212
Xu, J. (2023). Students' cognition of artificial intelligence in education. Applied and Computational Engineering, 22(1), 200-209. https://doi.org/10.54254/2755-2721/22/20231218
Yang, C., Chiang, F., Cheng, Q., & Ji, J. (2021). Machine learning-based student modeling methodology for intelligent tutoring systems. Journal of Educational Computing Research, 59(6), 1015-1035. https://doi.org/10.1177/0735633120986256
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 1970 Landry Dwiyoga Daniswara

This work is licensed under a Creative Commons Attribution 4.0 International License.




