20.500.12556/DKUM-96750 The impact of large language models on programming education and student learning outcomes Recent advancements in Large Language Models (LLMs) like ChatGPT and Copilot have led to their integration into various educational domains, including software development education. Regular use of LLMs in the learning process is still not well-researched; thus, this paper intends to fill this gap. The paper explores the nuanced impact of informal LLM usage on undergraduate students’ learning outcomes in software development education, focusing on React applications. We carefully designed an experiment involving thirty-two participants over ten weeks where we examined unrestricted but not specifically encouraged LLM use and their correlation with student performance. Our results reveal a significant negative correlation between increased LLM reliance for critical thinking-intensive tasks such as code generation and debugging and lower final grades. Furthermore, a downward trend in final grades is observed with increased average LLM use across all tasks. However, the correlation between the use of LLMs for seeking additional explanations and final grades was not as strong, indicating that LLMs may serve better as a supplementary learning tool. These findings highlight the importance of balancing LLM integration with the cultivation of independent problem-solving skills in programming education. large language models LLMs ChatGPT Copilot programming education react debugging programski jeziki učenje programiranja umetna inteligenca true false true MDPI Angleški jezik Slovenski jezik https://www.mdpi.com/journal/applsci © 2024 by the authors Članek v reviji 2026-01-27 12:38:36 2026-01-27 12:38:37 2026-01-28 03:13:08 0000-00-00 00:00:00 2024 0 0 15 str. iss. 10, [article no.] 4115 Vol. 14 13 May 2024 0000-00-00 Zaloznikova Objavljeno NiDoloceno 2024-03-24 2024-05-09 2024-05-13 004.43 195312131 10.3390/app14104115 2076-3417 applsci-14-04115.pdf applsci-14-04115.pdf 1 EE515D053FCA511F345B4B851C132CBD 7da8ff08d45d4939d71fb60681b7d6a45fec830e20c725c8b0d47d5016c809e9 e4126a5d-fb74-11f0-9c79-00155d00018d https://dk.um.si/Dokument.php?lang=slv&id=202619 https://www.mdpi.com/2076-3417/14/10/4115 1 b95c819c-fb74-11f0-9c79-00155d00018d https://dk.um.si/Dokument.php?lang=slv&id=202618 Fakulteta za elektrotehniko, računalništvo in informatiko 0 0 0