20.500.12556/DKUM-89785 Computer science education in ChatGPT Era: experiences from an experiment in a programming course for novice programmers The use of large language models with chatbots like ChatGPT has become increasingly popular among students, especially in Computer Science education. However, significant debates exist in the education community on the role of ChatGPT in learning. Therefore, it is critical to understand the potential impact of ChatGPT on the learning, engagement, and overall success of students in classrooms. In this empirical study, we report on a controlled experiment with 182 participants in a first-year undergraduate course on object-oriented programming. Our differential study divided students into two groups, one using ChatGPT and the other not using it for practical programming assignments. The study results showed that the students’ performance is not influenced by ChatGPT usage (no statistical significance between groups with a p-value of 0.730), nor are the grading results of practical assignments (p-value 0.760) and midterm exams (p-value 0.856). Our findings from the controlled experiment suggest that it is safe for novice programmers to use ChatGPT if specific measures and adjustments are adopted in the education process. large language models ChatGPT artificial intelligence controlled experiment object-oriented programming software engineering education jezikovni modeli umetna inteligenca objektno orientirano programiranje true false true MDPI Angleški jezik Slovenski jezik https://www.mdpi.com/2227-7390/12/5/629 © 2024 by the authors. Članek v reviji 2024-08-12 11:54:17 2024-08-12 11:54:18 2025-07-23 03:11:04 0000-00-00 00:00:00 2024 0 0 22 str. št. 5, št. članka 629 let. 12 2024 0000-00-00 Zaloznikova Objavljeno NiDoloceno 2024-01-19 2024-02-18 2024-02-21 621.3 187200259 10.3390/math12050629 2227-7390 mathematics-12-00629.pdf mathematics-12-00629.pdf 1 AA1B3164F86A12A0B49D153C5726EACE d1fc7125c2d1332875bb503a5cd82afabe3b1b59924c2e91c6452a3f3fb2f6fa c54a43d9-66d1-11f0-b4a7-00155d000105 https://dk.um.si/Dokument.php?lang=slv&id=193913 https://www.mdpi.com/2227-7390/12/5/629 0 ccb26dd5-66d1-11f0-b4a7-00155d000105 https://dk.um.si/Dokument.php?lang=slv&id=193914 Fakulteta za elektrotehniko, računalništvo in informatiko 0 0 0