| Title: | The impact of large language models on programming education and student learning outcomes |
|---|
| Authors: | ID Jošt, Gregor (Author)
ID Taneski, Viktor (Author)
ID Karakatič, Sašo (Author) |
|---|
| Files: |  applsci-14-04115.pdf (3,56 MB)
MD5: EE515D053FCA511F345B4B851C132CBD
 https://www.mdpi.com/2076-3417/14/10/4115
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | Recent advancements in Large Language Models (LLMs) like ChatGPT and Copilot have led to their integration into various educational domains, including software development education. Regular use of LLMs in the learning process is still not well-researched; thus, this paper intends to fill this gap. The paper explores the nuanced impact of informal LLM usage on undergraduate students’ learning outcomes in software development education, focusing on React applications. We carefully designed an experiment involving thirty-two participants over ten weeks where we examined unrestricted but not specifically encouraged LLM use and their correlation with student performance. Our results reveal a significant negative correlation between increased LLM reliance for critical thinking-intensive tasks such as code generation and debugging and lower final grades. Furthermore, a downward trend in final grades is observed with increased average LLM use across all tasks. However, the correlation between the use of LLMs for seeking additional explanations and final grades was not as strong, indicating that LLMs may serve better as a supplementary learning tool. These findings highlight the importance of balancing LLM integration with the cultivation of independent problem-solving skills in programming education. |
|---|
| Keywords: | large language models, LLMs, ChatGPT, Copilot, programming education, react, debugging |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 24.03.2024 |
|---|
| Article acceptance date: | 09.05.2024 |
|---|
| Publication date: | 13.05.2024 |
|---|
| Publisher: | MDPI |
|---|
| Year of publishing: | 2024 |
|---|
| Number of pages: | 15 str. |
|---|
| Numbering: | Vol. 14, iss. 10, [article no.] 4115 |
|---|
| PID: | 20.500.12556/DKUM-96750  |
|---|
| UDC: | 004.43 |
|---|
| ISSN on article: | 2076-3417 |
|---|
| COBISS.SI-ID: | 195312131 |
|---|
| DOI: | 10.3390/app14104115 |
|---|
| Copyright: | © 2024 by the authors |
|---|
| Publication date in DKUM: | 27.01.2026 |
|---|
| Views: | 260 |
|---|
| Downloads: | 4 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
