Uporaba transformer arhitekture nevronskih mrež za zapolnjevanje manjkajočih vrednosti v časovnih vrstah : magistrsko delo

Petelinek, Benjamin

| | SLO | ENG | Cookies and privacy

Bigger font | Smaller font

First page > Show document

Show document

Title:	Uporaba transformer arhitekture nevronskih mrež za zapolnjevanje manjkajočih vrednosti v časovnih vrstah : magistrsko delo
Authors:	ID Petelinek, Benjamin (Author) ID Vrbančič, Grega (Mentor) More about this mentor...
Files:	MAG_Petelinek_Benjamin_2025.pdf (2,27 MB) MD5: 533B25968C4A141CF7284AAC587FA4BE
Language:	Slovenian
Work type:	Master's thesis/paper
Typology:	2.09 - Master's Thesis
Organization:	FERI - Faculty of Electrical Engineering and Computer Science
Abstract:	V magistrskem delu se osredotočamo na problematiko manjkajočih vrednosti, ki pomembno vplivajo na napovedno uspešnost modelov strojnega učenja. V uvodnem delu magistrskega dela smo opisali problem manjkajočih vrednosti in teoretično predstavili osnove strojnega učenja. V eksperimentalnem delu smo razvili lastno arhitekturo za zapolnjevanje manjkajočih vrednosti. Pri razvoju smo se zgledovali po modelih GPT-2 in SAITS. Razvito arhitekturo smo ovrednotili, analizirali in rezultate primerjali z modeli linearne interpolacije, SAITS in KNN. Izkazalo se je, da je linearna interpolacija pri zapolnjevanju manjkajočih vrednosti PM2.5 najuspešnejša, vendar razlike med modeli linearne interpolacije, Transformer in SAITS ne presegajo 0,2 MAE. Glede na napovedno uspešnost se je arhitektura Transformer uvrstila na drugo mesto. Arhitektura KNN je ne glede na postajo ali delež manjkajočih vrednosti dosegla najslabši rezultat. Višje dimenzije vdelav so pri modelih Transformer izboljšale napovedno uspešnost, medtem ko pri modelih SAITS nismo videli podobnega učinka. Prav tako smo ugotovili, da višanje deleža manjkajočih vrednosti negativno vpliva na napovedno uspešnost modelov.
Keywords:	arhitektura Transformer, nevronske mreže, zapolnjevanje manjkajočih vrednosti, SAITS, strojno učenje.
Place of publishing:	Maribor
Place of performance:	Maribor
Publisher:	[B. Petelinek]
Year of publishing:	2025
Number of pages:	1 spletni vir (1 datoteka PDF (XI, 61 str.))
PID:	20.500.12556/DKUM-95819
UDC:	004.032.26:004.85(043.2)
COBISS.SI-ID:	262467331
Publication date in DKUM:	16.12.2025
Views:	144
Downloads:	28
Metadata:
Categories:	KTFMB - FERI
:	Copy citation

Average score:	(0 votes)
Your score:	Voting is allowed only for logged in users.
Share:

Hover the mouse pointer over a document title to show the abstract or click on the title to get all document metadata.

Licences

License:	CC BY 4.0, Creative Commons Attribution 4.0 International

Link:	http://creativecommons.org/licenses/by/4.0/
Description:	This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.
Licensing start date:	25.10.2025

Secondary language

Language:	English
Title:	Using transformer neural networks for missing value imputation in time series data
Abstract:	In this master’s thesis, we focus on the problem of missing values, which significantly affect the predictive performance of machine learning models. In the introductory part, we described the issue of missing values and provided a theoretical overview of the fundamentals of machine learning. In the experimental part, we developed our own architecture for imputing missing values, drawing inspiration from the GPT-2 and SAITS models. The developed architecture was compared with linear interpolation, SAITS, and KNN models. We evaluated the models and analyzed the results. It turned out that linear interpolation was the most successful method for imputing missing PM2.5 values. The results show that the differences between linear interpolation, Transformer, and SAITS do not exceed 0.2 MAE. Based on predictive performance, the Transformer architecture ranked second. Regardless of the station or the proportion of missing values, the KNN architecture achieved the worst results. We found that increasing the proportion of missing values negatively impacts the predictive performance of models. Larger input projection dimensions improved the predictive performance of Transformer models, while for SAITS models, we did not observe a similar effect.
Keywords:	Transformer architecture, neural networks, missing value imputation, SAITS, machine learning.

Comments

Leave comment

You must log in to leave a comment.

Comments (0)

0 - 0 / 0

There are no comments!

Back