| Title: | Ensemble-based knowledge distillation for identification of childhood pneumonia |
|---|
| Authors: | ID Vrbančič, Grega (Author)
ID Podgorelec, Vili (Author) |
|---|
| Files: |  electronics-14-03115.pdf (915,47 KB)
MD5: 9C1E2F1449121390503F2B2AA8CE2C30
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | Childhood pneumonia remains a key cause of global morbidity and mortality, highlighting the need for accurate and efficient diagnostic tools. Ensemble methods have proven to be among the most successful approaches for identifying childhood pneumonia from chest X-ray images. However, deploying large, complex convolutional neural network models in resource-constrained environments presents challenges due to their high computational demands. Therefore, we propose a novel ensemble-based knowledge distillation method for identifying childhood pneumonia from X-ray images, which utilizes an ensemble of classification models to distill the knowledge to a more efficient student model. Experiments conducted on a chest X-ray dataset show that the distilled student model achieves comparable (statistically not significantly different) predictive performance to that of the Stochastic Gradient with Warm Restarts ensemble method (F1-score on average 0.95 vs. 0.96, respectively), while significantly reducing inference time and decreasing FLOPs by a factor of 6.5. Based on the obtained results, the proposed method highlights the potential of knowledge distillation to enhance the efficiency of complex methods, making them more suitable for utilization in environments with limited computational resources. |
|---|
| Keywords: | knowledge distillation, convolutional neural networks, childhood pneumonia |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 26.05.2025 |
|---|
| Article acceptance date: | 03.08.2025 |
|---|
| Publication date: | 05.08.2025 |
|---|
| Publisher: | MDPI |
|---|
| Year of publishing: | 2025 |
|---|
| Number of pages: | 21 str. |
|---|
| Numbering: | Vol. 14, iss. 15, [art. no.] 3115 |
|---|
| PID: | 20.500.12556/DKUM-94527  |
|---|
| UDC: | 004.9 |
|---|
| ISSN on article: | 2079-9292 |
|---|
| COBISS.SI-ID: | 246010371 |
|---|
| DOI: | 10.3390/electronics14153115 |
|---|
| Copyright: | © 2025 by the authors |
|---|
| Publication date in DKUM: | 20.08.2025 |
|---|
| Views: | 138 |
|---|
| Downloads: | 9 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
