| Title: | Accuracy is not enough: optimizing for a fault detection delay |
|---|
| Authors: | ID Šprogar, Matej (Author)
ID Verber, Domen (Author) |
|---|
| Files: |  AccuracyIsNotEnough23.pdf (478,93 KB)
MD5: B863E205A9C82F493381E08681CF63A7
 https://www.mdpi.com/2227-7390/11/15/3369
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | This paper assesses the fault-detection capabilities of modern deep-learning models. It highlights that a naive deep-learning approach optimized for accuracy is unsuitable for learning fault-detection models from time-series data. Consequently, out-of-the-box deep-learning strategies may yield impressive accuracy results but are ill-equipped for real-world applications. The paper introduces a methodology for estimating fault-detection delays when no oracle information on fault occurrence time is available. Moreover, the paper presents a straightforward approach to implicitly achieve the objective of minimizing fault-detection delays. This approach involves using pseudo-multi-objective deep optimization with data windowing, which enables the utilization of standard deep-learning methods for fault detection and expanding their applicability. However, it does introduce an additional hyperparameter that needs careful tuning. The paper employs the Tennessee Eastman Process dataset as a case study to demonstrate its findings. The results effectively highlight the limitations of standard loss functions and emphasize the importance of incorporating fault-detection delays in evaluating and reporting performance. In our study, the pseudo-multi-objective optimization could reach a fault-detection accuracy of 95% in just a fifth of the time it takes the best naive approach to do so. |
|---|
| Keywords: | artificial neural networks, deep learning, fault detection, accuracy, multi-objective optimization |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 29.06.2023 |
|---|
| Article acceptance date: | 31.07.2023 |
|---|
| Publication date: | 01.08.2023 |
|---|
| Publisher: | MDPI |
|---|
| Year of publishing: | 2023 |
|---|
| Number of pages: | 18 str. |
|---|
| Numbering: | Vol. 11, no. 15, [Article no.] 3396 |
|---|
| PID: | 20.500.12556/DKUM-86403  |
|---|
| UDC: | 004.8 |
|---|
| ISSN on article: | 2227-7390 |
|---|
| COBISS.SI-ID: | 160904707 |
|---|
| DOI: | 10.3390/math11153369 |
|---|
| Copyright: | © 2023 by the authors |
|---|
| Publication date in DKUM: | 30.11.2023 |
|---|
| Views: | 558 |
|---|
| Downloads: | 54 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
