| Title: | Probability and certainty in the performance of evolutionary and swarm optimization algorithms |
|---|
| Authors: | ID Ivković, Nikola (Author)
ID Kudelić, Robert (Author)
ID Črepinšek, Matej (Author) |
|---|
| Files: |  mathematics-10-04364-v2.pdf (490,48 KB)
MD5: 31EA437B8F335262B5A1234F377A216C
 https://www.mdpi.com/2227-7390/10/22/4364
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | Reporting the empirical results of swarm and evolutionary computation algorithms
is a challenging task with many possible difficulties. These difficulties stem from the stochastic
nature of such algorithms, as well as their inability to guarantee an optimal solution in polynomial
time. This research deals with measuring the performance of stochastic optimization algorithms, as
well as the confidence intervals of the empirically obtained statistics. Traditionally, the arithmetic
mean is used for measuring average performance, but we propose quantiles for measuring average,
peak and bad-case performance, and give their interpretations in a relevant context for measuring
the performance of the metaheuristics. In order to investigate the differences between arithmetic
mean and quantiles, and to confirm possible benefits, we conducted experiments with 7 stochastic
algorithms and 20 unconstrained continuous variable optimization problems. The experiments
showed that median was a better measure of average performance than arithmetic mean, based on
the observed solution quality. Out of 20 problem instances, a discrepancy between the arithmetic
mean and median happened in 6 instances, out of which 5 were resolved in favor of median and
1 instance remained unresolved as a near tie. The arithmetic mean was completely inadequate
for measuring average performance based on the observed number of function evaluations, while
the 0.5 quantile (median) was suitable for that task. The quantiles also showed to be adequate for
assessing peak performance and bad-case performance. In this paper, we also proposed a bootstrap
method to calculate the confidence intervals of the probability of the empirically obtained quantiles.
Considering the many advantages of using quantiles, including the ability to calculate probabilities
of success in the case of multiple executions of the algorithm and the practically useful method of
calculating confidence intervals, we recommend quantiles as the standard measure of peak, average
and bad-case performance of stochastic optimization algorithms. |
|---|
| Keywords: | algorithmic performance, experimental evaluation, metaheuristics, quantile, confidence interval, stochastic algorithms, evolutionary computation, swarm intelligence, experimental methodology |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 24.10.2022 |
|---|
| Article acceptance date: | 15.11.2022 |
|---|
| Publication date: | 20.11.2022 |
|---|
| Publisher: | MDPI AG |
|---|
| Year of publishing: | 2022 |
|---|
| Number of pages: | 29 str. |
|---|
| Numbering: | Vol. 10, no. 22 |
|---|
| PID: | 20.500.12556/DKUM-92315  |
|---|
| UDC: | 004.8 |
|---|
| ISSN on article: | 2227-7390 |
|---|
| COBISS.SI-ID: | 130112003 |
|---|
| DOI: | 10.3390/math10224364 |
|---|
| Copyright: | © 2022 by the authors |
|---|
| Publication date in DKUM: | 28.03.2025 |
|---|
| Views: | 144 |
|---|
| Downloads: | 19 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
