20.500.12556/DKUM-99735 Fairness evaluation paradox how biased test data masks true group fairness assessment The EU AI Act makes fairness metrics for high-risk AI systems’ compliance evidence, turning their trustworthiness into a safety and accountability concern. Fairness audits assume that test data reflects the properties of real-world conditions, whereas standard evaluation protocols use test data drawn from the same biased records as the training data. Studies measuring how strongly this bias in test data distorts fairness metrics between the validation phase and real-world deployment are still very rare. We conduct an experiment on synthetic and real data, measuring this discrepancy across five fairness interventions on four unfairness types (1000 repetitions per combination, 20,000 total runs). Synthetic data lets us encode human bias in labels and compare fairness metrics on biased test labels (data available during development) against clean labels (conditions models face in deployment). We find that a systematic evaluation bias is present across all metrics, so the same models on the same test data can support opposite fairness conclusions and mask the mistreatment of the most disadvantaged groups. This pattern of fairness misevaluation is confirmed by a real-world validation on the Adult Census Income dataset. We conclude that trustworthy fairness auditing and regulatory standards should require bias-aware evaluation protocols, in which observed labels are not treated as ground truth. algorithmic fairness evaluation bias fairness metrics fairness interventions machine learning EU AI Act trustworthy AI model auditing algoritemska pravičnost metrike pravičnosti pristranskost pri vrednotenju strojno učenje true false true MDPI Angleški jezik Slovenski jezik https://www.mdpi.com/journal/mathematics © 2026 by the authors Članek v reviji 2026-08-25 13:29:43 2026-08-25 13:29:44 2026-08-26 03:09:51 0000-00-00 00:00:00 2026 0 0 28 str. iss. 16, [article no.] 2894 Vol. 14 2026 0000-00-00 Zaloznikova Objavljeno NiDoloceno 2026-07-06 2026-07-27 2026-08-11 004.8 288823043 10.3390/math14162894 2227-7390 https://www.mdpi.com/2227-7390/14/16/2894 1 46092dcd-a078-11f1-a977-00155d0001be https://dk.um.si/Dokument.php?lang=slv&id=210420 mathematics-14-02894.pdf mathematics-14-02894.pdf 1 2F4F92A129B0D74F8FDF3F31D8636251 79076561fe18d5ff83624a848f856429a6ae089020b7c92f6220c6af8bb0bf8e de8fe8e8-a078-11f1-a977-00155d0001be https://dk.um.si/Dokument.php?lang=slv&id=210421 Fakulteta za elektrotehniko, računalništvo in informatiko 0 0 0