Navigation and service

Utilizing Class Separation Distance for the Evaluation of Corruption Robustness of Machine Learning Classifiers

Robustness is a fundamental pillar of Machine Learning (ML) classifiers, substantially determining their reliability. Methods for assessing classifier robustness are therefore essential. In this work, we address the challenge of evaluating corruption robustness in a way that allows comparability and interpretability on a given dataset. We propose a test data augmentation method that uses a robustness distance ūĚúĖ derived from the datasets minimal class separation distance. The resulting MSCR (mean statistical corruption robustness) metric allows a dataset-specific comparison of different classifiers with respect to their corruption robustness. The MSCR value is interpretable, as it represents the classifiers avoidable loss of accuracy due to statistical corruptions. On 2D and image data, we show that the metric reflects different levels of classifier robustness. Furthermore, we observe unexpected optima in classifiers robust accuracy through training and testing classifiers with different levels of noise. While researchers have frequently reported on a significant tradeoff on accuracy when training robust models, we strengthen the view that a tradeoff between accuracy and corruption robustness is not inherent. Our results indicate that robustness training through simple data augmentation can already slightly improve accuracy.

Bibliographic information

G. Siedel, S. Vock, A. Morozov, S. Voß:
Utilizing Class Separation Distance for the Evaluation of Corruption Robustness of Machine Learning Classifiers
in: The IJCAI-ECAI-22 Workshop on Artificial Intelligence Safety (AISafety 2022) 2022. pages 1-10, Project number: F 2497, DOI: 10.48550/arXiv.2206.13405

download file "Utilizing Class Separation Distance for the Evaluation of Corruption Robustness of Machine Learning Classifiers"¬†(PDF, 2 MB, Not barrier-free file)

Research Project

Project numberF 2497 StatusOngoing Project Safety-related risk assessment of a cyber-physical model system for industry 4.0 applications

Find out more : Safety-related risk assessment of a cyber-physical model system for industry 4.0 applications …

Cookies help us to provide our services. By using our website you agree that we can use cookies. Read more about our Privacy Policy and visit the following link: Privacy Policy

OK