Alexander Hinterleitner, a doctoral researcher at the IDE+A Institute within the THK-AI Research Cluster at TH Köln, investigates how artificial intelligence can become more explainable. His latest publication in ACM Transactions on Evolutionary Learning and Optimization (TELO), co-authored with Oliver Niggemann (Helmut-Schmidt University, Hamburg) and Thomas Bartz-Beielstein (IDE+A, THK-AI), introduces a new approach to this challenge.
Different eXplainable AI (XAI) methods can offer conflicting explanations for the same AI model. Rather than choosing one method over another, the researchers ask: can neural networks be tuned so that different XAI methods agree more closely? Their approach incorporates agreement between explanations, termed “XAI consensus”, into multi-objective hyperparameter tuning alongside predictive accuracy. Experiments across 14 datasets show that this agreement can be improved, while also revealing trade-offs with predictive performance. Agreement alone does not guarantee that an explanation is correct, but it provides an additional criterion for evaluating and selecting models.
The practical motivation for this research grew out of close collaboration with Everllence SE. In critical engineering applications, accurate predictions alone are not enough: engineers also need explanations that are understandable and plausible in light of their domain knowledge. This industrial perspective motivates research into how explainability can become part of model development from the outset.
Link to publication: https://dl.acm.org/doi/10.1145/3858620