AI Toolbox Hackathon

As machine learning models become more complex and widely adopted, interpretable machine learning (IML) is essential for ensuring transparency, trust, and accountability. Many IML methods—such as SHAP and other Shapley-based techniques—rely on imputation strategies to estimate the influence of individual features. These include simple baselines, marginal imputations that treat features independently, and more advanced conditional imputations that preserve dependencies between features.

SHAP-IQ (Shapley Interaction Quantification) is a Python library that extends Shapley-based attribution to feature interactions, allowing users to quantify not only the individual effects of features but also how they interact in driving model predictions. While SHAP-IQ supports both baseline and marginal imputations for tabular data, its applicability to more complex domains like images, text, or time series is limited by the lack of robust, general-purpose imputation tools.

Currently, only a handful of imputers exist across data types and explanation methods, often with inconsistent quality—and no unified imputer supports all key use cases. This fragmentation slows progress and limits the reliability of IML in practice.

To address this, we are organizing a hackathon in collaboration with LMU Munich, focused on building high-quality, extensible imputation modules for SHAP-IQ. The aim is to improve support for baseline, marginal, and conditional imputations—including for structured inputs and feature groupings—bridging the gap between theory and practice. Participants will collaborate directly with SHAP-IQ developers and contribute to a project with real research impact and strong practical relevance.

Intuitively, the Lipschitz constant measures how much the output of a function can change in response to changes in its input. In machine learning, Lipschitz constants play a crucial role in several contexts: they can be used to quantify the stability and generalization of learning algorithms by ensuring that small changes in input data do not cause disproportionate changes in predictions; they serve as a tool for regularization, where enforcing a Lipschitz constraint stabilizes training; finally, bounding the Lipschitz constant of a neural network provides protection against adversarial perturbations. This concept therefore stands at an interesting intersection between theoretical foundation and practical usefulness. However, its spread is currently limited by the variety of different approximation approaches coupled with a lack of readily available implementations for estimating the Lipschitz constant of a given network. In this project, we will target a Python package with such a ready-to-use implementation of one (or more) algorithm(s) for Lipschitz constant estimation. The package should be compatible with a variety of common deep learning architectures implemented in industry-standard libraries such as PyTorch.

One way to address the challenge of small datasets or small subpopulations in machine learning is to incorporate relevant prior knowledge into the model architecture or the training process. A prominent approach to achieve this in practice is invariant representation learning, where the model is forced - usually by means of an adversarial training procedure - to learn a latent representation that is ‘invariant’, i.e., identical, across different values of a nuisance variable. Such nuisance variables may include a patient’s gender or ethnicity, technical recording parameters, or other factors deemed irrelevant to the medical prediction task at hand. The developer is thereby encoding the prior knowledge that this factor should be irrelevant to the prediction task into the training procedure. Unfortunately, such adversarial approaches notoriously suffer from training instability and convergence issues and are subject to important fundamental limitations and drawbacks. Empirically, they are not very successful. 

Here, we will explore a different but related approach to achieve a notion of (weak) latent space invariance. The method is based on the triplet loss commonly used in contrastive learning approaches. It circumvents many of the issues associated with adversarial invariance approaches, but it has not yet been evaluated in the field of medical image analysis.

The outcome of the hackathon will be a proof-of-concept based on an exemplary use case in chest x-ray analysis: does (a variation of) this method work well empirically, and does it result in more accurate and robust models? The implementation will be done in pytorch, and the experimental setup, dataset, and baseline model training code will be available as a starting point. If successful, the results of the hackathon may result in a joint publication of the findings (and the code, of course).