Edge 261: Local Model-Agnostic Interpretability Methods: LIME

LIME, Meta AI research on interpretable neurons and the Alibi Explain framework.

Created With: Stable Diffusion

In this issue:

1. An overview of the LIME interpretability method.

2. Meta AI’s controversial researcn about how interpretable neurons can negatively affect the accuracy of neural networks.

3. The Alibi Explain interpretability framework.

💡 ML Concept of the Day: Local Model-Agnostic Interpretability Methods: LIME

Continuing our series about ML interpretability, today we would like to cover one of the most popular local model-agnostic methods. Local interpretable model-agnostic explanations (LIME) is omnipresent in all literature and frameworks related to ML interpretability. Just like other local methods, LIME does not try to understand the complete behavior of a model but rather the changes in individual predictions.

What sets LIME apart, are the specific mechanics used to derive the interpretations. LIME introduces perturbations in the input features and observes the resulting impact in the output. Using a more technical nomenclature, given an example x, LIME tries to fit a local interpretable model that matches the output of the original model with features in close proximity to f(x). Once a local model is chosen, LIME tries to fit it to a new training set formed by adding perturbations to the original feature space.