Edge 265: Interpretability Methods for Deep Neural Networks

Interpretability methods optimized for deep neural networks, OpenAI's interpretability technique to discover multimodal neurons on CLIP and the Eli5 framework.

Created with Stable Diffusion

In this issue:

1. An overview ML interpretability methods optimized for large neural networks.

2. Explore how OpenAI discovered the role of multi-modal neurons in the famous CLIP model.

3. A deep dive into the Eli5( Explain like I am a 5-year old) framework.

💡 ML Concept of the Day: Interpretability Methods for Deep Neural Networks

To close our series about machine learning(ML) interpretability, we would like to discuss the evolution of the space given the fast growth of deep neural networks(DNNs). Most of the local and global model agnostic methods explored in this series are applicable to DNNs but they were mostly designed for simpler ML architectures. DNNs have some unique characteristics that provide new dimensions to explore by interpretability methods:

1. Concept Learning: DNNs are able to learn high level concepts in hidden layers which are relevant to the final output.

2. Gradient Interpretations: Gradients can be used as a computationally more efficient method to explore the behavior of the model.