Edge 421: A New Series About State Space Models

Diving into the best alternative to transformer models.

Created Using DALL-E

In this issue:

1. Introducing state space models(SSMs).

2. A review of the groundbreaking Transformers are SSMs paper.

3. An overview of the DeepChecks framework for evaluating and monitoring SSMs.

💡 ML Concept of the Day: A New Series About State Space Models

Coming out of one of our longest series, focused on autonomous agents, we are going to dive into a state space models(SSMs) which are considered one of the most interesting architectures for foundation models. What makes SSMs that interesting is that it is considered the most viable alternative to transformers.

While transformers are, by far, the most important architecture for foundation models they don’t come without limitations. The main one is the inference model that requires the entire sequence to be passed to the model every time a new output is generated. This posses major scalability limitations for long context tasks.