Open Source Scored the First Major M&A of the Generative AI Era
Next Week in The Sequence:
Edge 305: Our generative AI series continues with an overview of in-context, retrieval-augmented LLMs including Google’s original paper in the space. We also explore the Humanloop platform for LLMs.
Edge 307: We deep dive into the famous Open Assistant project for foundation models.
📝 Editorial: Open Source Scored the First Major M&A of the Generative AI Era
M&A activity is always interesting to evaluate the health of a tech market. While fundraising activity often forecasts the value of a company in the relatively long term, M&A activity provides a pragmatic view of what exit strategies might look like for a specific segment of companies. Having too much or too little M&A in a market is always bad; you want just the right level of deals to rationalize valuations in a sector. Well, last week, we witnessed the first high-profile M&A transaction in the generative AI space, and it went to the open-source column. Databricks agreed to acquire MosaicML for an astonishing $1.3 billion valuation.
MosaicML is a two-year-old company behind the open-source MPT-30B and MPT-7B models, and it has built a state-of-the-art platform for training and fine-tuning foundation models. This deal is incredibly significant for several reasons. Firstly, it demonstrates the real potential of open-source foundation models as a viable alternative to closed, API-based models. I mean, to pay $1B+ for something, you must be truly convinced that these open-source models will match the quality of GPT-4, Claude, and PaLM. If you haven't tried MPT-30B, I think you will be pleasantly surprised by its tremendous quality. Secondly, Databricks' enterprise distribution can act as a strong catalyst for the adoption of MPT models and eliminate barriers for open-source generative AI. Lastly, paying $1.3 billion for a two-year-old company in a highly competitive space might seem irrational, but it shows that Databricks believes the MosaicML platform can unlock $10 billion to $20 billion in value.
The MosaicML acquisition follows other significant transactions, such as Snowflake acquiring Streamlit for $800 million last year and Neeva for $150 million this year. Beyond the economics, I believe the Databricks-MosaicML deal is an incredible stamp of approval for open-source ML. Now we should see what Databricks' competitors (like Snowflake 😉 ) do.
🔎 ML Research
CoDi
Microsoft Research published a paper detailing CoDi, a generative AI model capable of generating content across different modalities such as language, image, audio or video. Together with the paper, Microsoft announced Project i-Code to foment multimodal generative AI —> Read more.
ZeRO++
Microsoft Research published a paper detailing ZeRO++, a high performance communication pipeline optimized for LLM training. As it names indicates, ZeRO++ is built on top of ZeRO but reduces the communication volume by 4x —> Read more.
A Unified Pretraining Strategy for Computer Vision Models
Google Research published a paper unveiling a pretraining strategy that combines image captioning and image classification. The strategy delivers amazing performance in zero shot classification tasks —> Read more.
XGen
Salesforce Research open sourced XGen, a 7 billion parameter LLM trained on 8K sequence length for up to 1.5T tokens. XGen achieved amazing results in both language and coding tasks —> Read more.
Textbooks is All You Need
In a fascinating paper, Microsoft Research introduced phi-1, a transformer model for coding trained in high quality text book data. Despite having only 1.3B parameters, phi-1 to match the quality of larger alternatives —> Read more.
🤖 Cool AI Tech Releases
LMFlow
LMFlow is an open source toolkit for fine-tuning large foundation models —> Read more.
Open LLM Leaderboard
Hugging Face provided an update about the helpful and controversial Open LLM Leaderboard —> Read more.
Chat Arena
Chat Arena is an open source game environment to enab,le research about autonomous LLM agents —> Read more.
MediaPipe Diffusion Plugins
Google Research open sourced text-to-image plugins for its MediaPipe on-device ML framework —> Read more.
🛠 Real World ML
Meta AI Cards
Meta AI released a series of cards that document the ML use cases across Facebook and Instagram —> Read more.
Real Time ML at Lyft
Lyft discusses the architecture behind Real-time Machine Learning with Streaming initiative which allow developers to incorporate real time ML capabilities into their applications —> Read more.
Declarative Data Pipelines at LinkedIn
LinkedIn provided an overview of the architecture and tech powering their declarative data pipelines —> Read more.
📡AI Radar
Databricks signed the agreement to acquire MosaicML for $1.3 billion.
Inflection, Reid Hoffman new venture, announced an astonishing $1.3 billion fundraise.
Generative-video darling Runway announced $141 million in new funding.
Generative-content enterprise platofrm Typeface raised $100 million in new funding.
Generative AI lab Reka announced $58 million to advance research in the space.
Google Deepmind’s CEO Demis Hassabis gave an interesting interview to Wired in which he he unveiled their plans for Gemini, a model that will rival ChatGPT.
Thomson Reuters acquired legal AI startup Casetext for $650 million.
Generative AI startup Voice.ai raised $6 million and announced a major user milestone.
Scriptic, a studio leveraging generative AI for content production scored a $5.7 million funding round.
AI-based API platform Speakeasy came out of stealth mode with a $7.6 million seed investment.
Gleamer, an AI platform for radiologists, raised $27 million.
Loora, a generative AI platform for language learning, came out of stealth mode with $9.25 million.
AI-developer platform Faros AI announced $20 million series A.
AI security platform CalypsoAI raised $23 million.