Open Source Generative AI is Experiencing a "Linux Moment" but it Needs an "Apache Moment"
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
Next Week in The Sequence
Edge 285: We do a complete recap of our series about federated learning.
Edge 286: We deep dive into Vicuna, the open source LLaMA based model that matches ChatGPT’s performance
📝 Editorial: Open Source Generative AI is Experiencing a "Linux Moment" but it Needs an "Apache Moment"
The momentum of open source generative AI in recent months is remarkable. Just last week, releases such as StableLM from Stability AI, OpenAssistant, MiniGPT-4, and RedPajama showcased impressive capabilities in language and computer vision. These releases add to an impressive list of projects such as Databricks’ Dolly, Stanford’s Alpaca, Auto-GPT, BabyAGI, and Berkeley University’s Koala, among others, which have chosen open source alternatives to platforms such as OpenAI, Claude, Cohere, or Google.
In the generative AI community, many refer to this open source momentum as a "Linux moment," a reference to the open-source operating system movement as an alternative to Microsoft’s Windows. The analogy makes sense since no other project in history has embodied the spirit of open source like Linux. However, I believe open source generative AI needs something like an "Apache moment." This reference is based on the fact that the release of the Apache Web server was the event that raised the profile of Linux and gave it an edge over Microsoft’s Windows for hosting websites. Prior to the release of Apache, Linux remained an incredible open-source OS with a small community of early adopters and enthusiasts.
Bringing the analogy to the world of generative AI, the open-source movement needs the equivalent of a ChatGPT application. The release of ChatGPT significantly raised the profile of OpenAI and helped closed-source generative AI platforms like Anthropic and Cohere. The open-source generative AI movement needs to find its ChatGPT and create its "Apache moment."
If you’re looking to adopt a feature store or platform for machine learning and don’t know where or how to start your research, then this guide is for you.
*Download the free guide to:
Access a comprehensive framework for understanding the capabilities of different feature stores and platforms
Get tips to on how to use a data-driven approach to evaluate vendors
Learn how the right solution can improve ML model accuracy
🔎 ML Research
DINOv2
Meta AI published a paper detailing DINOv2, a state of the art computer vision model based on self-supervised learning. DINOv2 can be trained on any collection of images without requiring metadata or fine-tuning —> Read more.
Visual Blocks for ML
Google Research published a paper detailing a visual programming platform for rapid development of ML multimedia apps. Formerly known as Rapsai, the mode consists of a node-graph editors that connects different components in an entire ML pipeline —> Read more.
Long Horizon Forecasting
Google Research published a paper discussing a time-series dense decoder architecture for time series forecasting. The model using a simple multilayer perceptron encoder-decoder that outperforms transformer models in long horizon forecasting —> Read more.
Generative Agents for Simulating Human Behavior
Researchers from Stanford University and Google Research published a paper detailing a model of generative agents that can simulate human behavior in interactive environments. The model was used in a virtual environment in which agents can interact with each other about daily activities using natural language —> Read more.
✨ Virtual LLMOps learning*
It’s not too late to register for two days of virtual LLMOps learning! Arize:Observe kicks off this Tuesday with tech talks from OpenAI, LlamaIndex, PromptLayer, Hugging Face, and others and a focus on building successful LLM applications for production. Register here.
🤖 Cool AI Tech Releases
StableLM
Stability AI launched a new suite of open source LLMs that include instruction following capabilities —> Read more.
MiniGPT-4
Researchers from King Abdullah University of Science and Technology open sourced a super interesting vision-language model that mimics the capabilities of GPT-4 —> Read more.
RedPajama
RedPajama, a project targeting the creation of open source LLMs launched with the release of a 1.2 trillion token dataset based on the Meta AI LLaMA paper —> Read more.
🛠 Real World ML
Spam Content Detection at LinkedIn
LinkedIn discusses the ML practices behind their viral spam content detection engine —> Read more.
Anomaly Detection at Lyft
Lyft’s engineering team discusses the process of building anomaly detection models in the LyftLearn platform —> Read more.
📡AI Radar
Alphabet consolidated Google Brain and DeepMind under a single AI research unit called Google DeepMind.
GPT cloud platform CoreWeave announced a $221 million series B.
Microsoft released Copilot for Viva, its employee engagement and performance management platform.
Google Bard incorporated code generation capabilities.
Cortical Labs, a startup working on a new approach to AI that combines lab grown brain cells with computer chips raised a $10 million round.
Warehouse robotics startup Robust.ai raised $20 million to scale its customer deployments.
Snap unveiled new generative AI capabilities such as an AI bot and generative AI lenses.
Vector database platform Qdrant announced a $7.5 million seed round.
Mediwhale raised $9 million for its retina scan AI technology.
Weights and Biases announced a new suite of LLMOps capabilities to support prompt engineering lifecycles.
Cyber security startup Fletch.ai raised $12.5 million to use NLP to scan threat landscape. .
Warehouse computer vision startup Groundlight raised a $10 million seed round.