The Undisputed Champion of Open Source Generative AI

Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.

Created Using StableDiffusion

Next Week in The Sequence:

• Edge 293: You are on for a treat! We get into the topic of instruction following LLMs that unlocked the power of ChatGPT! Review the InstructGPT paper and provide an overview of the increasingly popular Dust framework.

• Edge 294: We deep dive into StarCoder, the new open source model that can generate code in over 80 programming languages.

📝 Editorial: The Undisputed Champion of Open Source Generative AI

Stability AI is synonymous with open-source generative AI. The release of Stable Diffusion was a sort of Sputnik moment in the evolution of open-source generative AI models. For the first time, a company was trusting the benefits of open-source distribution ahead of the ethical concerns associated with generative AI models. Since the release of Stable Diffusion, we have seen a tremendous wave of innovation in the open-source foundation models space, and Stability AI has been at the forefront of it.

Little by little, Stability AI has been assembling one of the most impressive suites of open-source generative AI capabilities in the market. In addition to its hallmark text-to-image model, Stability AI DeepFloyd IF, there is another text-to-image model called Stable Vicuna, which can integrate text into images. Stable Vicuna is an open-source RLHF chatbot based on the Vicuna and LLaMA models. StableLM is a very comprehensive suite of small and efficient open-source LLMs.

Just this week, Stability AI released StableStudio, an open-source version of its DreamStudio platform. This release should accelerate the work on user interfaces for generative AI workloads. Additionally, Stability AI also released the Animation SDK, a very robust text-to-animation stack for developers.

Stability AI’s frantic pace of innovation is certainly raising the bar for open-source generative AI solutions. At least for now, the open-source generative AI movement has an undisputed champion.

🔎 ML Research

RL for Open Ended LLM Conversations

Google Research published a paper detailing dynamic planning, a reinforcement learning(RL) based technique to guide open ended conversations. The core idea is to use RL to extract information from reputable sources and that can be used with the output of LLMs to generate answers in multi-turn conversations —> Read more.

In-Context Learning in LLMs

Google Research published a paper analyzing the different factors that influence in-context learning in LLMs. Specifically, the paper focuses on semantic priors and input-level mappings as two of the core elements that can influence LLMs ability to learn from different input-label pairs before performing a task —> Read more.

Self-Aligned LLM

IBM Research published a paper introducing Dromedary, a self-aligned LLM trained with minimum user supervision. The model was also open sourced —> Read more.

Interpretability in Alpaca

Stanford researchers published a paper describing interpretability techniques used in Alpaca. Specifically, the paper focuses on the methods used to match actions with the causal dynamics of the environment —> Read more.

📢 Event: apply(risk), the ML Engineering Community Conference for Building Risk & Fraud Detection Systems

Want to connect with the ML engineering community and learn best practices from ML practitioners at Affirm, Remitly, Block, Tide, and more, on how to build risk and fraud detection systems?

Then join us at one of our favorite conferences – apply(risk) on May 30! This free half-day, virtual event focuses on the specific challenges of building ML risk and fraud detection systems, from low-latency streaming, to deploying features to production quickly and reliably. Need more reasons to sign up? Register and join us for apply(risk) for opportunities to win Visa gift cards!    

🤖 Cool AI Tech Releases

ChatGPT IOS App

OpenAI launched the first version of its mobile app for IOS —> Read more.

Guidance

Microsoft open sourced Guidance, a framework for controlling LLMs is a more effective way than simply prompting —> Read more.

Colab

Google integrated generative AI capabilities into Colab —> Read more.

StableStudio

Stability AI announced StableStudio, an open source version of its DreamStudio application —> Read more.

Animation SDK

Stability AI released Stable Animation SDK, a text-to-animation open source toolkit —> Read more.

MTIA

Meta announced the Meta Training and Inference Accelerator(MTIA) which  improve over traditional GPT for large foundation model workloads —> Read more.

🛠 Real World ML 

LLMs for Cloud Management

Microsoft details the architecture and technique used to leverage LLMs as part of the Azure incident management infrastructure —> Read more.

📡AI Radar

• OpenAI’s CEO Sam Altman delivered a quite interesting testimony to the US Senate.

• Together announced a $20 million seed round to accelerate their work in open source and cloud AI platforms.

• Anthropic and Zoom shared some details about their strategic alliance.

• Hippocratic raised $50 million to accelerate their vision of creating LLM models for healthcare.

• Union AI raised $19.1 million to accelerate its AI-first ETL platform.

• Yellow AI announced YellowG, a new conversational platform for workflow automation.

• Visual Layer announced a $7 million round to help enterprises manage datasets for computer vision models.

• Procurement platform Zip disclosed a $100 million fundraise to incorporate AI capabilities.

• ServiceNow and NVIDIA announced a partnership to scale generative AI IT workloads.