🌅 The Era of Foundation Models is Here
Weekly news digest curated by the industry insiders
📝 Editorial
The term ‘foundation models’ is becoming one of the hottest buzzwords in the machine learning (ML) lingo. Researchers from Stanford University originally coined the term to describe models that have been trained in large amounts of unlabeled data and can be fine-tuned to specific domains. Think about fine-tuning GPT-like models for domains such as law or science. Foundation models are shifting the ML development paradigm from creating brand-new models to fine-tuning large pretrained models.
The efforts around foundation models are increasing remarkably fast. Stanford University created the Center for Research on Foundation Models (CRFM), a new initiative focused on studying best practices around foundation models. Just this week, Snorkel AI released Data-centric Foundation Model Development, a new series of addition to the Snorkel Flow platform to fine-tune and distill foundation models. Meta AI also unveiled details about MultiRay, their platform for running foundation models at scale. Finally, the CRFM team unveiled a new benchmark to facilitate the holistic evaluation of foundation models. Foundation models efforts are popping up everywhere, from large AI labs to innovative startups.
Building by fine-tuning the new paradigm. The era of foundation models is definitely upon us!
🔺🔻TheSequence Scope – our Sunday edition with the industry’s development overview – is free. To receive high-quality content about the most relevant developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻
🗓 Next week in TheSequence Edge:
Edge#245: we start a new series about machine learning interpretability; discuss Manifold, an architecture for debugging ML models; explore Meta’s Captum, a framework for deep learning interpretability.
Edge#246: we discuss OpenAI’s best practices that they used to mitigate risks while training DALL-E2
📌 Our LinkedIn account
In this uncertain times for Twitter, we’d like to introduce to you TheSequence’s LinkedIn account. We are building a unique resource and support system for all ML&AI aficionados. Let’s connect!
Now, let’s review the most important developments in the AI industry this week
🔎 ML Research
HELM
Stanford University published HELM, a benchmark for the holistic evaluation of foundation models →read more
MultiRay
Meta AI discusses MultiRay, the architecture used to power large foundation ML models at scale across their different organizations →read more
Data Enrichment Practices
DeepMind published an insightful paper discussing human data collection best practices used in real-world ML scenarios →read more
MoE with Expert Routing
Google Research published a research paper proposing a routing algorithm in mixture of experts (MoE) neural networks →read more
🤖 Cool AI Tech Releases
Data-centric Foundation Model Development
Snorkel AI released Data-centric Foundation Model Development, a new set of capabilities in the Snorkel Flow platform to adapt large foundation models to domain-specific scenarios →read more
Data Cards Playbook
Google Brain released Data Cards Playbook, a toolkit for transparency in ML datasets →read more
🛠 Real World ML
Anomaly Detection in Prime Video
Amazon Science discusses the ML techniques used for anomaly detection in their Prime Video application →read more
Einstein Search Answers
Salesforce Research discusses the ML techniques powering Einstein Search Answers, a new search architecture for customer support →read more
Netflix Video Quality
Netflix discusses the neural network techniques used for video encoding optimizations in the media giant →read more
💸 Money in AI
ML&AI&Data
Chipmaker Astera Labs raised $150 million in a Series D funding round led by Fidelity Management & Research. Hiring globally.
Data platform provider WekaIO raised $135 million in a Series D funding round led by Generation Investment Management. Hiring in San Francisco, US/Tel Aviv, Israel.
AI-powered
Video and audio editor Descript raised a $50 million Series C fundraising round led by OpenAI Startup Fund. Hiring in San Francisco, US/Montreal and Quebec, Canada/remote.
Video intelligence service Spot AI raised a $40 million Series B financing round led by Scale Venture Partners. Hiring remote and in Lehi, Utah, US.
Video conference startup Owl Labs raised $25 million in a Series C investment round led by HP Tech Ventures. Hiring in Boston, US and remote.
Language learning platform Speak raised $27 million in a Series B round led by OpenAI Startup Fund. Hiring in San Francisco, US/ Seoul, South Korea/Ljubljana, Slovenia.
Contract intelligence platform Terzo raised $16 million in a Series A funding led by Align Ventures. Hiring in Los Angeles and Atlanta, US.