The NVIDIA GPU Scarcity Madness
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
Next Week in The Sequence:
Edge 319: Our series about foundation model techniques continues with a deep dive into the factors behind in-context-learning(ICL). We review Google’s paper about the elements that cause the emergence of ICL in LLMs and we provide an overview of the Guardrails AI framework.
Edge 320: WE discuss Meta AI’s I-JEPA, the first model based on Yann LeCun’s vision of autonomous AI.
Time to subscribe :)
📝 Editorial: The NVIDIA GPU Scarcity Madness
'GPUs are at this point considerably harder to get than drugs,' famously said Elon Musk a few weeks ago. The phrase summarizes the state of the GPU market, particularly in relation to one vendor: NVIDIA. Dependence on NVIDIA GPUs has become one of the main roadblocks to accelerating innovation in the AI space. GPUs are required in all aspects of the ML model lifecycle, but this is especially true for pretraining foundation models. Multiyear leases of NVIDIA GPUs by large tech companies have become the norm in the AI space, pricing out innovative startups.
This week, The New York Times published a well-researched article about the impact of the NVIDIA GPU scarcity on the AI startup ecosystem. The article presents a reality of hardware-software dependencies that haven't been seen in the tech industry for many decades, if ever. The scarcity of NVIDIA GPUs might be more due to supply chain issues and could be solved once supply matches demand, but it is certainly causing an interesting imbalance in the AI industry.
Obviously, the AI space is incredibly innovative, and we should expect to see some interesting developments as a result of NVIDIA's GPU scarcity. Here are some of the most obvious ones: • Marketplaces for GPUs will emerge. • Big tech providers such as Google (already doing it), Amazon, Apple, and Microsoft will develop their own GPU technology. • Alternative GPU vendors such as AMD or Intel will become highly attractive. • GPU startups will emerge as a hot area for VC investments. • Other GPU sources such as Bitcoin mining pools or gaming infrastructures will attempt to retool for AI (easier said than done 😉). Perhaps the combination of these factors might help balance the supply and demand needs in the GPU space. For now, NVIDIA GPUs are hotter than... finish the phrase 😉.
🔎 ML Research
Inferring Interrupted Questions
Amazon Science published a paper unveiling a model that can understand incomplete senteces. The model is actively used in Alexa and can be adapted to other audio assistants —> Read more.
BOLAA
Salesforce Research published a paper benchmarking LLM-augmented Autonomous Agents (LAAs). The paper evaluates the different LAA architectures across complex tasks —> Read more.
Neuralangelo
NVIDIA published a paper unveileling Neuralangelo, a neural surface reconstruction technique. The method can reconstruct structures of real world scenes from RGB videos —> Read more.
Pruning Pretrained Networks
Google Research published a paper outlinig CHITA(Combinatorial Hessian-free Iterative Thresholding Algorithm), a method for pruning large scale pretrained models. CHITA combines techniques such as high-dimensional statistics, combinatorial optimization, and neural network pruning in a single method —> Read more.
STUDY
Google Research published a paper detailing STUDY, an audio content recommendation system for educational audiobooks. One of the unique characteristics of STUDY is that it factors in the soacial nature of reading recommending books that improves student’s reading engagement —> Read more.
🤖 Cool AI Tech Releases
Dolma
The Allen Institute for Artificial Intelligence open sourced Dolma, a 3 trillion token dataset for LLM pretraining —> Read more.
Marqo
Open source vector search engine Marqo reached general availability —> Read more.
Arthur Bench
Arthur.ai released Arthur Bench, an open source framework to evaluate LLMs —> Read more.
🛠 Real World ML
GPT-4 for Content Moderation
OpenAI discusses their used of OpenAI for content moderation —> Read more.
📡AI Radar
The New York Times ran a remarkable story about the GPU shortage and its consequences to AI startups.
Anthropic has raised another $100 million in funding.
AI-based cloud observability platform Middleware raised $6.5 million.
AI-Privacy platform DynamoFL raised $15 million in a series A.
Voiceflow raised $15 million to boost its conversational AI agent platform.
Swedish company Apica acquired Logiq.ai to combine observability and synthetic data monitoring in a single platform.
AI-powered software mainteneance platform Grit emerged out of stealth mode with $7 million in funding.
Former Google CEO Eric Schmidt launched a new AI startup focused on advance scientific research.
AI-based microbiome startup Viome, raised $86.5 million.