Has OpenAI Hit Escape Velocity?

Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.

Created with StableDiffusion

🗓 Next week in TheSequence Edge:

Edge #265: Our series about ML interpretability explores interpretability methods for deep neural networks, discuss OpenAI’s research in multimodal neurons for CLIP explainability and the Eli5 framework.

Edge #264: Deep dive into the reinforcement learning with human feedback research that powered the ChatGPT breakthrough.

📝 Editorial 

Earlier this week I was at a dinner with some prominent VCs and I was asked my opinion about the balance of forces between the big AI labs. My answer was that I feel that, after the recent market events, OpenAI might have hit “escape velocity”. Celestial mechanics defines escape velocity as “speed needed to break free from the gravitational attraction of a massive body, without further propulsion”. In the venture capital space, the term is often used with companies that have achieved a growth momentum that can’t be stopped. Last week we devoted this editorial to the implications of strategic alliance between Microsoft and OpenAI. A few days ago, that alliance was cemented with Microsoft’s $10 billion investment in OpenAI’s recent tender offer. The capital, together with Microsoft’s infrastructure and distribution, might be last ingredient needed for OpenAI to catalyze a momentum that seems fundamentally stronger than other top players in the AI market.

ChatGPT has been at the center of OpenAI’s recent momentum. Initially planned as a demo of GPT-3.5 capabilities, the AI agent has taken the internet my storm challenging the imagination of creators, consumer and business application providers. ChatGPT is just a preview of what’s to come with GPT-4 which is rumored to have around 100 trillion parameters which is about 500x the size of GPT-3. The increase in size translates into a growth in capabilities that is far beyond exponential. Additionally, OpenAI is also taking leadership positions in areas such as text-to-image and text-to-code with models like DALL-E and Codex.

I certainly believe that the recent winds in the AI space have favored OpenAI but that doesn’t mean they will be the only company to capitalize on this momentum. In the current market, Google/DeepMind and Meta AI seems to be strongest alternative to OpenAI. Both tech giants have the research capabilities and compute infrastructures to compete with the Microsoft-OpenAI duo. Furthermore, both Google and Meta have been actively working in the generative AI space with models like LaMDA, Imagen, Muse, Make-A-Scene, Sparrow and many others.

Innovation in the foundation model space is happening everywhere but we might be on the verge of a race vastly dominated by OpenAI, Google and Meta. For now, OpenAI seems to have hit escape velocity.

🔎 ML Research

Reinforcement Learning for Mobile Manipulation

Berkeley AI Research(BAIR) published a paper presenting a reinforcement learning technique for mobile manipulation in robotic environments —> Read more.

 LLMs to Synthesize Training Data

Amazon Science published two papers discussing the use of LLMs to streamline the creation of synthetic training datasets —> Read more.  

Bandit Problem Improvements

Google Research published a paper detailing an ML algorithm that provides small hints to improve bandit-like decision making problems —> Read more.

Multimodal Transformers and Document Intelligence

Microsoft Research published an insightful blog post about their use of multimodal transformer architecture for extracting information from complex documents —> Read more.

🤖 Cool AI Tech Releases

3D Asset Generation

HuggingFace shows how to use language to generate 3D objects for games —> Read more.

🛠 Real World ML 

Model Serving at Walmart

Walmart Global Tech discusses an architecture based on TorchServe and PyTorch to serve large scale models for their search experience —> Read more.  

ML Payment Routing at LinkedIn

LinkedIn discusses the ML architecture used to optimize payment approval rates —> Read more.

💸 Money in AI

• Microsoft confirmed that it will invest an astonishing $10 billion in OpenAI new tender offer.

• Construction robotics firm Kewazo announced a $10 million series A.

• Robotics mower builder Scythe announced a $42 million series B.

• AI biotech startup Atomic AI raised $35 million series A to advance their RNA structuring platform.

• The red hot M&A environment in construction robotic continues with Built Robotics acquiring Roin technologies.

• AI powered anti-money laundering platform Hawk AI raised $17 million.

• AI note transcription platform Supernormal announced a $10 million fundraise.

• Inscribe raised $25 million to expand its AI document fraud detection capabilities.