Who Has The Vision?

On Sunday, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space

📝 Editorial 

What we have seen this week is truly remarkable. The war between two giants, Google and Microsoft, is hot. We saw a triumphant and unexpected comeback of Bing, Microsoft's search engine, powered by its Prometheus model, optimized to work with the “next-generation” OpenAI model. Bing, still unavailable to the public, reported its waiting list to be over a million people. Microsoft's aggressive move made Google rush into its presentation of LaMDA-based ChatGPT competitor Bard. One error of a model - Bard gave an incorrect answer during the presentation - cost Google $100 billion. 

As Microsoft prepares to roll out more integrations of OpenAI's language AI technology and its Prometheus model into its core productivity apps, Google seems to have everything except the vision.

It's worth noting that in 2017, it was Google's researchers who published the article "Attention is all you need," introducing the concept of the transformer, which gave a tremendous boost to the AI space. In the same year, a collaboration between DeepMind (acquired by Google in 2014) and OpenAI resulted in the paper "Deep reinforcement learning from human preferences", now known as RLHF - reinforcement learning from human feedback. This literally changed everything for ChatGPT. But Google seemed to overlook it for LaMDA.

Another Microsoft smart move was to invest in OpenAI, exclusively licensing GPT models but letting the company be independent and – therefore – much more agile. Google acquired DeepMind and though investing heavily in AI and hoarding up many top-notch AI researchers, the talent and AI innovations were first channeled into existing corporate products, without a try to reimagine them. 

Other players are now entering the game, such as Baidu and Meta. Baidu – the Chinese search engine leader – is set to announce its ERNIE bot, based on the "Enhanced Representation through Knowledge Integration" language model introduced in 2019. It is expected to be publicly rolled out in March as a standalone service and as part of the search engine. Meta, on the other hand, has adopted a wait-and-see approach after the disappointing performance of BlenderBot and Galactica, to avoid making costly mistakes.

The success of ChatGPT and the surge in interest in Bing showed that is not the size of the model that matters most, but the vision of how to implement it.

The new era of AI excitement is here, and there will be no shortage of AI news from the giants’ battle this year. As the commercialization of LLMs is looming over us, we continue to watch the independent research efforts, such as BigScience Research Workshop with its free language model BLOOM, and the fight for the ethics of LLMs. Whether we will see any collaboration of the giants around it is the question. Exciting and impactful times are ahead!

🔎 ML Research

Transformers’ In-Context Learning

Researchers from MIT, Google Research, and Stanford University published a paper discovering an algorithm that can describe the in-context learning of LLMs →read more

Sources of distribution inference attacks

Microsoft Research published a paper analyzing the sources of information leakage allowing distribution inference attacks →read more 

Anomaly detection

Google Research published an article detailing two research papers addressing the challenges of Anomaly detection using data-centric approaches →read more

🤖 Cool AI Tech Releases

TFX-Addons

TensorFlow detailed the work behind TFX-Addons. These are new components, libraries, and examples that extend TFX contributed by the community of developers →read more 

⚔️ AI vs. AI ⚔️ at Hugging Face

Hugging Face introduced a new tool for multi-agent competitions and explained how to use it with examples →read more 

ADEV for decision-making under uncertainty

Researchers from MIT and Oxford University published a paper introducing ADEV, a tool that can help to apply AI to a much broader class of problems. →read more 

IBM’s Vela

IBM publicly announced its first AI-optimized, cloud-native supercomputer Vela, the go-to environment for IBM Researchers creating to work on foundation models →read more

🛠 Real World ML 

Project Minerva for Modernization

IBM Research detailed their new efforts for making application modernization easier →read more 

Creative Insights in Promotional Artwork

Netflix detailed two approaches for data-driven recommendations for creating more effective asset suites of promotional artwork →read more

ML Innovation with Ray

Spotify described their today’s ML infrastructure and how they plan to transform it using Ray →read more 

💸 Money in AI

• The LLMs’ mistakes do not scare investors away. While some are betting on the fastest horse in the generative AI race, others are equipping their funds with AI experts. For example, this week, AI expert Fei-Fei Li joined Radical Ventures as a partner, and this trend is expected to continue.

• Inspection software company Zeitview (ex-DroneBase) raised $55 million in an investment round led by Valor Equity Partners.

• Data observability startup Acceldata raised $50 million in Series C funding round led by March Capital. Hiring in India.

• Customer experience automation startup Ushur raised a $50 million Series C funding round led by Third Point Ventures. Hiring in the US and India.

• Code-generating platform Magic raised $23 million in a Series A funding round led by Alphabet’s CapitalG.

• Document workspace Macro raised a $9.3 million Series Seed round led by Andreessen Horowitz. Hiring in New York City, US.