💃 New Week, New AI Super Model
Weekly news digest curated by the industry insiders
📝 Editorial
The current era of deep learning is, without a doubt, dominated by massively large models. Since the release of models like OpenAI’s GPT-3, research labs have immersed themselves in producing larger and more scalable neural network models. This trend is undoubtfully dominant in natural language processing (NLP) but has been rapidly expanding to other domains such as computer vision and speech intelligence. Big AI labs with virtually unlimited computation resources have been certainly pushing the scalability boundaries of deep learning architectures.
Last week, we saw two new examples of large models achieving impressive milestones. OpenAI released details about the latest version of DALL-E, one of the first deep learning models able to generate photorealistic, artistic images from natural language descriptions. Like the previous version, DALL-E 2 relies upon large pretrained models for language understanding and image generation. DALL-E 2 improves upon its predecessor by incorporating more knowledge about artists’ styles and better mechanisms to optimize images. Google has also been busy in the super model trend by unveiling PaLM, a new transformer model with an astonishing 540 billion parameters that can master many tasks, including language comprehension. One of the key innovations of PaLM is the flexibility to be trained in many hardware accelerators.
The trend of large deep learning models continues! While many experts argue that this trend is unsustainable and that these large models are just sophisticated memorization machines, it is unlikely that the trend of super models will end any time soon.
🔺🔻 TheSequence Scope is our Sunday free digest. To receive high-quality educational content about the most relevant concepts, research papers, and developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻
🗓 Next week in TheSequence Edge:
Edge#181: we start a new series about high scale ML training; explain SeedRL; overview Horovod.
Edge#182: we explore the topic of ethical data labeling.
Now, let’s review the most important developments in the AI industry this week
🔎 ML Research
DALL-E 2
OpenAI published a research paper detailing DALL-E 2, which improves its original model to generate artistic images from textual descriptions →read more on OpenAI blog
PaLM
Google Research published a paper discussing Pathways Language Model (PaLM), a 540 billion parameter transformer model that mastered many tasks and can be efficiently trained on different hardware architectures →read more on Google Research blog
VDTTS
Google Research published a paper outlining VDTTS, a visually-driven text-to-speech model that can generate high-quality audio from text and video frames →read more on Google Research blog
🤖 Cool AI Tech Releases
Google Cloud’s New ML Offering
Google Cloud announced new releases such as the GA of Vertex AI Workbench and the new Vertex AI Model Registry which provide a consistent experience to manage the lifecycle of ML models →read more on Google Cloud blog
BigLake
Google Cloud announced BigLake, a new service that unifies data lakes and data warehouses across different platforms →read more on Google Cloud blog
DINO’s SSL demos
Meta AI Research (FAIR) launched a new series of demos that showcased the capabilities of DINO, a self-supervised learning model for computer vision →read more on DINO demo website
🔺Open Roles at feature-oriented database platform Molecula🔻
Site Reliability Engineer: https://grnh.se/e2b29f1e4us
Principal Site Reliability Engineer: https://grnh.se/c1f4f1484us
🛠 Real World ML
Multi Text-to-Speech Model Framework
Amazon Research published a blog post detailing a framework to enable the coexistence of multiple text-to-speech models for a given task →read more on Amazon Research blog
💸 Money in AI
AI-powered
Contract management platform LinkSquares raised $100 million in Series C financing led by G Squared. Hiring in Boston/US.
Cyber security platform Coro raised $60 million in a Series C round led by Balderton Capital. Hiring in Chicago and New York/US, in Tel Aviv/Israel.
Business translation software Lilt raised $55 million in Series C funding found led by Four Rivers. Hiring globally.
Facility design and operation platform BeamUP raised a $15 million round of seed investment led by StageOne Ventures and Ibex Investors. Hiring in Tel Aviv.
Conversation platform Arena raised $13.6 million in a Series A round led by CRV. Hiring remote.
Sales enablement platform Amplemarket raised $12 million in back-to-back Seed and Series A investment rounds co-led by Comcast Ventures and Armilar Venture Partners. Hiring globally.
Data privacy automation platform LightBeam.ai raised $4.5 million in a seed funding round. Hiring.