👀👀 Self-Supervised Learning is Making Inroads 

📝 Editorial 

Self-supervised learning (SSL) is one of the most fascinating and exciting new areas of research in deep learning systems. Labeled by many experts as crucial to the future of deep learning, SSL is one of the disciplines that comes the closest to simulate human learning. Conceptually, SSL allows deep learning models to efficiently learn from unlabeled examples and build some general representations of an environment. The process of SSL methods resembles how babies develop early models of the world around them. The SSL school has gained important champions such as AI legends Yoshua Bengio and Yann LeCun, and powerhouse AI labs like Facebook. 

When you just start reading about SSL, it seems to be one of those futuristic techniques that is still impractical in real-world scenarios. This is actually very far from the truth. SSL applications are real and viable today and the research is advancing at an incredible pace. Just this week, Facebook open-sourced DINO and PAWS – two SSL models that are able to train transformer architectures in computer vision. Famous for powering models like OpenAI GPT-3, transformer models are incredibly large and complex and their applications in computer vision are still very nascent. If SSL methods can be used to train computer vision transformers, the possibilities are endless. SSL was certainly a dream two years ago but now, pushed by companies like Facebook, it is starting to become very real. Before long, we should have SSL frameworks readily available in mainstream deep learning platforms.  

Share if you agree

Comment if you have doubts

🔺🔻TheSequence Scope – our Sunday edition with the industry’s development overview – is free. To receive high-quality content about the most relevant developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#85: new series about Reinforcement Learning: the concept; how OpenAI trained RL agents to play hide-and-seek but they learned so much more; the TensorForce RL framework. 

Edge#84: deep dive into IBM’s use case with Snorkel Flow.

Subscribe to receive RL series

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

Self-Supervised Transformers 

Facebook AI Research (FAIR) published two research papers unveiling DINO and PAWS, two computer vision transformers trained using self-supervised techniques ->read more on FAIR blog

Viva Topics 

Microsoft Research published a paper detailing the research behind Viva topics, their AI solutions for organizing documents and other information sources ->read more on Microsoft Research blog

More Expressive Speech Synthesis 

IBM Research published a paper outlining a technique to improve the expressiveness of speech synthesis models ->read more on IBM Research blog

Reinforcement Learning for Multi-Agent Navigation 

Google Research published a paper discussing a reinforcement learning method to master navigation in an environment with multiple autonomous agents ->read more on Google Research blog


Facebook AI Research (FAIR) published a paper proposing a self-supervised learning method that can classify objects without the need for large labeled information ->read more on FAIR blog

🤖 Cool AI Tech Releases

RecSim NG 

Google Research open-sourced RecSim NG, an upgraded version of its framework to build recommender systems ->read more Google Research blog

TensorFlow Adaptive Frameworks 

The TensorFlow team open-sourced a framework for building recommendation systems that can run in mobile devices ->read more on their blog

Uber Data Platform  

The Uber engineering team published an insightful blog post detailing the real-time data analytics architecture powering Charon, a live monitoring tool for merchants >->read more on Uber blog

——>  If you'd like to talk about analytics, data engineering and data science, discuss different career options, and ask ML book authors any questions – we recommend DataTalks.Club. It's a community of people who love data. Check them out! They are fun.

💬 Useful Tweet

Follow us on Twitter

💸 Money in AI

AI implementation in business

  • AI-driven Cybersecurity startup Vectra AI raised $130 million at a post-money valuation of $1.2 billion. The round was led by funds managed by Blackstone Growth. Using AI, the company enriches network metadata it collects and stores, with the right context to detect, hunt, and investigate known and unknown threats in real-time. 

  • Digital therapeutics startup Kaia Health raised a $75 million Series C led by Optum Ventures. The company offers AI-assisted digital therapies via a mobile app for chronic pain related to musculoskeletal (MSK) disorders and for Chronic Obstructive Pulmonary Disease (COPD).

  • Computer vision for construction platform OpenSpace raised $55 million in a Series C round of funding led by Alkeon Capital Management. Their solutions combine 360° cameras, computer vision and AI to make it easy to capture a complete visual record of the job site, share it via the cloud, and track progress.

  • Customer intelligence platform Databook raised $16 million in Series A funding led by M12. They use AI to automate insight extraction of customer priorities, pain points, key buyers and relevant use cases.

  • AI-powered contract negotiator platform Pactum raised $11 million in a Series A round of funding led by Atomico. They develop AI tools that negotiate contracts between companies on a mass scale.

  • Decision intelligence platform Tellius raised $8 million in a Series A led by Sands Capital Ventures. They combine machine learning with a Google-like natural language interface to enable business teams to get insights on the data.

  • AI-powered social due diligence network Viso Trust raised $3 million. The platform automatically extracts data from source documents and audits to surface key information about third-party relationships.