🛰 Is Computer Vision About to Have its GPT-3 Moment?
Weekly news digest curated by the industry insiders
Machine learning (ML) is eating software, and natural language processing (NLP) is eating ML. This is one of the new catchy phrases within the ML community that captures the sentiment about the progress in NLP techniques. In the last few years, NLP has distanced itself from the rest of the deep learning field in terms of achievements both in research and practical applications. Language pretrained and transformer models such as GPT-3 have transcended the boundaries of AI labs to become part of the mainstream tech culture. With the high bar set up by NLP, we should wonder whether there is another deep learning school ready for a major breakthrough?
Computer vision seems to be following the footsteps of NLP and making astonishing progress in the last few years. Part of that progress has been triggered by the adaptation of transformer architectures to computer vision domains, but that’s not the only catalyzer. Self-supervised learning (SSL) is one of the fastest-growing areas of deep learning and seems to be a perfect fit for tackling different computer vision challenges. Unlike NLP, computer vision problems often lack highly accurate labeled datasets, which makes it an ideal candidate for SSL techniques. Meta (Facebook) has been the big champion behind the application of SSL methods to computer vision tasks. Just last week, Meta unveiled a new version of its famous SEER (SElf-SupERvised) model for computer vision which continues to achieve remarkable milestones. Even though computer vision is considerably more complex than NLP as a general domain, the combination of SSL techniques and transformers seems to push the space's boundaries. At this rate, computer vision could soon have its GPT-3 moment.
🔺🔻 TheSequence Scope is our Sunday free digest. To receive high-quality educational content about the most relevant concepts, research papers, and developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻
🗓 Next week in TheSequence Edge:
Edge#171: we introduce DCGANs; explain the DCGAN Paper; explore Nvidia Imaginaire, a GAN Library for image and video translation.
Edge#172: we deep dive into DeepMind AlphaCode that can generate code at the level of programming competitions.
Now, let’s review the most important developments in the AI industry this week
🔎 ML Research
Self-Supervised Computer Vision
Meta (Facebook) AI published a paper detailing advancements in SEER, their marquee self-supervised computer vision model →read more Meta AI blog
Co-training Transformers Using Videos
Google Research published a paper outlining a training technique that uses videos to improve action recognition in transformer models →read more on Google Research blog
Federated Learning with Differential Privacy
Google AI Research published a detailed blog post summarizing their journey to enable differential privacy techniques in federated learning models →read more on Google Research blog
Improving QA Models in Tabular Datasets
Amazon Research published a paper illustrating a pretraining method to improve question-answering models that work with structured tabular datasets →read more on Amazon Research blog
🤖 Cool AI Tech Releases
Next-Gen AI Chips
AI-first chipmaker Graphcore launched a new generation of AI chips and unveiled the plans for the new AI supercomputer →read more on Graphcore blog
🛠 Real World ML
Real-Time ML at LinkedIn
The LinkedIn engineering team published a blog post describing their ML architecture for real-time recommendations →read more their blog
Virtual Assistants at Walmart
Walmart Labs published a blog post detailing the multi-task models used to unify models powering different virtual assistants →read more on Walmart Labs blog
Using PyTorch at Amazon Ads
Amazon and PyTorch published an insightful blog post about their use of PyTorch and AWS Inferentia to develop and scale models for ad processing →read more on PyTorch blog
💸 Money in AI
AI supercomputer developers Luminous Computing raised $105 million in a Series A round, with participation from investors including Gigafund, Bill Gates, 8090 Partners, Neo, Third Kind Venture Capital, Alumni Ventures Group, and others. Hiring in California, New York, Texas/US.
*This news digest is presented by Superb AI’s team. We thank Superb AI for their support of TheSequence.
About Superb AI
Superb AI is an advanced DataOps platform looking to transform the way Computer Vision teams prepare and iterate on datasets. The Superb AI Suite provides automation products and tools across all steps of the data preparation workflow including data labeling, auditing, management and curation.