🛴🚲 The Race to Improve Reinforcement Learning

📝 Editorial 

Reinforcement Learning (RL) has been at the center of some of the most important milestones of the last decade of deep learning. DeepMind’s RL-based AlphaGo is considered by many the “Sputnik moment” in artificial intelligence (AI), responsible for sparking an innovation race between the top AI labs in the world. After AlphaGo, RL became sort of a pop-culture term in AI with many research papers making grandiose claims about RL applications that have little correlation with reality. There is something seductive about the idea of learning by trial and error that shares some resemblance with human intelligence. However, the fact is that, despite its popularity, RL techniques remain incredibly challenging and computationally expensive to implement, and most of the applications remain constrained to gaming.  

RL applications might not yet be mainstream, but research is accelerating at a frantic pace. Just this week, there were over four major RL papers published by AI labs like Google Research and Berkeley University proposing new methods to improve RL techniques. AI technology incumbents like Microsoft, Amazon, and Google have made RL a centerpiece of their machine learning product strategy. That movement should result in better frameworks and platforms that streamline the implementations of RL applications in the real world. RL might have been at the forefront of some of the most important recent milestones in deep learning but the race is just starting.  

If you are interested in RL, stay tuned and subscribe to TheSequence Edge if you haven’t yet because, in the next few weeks, we will be publishing a very extensive series covering RL research and technology. Some RL fun coming your way 😉

🔺🔻TheSequence Scope – our Sunday edition with the industry’s development overview – is free. To receive high-quality content about the most relevant developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#83: the final issue in our N-shot learning mini-series the concept of One-Shot Learning; Siamese Neural Networks architecture for one-shot-learning models; the review of the ONNX standard. 

Edge#84: deep dive into Snorkel Flow – one of the most complete machine learning platforms in the market.

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

Massive Training Scale

Microsoft Research published a paper unveiling a new technique that uses its DeepSpeed framework to achieve massive levels of training scalability ->read more on Microsoft Research blog

Hyperparameter Optimization and Reinforcement Learning

The famous Berkeley AI Research lab (BAIR) published a paper proposing a method that leverages AutoML to tune hyperparameters in model-based reinforcement learning solutions ->read more on BAIR blog

Improving Reinforcement Learning 

Google Research published a paper proposing a new computational graph method to improve the interpretability of reinforcement learning methods ->read more on Google Research blog

Multi-Task Reinforcement Learning for Robotics

Google Research published an amazing paper discussing MT-OPT, a new multi-task reinforcement learning method for training robots at scale ->read more on Google Research blog

Better Reward Functions for Reinforcement Learning 

Berkeley AI Research lab (BAIR) published a paper introducing EPIC, a more optimal reward function for reinforcement learning methods ->read more on BAIR blog

🤖 Cool AI Tech Releases


DeepMind open-sourced JAX, a new framework to accelerate deep learning research ->read more on DeepMind blog

Machine Learning for C++

Facebook AI Research(FAIR) open-sourced a new framework for building machine learning models using C++ ->read more on FAIR blog

💬 Useful Tweet

Follow us on Twitter

💸 Money in AI

Begin Capital invests from day zero to Series A and is looking for new pirates, winners, and robots to join their portfolio. Find out more about them at www.begincl.com. Statistically, 0.015% of website visitors get funding! Today, experts from Begin Capital commented on a few of last week’s investment rounds*:

  • Hive, a company providing pre-trained ML models for content moderation, raised $85 million in Series D co-led by Glynn Capital and General Catalyst.

Begin Capital says: Content moderation historically included a huge effort from the human side, especially for social networks where the problem comes at scale. Hive provides evidence that AI implementation can directly enhance cost-effectiveness. Models still have to be trained with a huge amount of labeled data though, meaning that the human element remains crucial.

  • Big data analytics company Unsupervised raised $35 million in a Series B round led by Cathay Innovation and Signalfire. The platform, built on unsupervised learning for analytics, helps organizations turn the complexity of data into business insights.

Begin Capital says: The robust Series B round of Unsupervised is a new chapter in the discussion of unsupervised ML efficiency and reliability. Unsupervised ML brings a bunch of advantages from a business perspective by diminishing human exposure and related costs. We see this as a continuing trend of marginalities convergence for AI versus traditional software business.

  • Computer vision startup Orca AI raised $13 million in Series A funding led by OCV Partners. Their AI-powered marine collision avoidance system provides real-time insights for data-driven decisions.

Begin Capital says: Commercial shipping has historically been relatively regulated, especially compared to other logistics submarkets like aviation or automobile freight forwarding. However, with the Orca AI round, we perceive a signal of increased technological penetration in the marine field, which will lead to increased safety and less challenging operations.

  • Real-time information system Applied XL raised $1.5 million in seed funding. Two founders are both ex-Wall Street Journal, in Applied XL they build information systems powered by editorial algorithms that combine the precision of data science with the high standards of journalism.

Begin Capital says: It was interesting to observe a talented product team looking for their focus. The startup has received a lot of buzz in 2020. The current focus of "Editorial Algorithms" that track data in real-time covers a few segments of the market. Even with modest success traction, the startup might easily raise a significant Series A in 1-1.5 years.

Leave a comment


Other interesting rounds:

  • Privacy-preserving platform for collaborative data science Cape Privacy raised $20 million in a Series A led by Evolution Equity Partners. On its platform, data scientists can collaborate with multiple parties on model development by using encrypted data.

  • Data infrastructure startup Era Software raised $15.25 million in a round led by Playground Global. They created the EraDB time-series database architecture for log management.

  • Data management startup CluedIn raised a $15 million Series A funding round led by Dawn Capital. CluedIn streamlines the process of making data ready for insights, without compromising its fidelity or flexibility.

  • AI video production startup Synthesia just raised a $12.5 million Series A funding round led by FirstMark Capital. Their promise is that creating a video with them is as easy as writing an email. A personal note from the editor: “I’ve been looking through so many AI startups the last ten months but this one really made me shiver. AI presenters look very real. It’s even eerie. Very impressive.”