The AWS DeepRacer is an almost toylike 1/18th scale race car. It comes with all of the sensors and software tools to help developers build machine learning models to drive the car around a course — or really do anything else they want it to do. The $399 DeepRacer launched at AWS’s massive re:Invent show in late 2018.
At the time, it seemed like a bit of a gimmick, but AWS has put a lot of its weight behind it and is currently running a DeepRacer league at its various events around the world. At these events, developers can pit their models against each other and learn more about building a specific kind of machine learning model in the process.
Why bother, though? It’s not like DeepRacer cars are likely to add to AWS’s bottom line anytime soon. DeepRacer, however, is part of a line of hardware products from AWS that started with DeepLens, a smart camera for developers.
“It really comes from the same place,” AWS general manager for Artificial Intelligence and Machine Learning marketing Ryan Gavin told me. “When you think about the stimulus for something like DeepLens, it was really about how do we put machine learning into the hands of every developer and data scientist. That’s our mission and we’re very consistent about that.”
Gavin argues that having a hardware device makes technologies like deep learning more approachable to developers. “We’ve always asked ourselves what are the ways we can take interesting and new and hot technologies in the world of machine learning and find ways to bring those to developers,” he said. “Coming off the success of DeepLens and all the positive feedback we received from developers and data scientists, we kind of looked at what was the next iteration of that and reinforcement learning as a very hot and emerging machine learning technique is certainly getting a lot of traction out there but still has a lot of barriers.”
Reinforcement learning does not require training data but instead, you create a learning agent and this agent essentially teaches itself through trial and error — and as it gets closer to its goal, it gets virtual rewards that, ideally, lead it down the right path.
For problems where no training sets exists, this is often an efficient way of creating a model. For the time being, though, getting started with reinforcement learning is still significantly harder than using other, more established techniques. And that’s where DeepRacer comes in.
Phillip Faraone/Getty Images for AT&T
“One of the most intuitive [applications for reinforcement learning] is in the world of on autonomous vehicles, in the concept of autonomous vehicles, and the idea that how do you train a car to learn how to drive itself,” Gavin argued.
Autonomous driving, he said, is something that everybody is familiar with, at least in concept. That, in turn, makes it easier for developers to take the next step and understand that there’s no historical training data here. It’s an iterative process and every lap around the race track represents another chance to have the model train itself.
“That takes a lot of iteration and practice for that model to learn and so this was just a very relatable concept in the world of reinforcement learning that we latched onto very early on,” he said. “We thought a car would be a fun way to bring that to developers.”
I watched some of the racing action at Amazon’s re:Mars event earlier this year. Every developer I talked to pretty much echoed what Gavin said. Having a physical device like these race cars made the process of learning these new concepts more fun and, indeed, approachable.
“The developers and data scientists that work very hard here at AWS, they’re just like all the other developers and data scientists out there in the world,” Gavin said. “And we saw them instantly playing with these deep racers and then starting to race. And it was just kind of that little moment of ‘Oh, this is a really fun and peculiar way to extend what we think is an interesting way to bring reinforcement learning to developers, but then extend that to this idea of a competition — this first global autonomous racing league — where developer can pit their skills against one another from around the globe.”
It’s no secret that a lot of the learning processes in real-world applications take place in simulations. It’s far easier, faster and safer to train a model in a virtual world, after all, where you can speed up time and have a race car could go around a track thousands of times an hour.
Unsurprisingly, Amazon is doing the same and making it easy for developers to train their models in a simulated environment in the cloud. Once they have tuned their models a bit in this virtual world, though, developers can then take this model and test it in the physical car.
That’s all great, but why is AWS actually building its own cars. It could just offer an SDK and let developers cobble together their own cars from parts. A lot of developers already do that, after all. Gavin, however, argues that the only way for the team to reach its goal of making this process approachable is to put a working car into developers hands and allow them to get started right away.
“There are places where focusing on our customer really requires a complete experience,” he said. The team didn’t want developers to have to spend a lot of time tinkering with the car but instead focus on getting started with reinforcement learning.
All of this, Gavin argues, is starting to pay off, in large parts because the racing league gives developers a goal to work toward. A number of companies, for example, have approached AWS to help them host their own internal DeepRacer league. These include oil and gas companies, financial institutions and others, all of which are trying to get their employees up to speed with modern machine learning techniques.
Gavin also noted that a number of colleges are looking at incorporating DeepRacer into their curriculum, though so far, it doesn’t look like you can take a DeepRacer class just yet.
A year ago, when AWS ran the first races at re:Invent, the winner took just over 50 seconds to get around the track. At some of the recent events, the top three finishers raced around the same course in under eight seconds.
Gavin noted that as the team looks ahead, it’s trying to keep in mind what it’s hearing from developers today. They, of course, want to continue to hack and tinker with the cars — maybe even the physical appearance of them. The company doesn’t have anything to share about this just yet, but re:Invent will host the finals of the racing league and chances are we’ll hear about what’s next for DeepRacer there.