At a press conference in Palo Alto, Calif., today, Toyota is announcing the first step of what is expected to be a major push into artificial intelligence and robotics, technologies that the company sees as critical for addressing current and future societal challenges. Toyota, the world’s largest automaker by sales, says it will establish two collaborative research centers at MIT and Stanford, with an investment of $50 million over the next five years. The initial focus will be on accelerating the development of AI with applications to smarter and safer vehicles, as well as robots that can make our lives better at home, especially as we age.
Toyota says an immediate goal is to figure out ways to save lives on the road. But the company is very clear that it’s not trying to develop a fully autonomous car in the same way that Google and many others are. Instead, they’re working on assistive autonomy: you’ll be driving most of the time (or at least in control of the vehicle), but the vehicle will be continuously sensing and interpreting the environment around you, ready to step in as soon as it detects a dangerous situation. Toyota believes this approach could make cars virtually crash-proof.
“Our long-term goal is to make a car that is never responsible for a crash,” says Dr. Gill Pratt, who was until just a few months ago the program manager at DARPA responsible for the DARPA Robotics Challenge (among other ambitious robotics programs) and will now direct this research at Toyota. He added that such intelligent cars will “allow older people to be able to drive, and help prevent the one and a half million deaths that occur as a result of cars every single year around the world.”
Dr. Pratt will be working with Professor Daniela Rus, head of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), as well as Professor Fei-Fei Li, director of the Stanford Artificial Intelligence Laboratory (SAIL).
Earlier this week, we spoke with Pratt, Rus, and Li to get all the details on what we have to look forward to over the next five years.
Toyota’s Announcement
Let’s begin with exactly what Toyota is announcing today. Starting this academic year (so, almost immediately) CSAIL and SAIL will begin an increased focus on researching AI for autonomous vehicles. This will involve providing funding for existing staff and students and possibly hiring new researchers to work on vehicle-related AI challenges, and Toyota will likely exchange some of its own researchers with the schools on a visiting basis.
The press conference today in Palo Alto. From left: Chuck Gulash, Toyota Senior Executive Engineer; Kiyotaka Ise, Toyota’s Chief Officer, R&D Group; Daniela Rus, director of MIT’s CSAIL; Fei-Fei Li, director of Stanford’s SAIL; Gill Pratt, a former DARPA program manager who will direct Toyota’s AI research program.Photo: Evan Ackerman/IEEE Spectrum
At first glance, $50 million over five years doesn’t sound like a huge amount if you consider the probable budgets of Google, Apple, Tesla, Uber and some of the other companies aggressively pursuing autonomous vehicle technology. However, we’re talking about $5 million a year that’s going directly into each of these university’s AI labs, funding specific lines of research, which according to Dr. Pratt will amount to “a very strong effort in terms of R&D.”
Furthermore, this is only the beginning for Toyota, and it’s not even the full scope of what the company has going on right now. Toyota isn’t yet ready to comment on what its entire robotics effort will consist of, but we’ve been assured that this is just the first move, and we’re expecting big things.
The other part of Toyota’s announcement is, of course, that Dr. Pratt will be overseeing the overall collaborative effort at the research centers, to “direct and accelerate these research activities and [their] application to intelligent vehicles and robotics.”
“It became clear to me that this was the place to have the greatest impact of applying the technology that I’ve been working on for the last few decades, both in my academic career and at DARPA, to really improve the human condition,” Dr. Pratt told us. Another thing that helped convince him to join the Japanese firm: “When I was younger I used to fix my own cars and lots of friends’ cars. Toyotas had a very good design and last the longest. They were my favorite cars to fix.”
A Focus on Vehicles
Like most other car companies right now, Toyota is very interested in using AI and robotics to make cars safer. And not just safer: completely, entirely safe: “a vehicle incapable of getting into an accident” is something we heard several times when we spoke with them. In our experience, it’s unusual to hear statements like these from large companies, because nobody seems to want to commit themselves to such moonshots, so this level of optimistic determination is pretty exciting.
Toyota also emphasized how its autonomous vehicle program will differ from Google’s and others. “Our goal, which is a little different than the approach that others take, is to build intelligence to help the car be really a guardian angel for you and keeping you from having a wreck,” Dr. Pratt explains. So Toyota wants teamwork between you and your car, as opposed to just trying to get the car to handle everything, and they believe that this collaborative approach will have the most significant near-term impact on safety. “We want to enhance the fun of driving for the human being while making it far more safe,” he says.
Essentially, what Toyota wants to do is to improve on the existing autonomous safety technologies that some cars already have—anti-lock brakes, blind spot warnings, pedestrian detection systems, and so forth—as well as invent new ones. Indeed, there is the potential to do much, much more, but at the same time it starts to get a lot harder, because the right thing to do in different driving situations, and the right way for the car to help you do it, becomes less obvious.
This is why Toyota is putting $50 million into AI research as opposed to (say) doing what Uber did (and arguably what Google did) and just buying a university autonomous vehicles lab straight up. At this point, they’re focused on the core AI component of autonomous driving, as opposed to the physical production of a self-driving car (although we imagine that they’ll be building prototype vehicles in the near future).
So, let’s take a look at what each of these AI labs has to offer, and what they’re (very generally) planning to start working on.
MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)
CSAIL, led by Daniela Rus, is well known for its diversity of robotics research. Just a few months ago at ICRA, they showed off their amazing little self-folding dissolving magnetically actuated origami microbot. I guess that counts as an autonomous vehicle (sort of), but this collaboration with Toyota on autonomous cars is still a surprise.
While SAIL will be focusing on recognizing what’s going on around the vehicle and understanding it, CSAIL will be developing predictive models to help the car decide what actions to take, especially in traffic, at high speeds, and in bad weather. CSAIL will also be taking into account the tediously unpredictable human component in all this, and finding ways for the car to interact with the driver, perhaps even adjusting its decisions based on its perception of your mood (if you’re distracted, for example).
“We hope to develop some of the new algorithms and systems and technologies that will take autonomous driving and robotic mobility to the next level, so that robots can deal with way more complex scenes than what they can deal with today,” Professor Rus told us.
Earlier this year, Rus gave one of the ICRA plenaries, so here’s a reasonably up to date snapshot of what they’ve been up to at CSAIL:
Stanford Artificial Intelligence Laboratory (SAIL)
SAIL, run by Fei-Fei Li, has an enormous amount of experience with computer vision, machine learning, deep learning, probabilistic reasoning, decision making, and all kinds of other things related to human-machine interaction. Professor Li says SAIL has a “holistic approach to artificial intelligence,” which she describes as a loop of five components: sensing, perception, learning, communication, and finally action (with the “thinking” part, the decision making process itself, occurring throughout).
“Autonomous vehicles are the perfect platform to realize this cutting edge research,” Professor Li says, “because they need all five of these AI components.”
SAIL is particularly strong in perception and learning: synthesizing huge amounts of data, finding patterns in those data, and using those patterns to understand what the data mean. This is critical in object recognition, and especially when it comes to recognizing objects in complicated or unfamiliar situations where you’ve got a lot of perceptual information coming in at once and your AI needs to be able to reliably parse all of it without missing anything.
These are only very broad directions, we know, but neither Stanford nor MIT have specific projects definitively lined up yet. What we can do is give you a sense of some of the specific things that Professor Li has been working on recently, as she gave a TED Talk earlier this year:
Robots at Toyota
Toyota may be a car company, but they have a notable history with robotics and automation in a variety of contexts and implementations. This $50 million AI-focused announcement has automotive technology at its core, but the results of the research will have much wider applications, and Toyota is well aware of this. They’re not ready to talk about it that much right now, but there are some hints about “life-improving robots” that may show up at some point.
For example, we have a vague expectation that we’ll be seeing more of Toyota’s Human Support Robot (HSR), especially since they posted this video showing human assistance applications just a month or two ago:
As far as autonomous vehicles go, Toyota hasn’t (publicly) put as much emphasis on research and development as other companies have, but they do have a little bit of history that we’re familiar with, and it’s exactly in line with the announcement today. At CES 2013, we checked out Toyota’s semi-autonomous Lexus Advanced Active Safety Research Vehicle. From our article on it:
“To be clear, this is a research vehicle, and it’s not designed to turn into a Google-style autonomous car. It’s more about a high-level of driver assistance, with the car using sensors and intelligence to augment what its human driver is doing. Think of it as a co-pilot, essentially, there to point things out and give you help when you need it.”
We’re not sure what Toyota has been doing with this thing for the last several years; it’s possible that the Lexus could reappear as a testbed, but that depends on what’s going to happen next.
What’s Next
Toyota, like many Japanese companies, tends to be cautious about big moves like this, so we’re reasonably confident that this commitment represents the product of a lot of careful thought. And historically, when Toyota commits, it’s scary: you may or may not remember how much of a risk the Prius was, but it’s fantastically successful now, at least partially because Toyota was able to correctly judge that hybrid cars (as opposed to all-electric) were both something that people wanted and something that technology was capable of delivering.
With vehicle artificial intelligence, Toyota is certainly not the first company to decide that it’s an important thing to explore, but that shouldn’t diminish the fact that they’ve decided to make a big move into AI and robotics for mobility more generally. While we’re not sure exactly what the entire scope of Toyota’s commitment is, we’re told that even for Toyota (a company with roughly $250 billion of annual revenue), it’s “substantial.”
Toyota’s HSR, a mobile robot with manipulation capabilities, is one technology the company is exploring to address the challenge of a rapidly aging population.Photo: Toyota
Substantial is good, because it’s going to take an enormous amount of work to fulfill Toyota’s promise of developing vehicle that will never crash. This is something that desperately needs to be done, and realistically, this type of safety-oriented collaborative automation is more important than the full autonomy that other companies are working on, because it directly tackles the problem of humans being terrible drivers (as opposed to just lazy humans). And furthermore, it’s something that we can take advantage of without having to solve all of the legal and ethical issues that obstruct the introduction of fully autonomous cars.
Beyond vehicle intelligence, we’re also looking forward to hearing more about the other steps that Toyota will be taking towards using robotics and AI to improve our lives. In Japan, in particular, that means finding ways of keeping a rapidly aging population as independent as possible. Vehicle autonomy will certainly be a part of that, but so will robots that can help people maintain their lifestyles, dignity, and happiness.
As this research progresses, we’ll be keeping up with it: we’re delighted to be able to report that Toyota isn’t going to be sucking these university labs into a black hole of secretiveness as so many other companies have done. There may be patents filed on certain things before results are published, but results will be published, and we’re expecting to see a lot of exciting conference papers over the next five years from Stanford and MIT that include the postscript “funded by Toyota.”
We have a lot more information from our in-depth interviews with Gill Pratt, Daniela Rus, and Fei-Fei Li, and we’ll be bringing you their perspectives on Toyota’s investment specifically, and on vehicle artificial intelligence in general, early next week.
[ Toyota ]
