MotionalandMITresearchers have built a system that lets self-driving cars explain their decisions in real-time, tackling the black-box problem in autonomous vehicle AI.
The work, published inNature, comes from a team at Motional that includes CEO Laura Major, working alongside researchers from MIT’s Computer Science and Artificial Intelligence Laboratory. Their proposed method, called the Concept-Wrapper Network or CW-Net, aims to translate the internal calculations of a self-driving system’s neural network into concepts a human can actually read.
If a current self-driving car brakes hard on a clear road with no obvious hazard in sight, neither the driver nor a passenger has any way of knowing why. Modern self-driving systems increasingly rely on neural networks trained on large volumes of driving data. Those networks can perform well, but they don’t expose their reasoning, which is why engineers describe them as black boxes.
CW-Net works by converting a self-driving system’s internal logic into concepts such as “Approaching Stopped Vehicle” or “Close to Cyclist.” These could, according to Motional, appear on a dashboard showing which concepts are influencing the vehicle’s driving decisions as they happen.
The system is designed so the explanations aren’t generated after the fact as a guess at what the network might have been doing. Instead, the vehicle’s final decision-making system takes action based directly on these human-interpretable concepts, so a braking event traces back to a specific concept that triggered it. Motional describes this as causally faithful, distinguishing it from approaches that generate natural-language explanations, which can read as plausible without necessarily being accurate.
Laura Major frames the case for this kind of interpretability against the alternative of relying purely on end-to-end deep learning to handle driving decisions.
“The general end-to-end only approach can get to a really good 80-90 percent – maybe even 95 percent – solution, but that’s not good enough to remove a driver or to earn the trust of cities, communities, and customers,” she said.
Explainable AI research has largely stayed confined to computer simulations in lab settings, according to Motional. The Motional and MIT team instead deployed CW-Net on an autonomous vehicle with an experienced safety operator in the driver’s seat, collecting data on a private test track and on public roads around Las Vegas.
The team used an earlier experimental version of its deep-learning-based planning system, described as showing competitive performance but with notable shortcomings that CW-Net could help surface. Two incidents from the testing illustrate what the system caught.
In one, the autonomous vehicle repeatedly stopped near a traffic cone, and the vehicle operator assumed the cone itself was triggering the behaviour. Researchers removed the cone and the car stopped anyway. CW-Net’s display showed the actual cause: the experimental planning system was hallucinating a stopped vehicle ahead, a pattern traced back to its training data. That explanation let the researchers understand, predict, and resolve the issue.
A second test involved a cyclist. The autonomous vehicle detected and stopped for the cyclist as expected, but CW-Net revealed that the experimental planning system wasn’t actually basing its decision on the cyclist’s presence. The safety driver responded by exercising more caution around cyclists after noticing this. Follow-up analysis confirmed that caution was warranted, because the vehicle’s braking in that case came from a safety backup system rather than the experimental deep-learning-based planner.
Adding layers of explainability to an AI system carries a known cost in speed and performance, and Motional acknowledges that risk. However, when researchers benchmarked CW-Net against leading autonomous driving algorithms, the difference in driving capability came in at less than one percent.