From Consumer Reports today.
Tesla's Navigate on Autopilot Requires Significant Driver Intervention
CR finds that latest version of Tesla's automatic lane-changing feature is far less competent than a human driver
Tesla recently updated its Autopilot software to allow certain cars to automatically change lanes. The automaker says it’s an attempt to make driving “more seamless.” But Consumer Reports observed the opposite in its own tests of the feature, finding that it doesn’t work very well and could create potential safety risks for drivers.
Tesla added the update to its Navigate on Autopilot feature last month as part of a promised upgrade to its package of driver assist features. To use it, a driver must first turn it on, essentially giving the car permission to make its own lane changes. A driver can cancel an automated lane change that’s in progress at any time by using the turn-signal stalk, braking, or holding the steering wheel in place.
In practice, we found that Navigate on Autopilot lagged far behind a human driver’s skill set: The feature cut off cars without leaving enough space and even passed other cars in ways that violate state laws, according to several law enforcement representatives CR interviewed for this report. As a result, the driver often had to prevent the system from making poor decisions.
“The system’s role should be to help the driver, but the way this technology is deployed, it’s the other way around,” says Jake Fisher, Consumer Reports’ senior director of auto testing. “It’s incredibly nearsighted. It doesn’t appear to react to brake lights or turn signals, it can’t anticipate what other drivers will do, and as a result, you constantly have to be one step ahead of it.”
Despite Tesla’s promises that it will have full self-driving technology by the end of next year, our experience with Navigate on Autopilot suggests it will take longer. In addition, experts tell CR that the automatic lane-change feature demonstrates the technological limits of Tesla’s current hardware.
Choosing It, Insuring It, Driving It
Get everything you need to know when it comes to your vehicle.
Join
What Tesla Claims
Tesla released the original version of the Navigate on Autopilot feature late last year. It added functions to the Autopilot partially automated driving system, which keeps the car centered in a lane and controls its speed, maintaining a set distance from vehicles in front. CR has previously expressed safety concerns about Autopilot.
Autopilot has been engaged during at least three fatal crashes in the U.S., according to the National Transportation Safety Board. The latest fatal crash occurred in March, when the driver of a Tesla Model 3 was killed after the vehicle struck the side of a semitrailer in Florida. The driver turned Autopilot on 10 seconds before the collision, according to the NTSB’s preliminary findings released last week.
More on Semi-Autonomous Technology
In the first version of Navigate on Autopilot, Tesla said the software could guide a car through highway interchanges and exits and that it could make a lane change if the driver confirmed it by using the turn signal or accepting an onscreen prompt. When CR first tested this version of the Navigate feature in November, we found that it lagged behind a human driver’s abilities in more complex driving scenarios despite Tesla’s claim that it would make driving “more relaxing, enjoyable and fun.”
On April 3, Tesla announced an updated version of Navigate on Autopilot that, according to the automaker, could automatically make lane changes on certain roads. This latest version gives drivers the option to hand over control for lane changes to the car’s computers. It’s available as a software update for Tesla owners who purchased Enhanced Autopilot or Full Self-Driving Capability, which, despite the name, does not make the vehicle a self-driving car.
If drivers turn it on, the car will automatically execute lane changes when Navigate on Autopilot is active, and it will continue to do so until drivers change the setting or turn off Autopilot. It works as long as Tesla’s Autopilot partial automation system is active and drivers have set a navigation destination. To turn it off, a driver will have to go into a menu on the vehicle’s touch screen and re-enable lane-change confirmation.
The first time a driver disables lane-change confirmation, a warning message pops up that reads, in part, “This does not make your vehicle autonomous.” However, this pop-up does not appear again when the system is in use.
A Tesla spokesperson told CR, “Navigate on Autopilot is based on map data, fleet data, and data from the vehicle’s sensors. However, it is the driver’s responsibility to remain in control of the car at all times, including safely executing lane changes.”
What We Saw
In early May, our Model 3 received a software update that allowed Navigate on Autopilot to make automatic lane changes without requiring driver confirmation. We enabled the feature and drove on several highways across Connecticut. In the process, multiple testers reported that the Tesla often changed lanes in ways that a safe human driver would not—cutting too closely in front of other cars, and passing on the right.
Navigate on Autopilot ready to pass on the left.
One area of particular concern is Tesla’s claims that the vehicle’s three rearward-facing cameras can detect fast-approaching objects from the rear better than the average driver can. Our testers found the opposite to be true in practice.
“The system has trouble responding to vehicles that approach quickly from behind,” Fisher says. “Because of this, the system will often cut off a vehicle that is going a much faster speed since it doesn’t seem to sense the oncoming car until it’s relatively close.”
Fisher says merging into traffic is another problem. “It is reluctant to merge in heavy traffic, but when it does, it often immediately applies the brakes to create space behind the follow car—this can be a rude surprise to the vehicle you cut off.”
Our testers often canceled a pass that had been initiated by Autopilot—usually by applying steering force to move the car back into the travel lane—when they felt that the maneuver would be unsafe.
Ultimately, even in light traffic, our testers found that the system’s lack of situational awareness made driving less pleasant.
“In essence, the system does the easy stuff, but the human needs to intervene when things get more complicated,” Fisher says.
When asked about the system’s performance, a Tesla spokesperson pointed us to a company blog post from April 3. “Through our internal testing and Early Access Program, more than half a million miles have already been driven with the lane change confirmation turned off,” the post said. “Our team consistently reviews data from instances when drivers took over while the feature has been in use, and has found that when used properly both versions of Navigate on Autopilot offer comparable levels of safety. We’ve also heard overwhelmingly from drivers in our Early Access Program that they like using the feature for road trips and during their daily commutes, and we’re excited to release the option to the rest of the Tesla family.”
Tech and Legal Concerns
Several CR testers observed Navigate on Autopilot initiate a pass on the right on a two-lane divided highway. We checked with a law enforcement official who confirmed this is considered an “improper pass” in Connecticut and could result in a ticket.
Navigate on Autopilot also failed to return to the right-hand travel lane after making a pass, which also could result in a driving infraction, the official confirmed. A Tesla spokesperson told CR that “it is the driver’s responsibility to remain in control of the car at all times, including safely executing lane changes.”
A Tesla Model 3 with Navigate on Autopilot ready to pass on the right.
Dorothy Glancy, a law professor at Santa Clara University School of Law in California who focuses on transportation and automation, told CR that making sure that automation is programmed to obey traffic laws is one of the many legal concerns that will crop up as vehicle automation increases.
“One of the issues we lawyers are looking at is the obligation of autonomous vehicles to obey all traffic laws where the vehicle is being used,” she said. “That can get tricky when there are variations from area to area, even within a state—for example, municipal speed limits.”
Glancy said that Navigate on Autopilot is unique because it allows drivers to change one setting a single time to enable automatic lane changes for every trip thereafter.
“There are a whole bunch of these lane keeping assistance-derived features, but this one is different because you commit to it when you start either the car or Autopilot,” she said.
Tesla Road Tests
That puts Navigate on Autopilot one step closer to the functionality of self-driving car prototypes being tested on public roads. The difference, Glancy said, is that self-driving car testing sometimes is regulated at the state level, while the safety of the Navigate on Autopilot system is not.
Shiv Patel, an automotive analyst at market research firm ABI Research, told CR that although he believes Tesla is ahead of other automakers when it comes to automated driving, the automaker’s current hardware may be working at capacity.
“I would say that Navigate on Autopilot would be pushing the upper limits of what could be achieved with the current computer hardware in the vehicle,” he said.
Additional features—which Tesla CEO Elon Musk has said would be available by 2020—would require new hardware, such as the new self-driving chip that Musk announced in April.
“The current hardware does not have the computing power required to support full self-driving features,” Patel said.
Musk said that owners who purchased the Full Self-Driving Capability option would get the new chip as part of a no-cost hardware upgrade.
The Verdict
David Friedman, vice president of advocacy at Consumer Reports, says that as it currently exists, the automatic lane-change function raises serious safety concerns.
“Tesla is showing what not to do on the path toward self-driving cars: release increasingly automated driving systems that aren’t vetted properly,” he says. “Before selling these systems, automakers should be required to give the public validated evidence of that system’s safety—backed by rigorous simulations, track testing, and the use of safety drivers in real-world conditions.”
Ultimately, automatic lane changes may be an interesting feature for Tesla enthusiasts, but the feature doesn’t provide any meaningful assistance to drivers—and it certainly doesn’t make Tesla vehicles “self-driving” by any means.
“This isn’t a convenience at all,” says CR’s Fisher. “Monitoring the system is much harder than just changing lanes yourself. Using the system is like monitoring a kid behind the wheel for the very first time. As any parent knows, it’s far more convenient and less stressful to simply drive yourself.”
Around there the company will be worth $12bio or so, below that someone will scoop it for sure.
Hopefully next summer or the one after I will be able to pull the trigger.
