With advancing technology, many new cars come equipped with automated features that appear to give them some degree of autonomous capability. The problem is, all these systems are slightly different and that has caused confusion and danger, and in some cases led to fatalities.
Vehicle safety testing house and ratings firms, Thatcham Research and Euro NCAP, have been monitoring these new advanced driver assistance systems (ADAS) and today release their first ever ratings of the tech.
That’s a lot of acronyms so let’s break this down. Thatcham has been researching how effective and safe systems like Tesla’s Autopilot are. Specifically, researchers have been looking at the features that allow a vehicle to accelerate, brake, and steer to maintain a position and a safe distance from the car in front, in a highway lane.
[Read: 5 things to know when you’re buying your first electric vehicle]
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
Thatcham Research tested a number of popular vehicles that feature driver assistance ADAS tech using three main criteria: vehicle assistance, drive engagement, and safety backup. Vehicles tested included the Mercedes-Benz GLE, BMW 3 Series, Audi Q8, Ford Kuga, Tesla Model 3, and Volvo V60.
The automated safety feature test
The vehicle assistance criteria refers to how well the speed, steering, and adaptive cruise control features work in collaboration to control the vehicle’s direction and velocity.
Driver engagement is an often overlooked metric when considering the effectiveness of ADAS systems. For Thatcham’s research, this rating considers how accurate the carmaker’s advertising and marketing materials are; how effective the car’s driver monitoring system is; how easy it is for the driver to control the ADAS feature; and how clearly does the car communicate its status when assisting the driver.
Finally, Thatcham also considers what safety nets the car has in its safety back-up rating. This considers how the driver is protected in the case of an ADAS failure, what happens when the driver is unresponsive or if the car is about to collide with another vehicle, and what happens when sensors malfunction. Typically, these kinds of functions are referred to as driver hand off procedures, or in other words, how does the system give control back to the driver.
With all that in mind, Thatcham Research considers a very good driver assistance system to be one that controls the vehicle safely, monitors the driver closely, and has robust safety features in the case of failure. But the researchers also recognize what driver perception of the system is likely to be, as this will affect real-world safety.
That’s an important consideration to make. ADAS technologies have been proven to improve vehicle safety, but only when used appropriately. We’ve seen all too often the disastrous outcomes that occur when these systems are misused because drivers thought they were more capable than they really are and didn’t pay appropriate attention.
The results
Take a look at the table below for full results from Thatcham. The short story, though: the Mercedes-Benz GLE scored the highest overall, with consistently high scores across each of the three testing categories. It scored the highest of all vehicles specifically for driver engagement. This suggests it’s an easy-to-use system that’s sold with clear marketing.
Speaking of the Mercedes GLE, Thatcham Director of research, Matthew Avery, said: “Our overall top scorer with consistently high scores across all testing categories. Keeps the driver engaged with plenty of clear communication regarding the assistance offered. Provides really useful assistance, but not so much that drivers will believe the car can drive itself.”
Let’s address the elephant in the room though, the Tesla Model 3.
Tesla fans will no doubt be loath to see Tesla not even make the top five. According to Thatcham’s tests, the Model 3 has the best performing automated driving features when it comes to how it controls the vehicle, and its safety backup in the case of system failure. Despite this, it’s only given a “moderate” score overall, and the reason isn’t surprising.
When it comes to drive engagement, the Tesla scores, well, terribly. It scores just 34 points, compared to the nearest vehicle, the Renault Clio, which scores 69 points.
Speaking of the Tesla, Avery said: “Many aspects of the Model 3 are exemplary; its vehicle assistance is the best we saw in testing and it also aced the safety back up element.”
“However, it achieves a ‘moderate’ rating for poor driver engagement, with a design philosophy that is very much about the vehicle doing the driving. That would be appropriate for an automated vehicle – but this is vehicle assistance. The big ‘self-driving’ sell in its marketing material, combined with the high performing assistance, encourages the driver to relinquish too much control,” he added.
Tesla isn’t alone in this, though. In conversation with SHIFT back in July, Avery said: “What we’ve seen with vehicle manufacturers, not only like Tesla with Autopilot, but also with Pro Pilot, you’ve got other manufacturers who are misleading consumers, not only with the naming of the systems, but also with the rhetoric they use in their marketing material.”
This phenomenon of overselling the capabilities of partially automated driver assistance systems has a name: autonowashing.
However, “Tesla should be recognized for its ability to update vehicles ‘Over the Air.’ Two years ago, its safety backup results would not have been market leading. This unique capability has seen it move the safety game on, across its whole fleet of vehicles,” Avery said in a statement about the results.
The future
Hopefully, Thatcham’s and Euro NCAP’s ratings will help car buyers better understand the capabilities of their vehicle’s driver assistance systems. The results should also highlight how car companies also have a moral and ethical duty to accurately and clearly describe their systems in marketing materials.
These results will no doubt be helpful to regulators too. Last month, the UK government put out a call for evidence to understand and define one component of ADAS systems — automated lane keeping (ALKS) technology — as part of a bid to eventually regulate its safe use.
It should be noted that ALKS won’t be recognized or regulated officially until next year, following the UK government’s consultation period.
Thatcham’s findings will clearly prove useful in highlighting where and when ALKS and driver assistance tech alike is most effective at improving road safety. The results could even encourage carmakers that oversell the capabilities of their systems to clean up their act and be clearer and more transparent about what their driver assistance features are and are not capable of.
Update October 1, 1014UTC: The original version of this article said that Thatcham has tested automated lane keeping systems (ALKS). Rather, the firm tested Level 2 (SAE Definition) automated driver assistance systems (ADAS). We have corrected the article to reflect this, and regret the error.
Euro NCAP also adjusted the Tesla Model 3 score from 87 to 91 for its vehicle assistance rating, the new table has been added.
Get the TNW newsletter
Get the most important tech news in your inbox each week.