An Interview with Peter Burns, Transport Canada

1. Your job involved evaluating ADAS systems – when you consider the holistic view, including not just sensing, but also the interventions and general HMI, would you say that most systems are fit for purpose?

At Transport Canada, my team conducts human factors and crash avoidance research to support the development of motor vehicle safety standards. From my experience, driver assistance and safety systems are quite capable, however not all systems are created equal. The results of our ADAS performance testing indicates that some systems exceed expectations while others do not live up to their claims.  Similarly, our human factors evaluations reveal that some manufacturers do an excellent job of designing ADAS displays and controls (i.e., human-machine interfaces HMI), while unfortunately, others could use improvement – and this is to be expected, not every HMI will be perfect. That said, poor HMIs can lead to potential safety risks, poor performance, and disappointed customers, so our evaluations on this front are ongoing.  

2. What factors impact the effectiveness of ADAS systems?

One of the most crucial things like we to underscore is that ADAS is here to help you, but it can’t replace you. Ultimately the driver is the best safety system in the car. That said, there are numerous factors that impact ADAS and warning system performance, including the sensors (quality, integration, calibration, maintenance), operating conditions (weather, road infrastructure), data processing, and algorithms. A host of human factors also impact the effectiveness of ADAS systems such as driver understanding, trust, driver monitoring capabilities, and HMI.  If driver assistance systems are not designed to be intuitive and understandable, drivers may not fully grasp their capabilities and limitations. This lack of understanding can result in drivers either underutilizing or over-relying on these systems, affecting their overall effectiveness. Clear communication of system status, limitations, and functionalities is essential to building driver trust and ensuring correct usage. Driver assistance systems need to be designed in a way that minimizes how much a driver needs to think about them. If these systems present too much information or are difficult to work with, they can distract drivers from the road, leading to lower situational awareness and higher collision risk. Driver assistance systems should provide effective feedback and alerts. Poorly designed alerts can be ignored or misunderstood, leading to missed warnings and reduced effectiveness.  Poorly designed user interfaces or controls for driver assistance systems can lead to frustration and difficulty in using them. This can result in drivers turning off or avoiding the use of these systems altogether. A key point is that the HMI’s are critical because it doesn’t matter how sophisticated your ADAS sensors and algorithms are if the system does not meet the driver’s needs

3. What are the common types of ‘fails’ you see in the design of driver assistance systems?

Bad ADAS HMI is more common than you may think, which is surprising considering its importance for safety and usability.  A common issue I see is the lack of feature integration that is inconsistent with other dashboard controls and displays. 

For example, we asked some outside experts to evaluate the safety and usability of driving assistance interfaces on three different vehicles.  They found numerous problems:

• Drivers could not find the controls for the lane keeping and adaptive cruise control. When they eventually found the controls, they were not easy to operate or were confused with other vehicle controls.  
• Vital information like current system status was never clear enough.
• Driver takeover requests were not signaled, only implied.
• The owner’s manuals said a lot more about system limitations than how to use them.
• Lastly, it was often difficult to determine what ADAS features were available on the car.  

Another failure we see concerns the utility of the ADAS features. Research shows 70% to 80% of new product development fails not from a lack of advanced technology, but because of a failure to understand users’ needs (Von Hippel, 2007). Innovation should be needs-driven. 

4. ISO 5283 is currently being drafted, related to how we view driver impairment and responsiveness. Can you tell us more about that?

The International Organization on Standardization (ISO) technical report 5283-1 will provide information to support the design and evaluation of driver monitoring systems for managing driver states and system interventions.  It is expected to include current state of knowledge and practice, future perspectives, and evaluation methods. Transport Canada has a member on the drafting team for this.  

5. How important is consistency of messaging, alerts, and ADAS function across car brands and vehicles?

I think it is very important to have consistent messages and interaction design across brands and vehicles. Unlike pilots who are trained and certified to operate different aircraft, drivers have one license to operate all light-duty vehicles. Drivers switch vehicles, so interoperability is critical to avoid confusion and errors. The reason we have standard road signage, road markings, and rules of the road is to avoid this type of confusion.  The same should apply to ADAS, but the challenge is that best practices in interface design have not yet emerged. However, Transport Canada is actively involved in the development of international standards for ADAS and automated driving systems (ADS) and standards for consistency are an objective of this work.

6. Can we learn from other industries or consumer markets where standardization in warning systems has proven to be more effective?

There’s a common assumption that we can adopt human factors design practices from other domains such as medical devices, aviation, and military specifications. While these other applications are relevant, ADAS users are not highly trained professionals. Nevertheless, there is a foundation of generic human factors knowledge about basic human abilities and limitations that can be applied. Unfortunately, in my opinion, developers are often overlooking this readily available scientific evidence and are unsuccessfully trying to reinvent the wheel. 

7. Where do you see educating drivers about the features the car has playing a role? Is it possible to design 100% intuitive systems, or should some level of ‘briefing’ always exist?

I think the objective should be to design driver assistance and automated driving systems that are intuitive (i.e., simple and easy to use safely without training), but this is a complex challenge. Unlike the engineers developing these systems — who know every tiny detail —, users often take a trial-and-error approach to operating new features in vehicles and don’t even refer to the manual.  Ideally, you’ll want a system that safely tolerates this trial-and-error approach to learning. Design features, such as limiting the number of ADAS modes and providing clear feedback, will help to support learning. Some recent research (DeGuzman, University of Toronto 2022) compared different types of ADAS training and found that informing drivers about their responsibilities was more effective than highlighting the limitations of the technology. In fact, a limitations-based approach to education left users with a negative impression of the ADAS.

8. What would be your advice for engineers at OEM and Tier 1 companies building ADAS?

I could write a book in response to this question, but will try to keep it clear and simple. In the race to automation, don’t leave safe and user-friendly behind. For success, human factors engineering must have a greater priority. Designs should adapt to driver needs and not the other way around.  Engineers should follow a human-centred design process when building automation. Human-centred is an approach to design that focuses on the users, their needs, and requirements, by applying human factors knowledge and techniques.  Basically, you want to have human factors engineers involved from the beginning of the development process and everything should stem from user needs.  Human-centred design is a win for all because it improves user experience (UX) and increases performance and usability. This makes products easier to use and reduces training, documentation, and product support needs (and associated costs). It also helps to build appropriate trust and makes products more accessible.  Most importantly, human-centred design helps to reduce driver errors and makes products safer. Canada tabled a proposal to develop a framework of key principles for automated vehicle safety and human-centered design at the UNECE WP.1 (Global Forum for Road Safety).  The goal of this work is to promote human needs in the design of automated driving systems.