The route towards safe, ethical AV deployment

We talk to Michael Woon, CEO and Founder at Retrospect, as he explores AV development with a focus on ethical safety solutions. Michael is presenting on October 13th at AutoSens 2020 on ‘Solving the Autonomous Ethical and Safety Dilemma’.

Your background is in functional safety consulting, what do you think about functional safety in AVs? Do you think the industry is making good progress in setting these guidelines?

Prior to functional safety I was in product development and I still think like I’m in product development, even though I’m in safety. At the end of the day we want a product that excites people and makes money – but paramount to all that, and this often goes without saying, sadly, but the product has to be completely safe. We don’t want anyone to be endangered, period. I don’t think about functional safety in AVs any differently than functional safety in transmissions or inverters. The scope and functionality is bigger and more complicated in AVs, certainly, but the principles of safety are exactly the same: This product can potentially harm someone, so we have to build into the design and into the controls these mechanisms that put the product into a safe state when faults occur, which would otherwise hurt someone. For planning and processes: we need the resources to analyse, design, and test these mechanisms right from the beginning, not added as an after-thought. And again, it’s an 8-speed transmission or an L4 autonomous vehicle, it’s the same functional safety principles.

Is the industry making good progress setting these guidelines… I’m just going to be blunt and say, no, but I’m not saying the industry isn’t making good progress today. In 2020 I see a lot of good collaboration and amazing efforts to tackle the problem of AV safety assurance head-on, and we will get there. I’m even optimistic that we’ll get there sooner than some estimates of 5-10 years. But it’s taken too long to get to this point we are at in 2020. That’s not good progress. From my perspective, as I saw things evolve, starting 2-3 years ago, the authors and committees who were drafting these safety standards for ADAS and AVs should have remained unified in their outward message and on the core principles in functional safety and worked out disagreements or misunderstandings internally. Instead, there were some very public and intentional rifts for potentially competing safety standards which has created mixed messages in the right safety approach: one group leaning more towards changes in the safety principles in order to accommodate the current AV technology; then there was another group leaning more towards changes in the AV technology to accommodate the trusted safety principles. The underlying issue, complicating all of this, is that we can’t “standardise” something before we’ve “built” something. We have to have real, tangible safety solutions to look at and see, first, then we can start to standardise. I think the mixed-messages may have caused misdirection on pursuing the right safety solution. If I’m wrong or misrepresenting anyone then I apologise, but otherwise I’d rather point this out because we can all learn from it and collaborate better in the future. And right now there is a lot of commonality in the recommended safety approaches being discussed, today, it seems like we’re all saying basically the same thing and it will be really exciting to see this safety solution take shape.

What do you see as the missing elements to successful commercial AV development?

I really think it’s safety, as of right now. I know there are other deep, technical issues in AV that developers and the industry are dealing with, and component costs, and pricing models, but I think some companies have figured out how to make a commercially viable autonomous product for their application, but the thing that is holding them back is the uncertainty in safety. The potential risks range from actually harming someone, to being out of compliance with industry standards or laws and having to go back and make costly re-architecting changes, or having to recall or take off the market this one huge thing you’ve invested in. It’s right to be cautious when these risks are present, but we can’t wait forever, and I am optimistic the right safety strategy is not far off.

Why do you think the industry is so fed up about hearing about the “Trolley

I think everyone already knows the problem is big and we all want to hear solutions. That’s one reason. There’s another reason, that I’ll go into in my presentation, and it’s that the “Trolley Problem” starts on a bad premise in terms of true accountability and ethics. I’m not a psychologist, but I don’t see the value in presenting someone with a no-win, hypothetical situation. Engineers work in the real-world, so hypothetical constructs always come to an end at some point, and this is an important point to cover because it helps explain the right ethical approach to safety.

In your presentation at AutoSens, you will talk about the route that you see existing towards safe, ethical AV deployment, and I know that you joined a couple of sessions as part of AutoSensONLINE. Did you learn anything in these that will influence your presentation later this year?

Yes, I’m excited about sharing a part of the solution, and one of the biggest things I’ve learned from the previous AutoSensONLINE sessions is how knowledgeable the speakers are and how engaging the discussion format is that AutoSens uses. So that is definitely influencing my talk in a good way, and I am looking forward to a very positive and constructive conversation with the audience, and with the other speakers at AutoSens. There have also been many changes throughout the industry in the past 6 months – 2020 has not been a slow year – so I do have a more up-to-date perspective, and certainly a more optimistic perspective, than when this talk was outlined at the beginning of the year.

Is there anything you would like attendees to consider before listening to your talk?

Yes, when we think about the driving function, we often think about how much to turn the wheel or press the pedals to get from Point A to Point B. That’s the actual “driving” function and that’s what’s gotten all the attention, so far. But what we do as drivers, and almost more as passengers, is we sense and feel how predictable – or not – the situation is around us, and how controllable – or not- the vehicle is, and we quickly react and constantly learn from those moments that send a little fear through us. We can even recognise we are making mistakes we shouldn’t be making. The autonomous vehicles are not going to sense or feel any of that unless we program them to. If we only program them to drive, that’s all they will do. I think understanding those separate concepts of driving will be helpful for our talk.