We caught up with Dr Marina Martínez, to share her journey to becoming a Virtual Testing Engineer at Porsche Engineering. She also discusses upcoming challenges in ADAS/AD software development. Dr Martínez will be presenting on ‘ADAS/AD virtual platform for end-to-end software development and testing’ on September 16th during our AutoSens Brussels Edition.
How did you get into the field of simulation?
After my three-year PhD and working experience in England, it was clear to me that I wanted to take part in ADAS and AV development area in my future career. With this idea in mind, I approached Porsche Engineering at the right point in time and they proposed me to join instead the simulation team and initiate the area of virtual development and testing of ADAS. At first I found it very interesting, a great chance and a suitable follow up from my previous projects, but slowly I realised it was much bigger than we all thought.
Most people would agree with me that developing ADAS functions is attractive from an engineer perspective, the use cases innovative and all the philosophical questions around it are popularly known, such as: “which is the correct behaviour and decision making for machines? In case of fatal accident prediction, who has the preference to be saved? Can a machine replace human-decision-making?”
However, these very relevant questions and facts are not to be ever tackled if we are not able to start from the beginning, test safely and reduce the development costs to make ADAS function actually possible. AV will never happen without simulation. I correct, without realistic enough, validated and certified simulation results that we can trust, or at least that we know how much we can trust. Although obvious, these facts are not yet understood in research and industry domains, which makes our job as simulation engineers harder, but doubly important.
What have you learnt from being one of the drivers of the Porsche Engineering Virtual ADAS Testing Center (PEVATeC)?
When I was hired in Porsche Engineering, I was asked to look for “the tool” that would suffice our requirements and help us with ADAS functions development.
I learnt after some months and a very intense market research that “the tool” does not exist. I learnt that this topic is wider than expected and involves many domains and disciplines, which were not traditionally linked to automotive. It was therefore very clear to us that we cannot tackle this alone. We need signals and sensor experts, 30 graphic designers, programmers and the support from software developers, experts on their area.
The first market research resulted in the initiation of PEVATeC. Based on the best tools available in the market we started hiring to fill in the gaps between tools, formats and our project requirements.
We have learnt that we need to remain flexible to integrate new tools and adapt to each projects. We keep up-to-date to new software releases and are able to build dynamically simulation environments able to integrate the suitable tools that meet requirements. Today we have a platform that can integrate software from third parties on project demand, we have colleagues with engineering background, but also programmers, 3D experts in game engines and a long list of interdisciplinary backgrounds that make PEVATeC possible.
Currently we are improving efficiency, we are learning with every task and we are currently defining the processes and methods that better suit our aims. In short, there is a lot still to learn, but all signs point that we are following the right direction.
What do you see as the biggest challenge in ADAS/AD software development?
Bring to market a function that is able to perform correctly under the expected use cases and is able to deactivate safely under unexpected situations.
ADAS/ AV software developers create software that has to interact with unknown drivers and with any road user, work in any country, under different traffic rules and cumbersome number of external disturbances and weather /lighting conditions. 100% safety cannot be ensured. Then, what is safe enough? Which is the acceptable error rate? How many miles are enough? And who should answer these questions?
On the one hand, ADAS/AV is a new area and it is lacking legislation, recommended processes and methodologies. On the other hand, the design of adequate legislation, processes and methods is lacking experience and data to prove then right. Furthermore, whatever the previous answers are, it is accepted the simulation necessity. However, it is not clear, how to certify and validate the simulation tools, what is accurate enough to replace real testing and a long etcetera.
The challenge resides on siting together OEMs, service providers, Tier x suppliers, standard organizations and governments and initiate collaborations to stablish legislation, best practices, methods and processes to guarantee user safety. Giving the complexity of the previous, many working groups have initiated activities with same target, which are of course of vital importance and a great contribution, but will have to converge eventually into a common understanding and be officialised by the relevant authorities.
You co-authored the paper “Feature uncertainty estimation in sensor fusion applied to autonomous vehicle location” what was the main finding of this paper and do you plan to follow it up or have you followed it up with further research?
his paper was the output of a three-month collaboration between fortiss, institute from TU MUnchen, and Cranfield University, where I was completing my PhD, and was initiated in the context of a bigger project focused on a flexible network of sensors involving both vehicle and infrastructure.
In this area, fortiss is a real visionary and understood very quickly that the set of sensor that we can install in a vehicle, and are affordable for the general public, will not suffice the requirements for AV. Sensor in the infrastructure are necessary and will not only improve safety, but also reduce costs, increase efficiency and reduce the complexity of vehicle systems.
Nonetheless, this means that perception has to be able to deal with a variable set of sensors, which can be plugged in and out the network sporadically. That is to say, perception could be improved by using cameras installed in the city, when these are in range, car-to-car communication and intelligent maps when approaching a traffic jam, for instance.
Within this project, one of the areas of interest was the adequate fusion and interpretation of data coming from “foreign” sensors sporadically available. In this paper, we tried to build models of the sensors uncertainty based on available measurements and situations where the “ground truth”, meaning by that very accurate measurements, were also available. Besides, this model would have to be able to update itself to changes in the sensor and external conditions and should provide enough information for perception.
The output of this paper is a preliminary model for estimation of vehicle localization uncertainty through odometry sensors using NNs. The lessons learnt are suitability of the approach using NNs and necessity of further research in the area. As sensor uncertainty is related to both sensor technology and external conditions, uncertainty estimation needs reflect these differences.
Although there is no specific follow-up planned on the formerly mentioned area, on eof our colleagues in PEG-AB is completing a PhD with TU Munchen and through this connection we are also collaborating in areas relevant to ADAS simulation. Besides, we are collaborating with universities and have ongoing MSc, Bachelor and internships to expand the capabilities in PEVATeC similar directions.
What are you most looking forward to about being a part of AutoSens 2020?
First of all, I am very thankful to have been given the chance to e-present what we do in Porsche Engineering in the area of virtual ADAS development at AutoSens. I hope we succeed in rising up attention to the main topics that we find crucial and build up the right connections to continue developing in the right direction. I also hop we can inspire colleagues to start working in this area and contribute to the common target of making Level 2+ to Level 5 a reality.
Last but not least, I am looking forward to hearing the presentations from other colleagues. There are many interesting sessions proposed and I am sure we will all benefit from this chance. It is a shame though that we can only e-meet, but I hope it is still very fruitful for everyone.
