In the corridors of the CRIStAL laboratory (Research Center in Computer Science, Signal and Automation of Lille) at the University of Lille, teams of researchers are working to create simulation environments capable of accurately reproducing the real world, paving the way for safer and smarter autonomous vehicles. We interviewed two researchers, Zaynab EL MAWAS, post-doctoral researcher, and Gérald DHERBOMEZ, CNRS Research Engineer and technical manager of the Lille Robotics and Intelligent Transport Research Platform (PRETIL), about the outlines of this project and the integration of Viametris' MS-96 mobile laser scanner in their research work. Before delving into the details of the work conducted by Zaynab EL MAWAS and Gérald DHERBOMEZ, it is essential to place this project in its broader context.
Maan EL BADAOUI EL NAJJAR, Scientific Director of the PRETIL platform: At PRETIL, we develop innovative tools and methods to address the challenges of autonomous mobility and robotics. In collaboration with CRIStAL laboratory teams and our academic and industrial partners, we explore advanced approaches, such as digital twins, to test, validate, and certify the perception and decision-making of autonomous vehicles. The integration of Viametris’ MS-96 mobile laser scanner illustrates our commitment to creating realistic simulated environments, ensuring an efficient transition between simulation and deployment in real conditions. Zaynab and Gérald, two of my valuable collaborators, share with you the stakes, challenges, and perspectives of this work that is shaping the future of intelligent mobility.
Zaynab EL MAWAS: The ToSyMA team focuses on fault tolerance in autonomous mobility systems, developing diagnostic algorithms and solutions for ten years to enhance the safety and accuracy of localization systems. The project aims to create a digital twin, a simulation integrating sensors (lidars, cameras, GPS, inertial units) and autonomous vehicles. Initially, with the PRETIL platform, we had a robotized Renault Zoe. Today, we have expanded the fleet to three vehicles to simulate multi-vehicle scenarios, test various faults, and environmental conditions before real deployment.
We used Viametris’ MS-96 to acquire geolocated point clouds in the scientific campus of the University of Lille. These aggregated data allow us to build a realistic simulated environment to validate our systems under diverse conditions. All this work is part of the PRETIL platform.
Gérald Dherbomez: PRETIL is a platform that brings together technical, human, and scientific resources around robotics and transport. Its main mission is to support theses and research teams at the CRIStAL laboratory, to which the platform is attached, and which brings together about 500 people, including more than 200 doctoral students. In this project, PRETIL’s expertise focuses on the digital twin part and interaction with real vehicles. We offer engineering services to researchers from the ToSyMA team, to which Zaynab belongs. For this project, we also worked extensively with the PIRVI platform, which is the virtual/augmented reality platform, human-machine interaction developed by teams with expertise in photogrammetry, 3D environment generation, etc.
Gérald Dherbomez: We have budgets that come directly from our supervisory institutions, namely the University of Lille, CNRS, INRIA, and École centrale de Lille. Then, we have more substantial funding from inter-regional plan contracts like CPER RITMEA, which allowed us to finance equipment purchases, such as Viametris’ MS-96. We have also established several funding mechanisms through collaborative projects with industrial partners on ANR, Horizon Europe, or Interreg type projects.
The most common funding mechanism with industrial partners is often through CIFRE theses (Industrial Agreements for Research Training). The objective is twofold for the doctoral student: developing solutions addressing industrial problems while generating new scientific knowledge. In this framework, they split their time between the laboratory and the company to develop solutions addressing specific issues.
Zaynab EL MAWAS: We worked with Nicolas BREMARD and Bryan EMERY, two engineers from the PIRVI virtual reality platform, directed by Samuel DEGRANDE. Nicolas is a photogrammetry specialist and designed the mesh files from the point cloud, while Bryan focused on virtual reality integration, particularly the VR headset and 3D model in the CARLA simulator based on Unreal Engine.
The technical challenges were numerous. First, the correct selection of coordinates and references. We collaborated with Viametris to understand the necessary steps: how to choose the right coordinate systems or assign and integrate external GNSS measurement references in the VIAMETRIS processing software, PPIMMS. Another challenge was choosing the output file format: LAZ, XYZ, or others, to ensure compatibility with photogrammetry tools and 3D model creation. Correct image extraction was also essential for producing accurate and consistent meshes.
Zaynab EL MAWAS: Nicolas uses existing tools. He integrates point clouds, images, and their positions and orientations extracted in CSV from PPIMMS. This data is then used as input in third-party software to generate meshes.
Gérald Dherbomez: This project is constantly evolving. Currently, we are working to expand digitization areas and improve our processes. Our research is part of national and international collaborations, particularly within the Equipex+ TIRREX project, which brings together about twenty French laboratories around robotic infrastructures. This project, funded until 2028-2029, focuses on developing an open infrastructure, an axis of which we are co-responsible.
In parallel, we are seeking to further automate the 3D environment generation process. Today, after producing a point cloud and 3D model, we still need to perform many manual steps to add layers of semantics and intelligence, such as traffic lane logic. Ideally, we would like these steps to become fully automated, but this remains a major technical challenge.
Gérald Dherbomez: Regarding perspectives, we are working on several axes, particularly with Equipex+ TIRREX. For example, for drones, it is necessary to digitize complex environments like forests, including the canopy. The GIPSA Lab in Grenoble is involved in this aspect, and we are considering using the MS-96 for these digitalizations.
Another laboratory in Marseille (ISM) is developing a flight tunnel to test drones in cavities and wants to obtain a precise 3D model of this environment. Discussions about the expected 3D format are still ongoing, but this shows the diversity of needs in digital twins. Furthermore, there is a recently validated PEPR Robotics program that covers topics such as air-ground vehicle fleets, localization, computer vision, and digital twins. This project is part of “France 2030” and represents a significant investment to develop these technologies.
In parallel, we are seeking to further automate the 3D environment generation process. Today, after producing a point cloud and 3D model, we still need to perform many manual steps to add layers of semantics and intelligence, such as traffic lane logic. Ideally, we would like these steps to become fully automated, but this remains a major technical challenge.
Gérald Dherbomez: The certification of autonomous vehicles is a complex technological challenge. Unlike traditional vehicles, where we proceed with classical mechanical tests and driving sequences, autonomy requires much more elaborate software validation. A study showed us that it would take more than 100 million kilometers of testing under different conditions. Imagine: with a fleet of 100 vehicles running 24/7, we would be at about 12 years of driving phase to cover all possible scenarios. This is simply impossible to implement. This is where the digital twin concept makes perfect sense, with some questions still pending: can we create simulation environments precise enough to certify an autonomous vehicle? What level of precision in 3D maps will certification bodies require to guarantee the reliability and market launch of autonomous vehicles in France? 10 cm? 20 cm? I don’t have the answer.
In a digital twin model, is there a notion of temporality? Can time be accelerated to simulate, for example, 12 years in a few weeks?
Gérald Dherbomez and Zaynab EL MAWAS: Yes, it’s possible. This allows for rapid simulation and testing scenarios in much shorter timeframes. For example, for an algorithm embedded in an autonomous vehicle, we can port the code to a data center with more powerful computers, such as GPUs, and thus accelerate processing.
However, we must be careful not to introduce bias in the results, as real-time is precisely dated. So there are precautions to take in calculations. But indeed, we can accelerate or parallelize simulations. For example, if we imagine 100 vehicles in a simulation over 12 years, we can significantly reduce its duration by increasing the number of computers or using massively parallel approaches.
Gérald Dherbomez: We define a target precision in the specifications and compare virtual sensor results to this precision. For example, we tested several versions of 3D maps: some included only roads, others added building facades with varying levels of precision.
To validate our simulations, we perform acquisitions with real vehicles and compare the data with that generated by virtual sensors. We then adjust the precision of meshes or simulated sensors to meet requirements.
In a digital twin model, is there a notion of temporality? Can time be accelerated to simulate, for example, 12 years in a few weeks?
Gérald Dherbomez and Zaynab EL MAWAS: Yes, it’s possible. This allows for rapid simulation and testing scenarios in much shorter timeframes. For example, for an algorithm embedded in an autonomous vehicle, we can port the code to a data center with more powerful computers, such as GPUs, and thus accelerate processing.
However, we must be careful not to introduce bias in the results, as real-time is precisely dated. So there are precautions to take in calculations. But indeed, we can accelerate or parallelize simulations. For example, if we imagine 100 vehicles in a simulation over 12 years, we can significantly reduce its duration by increasing the number of computers or using massively parallel approaches.
Zaynab EL MAWAS: We mainly work on safety, so the key criterion is that the simulated vehicle stays in its lane, without exceeding a 30 cm error margin, to avoid accidents. An upper bound is set, and the necessary precision depends on this objective.
Gérald Dherbomez and Zaynab EL MAWAS: It depends on the calculator load. When several vehicles are simulated in parallel, the load increases and can lead to control losses. Performance is therefore directly linked to the power of the graphics card used and the number of vehicles simulated simultaneously.
With a powerful graphics card or robust cloud infrastructure, it’s possible to manage several vehicles in parallel without compromising reliability. This opens interesting perspectives, particularly for cloud architectures allowing other users to test their models in simulated environments.
What we’re mainly looking for is to use the 3D environment as a tool. The goal isn’t to validate this environment to be precise to the millimeter, but rather to have a tool we can work with. In the vehicle’s real environment, we don’t control GPS satellites, weather, or other unpredictable elements like rain. In simulation, however, we control everything. This allows us to recreate specific conditions, sufficiently realistic, to test sensor-related phenomena, observe how the autonomous driving algorithm behaves, and verify if it detects faults, performs good diagnostics, then reconfigures itself to guarantee the mission safely.
Gérald Dherbomez: We’re often asked how to validate a simulator. It’s complex because these are two different worlds. The correspondence between the two involves cross-testing: we take a movement in the real world, recreate it in simulation, and compare. But it’s far from simple. And in simulation, we can include elements like satellite constellation changes or variable weather conditions, which is impossible in reality.
The team that contributed to this work:
Maan EL BADAOUI EL NAJJAR: Scientific Director of the platform (PRETIL)
Gérald DHERBOMEZ: Technical Manager of the platform (PRETIL)
Samuel DEGRANDE: Technical Manager of the platform (PIRVI)
Gauthier DESMET: Research Engineer (PRETIL)
Maxime DUQUESNE: Research Engineer (PRETIL)
Zaynab EL MAWAS: Post-Doctoral Researcher (PRETIL)
Nicolas BREMARD: Research Engineer (PIRVI)
Bryan EMERY: Research Engineer (PIRVI)
Learn more:
Videos:
