Strengthening the safety of self-driving cars with an object recognition system based on deep learning

Self-driving cars, or autonomous vehicles, have long been envisioned as the next generation of transportation. To enable autonomous navigation of such vehicles in different environments, many different technologies related to signal processing, image processing, artificial intelligence, deep learning, edge computing and IoT need to be implemented.

One of the biggest concerns with the popularization of autonomous vehicles is safety and reliability. To ensure a safe driving experience for the user, it is imperative that an autonomous vehicle accurately, effectively and efficiently monitor and discern its surroundings and potential threats to passenger safety.

For this purpose, autonomous vehicles use high-tech sensors such as Light Detection and Ranging (LiDaR), radar and RGB cameras, which generate large amounts of data as RGB images and 3D measuring points, a so-called “point cloud”.

The fast and accurate processing and interpretation of this collected information is crucial for the identification of pedestrians and other vehicles. This can be realized by integrating advanced computing methods and the Internet of Things (IoT) into these vehicles, enabling fast on-site data processing and more efficient navigation through various environments and obstacles.

In a recently published study in IEEE transactions of intelligent transport systemsA group of international researchers led by Professor Gwanggil Jeon from Incheon National University, Korea, has now developed an intelligent IoT-enabled end-to-end system for real-time 3D object recognition based on deep learning and autonomous driving situations is specialized.

“For autonomous vehicles, environmental awareness is crucial to answering a key question: ‘What’s around me?’ It is important that an autonomous vehicle can effectively and accurately understand its environmental conditions and surroundings in order to perform a responsive action,” explains Prof. Jeon.

“We developed a recognition model based on YOLOv3, a well-known identification algorithm. The model was first used for 2D object detection and then modified for 3D objects,” he explains.

The team fed the collected RGB images and point cloud data as input to YOLOv3, which in turn output classification labels and bounding boxes with confidence values. They then tested its performance using the Lyft dataset. The first results showed that YOLOv3 achieved extremely high recognition accuracy (>96%) for both 2D and 3D objects, outperforming other state-of-the-art recognition models.

The method can be applied to autonomous vehicles, autonomous parking, autonomous delivery, and future autonomous robots, as well as in applications where object and obstacle detection, tracking, and visual localization are required.

“Currently, autonomous driving is performed by LiDAR-based image processing, but it is predicted that a general camera will replace the role of LiDAR in the future. Therefore, the technology used in autonomous vehicles is changing every moment, and we are at the forefront,” says Prof. Jeon. “Based on the development of element technologies, autonomous vehicles with improved safety should come in the next 5-10 years be available.”

Provided by Incheon National University