Written evidence submitted by Dr M Nazmul Huda and Luiz Galvao,
Brunel University (SDV0015)

Executive Summary

• Road traffic problems have been a big challenge for many countries and with the estimated increase in vehicles and population, it is expected to worsen.
• Self-driving vehicles are a potential solution to solve road traffic problems, however, they still need further developments to make them available on public roads.
• Self-driving vehicles need improvements in their perception module since it is responsible to detect and predict the behaviour of other road users.
• State-of-the-art perception algorithms have been implemented using Deep Learning techniques which are considered a data-driven approach.
• A general dataset with rich labelling is still required to develop, train, and evaluate multiple perception tasks.
• There is a lack of public datasets recorded on the UK roads.

1          Introduction

This written evidence is based on our research findings on how to enable self-driving vehicles to effectively perceive hazardous situations. Human drivers develop hazard perception skills, which is an essential skill to help avoid or mitigate road accidents, therefore, we believe that the implementation of such skills into a self-driving vehicle perception system will enable self-driving vehicles s to be roadworthy on public roads. Our comments are focused on the shortcomings of the self-driving vehicle perception system, which is responsible for detecting static and non-static road objects and predicting their behaviour.

2          Road Traffic Problems

Many countries around the world are facing road traffic congestion, pollution, and accidents. It was reported in 2018 by WHO that 1.35 million people die each year due to road accidents. Human errors and imprudence are the main reasons for these accidents. In addition, road traffic accidents are the primary reason for children’s and young adults’ deaths[1]. 

3          Road Traffic Current Solutions

Current solutions are enforced legislation, road improvements, Advanced Driver Assistance System (ADAS) technology, and improved post-crash care. Although these solutions have shown a reduction in traffic accidents and deaths, the number of vehicles on the road is expected to be 2 billion by 2030[2], and the urban population is expected to double by 2050[3]. In addition, all these solutions would still depend on human actions.

4          Self-driving Vehicles as a Potential Solution

• A potential solution to these limitations is self-driving vehicles since they are not expected to be controlled by humans in any instance.
• There are a limited number of self-driving vehicles on the public road already, such as Waymo, Uber, Transdev, and Cruise.
• However, it has been reported that self-driving vehicles have been involved in several traffic accidents[4] [5]. Even though most of the accidents were caused by conventional drivers’ errors, self-driving vehicles are expected to predict these errors to avoid collisions.
• Anthony Lewandowski, Co-Founder and current CEO of Pronto has suggested that self-driving vehicles have not achieved automated levels 4 and 5 because they still have not achieved the human capability of predicting the behaviour of other road users[6].
• Chris Urmson who runs the Aurora self-driving company expressed the view that self-driving vehicles will greatly advance when a prediction system is able to anticipate what will occur in the next 5 seconds[7].
• Overall, self-driving vehicles still need improvement to reliably detect road objects and predict their behaviour in order to accurately prevent conventional drivers' errors.

4.1       Self-driving Vehicle Object Detection

• Since 2010, there have been several pieces of research to enable self-driving vehicles to detect static and non-static objects on the road and to predict the behaviour of non-static objects such as pedestrians and vehicles.
• State-of-the-art detection algorithms, use Deep Learning (DL) techniques that achieved impressive detection accuracy. For instance, the best monovision vehicle detection algorithm presented by Ren[8] and the best pedestrian detection algorithm presented by G. Brazil[9] were able to achieve 90.19% and 83.92% of detection accuracy in the well-known KITTI dataset, respectively.
• However, these algorithms still struggle to detect small, occluded, and truncated objects; and objects that are subjected to poor illumination and weather conditions (e.g., foggy, rainy, night)[10].

4.2       Self-driving Vehicle Non-static Object Behaviour Prediction

• Current vehicle and pedestrian behaviour prediction state-of-the-art algorithms also use DL techniques.
• The behaviour prediction can be the object’s trajectories or its discrete intention. Most of the research is focused on predicting the trajectories of the objects. There is only limited research focused on intention prediction.
• Most of the vehicle behaviour prediction research concentrates on lane change and turning manoeuvres, whereas other manoeuvres such as braking, reversing, parking, and emergency stop are not investigated. Additionally, there is limited research on recognising abnormal driving due to drunk, new, elderly, or sudden medical emergency drivers. 
• The best vehicle trajectory prediction algorithm can predict up to 5 seconds. However, they can only effectively predict for a short prediction horizon of 1 to 2 seconds[11].
• Most of the trajectory prediction works used a top-view camera dataset, which is recorded from high buildings and only covers a certain portion of the highways. Consequently, they cannot be directly applied to self-driving vehicle applications as they have onboard sensors.
• The best vehicle intention prediction algorithms can achieve 86.4%[12] to 91%[13] accuracy with a prediction horizon of 2 seconds. However, these algorithms do not meet real-time system requirements.
• There is still no benchmark dataset that can be used to make a fair comparison between the existing algorithms. It is important to highlight that nearly all the available public datasets are from other countries and only a small number is recorded in the UK, and the ones available have limited labelling.

5          Challenges and Shortcomings

5.1 Hazard perception is an important skill that new driver learns during their theory and practical lessons and keep developing through the years while driving. This ability has enabled drivers to detect potential or developing hazards and take the appropriate action to avoid or mitigate collisions. It would be beneficial for a self-driving vehicle system to be able to learn this skill. Yet, there are no works that researched how to enable self-driving vehicle systems to develop it. The reason could be that there are no specific datasets to develop, train and evaluate a hazard perception system.

5.2 State-of-the-art algorithms still struggle to detect small, occluded, and truncated objects; and objects that are subjected to poor illumination and weather condition (e.g., foggy, rainy, night).

5.3 There are limited works that investigate the prediction of braking, reversing, parking, and emergency stop manoeuvres, as well as, recognising abnormal driving due to drunk, new, elderly, or sudden medical emergency drivers.

5.4 Current behaviour prediction datasets are either specific to vehicles or pedestrians.

5.6 There are limited works that investigate the behaviour of other traffic road users such as cyclists, horse riders, and unleashed animals[14].

6          Recommendations

From our standpoint, the dataset is what currently drives the self-driving vehicle perception nowadays, considering that state-of-art algorithms are based on DL techniques, and they are data-driven. Even though there are multiple public datasets for the developments of self-driving vehicle perception system algorithms, still, there is no dataset that combines all the relevant information to solve multiple perception tasks. For example, there are datasets to train and evaluate vehicle or pedestrian prediction algorithms, however, there is no dataset that contains the information for both, as per our knowledge. Therefore, it is recommended to create a general benchmark dataset that combines all the relevant information with appropriate labelling to enable researchers to develop, evaluate, and compare algorithms that would solve multiple perception tasks. Moreover, it would be beneficial to record a dataset in the UK since the larger part of the datasets was documented in countries that drive on the left-hand side of the road.

August 2022

Endnotes