Written evidence submitted by Dr M Nazmul Huda and Luiz Galvao,
Brunel University (SDV0015)

Executive Summary

1          Introduction

This written evidence is based on our research findings on how to enable self-driving vehicles to effectively perceive hazardous situations. Human drivers develop hazard perception skills, which is an essential skill to help avoid or mitigate road accidents, therefore, we believe that the implementation of such skills into a self-driving vehicle perception system will enable self-driving vehicles s to be roadworthy on public roads. Our comments are focused on the shortcomings of the self-driving vehicle perception system, which is responsible for detecting static and non-static road objects and predicting their behaviour.

2          Road Traffic Problems

Many countries around the world are facing road traffic congestion, pollution, and accidents. It was reported in 2018 by WHO that 1.35 million people die each year due to road accidents. Human errors and imprudence are the main reasons for these accidents. In addition, road traffic accidents are the primary reason for children’s and young adults’ deaths[1]

3          Road Traffic Current Solutions

Current solutions are enforced legislation, road improvements, Advanced Driver Assistance System (ADAS) technology, and improved post-crash care. Although these solutions have shown a reduction in traffic accidents and deaths, the number of vehicles on the road is expected to be 2 billion by 2030[2], and the urban population is expected to double by 2050[3]. In addition, all these solutions would still depend on human actions.

4          Self-driving Vehicles as a Potential Solution

4.1       Self-driving Vehicle Object Detection


4.2       Self-driving Vehicle Non-static Object Behaviour Prediction

5          Challenges and Shortcomings

5.1 Hazard perception is an important skill that new driver learns during their theory and practical lessons and keep developing through the years while driving. This ability has enabled drivers to detect potential or developing hazards and take the appropriate action to avoid or mitigate collisions. It would be beneficial for a self-driving vehicle system to be able to learn this skill. Yet, there are no works that researched how to enable self-driving vehicle systems to develop it. The reason could be that there are no specific datasets to develop, train and evaluate a hazard perception system.

5.2 State-of-the-art algorithms still struggle to detect small, occluded, and truncated objects; and objects that are subjected to poor illumination and weather condition (e.g., foggy, rainy, night).

5.3 There are limited works that investigate the prediction of braking, reversing, parking, and emergency stop manoeuvres, as well as, recognising abnormal driving due to drunk, new, elderly, or sudden medical emergency drivers.

5.4 Current behaviour prediction datasets are either specific to vehicles or pedestrians.

5.6 There are limited works that investigate the behaviour of other traffic road users such as cyclists, horse riders, and unleashed animals[14].

6          Recommendations

From our standpoint, the dataset is what currently drives the self-driving vehicle perception nowadays, considering that state-of-art algorithms are based on DL techniques, and they are data-driven. Even though there are multiple public datasets for the developments of self-driving vehicle perception system algorithms, still, there is no dataset that combines all the relevant information to solve multiple perception tasks. For example, there are datasets to train and evaluate vehicle or pedestrian prediction algorithms, however, there is no dataset that contains the information for both, as per our knowledge. Therefore, it is recommended to create a general benchmark dataset that combines all the relevant information with appropriate labelling to enable researchers to develop, evaluate, and compare algorithms that would solve multiple perception tasks. Moreover, it would be beneficial to record a dataset in the UK since the larger part of the datasets was documented in countries that drive on the left-hand side of the road.


August 2022


[1] W. H. O. WHO, ‘Global status report on road safety 2018: Summary’, World Health Organization, 2018.

[2] A. Hart and C. Cox, ‘How autonomous vehicles could relive or worsen traffic congestion’, SBD HERE, Berlin, 2017. Accessed: Oct. 05, 2020. [Online]. Available: https://www.here.com/sites/g/files/odxslz166/files/2018-12/HERE\_How\_autonomous\_vehicles\_could\_relieve\_or\_worsen\_traffic\_congestion\_white\_paper.pdf

[3] M. COLONNA, ‘Urbanisation worldwide’, Knowledge for policy - European Commission. Oct. 2018. Accessed: Oct. 06, 2020. [Online]. Available: https://ec.europa.eu/knowledge4policy/foresight/topic/continuing-urbanisation/urbanisation-worldwide\_en

[4] \DJor\dje Petrović, R. Mijailović, and D. Pešić, ‘Traffic accidents with autonomous vehicles: type of collisions, manoeuvres and errors of conventional vehicles’ drivers’, Transp. Res. Procedia, vol. 45, pp. 161–168, 2020.

[5] M. Schwall, T. Daniel, T. Victor, F. Favaro, and H. Hohnhold, ‘Waymo public road safety performance data’, ArXiv Prepr. ArXiv201100038, 2020.

[6] Anthony Levandowski on lessons learned at TC Sessions: Robotics+AI, (Apr. 22, 2019). Accessed: Aug. 18, 2022. [Online Video]. Available: https://www.youtube.com/watch?v=fNgEG5rCav4

[7] P. Casts, ‘Chris Urmson: Self-Driving Cars at Aurora, Google, CMU, and DARPA - Lex Fridman Podcast’, Pocket Casts. https://pca.st/JD97 (accessed Aug. 18, 2022).

[8] J. Ren et al., ‘Accurate single stage detector using recurrent rolling convolution’, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 5420–5428.

[9] G. Brazil and X. Liu, ‘Pedestrian detection with autoregressive network phases’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7231–7240.

[10] L. G. Galvao, M. Abbod, T. Kalganova, V. Palade, and M. N. Huda, ‘Pedestrian and Vehicle Detection in Autonomous Vehicle Perception Systems—A Review’, Sensors, vol. 21, no. 21, p. 7267, 2021.

[11] X. Li, X. Ying, and M. C. Chuah, ‘Grip++: Enhanced graph-based interaction-aware trajectory prediction for autonomous driving’, ArXiv Prepr. ArXiv190707792, 2019.

[12] R. Izquierdo et al., ‘Vehicle Lane Change Prediction on Highways Using Efficient Environment Representation and Deep Learning’, IEEE Access, vol. 9, pp. 119454–119465, 2021.

[13] M. Biparva, D. Fernández-Llorca, R. Izquierdo-Gonzalo, and J. K. Tsotsos, ‘Video action recognition for lane-change classification and prediction of surrounding vehicles’, ArXiv Prepr. ArXiv210105043, 2021.

[14] S. Ahmed, M. N. Huda, S. Rajbhandari, C. Saha, M. Elshaw, and S. Kanarachos, ‘Pedestrian and cyclist detection and intent estimation for autonomous vehicles: A survey’, Appl. Sci., vol. 9, no. 11, p. 2335, 2019.