Written evidence submitted by the Met Office (SDV0042)
As the UK’s National Meteorological Service, a Public Sector Research Establishment (PSRE) and an Executive Agency of the Department for Business, Energy and Industrial Strategy (BEIS), the Met Office is responsible for monitoring and forecasting the weather and providing the National Severe Weather Warning Service (NSWWS) which warns of the impacts caused by severe weather ranging from disruption to transportation networks to danger to life. We are also responsible for providing the UK’s space weather monitoring and forecasting service through the Met Office Space Weather Operations Centre (MOSWOC), and running the Met Office Hadley Centre, which delivers climate science and services to help Governments, people and organisations understand and prepare for climate change. As a leading authority on meteorological science, Met Office also plays a key role in providing scientific insight and advice relevant to UK interests.
Safe operation of all vehicles requires an awareness of, and sensitivity to, the prevailing weather conditions. Increasing levels of connectivity and automation introduce additional considerations that must be understood to ensure safe and effective implementation of self-driving technologies. For example, the effectiveness of key sensors can be affected by certain weather conditions. To explore and address these challenges, Met Office has been collaborating with industry, other PSREs, regulators and academia; drawing on our expertise and capabilities to support decision making and technology across the Future of Mobility sector.
Having a sufficient understanding of the weather challenges is critical for the safe deployment and eventual operation of Future Mobility. For connected and autonomous vehicles (CAVs), considerations include:
Our deep expertise in technologies for observing weather phenomena is key to establishing how and where the weather may have an impact on critical vehicle components, such as perception systems. This is known as impact pathways.
It is in this capacity that we respond to this call for evidence. Our submission builds on evidence submitted to the 2021 Law Commission’s consultation on Automated Vehicles.
No MO Response
Met Office is engaged with a number of key initiatives and work programmes directly relating to autonomous vehicles on land, sea and air. Some examples relevant to this call include:
a) The Sensor Assurance Framework (SAF) project
As the UK’s national meteorological and metrology institutes, the Met Office and National Physical Laboratory (NPL) are jointly running the Sensor Assurance Framework project (current stage is due to conclude 31 March 2023). The project’s aim is to develop a usable and reliable framework for understanding how well CAV sensors perform in different weather-related conditions and is fully funded by the Centre for Connected and Autonomous Vehicles (CCAV). Building on previous work, the intention is that, when fully developed, this framework will be used to support validation, safety assurance and testing of automated vehicles across the UK, and beyond.
Notably, CAVs will see human senses replaced by perception sensors (typically cameras, lidars and mm-wave radars) and the brain by an automated driving system (ADS) and supporting decision systems. While technically complex, the successful implementation of automation will make transport more accessible; more sustainable through, for example, reduced private ownership and increased adoption of clean electric drive and significantly reduce fatalities associated with driver error. Our critical deliverable from the project is the demonstration of an external environment reference testbed. This testbed is based at our Met Office Research Unit in Cardington and primarily comprises a dense network of research-grade meteorological reference observation instruments alongside a broad array of representative vehicle sensors and supporting metrological instrumentation and reference targets. The comprehensive record of concomitant weather and vehicle sensor data being compiled provides a rich evidence base for investigating a wide range of critical weather impacts and establishing how those effects can be faithfully represented to enable robust safety assurance. The testbed has been collecting data since early 2022 and extensive, collaborative analysis and stakeholder engagement (regulatory and industry) is underway.
b) Developing standards for the Operational Design Domain (ODD)
The autonomous vehicle sector draws on experience and expertise from a wide range of disciplines. As such, development and description of standards can be a key activity for aligning technical, scientific and regulatory progress. The Met Office contributed significantly to a British Standards Institute Publicly Accessible Standard (BSI PAS:1883) in 2020. This sets out a framework for describing the operational design domain (ODD) for autonomous vehicles: the conditions within which a vehicle can safely operate in a self-driving mode. The Met Office is also a member of the Association of Standardization of Automation and Measuring Systems (ASAM) and is actively contributing to developing the OpenODD standard. This is aimed at representing the ODD specification syntax and semantics of taxonomy standards in a way that supports virtual simulation, which is a vital component of the assurance process. Through these activities, the Met Office is showcasing the UK as an international scientific thought leader on CAV, with outputs and learnings informing the Met Office’s own development.
c) Minimum Operational Design Domain Environment Specification Tool (MODDEST)
The MODDEST project brought together a consortium of key players across the CAV sector to combine independent, trusted data sources required to evaluate the ODD according to the BSI PAS:1883 standard. The project, funded under the BEIS Regulators’ Pioneer Fund and conducted in 2021/22, was the first attempt at a live implementation of the ODD and aimed to highlight and address key challenges in operational use of the ODD concept. In addition to trialling a novel road weather data API, the Met Office illustrated considerations for detailed weather descriptions and developed a framework for organising and prioritising ways in which the weather affects the ODD. This informed recommendations for how weather information may be best exploited for effective use in autonomous decision support. Having recently completed the first phase of the MODDEST project, the consortium is developing next steps for scaling up and refining the tool to provide an underpinning capability supporting the whole sector. This capability serves as an enabler for the wider CAV sector to develop and deploy safe self-driving systems.
It is important to account for weather and climate factors in planning infrastructure development, which could include changing exposure to weather hazards at decadal timescales and informing decisions around design and regulation. Mitigation can take place both during design and development, as well as at operational timescales. If this is not done, it exposes autonomous systems to risks that may not be tolerable or may result in costly losses. For example, weather related insight from the Sensor Assurance Framework, and related projects, may lead to recommendations for installing specific (physical) meteorological monitoring infrastructure to support autonomous system operation.
Being able to share updated information to vehicles in real time is essential for CAVs to mitigate fast-changing weather impacts in order to operate safely and efficiently within their ODD. It is reasonable to assume that neither an individual vehicle nor the supporting infrastructure will be able to uniquely or comprehensively observe and assess all relevant aspects of weather impact. However dynamic information sharing to facilitate weather-related data sharing in self-driving systems could mitigate risks. Methods for how safety-critical information is shared to all AVs should also be considered, to ensure they are able to benefit, in real time, from knowledge gained during an incident involving an individual vehicle.
From our position as the UK’s Space Weather Operations Centre and space weather risk owner, we feel it is also important to consider space weather impacts and the risk posed to automated vehicle systems by space weather events. Major space weather events can particularly disrupt or damage electrical equipment, including high frequency communication networks, digital systems (e.g. ‘bit flipping’), electrical networks (e.g. vehicle charging infrastructure) and satellite systems. For example, it is assumed that automated road vehicles will have a reliance on Global Navigation Satellite Systems (GNSS), such as GPS. During a plausible worst-case scenario event, GNSS signal may be degraded or completely lost for around 1-3 days. It is not possible to improve upon the GNSS technology to mitigate this risk as it is changes to the atmosphere which cause the degradation. However, we would support a mechanism to provide governance and guidance relating to possible impact from space weather events, including the requirement for back-up systems that do not share the same susceptibility to space weather. We believe the governance should cover, at minimum, the use of GNSS, communications systems and any safety-critical equipment.
The Met Office supports the need to quantify CAV sensor (and sensor system) performance with respect to the environment as part of any regulations developed for ADSs. This should follow both meteorological and metrological best practice and must complement any planned type approval via the CAVPASS safety regime (or similar). This would be most effective when implemented in the pre-deployment approval phase as well as throughout the continuous operation of vehicles.
We also believe a shared and unambiguous understanding of any descriptions relating to weather conditions would ensure there is consistency when designing and operating automated vehicles and the infrastructure within which they operate. A starting point was made in the publication of BSI’s PAS 1883 'Operational Design Domain Taxonomy for Automated Driving Systems' which defines a common language (or taxonomy) for describing the operating conditions, such as the environment and driving situations, an automated vehicle has been designed to function in. Through our experience working in the aviation industry, we are aware that they, amongst other industries, have similar protocols for the accreditation of instruments and systems used by air traffic control.
Sensor performance (and the consequent impact on the response of the ADS) should be quantitatively understood across the widest possible range of environmental conditions, most notably meteorological edge cases where sensor performance is likely to be compromised and the CAV is likely to exit its ODD. Twould need to capture sensor and system degradation in a traceable and quantitative sense. We believe that any safety case should consider how the CAV will acquire any necessary environmental situational awareness, with an acceptable level of uncertainty, in order to remain within its ODD. This is often described as the “measurability of the ODD”.
Furthermore, the Met Office sees an opportunity, through the regulation and mandatory requirement to share safety-critical observation data, for the UK to promote symbiosis between the mobility sector and weather forecasting capability. The way in which CAVs are impacted by the weather is novel and there is a unique opportunity to use vehicles and autonomous systems themselves to record weather and weather impact directly. For example, the output from CAVs and supporting infrastructure sensors is likely to contain some (indirect) information about the prevailing weather conditions. This could be a valuable source of information to meteorological forecasting agencies. Insights would support Automated Driving Systems (ADSs) in real time and, in the long-term, could be used to improve the UK’s understanding of dynamically changing weather and therefore benefit all weather forecast users. We would therefore suggest exploring whether the safety critical regulation of data sharing could be designed at outset, even if the facility is not used initially.
To enable data to be exchanged, we would support a minimum baseline of information being defined and shared across platforms in an interoperable way, including consideration of both mobility infrastructure and relevant data provision services. This means that the data could be exchanged in a useable and agreed way whilst also ensuring that any additional data input, such as that from forecast providers, can be assessed against pre-agreed standards to ensure levels of reliability. This principle is practiced by the International Civil Aviation Organisation (ICAO) who regulate the aviation industry to ensure that observations of weather and/or environmental conditions that pose a threat to safe operations are shared across the network. This is done both routinely for general weather and specifically for conditions which pose a threat to safe operations.
We would support the consideration of weather conditions, and the response of sensors and systems to these conditions, at the outset of investigations regarding high profile automated vehicle accidents. From our experience of working with the aviation industry, they have demonstrated expertise in considering the role of meteorology in accidents involving automated vehicles, an approach that could be adopted in other industries. For example, meteorology is one of the main factors considered by the Air Accident Investigation Board (AAIB) when investigating accidents. However, we acknowledge that any specialist meteorological service for accident investigation (whether that is sourced from the private or public sectors) would require appropriate resources to ensure its long-term viability.
Weather affects autonomous systems uniquely, compared to conventional human operated transport, and differ due to machine-led perception and decision-making behaviours. Not considering, or only partially considering, weather in these behaviours, risks CAVs being only partially safe. For example, a CAV may have been deemed safe to travel in rain, but if it experiences a different type of rain (drop size, distribution) from test conditions, the ADS may not be able to appropriately mitigate risks. From the Met Office’s work on the Sensor Assurance Framework, we would support a framework for testing components and whole systems against a comprehensive range of environmental conditions. The impact of environmental conditions on object detection skills could then be a quantified metric used as part of safety assessment.
In order to inspire public confidence in the CAV industry, we would support transparency in the safety testing of ADSs. By ensuring weather-related incidents are safely mitigated against, we can best ensure the UK CAV industry is protected and sets standards as global leaders. For example, if virtual testing technologies are used, the Met Office would support mechanisms for ensuring simulators generate “life-like” weather and weather impacts during testing processes.
The Met Office currently coordinates with National Highways to improve road safety and mitigate weather impacts. There is an opportunity to explore a similar regional-scale, regulated, joined-up approach for the safety operation of CAVs on the road during weather events. This ‘air traffic control’ network management approach could include speed restrictions or re-routing around weather hazards to avoid congestion.