LMP0004
Written evidence submitted by ‘Project X’
Key points :-
1. The scope of government major projects and programmes are always susceptible to variation; changes in political priorities and ministerial churn are recognised factors but we also know that ambitious transformation programmes, which account for a significant proportion of the Government Major Projects Portfolio (GMPP), require a sophisticated understanding of interdependencies and behaviour change, and that this is not always captured in the original scope.
2. The desire for reliable cost and schedule estimates must be tempered with a recognition that government projects and programmes are inherently complex; this introduces high levels of risk and uncertainty at the ‘front-end’ of the project or programme. The effective communication of risk and uncertainty is therefore crucial, as is the responsible use of probabilistic methods (models) in a format that is understandable to decision-makers.
3. We can understand complexity, to an extent, through an appreciation of the interdependencies that occur at any given time during the life-cycle. This is not an easy task however, particularly in very-long duration projects and programmes where changes in policy may interact with budget constraints. Complex systems are by definition ‘unknowable’ and thus, we must learn to live with the consequences of complexity in the context of major government programmes. We have produced the world’s largest systematic review to date on the performance of major and megaprojects with > 6000 academic summaries analysed, generating a map of our understanding so far and charting future avenues. The paper identifies 18 main causes of poor performance and proposes 54 ‘cures’, concluding that no isolated factor is responsible for failure in megaprojects, with a number of interrelated factors contributing equally to poor performance.
4. Effective and robust governance of major projects and programmes is crucial to the accountability of state spending. Our soon to be published research on GMPP assurance demonstrates that ‘delivery confidence’ of the major projects and programmes included in the study improves during the time that they are assured on the GMPP. This would suggest that ‘enhanced’ governance routines are desirable in programmes and projects that exhibit high degrees of complexity and scale.
5. World-class research into project delivery is crucial to supporting the government's post-pandemic recovery strategy . We are concerned that the availability of UKRI funding for research in project studies doesn’t match the scale of the UK government’s ambition for the delivery of the major project portfolio and the crucial role of successful government projects in a post-pandemic world.
1. Project X
1.1 In 2015, and in response to the Public Accounts Committee (PAC) ‘Delivering Major Projects in Government’ inquiry, a collaboration of academics, civil servants, industry and professional body representatives formed a working group to identify research ‘grand challenges’ in the space of major project and programme delivery.
1.2 With support from the Economic and Social Research Council (ESRC) and The Infrastructure and Projects Authority, the collaboration led to the launch of ‘Project X’.
1.3 ‘Project X’ seeks to generate unique insights into the performance of major projects and programmes within and outside of government through excellent ‘co-produced’ research. Our long term ambition is to build a community of next generation scholarship through advanced training in research methods that enable a more sophisticated understanding of the nature of major projects and programmes.
1.4 Our aim is to improve the evidence base by fostering distinct communities across disciplines and knowledge areas, united by a common interest in project and programme management and delivery. We aspire to translate this research into ‘useable’ outputs that support continuous project and programme management improvement in government and beyond.
1.5 Our researchers have contributed to recent parliamentary enquires into the delivery of major government projects and programmes including the 2019 Public Administration and Constitutional Affairs Committee through oral evidence in 2019 http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/public-administration-and-constitutional-affairs-committee/the-governments-management-of-major-projects/oral/103738.html and the written evidence submission at http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/public-administration-and-constitutional-affairs-committee/the-governments-management-of-major-projects/written/92642.html and in 2020 https://committees.parliament.uk/publications/2162/documents/20079/default/
1.6 This submission is intended to provide an update to our evidence in the context of the National Audit Office report published in November 2020.
2. Project X research themes
2.1 Following a strategic investment of £1.1m from the ESRC, ‘Project X’ transitioned from the IPA into a wholly independent research platform, led by The University of Sussex in collaboration with academics from The University of Manchester, University College London, The University of Oxford, Cranfield University, The University of Bath, The University of Hull and The University of Brighton.
2.2 In early 2020, The University of Leeds and The University of Warwick joined the platform.
2.3 Project X research continues to be organised into thematic areas that reflect the grand challenges we have identified through extensive consultation and engagement with our stakeholders;
● Theme A: Defining value, understanding and measuring success and the identification of critical success factors (led by The University of Oxford and The University of Leeds)
● Theme B: Front-end and back-end management practices and their influence on project performance (led by University College London and The University of Bath)
● Theme C: Data quality and use and their connections to project performance (led by The University of Manchester - Alliance Manchester Business School and The University of Warwick)
● Theme D: Assurance, reviews, reporting and governance and their connections to effective decision-making (led by Cranfield University)
● Theme E: Capability & knowledge management (led by Brighton University and The University of Sussex)
● Theme F: Government transformation (led by The University of Manchester and The University of Sussex)
3. Our view on the National Audit Office ‘Lessons Learned from Major Programmes’ Report
3.1 Scope
Our research on the ‘front-end’ of projects, led by The University of Hull and funded by the Project Management Institute (PMI) suggests that project success criteria is a contentious issue for scholars and practitioners alike; this is a consequence of the multiplicity of value-drivers and the causal interactions that occur during the initial phases of project delivery. Key to these interactions are the higher-level success criteria established prior to project definition and the efficiency measures (‘iron triangle’) to deliver the subsequently-defined project. Literature in this area suggests many criteria other than simply meeting iron triangle objectives, including end-user satisfaction and other stakeholder satisfaction metrics; the triple bottom line (economic, social and environmental criteria); flexibility - the project’s ability to deal with changes in the project definition or scope with minimal impacts on schedule, budget and quality; and the performance of the project manager as a leader and the controllability of the procedure between the front-end stage up to project delivery and transition into operations and business-as-usual. The latter point is particularly pertinent to our understanding of the relationship between policy and project delivery.
Defining the scope is absolutely crucial to a positive future project trajectory. Strategic intent is clearly important and our understanding is that revisions to the HM Treasury Green Book due in late 2020 will seek to strengthen business-cases through improved appraisal of strategic intent. Our researchers have problematised the conflicting demands in achieving scope clarity at the very early stages of the life-cycle - highlighting the importance of unambiguous and clear objective setting and appropriate measurable objectives that are aligned with the policy goals of the government of the day. The effect of parliamentary cycles should be factored into conceptualisations of successful scope development, and an appreciation that the availability of funding from HM Treasury will influence the capability of the project delivery team to deliver against the agreed scope. Our research led by Cranfield University and funded by the Association for Project Management (APM) on ‘the practice of governance’ illustrates this problem in the context of ‘fixed goal’ and ‘moving goal’ projects and programmes.
3.2 Cost and schedule
Deterministic estimates of cost and schedule are often quoted by Ministers when announcing new projects as a vehicle for the delivery of a policy intent, yet history tells us that the accuracy of these estimates are often woeful. Strategic misrepresentation is often quoted by scholars - this may manifest itself in the strategic outline business case where the quality of assumptions can be best described as ‘underdeveloped’ and are presented alongside optimistic forecasts of future long-run benefits. Whilst an unrealistically low initial cost estimate may increase the chance of the project being initiated - future problems are baked in and are often irreconcilable. Our research points to some of the reasons why costs, benefits, and time forecasts particularly for complex projects are systematically over-optimistic in the front-end; this is termed ‘normalization of deviation’ by Pinto and Slevin (2006). This variation during the front-end is important for the concept of ‘lock-in’: costs are considered at the formal decision to execute a project - but often the actual decision to execute precedes this, so decisions are made on earlier, lower, cost estimates. This ‘escalating commitment of decision-makers’ can occur both at the decision-making level (before the decision) and during project execution. Public projects are particularly prone to wider influences such as political and legislative factors in the project front-end and this can lead to entrenched problems as the project evolves..
Benchmarking is widely accepted as useful in the development of assumptions and early estimates of costs, but we are concerned that historical benchmarking requires careful analysis and clear ‘health-warnings’ where applied to projects and programmes of high complexity, particularly those that currently feature on the Government Major Projects Portfolio.
3.3 Interdependencies
Our research into the performance of megaprojects led by University College London identifies a number of interdependencies causing the poor performance and provides solutions as to how they can be avoided. The research highlights the importance of changing the narrative of measuring megaproject success only by its final cost, shifting towards unpacking the problems and solutions throughout different stages of such long endeavours. This paper is one of the first studies to systematically unpack what goes on inside the “black box” of megaproject delivery, recognising that successful performance depends on what happens (strategies and practices) during megaproject planning, execution and handover to operations. The study found that no isolated factor was responsible for failure in megaprojects, with a number of interrelated factors contributing equally to poor performance. The paper identifies six themes, which reveal 18 causes of poor performance and 54 solutions: 1. Decision-making behaviour, 2. Strategy, governance and procurement, 3. Risk and uncertainty, 4. Leadership and capable teams, 5. Stakeholder engagement and management, 6. Supply chain integration and coordination. The work concluded that what is missing in current research is an understanding of megaprojects as a complete production system—from planning, through design, manufacturing, and construction, to integration and handover to operations. The research suggests exploring megaprojects as dynamic inter-organizational systems, including stages of development, delivery, and operations, identifying strategies to design the architecture of the evolving system.
The study is the world’s largest systematic review of megaproject performance to date, and broke records with a very high Altmetric score in the project management field, being classified as the most downloaded of the entire history of Project Management Journal (over 16.300 downloads as of 25th November). The paper attracted significant media coverage across the world (e.g. The Tribune, New Civil Engineer, Engineering and Technology) and attention from senior industrial leaders.
"We at HS2 were pleased to contribute to what is an impressive and exhaustive piece of research into the world of megaprojects, covering both the practical and academic arenas. But what is really crucial is it has concluded on some key themes that those of us who wrestle with these challenges every day can apply as a useful guide to help drive success, something that both the profession and wider society will benefit from."
Mark Thurston, CEO HS2 Ltd. stated in the Daily Mail (14th February 2020).
Benefits are the principal reason why an organization may seek to enact change through programmes and projects. The discipline of identification, definition, planning, tracking and realization of benefits is recognized to be instrumental in achieving organizational strategy. Our study, led by The University of Hull, presents a cross-national comparison of public sector benefits management (BM) practices in Australia, Canada, the UK and the USA and explores ‘BM practices in action’, considering to what extent ‘espoused’ or ‘mandated’ frameworks are actually practised and perceived by their users. Employing qualitative analysis, semi-structured interview data were analysed from 46 participants with experience in sponsoring, managing and/or reviewing government projects. The results expose considerable variation in the adoption and standardization of BM frameworks from inter and intragovernmental perspectives. We evidence a strong focus on benefits identification across the data set, specifically at the outset (the business case stage seeking project approval) and observe deterioration in focus as the project or programme progresses through the authorization (or assurance) approval gates towards close-out and operations. The results further emphasize the prominence of political interest, leadership buy-in, a benefits-driven culture and a transparent benefit reporting mechanism in the implementation of ‘effective’ frameworks - and a recognition that temporal changes in interdependencies across the project system will create unintended benefits.
There is a wider debate opened up with these findings about the accountability for benefits realization. In the UK system, it is the Senior Responsible Owner who is responsible both for delivery of the project and for the benefits resulting from this – indeed, the SRO has to sign a letter agreeing to this (see UK Government 2019 for an example). However, we have already seen that there is not only sometimes a diversion between the pre-defined project and the emergent and sometimes fluid nature of benefits, but also that evaluation of the benefits of a major public project can be difficult when the economic environment within which it operates has itself changed significantly over the lifetime of a project. Sometimes the project merely facilitates benefits and it is up to other bodies to harness those benefits (e.g. construction of infrastructure might enable economic development, but it will be up to local authorities to take advantage of the infrastructure). In these circumstances, it is not clear how realistic it is for SROs to take complete accountability for the project benefits.
3.4 Governance
Government policy is generally enacted through portfolio, programme and project management, ideally framed by a holistic approach. The successful delivery of outcomes – that is, the benefits that projects were set up to achieve – is essential to achieving policy objectives rather than the operational delivery of outputs on time/scope/budget. Our recent research has illustrated how commercial organisations assess projects/programmes/portfolios based on outputs, without consistently tracking whether these align (or remain aligned) with the organisation’s strategic goals. These two definitions of ‘project success’ – achievement of benefits and delivery/operational success (often called “project management success” going back to de Wit [1988]) – are distinctive (although can be, and possibly often are, confused with each other and overlap) and both need to be considered in the context of effective governance.
Many problems within major public investment projects have their origins before the final decisions to go ahead, which means that there are opportunities to re-scope and improve projects. Our research on ‘developing the practice of governance’ led by Cranfield University and funded by the Association for Project Management illustrates how ‘soft’ analysis methods are important in helping sponsors or owners to ‘see through complexity’ and inform their decisions. We argue that soft analysis methods should be applied to all major projects to identify the most critical issues and risks ahead of time and ahead of the final decision to go proceed. The research also uncovers two types of project, fixed-target and moving-target, and recommends that the governance and assurance system will be very different for these types of projects because they are fundamentally different entities. In a similar vein, we continue to look at the efficacy of assurance routines in GMPP projects through access to IPA data; studies funded by the Project Management Institute and led by The University of Hull present s framework analysis approach to the evaluation of a subset of the GMPP assurance database leading to original insights into i) the framing of assurance review recommendations, ii) the treatment of assurance review data and iii) the subsequent tracking of the implementation of actions arising from the assurance review process. Our analysis reveals that the ‘delivery confidence’ of the major projects and programmes included in this study improves during the time that they sit on the GMPP. This would suggest that ‘enhanced’ governance routines are desirable in programmes and projects that exhibit high degrees of complexity and scale.
3.5 UKRI funding for project studies research
The availability of funding from UKRI to enable the further development of project studies research is crucial to building the evidence base necessary to support the government’s ambition of a world-class project delivery system. Project X is engaged in a stream of fundamental research to identify new practices and capabilities to mitigate project delivery failure. Work is underway to update our understanding of early-stage strategic decision-making, interdependency and the governance of emergent risk and contemporary approaches to leading change and managing diversity. The initial investment of £1.1m by the Economic and Social Research Council is welcome, but further investment comparable to other ESRC priority areas is needed. By way of example, analysis of COVID19 research project funding data provided by UKRI shows that no substantial funding awards for project studies were made, despite the fact that government spending on unexpected and urgent projects and programmes during the pandemic are at unprecedented levels and of significant concern to citizens and society. Furthermore, the Cabinet Office ARI explicitly states the strategic importance of project studies at the heart of government. Our recent publication ‘Project and programme research centres: Lessons for scholarship, policy and practice’ encapsulates a programme of research led by The University of Bath and funded by the Association for Project Management presents exploratory studies of different institutional models for project-studies research that have been adopted globally as a vehicle to facilitate formal collaboration between universities, professional bodies, government and industry. The report provides useful insights and features illustration of successful research platforms through a series of ‘deep-dives’.
Professor Mike Bourne, School of Management, Cranfield University
Professor Andy Davies, Science Policy Research Unit, The University of Sussex
Dr Juliano Denicol, The Bartlett School of Construction and Project Management, University College London
Dr Richard Kirkham, Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester
Professor Mike Lewis, University of Bath Management School
Professor Giorgio Locatelli, School of Civil Engineering, The University of Leeds
Professor Paolo Quattrone, Alliance Manchester Business School, University of Manchester
Dr Rebecca Vine, University of Sussex Business School
Professor Terry Williams, Hull University Business School, The University of Hull
November 2020
Relevant publications arising from Project X
Msulwa, R., Bloomfield, K., Young, P., and Lewis, M. (2020). Project and programme research centres: Lessons for scholarship, policy and practice, Association for Project Management. https://www.apm.org.uk/media/46611/ppr-benchmarking-report_final.pdf
Williams, T., Vo, H., Samset, K., and Edkins, A. (2019). The front-end of projects: a systematic literature review and structuring. Production Planning & Control, 30:14, 1137-1169. https://doi.org/10.1080/09537287.2019.1594429
Williams, T., Vo, H., Bourne, M., Bourne, P., Cooke-Davies, T., Kirkham, R., Masterton, G., Quattrone, P., and Valette, J. (2020). A cross-national comparison of public project benefits management practices – the effectiveness of benefits management frameworks in application, Production Planning & Control, 31:8, 644-659. https://doi.org/10.1080/09537287.2019.1668980
Eltigani, A., Gardiner, P., Kirkham, R., Williams, T., Ou, L., and Calabrese, A. (2020). Learning in and from projects: the learning modes and a learning capability model. Production Planning & Control, 31:10, 786-798. https://doi.org/10.1080/09537287.2019.1690175
Bloomfield, K., Williams, T., Bovis, C., and Merali, Y. (2019). Systemic risk in major public contracts. International Journal of Forecasting, 35(2), 667-676. https://doi.org/10.1016/j.ijforecast.2018.10.005
Bourne, M., and Parr, M. (2020). Developing the Practice of Governance, APM Research Fund Series Report, Association for Project Management https://www.apm.org.uk/media/41106/developing-the-practice-of-governance-report_web1.pdf
Denicol, J., Davies, A., and Krystallis, I. (2020). What Are the Causes and Cures of Poor Megaproject Performance? A Systematic Literature Review and Research Agenda. Project Management Journal, 51(3), 328–345. https://doi.org/10.1177/8756972819896113
Makridakis, S., Williams, T., Kirkham, R., and Papadaki, M. (2019). Forecasting, uncertainty and risk management. International Journal of Forecasting,
Volume 35, Issue 2, Pages 641-643, ISSN 0169-2070, https://doi.org/10.1016/j.ijforecast.2018.10.002
Kalra, J., Lewis, M., and Roehrich J. (2020). Coordination failures in Complex Service outsourcing. Supply Chain Management: An international journal, In Press.
Kortantamer, D. (2019). Leadership: Responding to complexity, APM Research Fund Series Report, Association for Project Management, https://www.apm.org.uk/media/35047/leadership-complexity-research-report-final-lr.pdf
Vo, H., Kirkham, R., Williams, T., Cook-Davies, T., Howells, A., and Forster, R. (2020). (in press). An empirical study of assurance in the UK government major projects portfolio: from data to recommendations, to action or inaction. International Journal of Managing Projects in Business. https://doi.org/10.1108/IJMPB-10-2019-0242