HoC 85mm(Green).tif


International Trade Committee 

Oral evidence: UK trade negotiations, HC 127

Wednesday 23 March 2022

Ordered by the House of Commons to be published on 23 March 2022.

Watch the meeting 

Members present: Mark Garnier; Paul Girvan; Tony Lloyd; Anthony Mangnall; Lloyd Russell-Moyle; Martin Vickers; Mick Whitley; Mike Wood.

In the absence of the Chair, Mark Garnier took the Chair.

Questions 350 - 390


I: Dr Jonathan Cave, Member, Regulatory Policy Committee; Richard Price, Chief Economist, Department for International Trade; Tammy Holmes, Deputy Director, Trade Agreements Analysis, Department for International Trade; Stephen Gibson, Chair, Regulatory Policy Committee.

Examination of witnesses

Dr Jonathan Cave, Richard Price, Tammy Holmes and Stephen Gibson.

Q350       Chair: A very good morning to everybody at this International Trade Committee meeting. We are looking at economic modelling on trade deals. We have with us a very distinguished panel. Perhaps I could ask you to give your name, rank and serial number.

Richard Price: I am Richard Price. I am the director for analysis at the Department for International Trade.

Tammy Holmes: I am deputy director for trade agreements analysis. My team deliver the impact assessments that we are looking at today.

Stephen Gibson: I am Stephen Gibson. I am the chair of the Regulatory Policy Committee. We have reviewed the impact assessments that the Department has produced.

Dr Cave: My name is Jonathan Cave. I am a member of the economics department at Warwick University and economist member of the RPC.

Q351       Chair: Richard, what are the Government’s impact assessments and how do they work?

Richard Price: Chair, first of all, could I thank you all for the opportunity to be here to provide evidence on our impact assessments from our free trade agreements with Australia and New Zealand? Before talking about the impact assessments specifically, would it be helpful if I described a little bit of the process that leads up to it, just to give you some context?

Chair: That would be really helpful, yes.

Richard Price: Prior to negotiations, as you know, we produce modelling of a possible agreement to help give a sense of the size and the pattern of potential benefits for the UK and for our trading partner. That helps to inform our priorities and our overall approach to the negotiation. We publish that as a scoping assessment that we submit to you and make public, so that there can be a proper debate and people have insights into the potential impacts of the negotiation.

During the negotiations themselves, my team of analysts works very closely with the negotiating teams and the chief negotiators to get a sense, on the basis of that analysis, of where value potentially lies in the negotiation, to support decisions and to help them to understand potential trade-offs as they negotiate. That really draws on support from right across Government, because a lot of the sectoral expertise that we draw on resides in outlying Departments.

Towards the end of the negotiations, we look at what has been agreed and try to assess the impact of that on barriers to trade and, consequently, on the economy. We then turn it into impact assessments, which is what we are here to discuss today.

Finally, as set out in the impact assessments, the Department will, for the future, be conducting monitoring and evaluation so that we have a real understanding of whether what we have assessed in our impact assessment turns out to be correct and, if not, whether there are things that we can do about it through policy.

To come to the impact assessments themselves, their purpose is to make sure that Parliament and the public have a clear set of analysis on the potential impacts of what we have negotiated on the economy, and that analysis is used to inform Ministers and Government more widely on the potential impacts of a deal.

There are a few really important things to say about what they are and what they are not. They assess the incremental impact of the agreement itself. They are not looking at lots of other policy changes or things that might be changing in the global economy. They are trying to separate out, as is normal for impact assessments from Government Departments, just what the agreement itself is achieving.

Chair: All other things have to remain the same

Richard Price: Broadly speaking, yes, that is right. They assess the gains in the long term, because the thing about free trade agreements, as you will know, is that they affect large parts of the economy in all kinds of ways and it takes time for businesses and regulations and so on to adjust. We look at the effects over the long term, which we normally interpret as around 15 years’ time.

Of course, lots of other things will be changing in the economy over that period, so this is not a forecast of what will happen; it is an attempt to assess the specific impact of the FTA. It gives a broad order of magnitude and direction rather than a series of really very precise estimates, and we try to reflect those sources of uncertainty in the impact assessment.

Our approach to assessing the main economic impacts of the agreements focuses on our main global economic model. It is something called a computable general equilibrium model, CGE; that might come up a few times in this discussion.

Chair: It is going to come up in the next question.

Richard Price: I will not go into detail on that, because I am sure you will have questions about it. Essentially, for the agreement that we have negotiated, it gives us a sense of what is happening to the economy overall and what is happening sector by sector. From that, we interpret what is likely to happen across the regions and nations of the UK.

We also look, in the impact assessment, at a range of other analysis, including things like the environment and social impacts, and impacts on different parts of the labour market.

Q352       Chair: We have a lot of questions going into a lot of these details, including the regional distribution, but I just wanted to come back to something you said a bit earlier. You start off by looking at what the opportunity is. For example, if you are looking at an economy that is largely agricultural, such as Australia, which has a lot of agriculture, you will look at it from that point of view. You will come up with an assessment of where the opportunity lies and then there will be the outcome. You have the objective and then the outcomes.

Do you mark the Department’s homework in terms of looking to see that it has maximised the opportunity that is presented by a specific trade deal? If you sit there as an economist and say, “We can do really well out of x, y, z”, and the assessment comes out saying it is going to be a 0.5% increase in GDP, and when they come to the actual one, it is only a 0.1% increase in GDP, do you then turn around and say that your negotiators have failed in their duty to maximise the opportunity of global Britain?

Richard Price: No, we are not doing that. What we are doing is working closely with the negotiators through the process to help them understand at a fairly sectoral level where the likely opportunities and trade-offs are, so it helps them to fine-tune the prioritisation within the negotiation. That also informs the mandate that we start with. We also then support them in interpreting what is going on within those sectors, looking at individual tariff lines and assessing non-tariff barriers and so on, so trying to work out specifically how you realise that potential gain by, essentially, agreeing lines of text in a negotiation, which is what it really comes down to.

Q353       Chair: If I may tell you what is behind my question, one of the problems that we find on this Committee in terms of analysing trade deals is that it is quite difficult to know what, ultimately, the UK’s negotiating objectives are in terms of our commercial relationship with the rest of the world. That is your starting point. When it comes to an individual trade deal, it is then quite difficult to be able to work out how that trade deal has done against the global objectives and then, when it comes to the more micro level, whether we have done a good deal on those global objectives and on the objectives of a specific country. What we are trying to find at every point is how we can scrutinise the work of DIT in a methodical way, because everything is very different in each trade deal and we are trying to find continuity between one and the other.

Tammy Holmes: I was going to add something, but I am not sure it gives you a full answer to what you are trying to achieve there. It is just to note that, when we publish the scoping assessment, which is the analysis we do at the launch of the negotiations, that is based on some broad, more or less hypothetical scenarios of what a deep or a shallower agreement might achieve. It does not set out, in any specific provision, what we would hope to negotiate. In that sense, to back up Richard’s point, that is not really what we are trying to achieve at that point. We are trying to give a sense of scale. It does not allow you to trace across the two.

The other thing that I would say—and perhaps we will come on to this later—is that we will be conducting monitoring and evaluation once the agreement is in place in order to try to understand the extent to which those benefits are delivered.

Q354       Tony Lloyd: In terms of your initial assessment, one of the things that you are looking at is where the opportunities are for the UK to enhance its trade, but there is also an impact in terms of trade in the other direction. One of the critiques of the Australia agreement, for example, is that it may be detrimental to UK agriculture. Is that something that you model, so that Ministers have a full range of options when they begin the negotiating process?

Richard Price: Absolutely. The model looks at the effect of both imports and exports, and the likely effects of the changes in trade barriers on both sides—both non-tariff barriers and tariffs. It is important to emphasise that the way we approach it in the model is that both exports and imports make a positive difference to the UK economy, in the sense that, as you export, you are steadily creating better jobs and raising additional income and opportunities for UK businesses. Imports also have significant benefits for consumers. They give UK businesses access to imports that improve their own competitiveness, which means that they perform better in exports. We look at that overall. The scoping assessment allows us to identify risks and issues across the whole economy, which means that we can then do deeper dives into individual sectors as we go. Yes, we do that.

Q355       Anthony Mangnall: Sticking with you, Richard, how accurate do you think you are?

Richard Price: In terms of broad orders of magnitude, relativities across sectors of the economy and the trade flows in both directions, we do a reasonably good job in that they certainly indicate the likely outcome. In a sense, we are as good as the inputs that we put into the model, and Tammy might want to say a bit about what those are. I know we are coming on to the CGE later. We know what the tariffs are. We have a pretty good assessment of what the non-tariff barriers are, economy by economy.

The modelling that we do is as good as any modelling that is done around the world. When we set up the Department, we had long conversations with other existing trade Ministries to see how they went about this. We talked with our colleagues in the US system, in Korea, in DG TRADE and in Canada—a whole bunch of countries that we think do this pretty well. We have learned from that and have gone further in terms of developing the better technology of the modelling that I have been talking about. There is also lots of off-model analysis that we have to do to understand what is really driving performance in the sectors.

Q356       Anthony Mangnall: That slightly answers my next question, which is how you think you compare to other countries and what they are doing on this.

Richard Price: I think we are pretty good. I will not say which one it was, but there was one negotiation where our counterparts saw what we had done after the event and said, “If we had known how much progress you had made, we would have gone in a lot harder on you”. There are lots of things that the pandemic has made more difficult. One of the things that we have benefited from is more time talking to a range of our counterparts around the world in virtual meetings, so we are learning from each other.

We also set up our modelling review panel, which has kicked the tyres on what we are doing and has suggested a number of ways in which it thinks we can improve. We are already implementing some of that. Stephen and colleagues at the RPC then look at the way we interpret and present the results. We have deliberately put in place a series of things that open us to scrutiny and challenge and create that environment of continuous improvement in our modelling.

A final thing to say is that, looking across Government, as we do, this is pretty good value for money. There are not many policies that get you significant gains permanently in every year to GDP for a relatively small amount of public money.

Q357       Anthony Mangnall: Tammy, can I bring you in? We have talked about CGE already. Can you just explain it and how it is used to model the impact of trade agreements?

Tammy Holmes: First, it is important to remember that CGE sits among a range of other analysis that we carry out, from simple, descriptive analysis of trade flows to other, more complicated techniques that I will not go into in detail, but some of those are included in the impact assessment as well.

The big advantages that CGE has—and it stands for computable general equilibrium modelling—are, first, that it is the best technique for capturing the impacts at a wide economic level. It is really the only technique that can tell you, “What is this going to do to GDP?” For total exports and trade flows at that kind of level, capturing the impacts across all sectors, it is the CGE model that you would want to be using. When we are trying to produce an impact assessment so that Parliament, Ministers and so one can understand the effects at that level, that is where we go.

The other particular advantage that CGE modelling has—there is a hint in the name, “general equilibrium”—is that it captures the links between all of the different areas of the economy. The different sectors, actors, the labour market, prices and sectors across countries are all captured in terms of the inputs and outputs to different sectors. That is all in there and is not something that other models are able to do. That is one of the things that enables it to then give you that high-level economic picture.

Anthony Mangnall: It is not a forecast though. You expressly say it is not a forecast.

Tammy Holmes: It is not a forecast. It is a sense of the incremental impact, and I will come on a little bit more to what we mean by that as I talk through how it works.

The first stage in this type of modelling is to build up a picture of the world as it looks now, which is done primarily through a very large dataset that contains about 140 countries and about 65 sectors within each of those countries. It is a very big and detailed dataset. It is produced externally—we do not produce it ourselves—and that data takes quite a long time to put together.

Once we have that picture of the world as it is now, we then look at the agreement and translate what the impact of that agreement would be on the world. That is primarily in the form of what the tariff changes are and what the changes to other trade costs are that would be brought about as a result of the agreement. Within the model, you are changing those things, which then allows you to turn the handle on the model and see how things change as a result of those reductions in costs.

You then have the world as it was before and your variables such as GDP and trade, and then a sense of what the world looks like afterwards. We think of that, as Richard already said, as a long-run change, because we know that, in practice, it will take years for all of those complex interactions to feed through the economy. That is why it is known as a comparative static model, because it compares a situation at one point in time with a situation much later in time.

There is a lot more complicated stuff going on inside the model; there are assumptions about how production responds to reductions in trade costs and so on, and some of the structural assumptions vary a little from model to model, which we will go into in a bit more detail later. Broadly speaking, these CGE models all work in that way.

As Richard has said, we would not want to be too precise in our interpretation of a single number that comes out of it, but it is much better to have this than not to have it. It is going to give you a sense of what an impact might be in a way that no other form of analysis could. As we have also said, we combine it with other types of analysis and modelling to give you that picture that you see in the impact assessment.

Q358       Tony Lloyd: The point you just made is that, realistically, there is going to be a variation of possible outcomes. Why do you not produce a confidence interval rather than a single figure?

Tammy Holmes: We do produce a confidence interval.

Q359       Tony Lloyd: But you do not publish that for us. We have not seen that, have we?

Tammy Holmes: There is a confidence interval in the impact assessment. There is a sensitivity chapter. I think it is referenced in the summary as well. There is a confidence interval around the GDP impact. In terms of what we do to arrive at that—we might talk about this a bit later as well—there is a huge amount of uncertainty in the world. We see that all the time. There are lots of things within the model that might be different from how we assume they are. When we conduct that sensitivity analysis, we look at the key elements and assumptions in the modelling that we think are most likely to have an effect on that incremental effect of the agreement. We run the model many times—it is called Monte Carlo simulation—and adjust those assumptions and then create a confidence interval from that.

Q360       Anthony Mangnall: How deep do you go into your analysis on this? Coining one of your phrases, it is about second-degree changes. There is a second level. You look at agriculture and you say, “We have changed the tariff rate quotas”. How much further down do you go in your modelling?

Tammy Holmes: It is a complex model that captures the interlinkages between sectors. For example, when, as a first order, the trade cost falls, that will change prices within the economy and there is what is called an input-output matrix. When one sector draws on another, the prices within that will all change, and all these rounds of effects are reflected in the model.

Richard Price: Another thing to bear in mind also in regard to your question is that the CGE model is not the only thing that we are using here. Where we know that there are sensitivities within sectors, we will also do deep dives, often jointly with Departments, into what is going on in those sectors. In fact, this is one of the things that we are building on as a result of one of the recommendations in the modelling review. We use a range of other techniques to look not just at the sector level but, in some cases, right down to the commodity level, in order to understand where the risks and opportunities are and to help negotiators think about how those factors could be reflected in the negotiation.

Anthony Mangnall: We politicians are simple folk—I am sorry; I speak for myself—and we like a number and an idea of what the value of these trade deals are. When we have the scrutiny process, we always hear the Chair, who is not with us today, saying, “It is worth only 0.8%, or something like this, so it is important to understand how you reach these numbers. You say you do it on a 15-year timescale. Do you think that they stand the test of time? In 2035, if none of us have moved on and we are still here, which I am very hopeful for—

Chair: Again, speak for yourself.

Q361       Anthony Mangnall: Will you be saying that they stand the test of time and you are accurate on them?

Richard Price: It goes back to your previous question. Yes, in the sense that we will have seen a number of the things that we have identified in the model playing out. We know how the negotiation will affect different trade barriers—tariffs and non-tariffs. We have a good and increasingly sophisticated sense of how businesses in the UK and in the partner country are likely to respond to those. We know how those sit in the context of broader global markets.

Will the numbers be absolutely precise? No, of course not. Will changes have happened in the broad ways that we expect? Yes, I am pretty confident that they will.

Q362       Chair: On the technicalities of how this thing works, you talk about the model but there are different variations of this model. I was trying to work it out in my head. If you are looking at 145 countries and 47 sectors, and all of those have variables, is that not 145 to the power of 47 different variables on just those two factors? Then you add in commodity prices and all the rest of it, so there are billions and trillions of different ways that you can run this model, depending on what your inputs are and the weightings you give of certain areas. You say you adjust these things. That is the first question.

The second question is whether there is a standard way that this model is written, or is it like an internal combustion engine, where you can have a single cylinder model aircraft engine all the way up to a V12 or W18 Ferrari engine? Give us a sense of how it works.

Tammy Holmes: You are right that there are a lot of variables in the model. It is complex. In terms of whether there are many different versions of it, we will talk about this a bit later but we rely on a standard dataset.

Q363       Chair: Yes, but this is your slide rule in the background. Are you writing this model yourself or are you buying an off-the-shelf model from the LSE or something that everybody else is using?

Tammy Holmes: We procure the model. We are using a model called the GTAP model, which is a fairly standard version of the CGE model that is quite widely used.

Chair: Economists would recognise it.

Tammy Holmes: Yes. We have not written the code or constructed it ourselves.

Q364       Chair: How do you go about selecting which variations? Ultimately, that is where the judgment comes into it. Although you are using the same pocket calculator or spreadsheet, as it were, to run it, the difference is how you as economists within DIT will do this. It is entirely possible that the economists in DEFRA might be running the same model but they can come up with different outcomes because they put different emphasis on certain elements of it.

Tammy Holmes: There is variation in how you could capture the agreement. For that, we do a lot of work with our policy colleagues and across Government to try to make sure that we have a good common view on what the impact of an agreement on trade costs would be. If you are referring to where we do the sensitivity analysis and try to test the sensitivity of results to the variation of some of the key assumptions, we have identified the assumptions.

For example, one of the things we look at is the trade costs as they are captured in the agreement and how we have captured those in the model. One of the things we do is to look at what would happen if we represented those as a slightly higher or lower reduction in trade costs, and that is reflected in that sensitivity analysis that we do. There are a couple of other things that we put in there.

Q365       Chair: But it is a judgment call, is it not, on your part? If you are looking at the cost of trade—for example, the cost of shipping lamb from New Zealand—that is dependent on shipping prices, which are now up 20 times as a result of the pandemic. Presumably you have to make a judgment call on what is going to happen to shipping costs over the 15-year period.

Tammy Holmes: That is included in the model. We do not make a judgment on that.

Richard Price: There is a lot in the data here. The data we are using is drawn from an international consortium that lots of our counterpart trade ministries look at, and so, to some degree, we are all looking at the same data. It also means that we are not required to make a judgment on things like that. This is reasonably reliable, objective international data.

Q366       Mick Whitley: Richard, could you summarise the headline economic conclusions of the impact assessments for the trade agreements with Australia and New Zealand?

Richard Price: For Australia, what we are expecting is a 0.8% increase in GDP, amounting to an addition to GDP of £2.3 billion every year in the long term, in perpetuity. That involves an increase in total bilateral trade between Australia and the UK of £10.5 billion, which is a 53% increase in our current bilateral trade. We see slightly bigger UK exports to Australia than the increase in imports from Australia as a result of that, and nearly an additional £1 billion in terms of workers’ take-home pay across the economy as a result of that.

The particular sectors that we expect to gain from the agreement with Australia are in services and in the manufacturing of equipment. We expect all regions and nations of the UK to gain. In proportionate terms, the biggest gains are, as I recall, for the north-west and the midlands, with gains also for the south-east and the north-east.

Q367       Mick Whitley: Is that in respect of shipping?

Richard Price: No, it is across the range of sectors that gain from this. There is a series of sectors that gain, particularly the manufacturing of machinery and equipment on the goods side. There are a number of gains for service sectors, such as construction and business services, on the services side.

Q368       Chair: I think you mentioned the increase in GDP was 0.8%. Is it 0.08%?

Richard Price: 0.08%, yes.

Q369       Anthony Mangnall: What expectation do you have for job creation between the two sides, and will this increase the improvement?

Richard Price: The way we assess the impact of an agreement is to look at the way the labour force is deployed across the economy. There is no change in the aggregate number of jobs. What changes is what people are paid for doing different jobs.

Chair: The £1 billion.

Richard Price: Yes, exactly. The £900 million, to be precise, comes from two things. First of all, it is, over that long period, a shift in the way workers are employed around sectors of the economy and, secondly, productivity improvements resulting from competition. Businesses that export tend to be more productive, so if you have more of them exporting and they are exporting more, that will also have an effect on economic productivity. It is not that there are more people employed in the economy, but that they are typically doing jobs where the returns to the activities that they are doing are higher, because we have removed constraints to trade with the rest of the world.

Q370       Anthony Mangnall: Because you are focusing on these individual deals, just from the brief that we have had, why do you not include in something like an agreement with Australia and New Zealand the follow-on impact with CPTPP?

Richard Price: That is a very good question, and the answer is that, in the impact assessment, we are trying to isolate the effects of the individual agreement, so that you can judge what impact it is having. In a sense, we are deliberately trying to look at the incremental impacts. However, it is absolutely right to say that the pattern of agreements that we are doing here reflect where we expect growth in the global economy to come from.

In September, we published a contextual document called the Global trade outlook, which is a really good read, apart from anything else. Some of the key things that it says are that we expect south and east Asia to represent more than half of global growth out to 2050. We expect China, south Asia and Asia-Pacific to account for 40% of global demand for importsso the size of the market that we are looking at—by 2050, compared with 32% in 2019. With Australia and New Zealand being part of CPTPP, there are clear economic gains in both of those agreements. There is a significant strategic gain for us in accession to CPTTP beyond those immediate economic gains, because that gives us better access to rapidly growing adjacent markets.

Anthony Mangnall: From my point of view, it is just not looking at it in a siloed way and seeing where there is a covered point, but I absolutely appreciate your answer.

Richard Price: It is probably worth saying that, for each impact assessment we publish, we factor in the effects of the previous agreements that we have signed. In the Australia impact assessment, for instance, we are including the agreement that we signed with Japan. In the New Zealand impact assessment, we are including the Australian impact assessment, so we are avoiding double-counting, in that sense, and we will be doing the same with CPTPP.

Q371       Anthony Mangnall: Mr Gibson, sorry; we have taken a while to get to you. What role does the RPC have in scrutinising impact assessments on trade agreements and what concerns are you reaching on those regarding Australia and New Zealand?

Stephen Gibson: Perhaps I can just introduce the RPC and then answer your question. The Regulatory Policy Committee is the independent better regulation watchdog and our core role is to independently review Government impact assessments and to assess whether they are fit for purpose. We rate the assessment of the costs to businesses—and particularly to small and microbusinesses—and the quality of the analysis of the proposed regulations.

In September 2020, Liz Truss, who was the Secretary of State for International Trade at the time, wrote to Alok Sharma, who was the BEIS Secretary of State, and asked if the RPC could extend its role to provide external independent scrutiny of trade deal impact assessments, which we agreed to.

We worked with our colleagues in the Department for International Trade to develop a checklist of our expectations of what a good free trade agreement impact assessment would look like. That was the basis for our rating of free trade agreement impact assessments. The checklist has 20 elements and includes things like sectoral and regional aggregation, the justification for the model assumptions, the data sources, the assessment of tariff versus non-tariff barriers, and many of the points that you have already questioned and raised this morning.

So far, we have reviewed three new free trade agreements—the two this morning with Australia and New Zealand, and previously the Japan free trade agreementall of which we have rated as fit for purpose. Although we had some initial concerns with the Australian free trade agreement, those were subsequently addressed by the Department, which shows that the purpose that we have and the role that we play is working. They produced a final impact assessment that we were happy with, which addressed our concerns and which we were happy to rate as fit for purpose. We publish all our opinions very transparently on our website.

Q372       Anthony Mangnall: What was your initial rating of the Australia agreement?

Stephen Gibson: Not fit for purpose.

Q373       Anthony Mangnall: What were the requested changes? What did you ask of the Department to get it to the stage where it was fit for purpose?

Stephen Gibson: We raised three areas of concern. The first was that we did not feel that the draft impact assessment provided a balanced presentation of the impacts. It tended to exaggerate and presentationally focus on the benefits, with very limited mention of the costs. We believed that the impact assessment should be a neutral presentation of the outcomes rather than something that tried to play out more the benefits.

Secondly, it did not properly, in our view, describe the range of risks and uncertainties associated with the impacts—some of the points that you have been teasing out this morning about what would happen if things did not go according to our plan.

Finally, it presented a level of certainty and accuracy that we did not believe was supported by the underlying evidence and modelling. As you have heard, much of the modelling is directional and order of magnitudes, so it gives you an idea of what the benefit would be, but not quoting individual millions of pounds or, indeed, hundreds of thousands of pounds.

All of those three points were accepted and taken on board, and the Department did a very good job of re-presenting the work. We thought that the underlying modelling was well researched, but the Department did a very good job of re-presenting the work to an impact assessment that we believed was fit for purpose.

Q374       Anthony Mangnall: This is a good opportunity to bring in Dr Cave, who was quite vocal about some of his concerns with the presentation of the results. Do you want to add a little bit to that, and give your feeling of how this affects future agreements?

Dr Cave: Thank you for the opportunity to expand a bit on what Stephen has said. It is important as a starting point to consider the purpose for which the impact assessments are fit or not fit. In other words, it is slightly different from domestic regulation, where you are making independent decisions about what will be in the regulation. The idea here, as Richard put quite clearly, is to provide a sense of the thinking behind the negotiations, their objectives and the basis on which the proposed agreement was reached.

In that sense, because this is such a complex situation and there are so many assumptions in it, it is quite easy to not wish to overwhelm the reader with too much detail and, in the process, it is possible to give a slightly imbalanced or tendentious picture of what is going on.

In particular, in terms of some of the language, as Stephen has said, there was more talk about the potential benefits than about the potential risks or losses, and this fed through into the language in which things were talked about. Coming to Stephen’s third point about giving the impression of accuracy, a numerical model will produce numerical results down to the nth decimal place. The uncertainty means that those results are not necessarily significant or statistically reliable, which is why the model includes confidence intervals, but the understanding of confidence intervals is not always complete.

The other point is that some of the numbers were presented as monetary sums and others as percentages. We felt quite strongly that all relevant numbers should be presented both as numbers and as percentages, so that there was no confusion about the magnitude of the effects.

In terms of the balance, we raised a number of issues, which came from our reading this document as somebody not steeped in the analysis would read it and trying to avoid the impression of providing a business case.

The other element that I want to briefly comment about is the risks and uncertainties. As Tammy discussed, the model uses Monte Carlo simulations to capture some of the risks, which are small variations in the key parameters that might affect the conclusions of the model, either numerically or in relation to any decision. We had some technical concerns with the way in which those variations or modifications to the model were conducted, in particular in relation to uncertainties—the things that cannot be quantified.

The world, as we know, is a very uncertain place. Many of the things that dominate the numbers that we see today could not have been foreseen at the time the analysis was done. We felt quite strongly that it was important to be clear about which things could be relied upon and which things would have changed, both in a world with the free trade agreement and in the counterfactual world of the free trade agreement not going through.

Q375       Anthony Mangnall: There is a large part here, which is that Departments and, I suppose, politicians are usually more optimistic than they are pessimistic, and hope springs eternal. I am wondering if that is what you are suggesting happened when we first caught a glimpse of the Australia agreement.

Dr Cave: It is inevitable that, if you are involved in a negotiation, you are focused on the successful outcome of that negotiation, which is one of the differences between a regulation, for example, that address a present deficiency in policy, and a negotiation that reaches towards something that is more positive. In the Treasury Green Book, there is quite an extensive discussion of the optimism bias that affects people who are doing things that they regard as positive. I do not think that it was, in a sense, disingenuous, but you are right that it is quite natural.

Q376       Anthony Mangnall: I am going to bring in Richard in a second but, before I do, Dr Cave, you mentioned the Monte Carlo formulations. I wonder if you could just expand a little bit on that. Also, are you optimistic that the Department has learned not to be overly optimistic about its projections on its free trade agreements? We are clearly about to see a lot more come through. Are they learning from your findings and your vocalisation about the concerns with their report?

Dr Cave: Let me address the second one first. The evidence from the New Zealand free trade agreement and our continuing discussions with Richard, Tammy and their colleagues show that they have taken this on board absolutely. I have no problem with that whatsoever. The fact that they have consulted with people like Joseph Francois and Tony Venables and that they have incorporated results from the modelling review shows that they are not only taking on board our advice but, in many ways, are world-leading in what they do.

On the Monte Carlo simulation, the way it works is that, if you have parameters in the model about whose value you may not be certain, the question is whether a mistake in those parameters would lead to a change in the outcomes in a way that might be significant, not for a numerical account of what is going to happen but for some decision or understanding on a complementary policy you might want to take.

The approach that was taken by the Department was to identify a few key parameters and vary them independently and to do many runs of the model to get a sense of the response surface. In other words, it is different if you are wandering about at the edge of a cliff or not, so it is useful to feel around for where you are.

There are other approaches, though, because many of these parameters do not vary independently. In a world in which one of them was high, another one might necessarily be low, and so there are other techniques, such as exploratory modelling, where you simply change everything in the model.

These are, necessarily, small changes. Many of the changes we are seeing in the world right now are not small. Beyond a certain quantitative limit, the changes become qualitative. Trade networks rewire themselves, for example. The model is a starting point for a discussion. It cannot be a substitute for it.

Q377       Anthony Mangnall: That is really helpful. I know Richard wanted to come in. Can I just add to that and ask what you were doing to address the concerns from the RPC on this?

Richard Price: The first thing to say is that the RPC is a really important part of this process. We voluntarily submitted our impact assessments to the RPC, which has responded incredibly well in developing the capacity and capability to scrutinise what, frankly, is a whole new area of activity for Government. The reason why it is important is that, first, as Jonathan and Stephen have demonstrated, they genuinely improve what we do. Secondly, it is vitally important that there is an independent, credible source of expert opinion that is capable of looking through what we are doing and giving you and the public assurance that we are making sensible judgments. We are really grateful to them for that.

I can tell you exactly what happened with the rating that we got of not fit for purpose, and what led to it. Essentially, this was the first from-scratch negotiation that we had done and we were thinking very hard about how we present this. We have to produce something that is intelligible for people who are going to look at it at a glance, without any detailed understanding of the way the trade system works and what a negotiation is. There is a very substantial army of trade experts who crawl all over this and look at it really carefully, as they should.

We were trying to think, “What is right balance and how do we get this across to those two groups?” In the document itself, we erred on the side of trying to simplify, which meant that, in my view and in that of the RPC, we took too much out. We had a good discussion with the RPC around that and ended up restoring a lot of the detail that we had had in the Japan impact assessment, but also bolstering the description for sectoral impacts. You will see that there is a section specifically on agriculture, which we worked very closely on with DEFRA, and DEFRA made a big contribution to that.

There is a lot more discussion of uncertainty and we have put additional sensitivities in. If the RPC were not occasionally giving us a hard time, you would know that the system was not working, but they certainly did in this case and we are all the better for it, honestly.

Specifically on the Monte Carlo analysis, the way that Jonathan describes it is spot-on. There is a judgment as to which things that could change you feed into that analysis of uncertainty. Lots of things will be changing in the global economy. We focused on those that we think will be most germane to the negotiation, the agreement that we end up with and the specific trade relationship.

The things that we focused on are the value and impact of non-tariff barriers to trade, which are hard to observe—there is lots of uncertainty around those—and also how businesses respond on both sides in response to changes in those trade barriers. Lots of other things are going on in the global economy that we could throw in, but we think that those are the things that are most important and create the most uncertainty specifically for those incremental effects of the deal.

Q378       Anthony Mangnall: Just so that I know that you are all being up front with us, when the New Zealand deal was published, was it rated as fit for purpose? Was that the first rating rather than the second?

Richard Price: Absolutely. I just have a couple of things for the record, if you do not mind. First, I misspoke earlier: the greatest proportional gains from the Australia FTA are in the north-east and west midlands. There are still big but slightly lesser gains in the north-west and south-east.

Secondly, I just want to put on record that all of the work that you have just heard described is the product of an incredibly talented team at the DIT. I am remarkably lucky to have an extraordinary number of very talented people in the Department, and also people across Government who work with us to make all this happen. I am enormously proud of and grateful to the team.

Chair: My experience is that they are brilliant people as well.

Q379       Lloyd Russell-Moyle: Richard, for the Australia and New Zealand deals, you used a different assessment for the initial scoping in terms of the model compared to the final outcome. Why was this?

Richard Price: As I said, the approach that we have taken to analysing deals has been improving progressively. We have made a series of incremental improvements as we have gone through. The scoping assessments were at a very early stage and we have learned a lot about the way the model operates and some of its properties since then.

A number of things have changed. I would split them into two categories. There is housekeeping stuff, which is things like making sure we are using the most up-to-date version of the data and the most up-to-date international database on non-tariff barriers and tariffs. We have updated it to include the UK global tariffs, so that we are not double-counting the effects of changing tariffs through the adoption of our global tariff regime and also changes in the FTA.

Then there are things that are more methodology-based, where we are trying to make sure that we are doing things that are robust and improve the accuracy of the model, so that the way we use the model is a better representation of what we think will happen.

The principal change here was that, for the scoping assessments, we had adopted quite a sophisticated approach to trying to understand how difference sorts of businesses are likely to respond to removing trade barriers. However, first of all, it is fair to say that not many of our counterparts around the world use that particular approach. Also, looking at it carefully, we concluded that it was not completely consistent with the empirical evidence that we were seeing. There is lots of data around the world on the relationship between trade barriers and how firms respond to new opportunities, so we switched to that.

Q380       Lloyd Russell-Moyle: What are the big differences between the two in terms of outcomes?

Tammy Holmes: As Richard said, the scoping assessment analysis was based on a model that is, in some senses, more sophisticated. It tries to capture the fact that some industries and businesses within the economy have what are known as increasing returns to scale. As they produce more, they become more productive.

However, in that model, in order to understand the degree of responsiveness for that type of business, you have to make an assessment or a judgment on a particular assumption about that responsiveness. As Richard has said, this model is not that widely used, so we were left in a position where we did not have good evidence or literature to support a judgment on what that particular assumption should be. As it turned out, the results of the model are very sensitive to that assumption.

In the model that we now use, we do not attempt to account for this possible increase in returns to scale. We do not have this complication in the way that business responds, which means that we are able to use the published GTAP dataset without making any judgments of our own about how businesses are going to respond to reductions in trade costs.

Q381       Lloyd Russell-Moyle: Would you say that the model you have settled on is possibly more conservative in its assessments of how businesses might scale up?

Tammy Holmes: As it turns out, as we were running the model, the results were coming out lower in the original model. We think that we were understating the responsiveness of this particular set of industries, but we were not able to have confidence in the degree to which we should adjust that responsiveness, so we felt much more comfortable in the slightly simpler model, where all of these responsiveness assumptions are given to us in the dataset.

Q382       Lloyd Russell-Moyle: In the model you use now, you just have a fixed responsiveness that is embedded in the dataset, rather than having to make tweaks for everyone.

Tammy Holmes: Yes.

Q383       Lloyd Russell-Moyle: The DIT said to us that the changes were informed by discussions with a modelling review expert panel, but the review was not published until after the Australia impact assessment, and the Government response to the review still has not been forthcoming. Why was the change done before some of that public due diligence had been published?

Richard Price: The modelling review was running for about a year. Although we did not involve it in any of the specific negotiations that we were working on, some things became pretty clear quite quickly. We were already working, as Tammy has described, on trying to understand what was driving some of the results that we had seen in the scoping assessments. We specifically put some of these questions to the panel and we got their judgment. We knew that we needed to do something to address this, so we have not been waiting around, as it were, for a lot of things to be published. We have been looking at how we take advice quite quickly to make those changes.

There are a number of things that give me quite a lot of comfort on this. First of all, what we are doing now is more in line with what other Ministries around the world do. As Tammy has described, it draws on actual data rather than us having to make a series of assumptions which are controversial and contestable. Thirdly, it is really hard to make these sorts of comparisons but, very broadly speaking, looking at similar sorts of agreements and analysis from other trade Ministries, the orders of magnitude that we are producing now look more like what we are seeing from other trade Ministries around the world than the scoping assessment numbers.

Q384       Lloyd Russell-Moyle: Do you expect to now use this model going forward in a fixed way? Nothing is fixed forever, is it, but do you expect to use it for the foreseeable future?

Richard Price: Yes. We will continue to make some changes. As I said, we will always use the most up-to-date data, but it will be relatively stable for now. What we will also be trying to do, though, acting on the model review’s recommendations, is to do more off-model analysis and perhaps publish more of that, looking at sectoral modelling as well as the CGE that Tammy has described.

Q385       Tony Lloyd: If this has now become the conventional global model, we have a problem if it is not right. What is the mechanism by which you will check the usefulness, not simply here in the UK but globally?

Richard Price: There are two answers to that. First of all, the modelling review itself drew in quite a wide range of experts, often with differing views on the approach. They looked carefully at what we did. They looked at a range of established approaches and of more cutting-edge approaches. As you can see in their recommendations, they have advised us to take a range of approaches, so that we have more and better crosschecks on our main results and can flesh out some of those sensitivities and uncertainties that we talked about before. That will make a difference.

The other thing that will make a difference is having really good evaluation and monitoring of the agreements. In two years’ time and two years after that, we will be publishing monitoring of the agreements, so that we can see whether we are on track to get towards the sorts of benefits that we have described. We will have a full evaluation in five years’ time, which will give us an even clearer sense of whether that—

Tony Lloyd: We will see you in five years.

Richard Price: That seems like a long time, but we will be monitoring these agreements in a shorter time than that, and we are constantly trying to see whether things are going as we expected. We will be reviewing this.

Q386       Mick Whitley: When we came out of Brexit, the Japanese agreement was the first one that we did. There was a side letter in that agreement, I am led to believe. If the EU was disenfranchised in any way, or if that agreement was better for the UK and Japan, that side letter would be invoked. What would the benefits be? At the moment, the Japan agreement was the first one. When we scrutinised it at this Committee, some of the answers we were getting from the experts, particularly on agricultural goods, were that the EU would be looking at and scrutinising that agreement and saying, “Overall, if that agreement is better than ours, we want to renegotiate”.

Richard Price: I am not briefed on the side letter, so I cannot answer that directly. What I can say is that the agreement that we did with Japan had a number of elements that were bespoke to maximise the advantages of the relationship between us and Japan, particularly going further than any other agreements in the areas of digital co-operation and regulation, so that was really important and it will give us bigger benefits than if we had not had that.

Secondly, we will be assessing and monitoring the agreement with Japan. DG TRADE will be doing the same for its agreement. Countries like us all around the world are constantly thinking, “What have we negotiated? What is so-and-so over here negotiating?” You are always trying, in trade negotiations, to develop precedents in a particular area that other countries can pick up and import into their own trade agreement, so that is really important. There is a lot of legal text precedent.

There is nothing unusual in that. We are always looking over each other’s shoulders, thinking, “What is going on over here?” and looking to see who is innovating and what difference it makes.

Q387       Chair: Richard, can I turn to the fact that the Australia impact assessment of the outcome to UK GDP led to a fourfold increase in the expectations of the benefit when it was reassessed? Coming straight to the point, David Henig, a former colleague of yours and mine, said, “It is hard to avoid the suspicion that Ministers had put pressure on the hitherto scrupulously impartial”—I agree with that—“in-house trade economists’ team to come up with better results”. Richard, discuss. Is David right?

Richard Price: I have the greatest respect for my dear former colleague and friend David Henig. He is quite wrong about this. I am responsible for the integrity of the analysis that we produce in the Department and I sign off the judgments that we make in the analysis, and we have been trying to set that all out for you today.

No, that is quite wrong. The things that made the difference to the scale of the benefits were as we have described them—data, assumptions about tariffs being updated, and those changes in the model that make it less controversial and more reliable.

Q388       Chair: I am going to come to Paul in a second, but just following up on the comments a bit earlier about the Venables review, perhaps I could ask you to take a message back to the Department, and particularly to the parliamentary liaison unit. We were absolutely promised that we would see not only the impact assessment publication but also the Government’s response by 31 January. Although the impact assessment was published on 31 January, we still have not seen the Government’s response. The message, which I am quite happy to say in public, is that the relationship between this Committee and the parliamentary unit is very strained at the moment. We really do need to crack on and there need to be some heads knocked together.

Richard Price: That is noted. I will take that back.

Q389       Paul Girvan: Why do the impact assessments not explicitly take into account the impact arising from the interaction of free trade agreements and the Northern Ireland protocol? I am not particularly worried about Ireland but about Northern Ireland and how it impacts directly. We cannot take advantage of some of those trade deals that are out there.

Tammy Holmes: The model, as I have described, includes many countries around the world, but it does not include the different nations of the UK, so there is no capacity within the model, as it currently stands, to distinguish between the impacts on Northern Ireland and those on any other nation. The modelling, as it is presented, assumes that all nations of the UK benefit from the agreement like one another.

Q390       Paul Girvan: Unfortunately, the reality is that we cannot. It has already been identified that the EU will challenge any benefits that are there. I have had a conversation with businesses that export machinery mainly to Australia in relation to mining; 60% of the world’s equipment used within the mining and quarry industry is manufactured in Northern Ireland. It is going to have a major impact if we cannot take advantage of a straight Australia trade deal that is of benefit to the United Kingdom and Northern Ireland. It is looking like they might well be challenged in relation to the outworkings of that.

I just want to know why we have not focused on a specific area, because the protocol is doing damage to our opportunity to do deals and take advantage of free trade deals that this nation is drawing up at a phenomenal rate. It is great that we are able to do it, but not all of the United Kingdom, unfortunately, is able to benefit. The modelling and the financial benefits to the United Kingdom should have taken into account the impact of the protocol. I am just wondering why it did not.

Tammy Holmes: It is something that we will look at and try to achieve. It is not something that we have been able to achieve so far.

Chair: Thank you very much indeed for coming in. Dr Cave, thank you very much. I think you are coming to us from America, which is incredibly gratefully received. Thank you all very much indeed for coming in.