International Trade Committee
Oral evidence: UK trade negotiations, HC 127
Wednesday 23 March 2022
Ordered by the House of Commons to be published on 23 March 2022.
Members present: Mark Garnier; Paul Girvan; Tony Lloyd; Anthony Mangnall; Lloyd Russell-Moyle; Martin Vickers; Mick Whitley; Mike Wood.
In the absence of the Chair, Mark Garnier took the Chair.
Questions 391 - 403
Witnesses
II: Professor Joe Francois, Professor of International Economics, University of Bern; Professor Tony Venables, Senior Research Fellow, University of Oxford.
Witnesses: Professor Joe Francois and Professor Tony Venables.
Q391 Chair: Welcome to the second session of this International Trade Committee meeting. We are going to be talking to some academics. Perhaps we could ask you to introduce yourselves.
Professor Venables: I am Tony Venables, professor of economics at the Productivity Institute at the University of Manchester, and at Monash University in Melbourne, where I currently am.
Professor Francois: I am managing director of the World Trade Institute at the University of Bern, and professor of international economics at the university as well. I have been doing this modelling for way too long.
Q392 Chair: Starting with Professor Venables, how much certainty is attached to the outputs produced by the CGE modelling? What are the key uncertainties? Perhaps you were listening to the earlier session, in which we tried to go through this, but it would be very interesting to hear from an academic point of view just how reliable these are. On an important point, are we, as laymen, interpreting them properly? Are we perhaps misunderstanding what can and cannot come out of these models?
Professor Venables: They are good central estimates, but there is considerable uncertainty around the estimates. I see it as coming from three distinct sources. One is the data, which is always full of errors, out of date and imperfect. However good the GTAP database is, it is imperfect. That feeds into the model in a couple of different ways. One is the key parameters of the model—the power of the relationships. The magnitude of the effect of this variable on that variable depends on the data, so there is uncertainty around those numbers. Another way that data uncertainty feeds in is quantifying experiments. If you remove a non-tariff barrier, you have to know how costly the non-tariff barrier was in the first place, or what the barriers were, which we do not. There are those data uncertainties, which are completely inevitable, and one response to them, as Tammy said, was building confidence intervals around the model and doing the Monte Carlos. That is one source of uncertainty.
Another one that is a bit more fundamental is in the model’s structure. Inevitably, the model is very stylised. A one-to-one map is of no use to anyone, so it abstracts from a lot of things. It does some things very well. In terms of other things, like relationships with other sectors, we know that it does less well. Foreign direct investment is an example. It is something that we have not talked about, but it is rather important as a possible response to a trade agreement.
Trade in services is more difficult to model than trade in manufacturing or agriculture, as is the extent to which innovation is triggered. These are things that are not all captured equally well in the stylised representation of the economy, so uncertainty arises there. A response to that is to zoom in on these difficult areas with the complementary modelling that you have been talking about already.
A third one is just events. Given the very long time that it takes for trade agreements to feed through, there is enormous uncertainty around other events in the world and the domestic economy. A response to that is to be willing to do scenarios and being quite explicit about, “If A happens, the effect will be B. If C happens, the effect might be D instead”.
Coming to the last aspect of your question, quite a good way to communicate things is to be very up front that, “If this, then that, but there is uncertainty; if something else, then something else”. The uncertainties are considerable, but the models are producing central best estimates.
Q393 Chair: The Bank of England started producing fan charts, which is quite a useful way of doing it, but it is interesting that there is an assumption by the general public that these are predictions rather than guides as to what the outcome is.
Professor Venables: Yes, they are estimates. I am a little sceptical about fan charts, because it is not just a matter of confidence intervals. It is really, “The world might change. If that, then something else”. Give me two scenarios and we can make informed judgments about which is more likely, which is quite different from putting up a fan chart.
Professor Francois: If I could follow up on Tony’s comments to the question, CGE modellers look at climate. One thing that they do on a regular basis is they get all the models they have together. There are different teams at different universities, and they will do stress tests. They will run a particular exercise—“What if this happens?”—and they can identify where differences are coming from if somebody’s model is an outlier, and that gives a bit of confidence in that you get a range of results from different approaches to how you model things.
That has been done with some of the trade models but not on a regular basis, and it is one of the things we have talked about trying to set up. This was done for the NAFTA model and was done in the Uruguay Round. In one case, the USITC organised it. In another one, it was the World Bank.
There was a nice recent piece that goes through and redoes all of the major TTIP studies to see where differences are coming from in terms of where the barriers’ estimates are from and what the differences are in the modelling structure.
It seems that most of the results are driven by the underlying data, and so that quality issue is really important. I know that DIT has joined the GTAP consortium, so they are in that, and that is a good thing. In addition, in terms of defining scenarios, you do not know what is going to play out. Indeed, the scoping studies are designed to figure out which things you can make concessions on and push and which ones you cannot, and so the final results are going to be different. Indeed, there is going to be uncertainty, depending on whether you make concessions on beef or not, or whether you think that other agreements are in the background or not. That gives a good bit of uncertainty, and doing this range of scenarios makes sense. That is the only way that we can get at it.
Q394 Chair: Given all the variables, as far as you are concerned, the CGE model is about as good as it is going to get for the purpose of assessing.
Professor Francois: Yes, exactly. One other value of the exercise, apart from the flanking work that was mentioned in the previous session, is that about 80% of the work in the CGE study is getting the data together in the first place, including the scenarios, in terms of what you are going to assume. That is a valuable exercise.
Even before you push the button, in a sense, and get the numbers out, there is a lot of information there that is part of the value of the process in terms of identifying what you do not know. We do not know how big these barriers are, so we should talk to industry, or some things are bigger than others. That is a useful part of the exercise and it is worthwhile spending time on what got you to be able to run the model, because the end result is you have these profiles on underlying tariffs. You have mapped those to the trade flows. You know where the industries are exposed or not.
One nice thing with these data that we have been doing is what is called multi-region input-output—MRIO, just to throw in another acronym. That means how much machinery from the UK is used in mining in Australia, for example. Those kinds of things are also working inside the data to get to the model before you even run it, and so there is a value in that whole process, because it flags institutionally where the uncertainties are that you have—what we can and cannot concede on, and who is knocking on our doors and asking for things—and this forces you to put that information together in an organised manner. That is of value, in that, before you run the model, at least you also know, “These are things that we are uncertain about. We do not know if we can get these concessions. We are not certain if these costs are high or low”. The process of getting there is valuable.
Q395 Mick Whitley: What potential is there to complement CGE modelling with the qualitative forms of evidence, such as information from engagement with stakeholders and sector experts, and in-depth case studies? What would be the advantage of doing so?
Professor Venables: That is enormously valuable. There is this big GTAP database, but there is lots more information out there in the world in the heads of stakeholders and businesspeople and in academic papers of past case studies. There is vastly more information out there, and it is particularly valuable in sectors that are difficult to model and to think about, such as foreign direct investment, innovation and services, which I talked about earlier.
It is about getting ways to extract this information and to work with stakeholders, who have a vested interest in saying one thing rather than another, and thinking of how to use it rather creatively, and qualitatively as well as quantitatively. If you want to build a model of a particular sector in detail, which will remove some of the very stylised assumptions, getting the information for that from all sorts of stakeholders and zooming into a small, bespoke model for a particular sector are very important further developing steps that the Department has already taken in that direction.
Q396 Tony Lloyd: Professor Venables, your report recommended that the presentation and communication of the results of economic modelling should be “as transparent and clear ... as is possible”. How should that be done? Can you also talk a little about what that means in terms of exposing the underlying assumptions and the underlying doubts about the data source that you referred to earlier on?
Professor Venables: The way in which these results often get reported in the press, as you know, is in one central number. That is not the way they come out of the Department, but it is the way they get reported. It is really important that, first, a narrative is built around this: “The direct effects of the policy change are potentially on exports here and imports there in this sector”. That is the first step. From that, production patterns change and an ultimate GDP gain, or change—it could be a possible loss—comes out at the end. Going through those steps is important. That is one part of it.
Another part, which is for people with some technical knowledge but is really important, is being able to do a back-of-the-envelope calculation to check that the orders of magnitude are sensible. The CGE is fundamentally quite simple. It may come as a surprise, but they are fundamentally quite simple ingredients. A reasonably trained economist should be able to do back-of-the-envelope calculations and say, “Yes, this number, that number—this adds up and makes sense”, and that is an important part of it as well.
The third part is being frank about the assumptions that are being made in the context of scenarios. If we make one set of assumptions, we are going to get one number. If we make another set of assumptions—they may be about the world changing or about whether we rely wholly on the CGE model or take stuff out of smaller accompanying models—we are going to get a different number. It is about presenting a range of results and explaining why they are different. One central number that makes it through to the media at least is just not a good way to do this.
Q397 Tony Lloyd: Professor Francois, are there good international examples that the UK should be looking at to improve our own modelling expertise?
Professor Francois: I would not say that it is expertise. You guys have outstanding people. In my experience of talking to DIT, the people there are quite good. Maybe in terms of the process, let me describe for a minute what goes on at the USITC, for example. There is a big CGE model team that runs things, but there is a whole flanking effort. You have what I will call qualitative modelling, with a set of questions that you ask industry and agriculture experts on supply and demand conditions and the similarity of products. They know it in terms of the industries that they look at. We think of them as elasticities. It is a supply elasticity. They think of it as excess capacity.
With those kinds of things, you can then almost go through a mental checklist and say, “This suggests that we will get a big effect or a small effect”. Those kinds of things where you can go to a very detailed level— looking at Chinese votive candles or beef or whatever it is, with different HS codes—give you a way to bring in the industry types who have expertise that is more qualitative. They should then also be involved in looking at the results that you get, to say, “That is crazy, because there is no capacity”, or, “Yes, that looks plausible”.
You want to involve the modelling in a broader process, which Tony hinted at too, where you are talking to the experts, who may be other Ministries. You want to have them in an inter-agency process, where you try to identify where sensitive sectors may be from the beginning. A previous question referred to talking to industry or having consultations. This gives you a basis for some of those questions too: “We think we are going to see large effects here. We have identified these as issues. Does this make sense to you?” This should then feed back into defining what your scenarios are. If I am running a model, the scenarios are, in the end, policy-determined and, in a sense, reflect what I will call institutional calibration in terms of the expectations of what might change what you can get out of the agreement.
Just like I said, the model is a framework for organising data. It is also a framework for discussion. We need to talk to the ag people and to the guys who follow industry. They are not modellers, but they know things qualitatively that we do not. In that sense, they can provide a gut check in terms of whether the numbers make sense. They can also provide a gut check in terms of, “That may be too sensitive to put in the list of things to ask for”. That is an example that I would look at.
To some extent, the EC—the Commission—has got better at this too, in terms of the public consultations and the scoping studies at the beginning, so that they bring people in to talk about what comes out of those. My sense is that the US probably does that a little better, but I would look at both. In both cases, it means that you have a broad set of interests and you need to have a framework to bring these people in from the beginning. The modelling can then integrate that stuff. In a sense, the role of these studies—and I heard your questions about whether it was fit for purpose—is that, when the study is done, you toss it on the coffee table and everybody disagrees with it, but they disagree for different reasons. It provides a focus or a basis to discuss why you disagree. That is part of the formulation strategy and so on.
I would look at those. The message from those processes—and you are finding this already with your initial reports out and getting feedback from, say, agri business worrying about what is going to happen to the UK industry—is that you need to talk to these people up front. This process and the scoping assessments should include that qualitative stuff.
Q398 Chair: Just following up on that point, a lot of economics is about intuition, but what you are doing with modelling, ultimately, is coming down with a mathematical formula, albeit a very complicated one, to come up with a result. If you have, for example, the National Farmers’ Union coming up with a finger-in-the-air type of approach to this, how do you convert that from being intuitive by the NFU into something that you can then pump into a model that then comes out with a mathematical outcome?
Professor Francois: Getting back to the ITC for a minute, one thing that Bob Feinberg, an economist at American University, cooked up was literally a qualitative modelling check sheet, with those questions that I mentioned. You go through and combine information on supply and demand conditions and the size of industry and so on. It is not even a quantitative model. You get adjectives for results but it gives you a sense of where things are.
That kind of information gives you a way to cross-check or gives you a gut check. You have an estimate for the econometrics and it says that there is, effectively, a 20% trade cost reduction from PTAs in dairy, and the dairy people say, “No, it is not so much” or “It is covered by TRQs”. They bring institutional knowledge that may then inform how you organise the model and which sectors you want to break out.
Some of the information is going to remain qualitative. Being honest, we do not have the data, first off, or we are not sure how the data fits into what we are doing, but there should be room for those qualitative insights as well. Where we can, we try to bring it in. In the scoping work on TTIP, which is a while ago now, there were industry surveys, industry expert comments and econometrics, all of which were used to cross-check each other. Indeed, the industry survey results were fed into the econometric results, and they were consistent, which is kind of odd. Triangulating and finding different ways to get the same information is also a way to pull this stuff together.
Professor Venables: It is really important to interrogate the member of the farmers’ union or whoever it was in terms of what their thought process was, to go into that and to use it to understand the industry. The industry probably does not fit the stylised representation in the model. Maybe it takes some quite creative economic thinking to say, “The industry is different in this way and that way. Let us take that and see if we can formalise it as a sector-specific module for the model”. That is really important input. Interrogating, understanding and modelling it is important.
Q399 Mike Wood: Professor Venables, your report also recommends developing extensions to a core model, “to capture regional impacts of trade, the dynamics of adjustment, and environmental impacts”. Can you talk us through some of the practical benefits of taking that approach?
Professor Venables: First, these are all really important questions that politicians, and certainly the public, want to know about. On one level, that is the motivation, but let me talk through that in a bit more detail. Earlier this morning, Tammy Holmes used the phrase “comparative statics” for the model, so it computes where we are now and where we will be when everything comes to rest after all these changes have worked through.
The dynamic extension that the review discussed and recommended is to capture the adjustment process from where we are now to where we think we might end up. That is important because that will presumably involve reallocation of labour from one sort of activity to another. As we know, that is probably not going to be instantaneous and painless. It can be extremely painful and take a long time, so it is really important for understanding the full implications of these changes and for designing policy to make sure that that path is as smooth as possible.
By “dynamics”, we could have meant one of two things. One is the short-to-medium-run adjustments that I talked about; the other could have been very long-run and what is it going to do to growth and innovation. The review did not come down in favour of that. Economists are doing it, but it is too speculative. It is about a better understanding of that adjustment path. People want to know and policy makers need to be aware of it.
The regional is really linked to that. We know that trade policy, as it affects sectors, will have different impacts in different sectors and, therefore, in different places. That is where we see the labour market impacts sometimes looking pretty acute and pretty severe, as we all know. Thinking through those regional impacts is important for the levelling up agenda. We know that trade changes in the 1970s and 1980s were an important part of the regional problem, as we now experience it.
The Department has already done some of that. It does it in a simple, top-down way, just saying, “The UK model is giving us the changes to these industries, and we know where the industries are, so we can map it into the regions”. They need to take it a little bit further than that in terms of looking at not just the industry itself but its suppliers, and mapping a bit further down the supply chain, and then to link it to the labour market in terms of the short-run adjustment dynamics that I was talking about a moment ago.
Those things are important and doable without a huge modelling effort. There is a more ambitious version of doing the regional modelling, which I do not think would be proportionate, but an extension of what the Department does now would pay off and be valuable.
The third extension that you mentioned from the report is environmental modelling. We need to know the carbon impact of changes in trade and the location of industry. We need to know how that interacts with carbon markets. Carbon border adjustments are likely to be trade policy instruments in the not too far distant future, so we need to be ready and in a position where we can think through and quantify CO2 emission implications of changes in trade policy, and possibly implement carbon border adjustments in terms of an environment-based trade policy. That is necessary.
Incidentally, CGE modelling is rather well suited to addressing those issues. You have outputs from lots of industries that are changing, and lots of economic activities, and attaching emissions to those outputs is relatively straightforward.
Q400 Anthony Mangnall: I am conscious that time is marching on and we have the spring statement here, so I am going to be quite brief and just push you for quite brief answers as well. First, Professor Francois, do you think the monitoring system that they were talking about in the previous session is effective and is keeping up with your expectation? Also, what was your input to DIT’s modelling for the impact assessment on the UK-Japan trade agreement in 2020?
Professor Francois: First off, the quality tracking makes sense. I do not know how else you would do this, to be honest. You could invite stress testing. If you have different teams that are contributing support—LSE or wherever—once in a while having them do an exercise where you can identify whether you are getting comparability out of the different approaches would make sense. That is on top of what is being done now. It is clearly a developing process as you are getting reports back and see if they work or not, but it is the right direction to go on, and something like that is necessary.
On your second question, my input was running a standard GTAP-type model with inputs from DIT in terms of what they thought the scenarios would be, and going back and forth a bit on that in terms of framing how you come up with these ranges of scenarios and how you treat the NTBs. I have done this stuff for a long time. Part of it was working through how you define your experiments and, going back to what Tony was saying, getting some sense of uncertainty by thinking about the different parts of the agreement, and about whether you are going to be ambitious and what “ambitious” means. It was partly both. Part of it is running it but also then walking through how you are setting up the scenarios.
Q401 Anthony Mangnall: We are hearing from the Department for International Trade that the Australia and New Zealand impact assessments used the same modelling approach as the Japan impact assessment, but that direct comparisons with the Japan agreement are not possible as some of the inputs used were not the same. Is this a valid argument or are we being given the run-around?
Professor Francois: It is and it is not. The underlying assumptions changed a bit between the assessments, but that is because, for example, we have had Covid since then, which affected macro baseline projections taken externally. The basic messages are still similar. In the case of Australia and New Zealand, a lot of the potential trade is in sectors where there are relatively high NTMs. With Japan, that is not the case. Notwithstanding how big the economies are, the trade flows in one are subject to more barriers than in the other. That kind of stuff, which is not model-specific, is the part that gets you to running the model and you can certainly talk about it.
It is true that when you then run the model there are differences. As I heard earlier this morning, you try to incorporate the most recent information you have in terms of how things are now. We are not making a forecast but we want to reflect how things are. It has been an interesting few years, unfortunately. Even right now, we have another big shock in the food markets, because things are not moving through the Bosporus in terms of grain exports from Ukraine. That is not in any of these assessments either.
We have had some substantial changes in the underlying assumptions that cannot be helped, but we can make comparisons on the part that goes before that, which is looking at the moving pieces in terms of the trade flows and the assessments of the underlying barriers.
Q402 Tony Lloyd: We know that the global economy will have shocks in the next 15-year period. You make the point about food prices and energy. It is a very wise person who can predict what the energy market will be doing and who the winners and losers will be even over that 15-year period, never mind longer. How robust can we make our economic models or are we using them in the wrong way? Ought they to be indicators of direction of travel rather than about trying to give a fixed point of arrival?
Professor Francois: The answers we get are both. It is a fixed point but we get there in some direction, and we are trying to say that adjustment cost involves how we get there. The trade community is freeriding a bit on the climate work. The OECD and IIASA do a lot of work together on medium-term macro projections that feed into the climate assessments, and we use those. That is where these macro-based lines are coming from. That exercise involves the economics directorate at the OECD and the macro guys at IIASA. It involves the people who do the IMF projections.
We are freeriding on that, but there is an effort by people who are better qualified than me to think about how those medium-term baselines are going to be affected by these things. It is not a forecast. It is going to be wrong. We do not know why it is going to be wrong; we just know that it is. It is as robust as we can be with these timeframes. It turns out that, whether you do the projections or the purely static assessments with endogenous capital, the steady states are comparative. You get the same results that are driven by the shares in the trade flows and by the structure of the input-output tables. They are relatively robust to these kinds of temporary shocks, assuming that they are temporary. If they are permanent, it is a different thing.
Professor Venables: Are big shocks particularly going to interact with trade between the UK and New Zealand? Most of them are probably not. Going back to what Richard said at the beginning, we are looking at an increment. Maybe that increment is the same or similar across quite a wide range of shocks, so we should not assume that every shock to the world economy validates what has been done.
Q403 Chair: We are coming to more or less the end of our session, but if we could finish with one question, which is more or less where I started in the previous session, that is how we can facilitate comparisons on a like-for-like basis with these deals. It comes down to how we can be better at scrutinising these trade deals. It is really the basis of trying to work out whether the negotiators have done a good job compared with what their initial negotiating objectives were.
Also, slightly more complicatedly, one of the problems that we have is that the Department for International Trade has not published a simple set of negotiating objectives in terms of what it is trying to get out of international trade on a global basis. Starting with the big picture, what we do not know is how the negotiating objectives work within a specific country and how the negotiators perform compared with what the negotiating objectives were in the first place.
Is there a way that the Department for International Trade can set itself up and give us more information that would make our job a lot easier in terms of trying to work out if we are getting value for money for the taxpayer out of these guys in Whitehall?
Professor Venables: There are some real choices. In a negotiation, you prioritise this or you prioritise that. The model that you are using within a negotiation with one particular country is presumably the same, so the modelling approach is a useful tool for comparing this and that. I am not quite sure what the Department does here, but in terms of how much is published or made available about the alternative negotiating positions that were tested in the model, I know that alternative negotiating positions are tested, and maybe that should be made more explicit.
Across FTAs, the model will and should evolve, and the data will change. Frankly, I do not think that there is a real choice between either doing this FTA with this country or that FTA with that country, and it is going to be decided on the basis of what comes out of the model. I do not think that that is the way that those choices are made. Asking for uniformity across choices like that does not seem to me a priority. In terms of different negotiating strategies within an FTA, the approach is useful, is there and is being used.
Professor Francois: You could ask, when the agreement is reached as part of the basis for discussion and you then debate it, whether it has been successful and whether it fits as closely as possible to what the final agreement is. The scoping stuff, as you heard earlier, was guessing what you think it is going to be. What are the scheduled tariff concessions and such at the end? Whether that is the best agreement is a political process. You may identify things that are sensitive. Some industries have vested interests, and others you have to accommodate, and so in terms of defining what the best agreement is, those things are not in the model.
There are political weights that are assigned to different groups. You may be worried about some regions over others. You can identify those things by writing a final assessment, but whether it is good or bad is part of the process by which a decision was made on what things to press or to concede on. You can produce information for that discussion, but it is not going to tell you whether something was good or bad, because the process was driven by factors that are not in the model, in terms of the decisions that were made to get to what the final agreement actually looks like.
Chair: Very helpful, although I do not know if it necessarily gives me the answer that I was looking for. That draws us to a close, so thank you both very much indeed for joining us. Your insight is incredibly invaluable and we are very grateful to you.