The Philosophy and Culture of Aid

In the Justice Sector

Evidence paper for the UK parliament’s International Development Committee

Abstract

When you go to a doctor because you are not feeling well, s/he will ask some questions and run a few checks before prescribing a treatment. The diagnosis is sometimes surprising: the original complaint turns out to be a symptom of something deeper and wider that needs a concerted course of treatment over time.

So too with criminal justice systems: the symptoms of dysfunction (the length of time it takes to process cases, low conviction rates, prison overcrowding etc) that shout for remedial action are seldom solved by attention to the particular institution alone because all need to function if the system as a whole is to work.

A determination of the appropriate course of treatment, requires first an assembly of data from across the system and benefits from wide consultation with practitioners and users. International aid programmes in the justice sector have at best been inconsistent in observing this basic step.

Advances in software development are on the way to replacing the narrative report with its dizzy-making pie-charts, bar columns and spiderweb graphs (outdated the moment it is published) by an inter-active, visualisation of the justice system that enables planner, practitioner or the public - at a click of a mouse - to see whether justice is being done.

The Governance and Justice Group (UK) with Justice Mapping (USA) sought to answer the call for a ‘data revolution’ issued by the report of the High Level Panel of Eminent Persons on the Post-2015 development agenda. They developed the ‘Justice Audit’ which offers an independent and systematic review of institutional data checked against original records to produce a national evidence base in an updateable data bank that permits monitoring change and progress over time.

The Importance of Data and information technology in the Justice Sector

“There is an inseparable link between the protection of individual and collective human rights and democracy. The field of battle in which democracy and human rights are tested is the administration of criminal justice, which encompasses all processes and practices by which a state affects, curtails, or removes basic rights.”[1]

Introduction

For years it seems, we have been tinkering nervously at the edges of an area that goes to the essence of human need. Why have we not grasped the criminal justice sector as we have, say, the supply of food, education or health services? Why has ‘justice’ been left in the hands of judges and lawyers as if only the initiated with their ‘magical knowledge’[2] are able to determine ways of resolving right and wrong in a way that society recognises as just and fair?

The establishment would respond: because it is more complicated than that. Clearly it is complicated, but are the intricacies of the human body any less complex? And the health ‘sector’ seems to have done a pretty good job at getting its services out to the most remote village, so why is the justice sector so far behind? One academic with long experience of justice development in three continents put it this way:

“Today’s heavy emphasis on judges, lawyers and courts is analogous to what the public health field would look like if it mainly focused on urban hospitals and the doctors staffing them, and largely ignored nurses, other health workers, maternal and public education, other preventive approaches, rural and community health issues, building community capacities, and nonmedical strategies (such as improving sanitation and water supply).”[3]

While efforts have been made in the last 20 years to bring justice closer to the people, the emphasis on themes or target constituencies that match the current priorities in donor capitals, or perceived needs, has in general produced a fragmented set of outcomes. The notion that the ‘rule of law’ is stronger today or that it is central to economic development are moot points when one glances around the world.

Understanding people’s ‘unmet needs’ and the justice gap

We know from multiple research papers that people consider safety and security fundamental to their wellbeing and that they form essential components along with access to food, water, education and health services.[4] We also know that the needs are simple: people want lawful remedies to their disputes that they can access nearby, at low cost and quickly. Attitudes towards most offending are not punitive: they want remedies that restore the harm done and enable them to get on with their lives and live in harmony with their neighbours.

Yet the international community continues to offer high cost, largely Big Picture solutions, implemented through a growing industry of corporations that tend to fall far short of expectations in large measure because the demands of the institutions (for more buildings, equipment, training as ‘capacity building’) do little to ease access to the services the voices of the poor call for. [5] They are more easily measurable and consume more ‘spend’ than smaller approaches which are responsive to the demands of people and may not require large sums of money, but rather time and flexibility. Small wonder a judge of the Supreme Court of Mozambique lamented: ‘we appear to be catering for 10% of the justice needs of the population’.[6]

Three problems with the way international assistance programmes in the criminal justice sector are fashioned.

Design: missing the context

“Law reform projects seem to be based on the unspoken Weberian assumption that because a particular legal system is found in countries which are developed, that legal system will help countries to be developed. There is no proof of this.”[7]

International donors love sending in teams of consultants to design interventions.[8] The consultants are ‘experts’ in their own countries and, as experts, will therefore know the answer to the justice problems confronting a country which may share their legal system and language but otherwise will have nothing else in common. A British police officer and specialist in finger prints sat forlorn in an office in Lilongwe, Malawi. He explained that he had come to assist the Malawi Police Service develop their capacity in forensic evidence to solve crime, but had discovered that the most common building materials used by people were wood and straw. He was cutting short his visit.

A nice and discreet project is to review and reform the key laws. National draftspeople will be in short supply, so non-national consultants will be drafted in. They will produce a piece of legislation that reads well, meets international standards and incorporates emerging (progressive) trends: it will be a ‘model’. And it will not work, because the ‘stuff’ needed to enable it to work is not in place and won’t be for years to come. In any case, in most countries, it is more often the implementation of laws that is the problem, rather than the laws themselves.

Those commissioning a ‘design’ like symmetry and coherence. They like to give a bit to everyone (addressing the ‘whole’ sector) and favour the odd set piece they can wave a flag over. Donors have their pet projects, agendas and biases. But for all the talk of ‘harmonisation’ and ‘co-ordination’ the aid provided rarely amounts to either, in part because the sector is so fragmented (and operates in siloes rather than as a coherent unit) and in part because no one wants to be co-ordinated.

Implementation: quick wins or quick profits?

Maxims and aphorisms abound on the importance of maximising participation and even culinary metaphors are introduced citing the wonders of ownership in ‘cooking’ change. As with most truisms, they are true but they ring hollow to local ears. Time is not taken to consult with a broad range of ‘stakeholders’ and there is deep confusion in whom to invest ‘ownership’ in any justice reforms. On the one hand there is a risk that by investing in the legal establishment, you risk elite capture (and a return to the status quo ante) and by investing in society, you alienate the establishment who push back against any of the reforms tabled. For large corporations administering multi-million dollar contracts, the time and expense taken in consulting with stakeholders may not sit well with their shareholders.

Then there are the practical considerations. Nowadays the average governance / justice adviser will take up residence in a peaceable country for approximately three years (shorter in a post-conflict environment) – and how long a Chief of Police, head of prisons or Minister? The interventions the adviser designs need to be SMART,[9] with a nod of recognition that justice reform is a process over many years. These conditions have frustrated the best of intentions on more than one occasion.

So, a building here and bit of training there can appear a smart choice, even if the building is in the wrong place or the training has little chance of impacting on the way business is done in the real world outside the classroom. There is plenty of ‘low hanging fruit’ to pick from and a ‘quick win’ is anything you want it to be. This is the beauty of jargon: an outright failure becomes a ‘lesson learned’.

Experience strongly suggests that the process of reforming justice and security systems and processes benefits from the inclusion of all those with a stake in the process. While it may be expedient for a small group of people to direct a top-down approach, there may also be a push back from those on whom the policy impacts or those charged with implementing the policy (ie those on the ground). Solutions that are imposed and centrally-planned have proved to be costly and questionable in terms of impact. On the other hand, participatory processes where input is actively sought from practitioners and the population about strategies or approaches to building the rule of law locally and nationally have proved less costly and highly effective. They also allow for piloting a range of small-scale initiatives, which may be scaled up where they demonstrate their worth.[10]

In the short term, this helps in several ways: the act of consultation is itself a confidence building measure and sets the standard for how the government intends to conduct its business. At the same time, it assists government allocate scarce resources where they are available to ensure core services are delivered and draw on the good will and experience of civil society groups (and other non-state service providers) where they are not.

Sustainability: a donor mantra

The international community has directed millions of dollars at justice systems (erecting buildings, providing fleets of cars, boxes of computers, etc) and loaning even more (through the World Bank) to install hi-tech case management systems and establish new ‘speedy trial’ procedures and special courts – and then throwing up its collective hands when the host country proves unable to maintain it all or service the loan.

Similarly, effective but low-cost approaches have been discontinued because funding through non-governmental organisations is deemed ‘unsustainable’ since these organisations rely on development funding for their existence (as indeed does the aid ‘industry’ managing the funds). This narrow interpretation of sustainability (money) ignores the high value placed on this work by beneficiaries (impact). If the work is effective and economical, surely the argument for sustaining it is made?

The health sector needs drugs and kit at all levels, minor and more serious, so it needs money. In the justice system, minor disputes and offences can be solved without great expense and there is increasing realisation around the ‘developed’ world that they must be (witness the rapid growth in restorative justice approaches to minor criminal offences and use of community courts to deal with narcotics). A malfunctioning justice system owes as much to the ‘the continuing failure of the players within the system to communicate and co-operate effectively’[11] as to a lack of funding. But still the tendency is to throw money at the problem, or at least the part of the problem with which donors are comfortable with engaging. In general, it’s a case of plus ça change plus c’est la même chose.

Time for a data revolution

‘The proper role of government is to provide the framework, while those who have first hand experience of how the system works in practice provide the detail. Unless those who have the experience and the day-to-day responsibility on the ground are intimately involved in the process of change, they will not feel the sense of ownership which is required to make the change a success.’[12]

The criminal justice system comprises a range of actors (police, prosecution, legal aid services, courts, probation, social services, prisons) and delivery mechanisms: those provided by the state (dealing with public order, personal matters, ‘morals’, property etc) and those provided by non-state mechanisms (dealing with customary laws, ‘traditional’ practices and restorative justice, offering mediation, diversion and other alternative methods of resolving disputes).

A lot of energy in recent times has gone into the development of justice ‘indicators’[13] and criminal justice toolkits[14] that seek to provide templates or ‘golden threads’ for countries with broken or dysfunctional justice systems to follow. There are all sorts of problems with this approach. The first is that it assumes there is a ‘model’ justice system anywhere that posits a standard, or standards, against which to measure – and there is not: criminal justice systems all over the world are messy. Secondly, it is putting the cart before the horse, prejudging without taking account of the evidence. You cannot monitor and evaluate what you do not know – you need the data first.

At the institutional level, there will be data covering infrastructure (ie the buildings and state they are in), resources available (human as well as material) and pay and grade scales. There will also be data on case management (sometimes disaggregated, often not) that will show the number of cases entering the system, the number disposed (though type of disposition is not always clear) and the number pending at the end of a period. Prisons will provide figures on who is inside them, how long those on remand have been there and the length of sentence of those convicted. Finally there will be data on such governance issues as inspection, reporting and accountability.

The first ‘But’: these data will be ‘siloed’ and sit inside each institution. They will not be joined-up to provide a system-wide perspective and it is unlikely they will be disaggregated to show how the system relates to women or young people, or how the system impacts on the disabled, at national, regional and local levels. In short, it will be limited to the state’s perspective of the justice system, and within this, the perspectives of separate ministries, as well as the suppliers of those services.

In many African and Asian countries, the formal (state) justice system is invisible. Poor people in their villages will have recourse to local leaders, Chiefs and village headmen, or locally elected or respected officials to deal with their problems, even when they amount to serious criminal offences. Figures are anecdotal, but few would argue against the proposition that the ‘vast majority of disputes’ are dealt with in this way.[15] Even if these systems are ‘traditional’ (and subject to ‘serious shortcomings’[16]), they nonetheless form part of the justice system and need to be taken into account.

The second ‘But’: the ‘hard’ data will lack context and nuance if not triangulated with data from the practitioners in – and users of – the system. Both categories have first hand experience of the system and can provide insight into reform processes that mere contemplation of the hard data cannot. They can talk to the reality on the ground as opposed to the theory or official story. For instance, Bangladesh currently has over 3 million cases pending disposal. They sit in a backlog which shows every sign of increasing unless remedial action is taken. On the one hand, the data show very few judicial personnel available to deal with the caseload (less than half a judge/magistrate for 100,000 people[17]) and few courts. However it is unlikely that the appointment of more judges/magistrates and construction of more courts will solve the problem because a significant contributing factor in inflating the caseload is the tendency of citizens to use the criminal justice system to ‘get at’ business rivals, neighbours and others against whom grudges are held by preferring false, vexatious or trivial charges which take them out of commission for a time.[18]

The aim of the data collection is to get as accurate and complete a picture as possible as to how justice and security work in the country. The data hold a mirror up to the criminal justice system and show how the system should work in law and contrasts how it actually functions in practice. Its compelling feature when triangulated in the manner described is that the data are neutral.

The Justice Audit: short background

In 2010, in recognition of the problems mentioned, the GJG conducted a baseline study of the formal justice system in Helmand province of Afghanistan for DFID. Rule of law advisers in the Helmand Provincial Reconstruction Team (PRT) cast doubt over the existence of any data worth the noun. 

This first ‘Justice Audit’ gathered reams of data by visiting the courts, police, prosecutors, prison, lawyers and other justice actors and listing the data under institutional headings. It established that even in a ‘broken’ justice system, data exist and, furthermore, the data are instructive.

It pointed to ways in which, for instance, the popular view of the formal justice system (in summary: corrupt and ineffectual) might be addressed by: exposing the poor terms and conditions of state justice providers that left them susceptible to bad or corrupting practices; and revealing the gaps in service delivery in terms both of geography and articulated need.

Had these baseline data been available at the time of designing interventions, the international community might have avoided the high expense of constructing justice ‘forts’ at strategic sites (recording barely 2-3 cases per month) and questioned the assumption that while the judges and prosecutors servicing them were safe once inside the walls, they were at risk on the roads to these places of justice. The data might also have indicated that as a counter-measure to the Taliban’s highly mobile, instant (and often brutal) ‘justice services’ the approach was unlikely to succeed.

In the event, the data gathered had little impact. In major part because the prevailing political situation was too big (focused as it was on containing the insurgency), but in very small part because the data as listed were hard to read and digest. The GJG approached Justice Mapping in New York[19] which had made inroads into policy thinking on ‘reinvesting’ in penal reform by developing software to map where offenders lived in state capitals around the USA and show that the overwhelming majority lived (surprise) in the same few blocks where unemployment was high and social services low.

The GJG worked with Justice Mapping following a commission from the Malaysian government for a Justice Audit of its criminal justice system. The result was a visualised presentation of the data around four dimensions: infrastructure, resources, case management and governance.[20] At the presentation to senior government officers, the Inspector General of Police expressed his regret that he had withheld his consent to participating in the Justice Audit (as he had feared ‘another’ attack on the police) and saw it as a useful planning tool. The Minister of Justice expressed concern at the high number of drug users in the prisons, especially as his Ministry’s policy was to treat drug use rather than punish simple drug use.[21] He also brought forward legal aid reforms on noting that the high rates of conviction in the lower courts were due to the total absence of defence services in these courts.

This led in May 2012 to a review of the methodology applied at a meeting of practitioners from around the world at Wiston House in Sussex (aka Wilton Park) with the support of the US institute of Peace (USIP).[22] At this meeting, the methodology was further refined and emphasis placed upon a partnership with government and the citizens of the country. In other words, the key word was (national) ‘transparency’ rather than (government) ‘ownership’.

In 2013, the Bangladesh Ministry of Law, through its programme of co-operation with GIZ, invited the GJG to conduct a Justice Audit of five districts. The GJG invited assistance from Northwestern University (USA) both to tap into the university’s research capacity and to open up to clinical law students this field of work. The Justice Audit was presented to the Ministers of Law and Home Affairs and other heads of institutions. This led in 2016 to a commission from the Minister of Law to conduct a national Justice Audit.[23] The data contained in the Justice Audit formed the evidence base for the justice sector in the next five year plan.[24]

In 2018, DFID working through UNOPS approached the GJG to present the Justice Audit in Nairobi to form the basis for a significant 8 year programme of support by DFID as well as other donors with an interest in the sector. Considerable importance was attached to getting national justice stakeholders as well as UN agencies on board.

The justice needs in countries recovering from conflict (or natural disaster) differ significantly from peaceable countries. The GJG adapted the Justice Audit methodology to provide a quicker and nimbler ‘Justice Snapshot’ of the political economy of justice, the geography of existing justice services and movement of people and focusing less on case management (since most cases will be dealt with at customary law) zoom in on existing infrastructure (supported by photographs) and resources (material and human) available as well as governance mechanisms at work. And in this way provide the national authorities and development partners and agencies with an evidence base around which to plan.

The Justice Snapshot South Central Somalia was delivered in 2020[25] and a further Justice Snapshot Somaliland is to be delivered in April 2021. Both took place during the COVID pandemic. The intention of DFID is that both will be ‘front and centre’ to donor engagement in the justice sector and planning with the national authorities. Furthermore, that both will provide a monitoring mechanism against agreed benchmarks and provide investment opportunities for based on gaps and needs identified by the data (rather than the whims of ministers).

Providing government, donors and citizens with an evidence-base from which to plan

The Justice Audit is a living tool: it is updateable. It creates a central, web-based, resource for governments, justice service providers, civil society and external partners. There is no thumping great report to wade through – the practitioner can pick up the salient points in minutes. Its uniqueness lies in the focus on a data-driven approach shorn of comment and direct engagement with government and practitioners in the collection and analysis of their own data. Its added value is that it leaves the authority deputed with co-ordinating the data going forward with the software infrastructure to conduct periodic audits of future years’ data going forward.

The organization of a spectrum of data on the criminal justice system into format using graphical user interface (GUI), including geographical information system (GIS) mapping, provides the viewer with a ‘snapshot’ of the criminal justice system illustrating how the justice process works, the points of pressure and challenges that emerge and options for improvements based on good practices – at the click of a mouse.

Where the viewer doubts the way in which the data has been presented or cut, s/he can click on the baseline data and check the source of that data. Where data is incomplete (and in many countries it will be) or inaccurate (and so giving a false reading), the incentive is provided to invest in the data collection to inform sound planning.

There are several differences with this approach from others. A Justice Audit is not an event, or one-off assessment, conducted solely by an external team of evaluators; nor is it a finger-pointing exercise; nor an index of performance against which the country may be measured against other countries.

Emphasis is placed throughout on a collaborative venture between state actors, practitioners and civil society. At the beginning of each Justice Audit, a small group of national stakeholders is formed to act as a ‘sounding board’ and play an advisory role for the team at each stage of the Audit to build trust, ensure accuracy and promote acceptance and local ownership of the process.

This contributes to good governance in the justice sector in several ways. Firstly, it illustrates the value of accurate data collection and so catalyses the start of a data collection system and culture throughout the justice and security sector using a process approach; secondly, it provides a strong evidence-base to inform planning and budget allocation – for the national authorities and international donors; thirdly, it enables practitioners to see the interlocking needs and inter-dependent nature of the parts and stages of the system and to see themselves as part of a greater whole; fourthly, it enables the public to understand the justice ‘story’ and, by improving the quality of statistics and information available to citizens empowers them with information on the progress towards the targets and the contribution they can make; and finally it provides a key tool for policy-makers in leading the development of a long-term plan for the (re)establishment of a fair and efficient justice system. A short PowerPoint by way of illustration is attached at Annex 1.

Conclusion

In a sector where it appears there is everything to do, time taken to study the situation as a whole, on the ground, is a good place to start. The report of the High Level Panel of Experts on the Post-2015 Development Agenda called for a ‘new international initiative to improve the quality of statistics and information available to citizens…to take advantage of new technology…to empower people with information on the progress towards the targets.’ [26] The Justice Audit aims to do just that. It is ‘the justice equivalent to a full medical check-up’ providing ‘a survey of all factors that contribute to the functioning of the body politic’. Its value is that it is ‘complete’ and presented in an accessible way ‘without being prescriptive.’[27] This service is one the international community can facilitate. It can then get around the government and support it take forward a reform process at its own pace, based upon an agreed evidence base. In short, it provides, in our submission, smarter and Better Aid.

Adam Stapleton

Director,

The Governance and Justice Group

London

www.governancejustice.org

Annex 1: PowerPoint on the Justice Audit.