Coalition government and the evidence base
1 Sep 2010
Nigel Meager, Director
Details of the new UK government’s policy priorities in areas of interest to IES (including welfare reform, skill development, further and higher education, workforce health and well-being, older workers and pensions, and many others) are only just emerging. However, its approach to the generation and analysis of the evidence base to underpin these priorities remains wholly unclear.
A pervasive Leitmotiv of the post-1997 Labour administration was its devotion to (some might say obsession with) so-called ‘evidence-based policy-making’ (EBPM). Despite the warm words, however, the objective of EBPM was rarely fully achieved during this period, partly because the research evidence was rarely considered at the policy design stage, and also because the political imperative for quick results barely allowed time to collect robust evidence.
Nevertheless, the Labour years saw a major expansion in government commissioning of applied social and economic research, and the growth of a ‘market’ of research suppliers. Not all the research was of high quality, or made best use of public resources: many costly and overlapping surveys were commissioned; there was a tendency to treat every policy question as if it was a new one, often ignoring existing research evidence and ‘reinventing the wheel’; and learning from overseas evidence (apart from the US) was distinctly unfashionable.
Attitude to evidence-based policy making
Today’s focus on value for money and effective use of public resources could, therefore, deserve a cautious welcome in the area of EBPM – or whatever the coalition-approved term for this activity might be .... However, as I write, in the early months of the new administration, little or no research or evaluation is being commissioned by government or its agencies (and many of the latter are disappearing in the ‘bonfire of the quangos’). It remains unclear how far this reflects a short-term, post-election refocusing of activity combined with financial stringency, and how far it reflects a permanent halt to evidence-gathering and analysis for policy development purposes.
Some eminent voices are already arguing that the latter is the case, that the new government will pay no more than lip service to the role of evidence in policy-making, and that political expediency and/or ideology are again in the ascendancy. If true, we may be about to see the end of an era of involvement of the social and economic sciences in public policy-making.
This could be too pessimistic a view, however: first, a recognition of the importance of evidence to policy-making has become deeply embedded within the policy infrastructure, and would be difficult to reverse. Further, it is clear that the new administration is not intrinsically hostile to social science and research: indeed several government ministers are highly social science-literate – David Willetts, Michael Gove and Steve Webb spring to mind. For those who remember the ‘evidence-light’ policy-making of the Thatcher years (not to mention the threat to the then Social Science Research Council; the vendetta against the Industrial Relations Research Unit at Warwick; and the refusal to fund major social surveys with public health objectives), this administration feels very different.
Risk of loss of capacity
Nevertheless, it is important that the new government realises that, while there may be a case for rationalisation, if the current lack of attention to evidence-gathering persists for an extended period, there is a real risk to the continued existence of an independent applied research and evaluation capacity outside government. Given the degree of external scrutiny to which policy-making is now subject (through Freedom of Information, the press and the Internet, and the attention of select committees, the National Audit Office, and international bodies such as the OECD and the European Commission), it is difficult to imagine any administration choosing to perform without the safety net of externally-validated evidence. Not having a rigorous evidence base risks embarrassment, or worse, when things go wrong.
When faced with the need for rationalisation and cost-saving in policy research, a strong temptation, perhaps reinforced by civil servants’ anxieties about their own jobs, would be to bring government analytical capacity back in-house. This would be a mistake. The political risks of not having a fully independent, arms-length supply of research and evidence have been adequately demonstrated by the recent debates about the independence (or lack of it) from the Treasury of the new Office of Budget Responsibility. Equally risky would be over-reliance on the siren voices of the political ‘think tanks’; while the latter provide a good supply of innovative policy ideas they are campaigning organisations that rarely meet the test of real independence, and lack the scientific capacity (both social and economic) to test those ideas against real empirical evidence.
Rather, the government needs to find ways, in straitened times, to maintain its capacity to commission independent policy-relevant research, while still saving costs. How can this be done? Inevitably it will involve a far more parsimonious use of large-scale primary data collection through sample surveys: the tendency to throw a survey at any and every policy question has not only generated huge quantities of poorly focused, overlapping and under-utilised data, but has contributed to growing ‘survey fatigue’ among households, businesses and other respondents, reducing response rates and the relevance of the data. Expensive and burdensome surveys should be saved for those questions for which there is no alternative research strategy, and which are sufficiently important to justify the expense. Much greater use needs to be made in policy research, of administrative data collected as part of the normal process of delivering public services, particularly where data sets from different sources can be linked at micro-level. There is a case for far more extensive reliance on the existing evidence base (including, importantly, the international evidence), in the academic and broader literature when examining policy questions, using improved methods of systematic and rapid evidence reviews. Similarly far more can and should be done through small scale in-depth qualitative research.
It is already clear from the ‘Structural Reform Plans’ produced by government departments, that the coalition has ambitious policy objectives that will be extremely challenging to achieve alongside drastic public spending cuts. In such an environment, the need for robust, timely and accessible evidence of ‘what works’ and what offers best value for money must be greater than ever.
For more information on this work, please contact Nigel Meager at IES.