The tyranny of innovation?

by Colin Talbot, CBR Research Associate, University of Cambridge

Colin Talbot.
Colin Talbot

‘Innovation’ is cool. In the airport bookshop business section it’s probably about equal with ‘leadership’ as the hot topic (with AI coming up fast on the outside).

I mean, who can be against innovation, right? It’s brought us smart phones, Twitter, Instagram, Google, Netflix and take-away meals in the supermarket. What’s not to like?

In my own main field of expertise – Government Institutions and Public Policy or what used to be called ‘public administration’ – innovation is now also ‘king’.

You can’t move for tripping over learned academic articles or gushing Government papers about the wonders of innovation in public administration and how it is going to solve the problem of ‘doing more with less’ (again).

At the risk of being accused of conservatism, let’s stop and think about this “innovation good” fashion?


Life on Earth is, undeniably, a triumph of evolution and adaptation. Life has changed, and spread, through varying climates and environments and diversified in phenomenal ways. It has survived massive challenges from asteroid strikes to climate change, and still thrived, if often radically changed.

But it is crucial to understand that life does not just survive by innovation and adaptation, but also by conservation. Without a high degree of conservation in species there would be chaos. Change has to occur incrementally and slowly, or it risks self-destruction.

It is the balance between innovation and conservation that is key. And this is in turn determined by environmental (exogenous) factors as much as by internal (endogenous) ones.

If the external environment is changing rapidly then mutations (innovations) might also be rapid and, in some cases, beneficial. Or at least the environmentally beneficial ones might spread more rapidly?

Conversely, if the external environment remains relatively stable there is little advantage in a ‘winning’ formula changing. Some species have remained remarkably unchanged for millions of years because their environmental niche has also changed very little. “If it ain’t broke, don’t fix it” holds true for evolutionary pressures. This holds true for some areas of Government and Public Policy.

Conservation & innovation

The late Larry Terry, a US public administration scholar, wrote an under-appreciated book entitled “Leadership of Public Bureaucracies – the Administrator as Conservator”. It’s the second part of the title that is important here.

Terry’s argument was that it is an important role of public bureaucrats to protect – conserve – their organizations and institutions against attempts to undermine them, including by, if necessary, elected politicians.

Some may see this as almost undemocratic, but it is an important part of the ‘checks and balances’, Terry argued, that various democratic governments have evolved to protect democratic institutions from ‘the tyranny of the majority’ or corrupt politicians.

The USA went through a period of corrupt public administration – both politically and financially – as did most advanced countries. Part of the remedy for that corruption was the granting of a certain level of ‘bureaucratic autonomy’ to some public agencies – especially those that need near universal public consent to be able to function – things like regulators, tax collectors, social security administrators.

These agencies are also usually very slow to change – for good reason. The UK economist Tim Harford’s book “Adapt” has the subtitle “why success always starts with failure.” In evolutionary terms he is of course correct – most biological mutations (innovations) fail – only a lucky few survive and thrive. And Harford also points out that innovative failures are more common than successes in business and markets too – and there they are a good thing, speeding up innovation and redistributing scarce resources to ‘what works’.

But in some public agencies, especially ones that require regularity to deliver equitable outcomes (like tax collection) failure is rarely an option. Standardised success, even if it’s not optimal, is better than failure. Risks associated with innovations – such as ‘post code lotteries’, unfair outcomes, corruption – are all failures to be avoided at almost any cost.

Innovation in bureaucracies?

So how can bureaucracies achieve successful innovation, whilst avoiding the risks and conserving what needs to be conserved?

Well it’s certainly not by adopting the relentless rhetoric of ‘innovation is good, conservation is bad’, which seems to be happening all too often.

Public bureaucrats are attacked for being ‘too risk averse’ and then denounced when they risk innovations that fail, sometimes costing billions and damaging people’s lives and trust in government.

Public bureaucracies need to adapt and innovate, but they need to balance that with preservation of what’s good and necessary. And in that need for balance lies the clue.

One way to innovate where there are multiple similar units in a field – like schools in education, prisons in corrections, hospitals in health – is to allow a small number of them to innovate and spread ‘what works’ to the rest?

Under the last Labour Government (1997-2010) this was done by an unofficial policy of allowing a portion – usually not more than 10% – to do things differently. In prisons this was done by contracting out their management. In schools it was done by allowing ‘academies’. In both cases the number of ‘experimental’ units was limited to roughly one in ten.

(Opponents on both right and left failed to notice this limitation was intentional and criticized it for either not going far enough or for happening at all).

Other options were also tried – such as pilot schemes in a limited number of areas. In some instances innovations simply ‘emerged’ somewhere in the system and then were spread by various means. Back in 1992 John Kimberley and colleagues published a fascinating case study – The Migration of Managerial Innovation – on how ‘diagnostic-related groups’ had spread almost spontaneously through western European health care systems.

Under the previous Conservative government (1979-1997) another approach was applied in the introduction of a new system – housing benefit – into local government. Individual local authorities were left to their own devices to let “a thousand flowers blossom” – and they did. Many different ICT companies jumped into the new market with gusto.

Most failed, at some cost, but within a few years the number of different systems rapidly dwindled until only a few survived. We can argue if these were the ‘best’, but in this ecological niche they were clearly the ‘fittest’?

I’ve written elsewhere about another approach – the use of what are called “parallel learning systems” within organizations to innovate without risking catastrophic systems failure.

The point is innovation and conservation, or to use simpler language continuity and change, is both desirable and possible if it is approached in a balanced and considered way. The British Civil Service used to know this – at least it did when it published its ‘Continuity and Change’ White Paper on civil service reform in 1992. What usually ends in tears is one-sided, gung-ho, ‘innovation is good’ – as Tim Harford observed, it usually ends in failure?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.