NHS IT: When will we ever learn?

So, NHS NPfIT / CfH is put out of its misery. I come to bury it not to praise it. There have many enquiries and reports on the NHS IT programme. Its scope, procurement and management have been put under the microscope over 10 years.

However, I believe there are some important lessons that are still to be fully understood. I believe that these lessons are important, if future projects in health IT are to deliver real patient benefits and value for the public purse. Back in 1990, with my late colleague Hugh MacDonald of ICL, we gave a session in Whitehall to senior civil servants on the findings of the Sloan School of Management, MIT landmark study, ‘Management in the 1990s’. Our case was to distinguish the characteristics of success and failure in business-IT.

  1. IT projects were most likely to succeed where the project was owned by business, not seen as a technical discipline.
  2. Alignment between business strategy and IT was a key to realising business value.
  3. Engagement of users at an early stage and then throughout the project reduced project risk.
  4. The organisational costs, notably training and change management, were larger than the hardware and software costs measured over the business life-cycle and needed to be managed alongside the capital costs of IT.

So, does that sound familiar? Hence my title, ‘when will we ever learn?’

Back in 2002 I was involved in assessing the possibility of bidding for the NHS IT programme. To understand the scope and challenges of the proposed programme, various research projects were undertaken and external experts consulted to build up a picture of the background for a proposed consortium.

The data available at the time showed some inconsistencies and gaps which made for difficult planning for a future bidder.

In turn, this has led to various misunderstandings and complexity since then which have impacted on the programme, private sector suppliers and the NHS staff, leading to a widespread disappointment with the delivery and benefits realised by the programme.

The state of play in 2001-02 in NHS IT

Much has been written about the cost over runs and delays in the delivery of NPfIT/Connecting for Health.

In 2001-02 the conclusion I made for the existing expenditure on IT within the NHS was that the NHS was spending around £1bn per annum in the areas covered by NPfIT. This estimate came from the salaries of the IT staff employed by the NHS and adding up the health turnovers of companies engaged in selling IT products and services to the NHS. Figures from market researchers using other methods confirmed this figure to be a good starting assumption. Looking at IT expenditure in areas not covered by the NPfIT, a further figure of around £500m was suggested. These areas included dentistry, opticians and some GP-related IT.

This implied that the total NHS IT expenditure at £1.5bn. was running at 3-4 % of NHS expenditure. At that time, information intensive sectors of the economy typically would be investing around 3-5% of expenditure on IT, so the NHS was at the low-end but in range.

Given the rapid growth in total NHS expenditure during the decade, if NHS IT expenditure stayed at around the same percentage of total turnover, then expenditure by 2010 would be around £3-4bn per annum.

To kick start the programme, the Wanless review for the Treasury had suggested a ring-fenced sum of £2.3bn for three years.

The first bidding document OBS1 (output-based specification 1) outlined the scope for the NPfIT programme.

From the base work, it became possible to estimate what the full implications of implementation would be. The overall estimate I came to, was that OBS1 implied a rise from £1bn to £3bn per annum if fully implemented. OBS1 was much broader in its scope than that which was finally contracted for in terms of the spine and the local service providers, LSPs. Areas such as telemedicine and telecare were not carried forward to the initial contracts.

There were some serious areas where OBS1 and its successor document OBS2 were lacking in clarity. The costs of change management in implementing the national systems at a local level were outside the NPfIT scope. However, talking to the local bodies, they seemed unaware of their responsibilities in this regard and were not budgeting for them.

The initial contracts

When the contracts were signed they were for a total of £7.4bn over 7 years. This was when the first headlines appeared implying that NPfIT was already £5bn over budget. This figure was derived by assuming that the Wanless funding was the project budget. The difference actually arises from the difference between the 3 year and 7 year terms.

With the late delivery of the national programme, the overrun has been reported as anything from £5- £12bn over budget.

Using the OBS1 figures over 10 years would be approaching £30bn, so the programme could be described as over £25bn over budget, double the current claims.

On the other hand the projected £12bn cost to 2015 was close to the run rate of IT expenditure back in 2002. So it could be spun that there was no overrun, only late delivery. It would be possible to spin the claim that the 2002 situation was itself the scandal.

I’m not trying to defend any one position here. It is the certainty which various reports claim over having found the problem that I am sceptical about.

As with capital expenditure in defence, what the narrative above shows is that it is difficult to get to a consistent and complete set of figures to understand the long term impacts and commitments in programmes like NPfIT.

The publication of the full data via data.gov.uk is to be welcomed in the future, but building a model which is more robust than that described above for NPfIT/CfH will not be solved by the data alone. There is too much room for interpretation. Missing commitments, such as the change management costs outlined above, will not be surfaced by publishing data alone.

What is needed is for a model or template for major IT projects which outlines scope and external dependencies as well as life time commitments to major capital projects. Only in this way can the data be organised to understand the ‘big picture’ issues.

Top-down or bottom-up

Returning to 2002, it is instructive to look at how IT expenditure was organised and the implications for the operation of the service.

IT expenditure was very fragmented across the NHS. That was why it was so difficult to get a baseline figure for the expenditure across the service.

In terms of functionality, looking at London as an example there were some implementations of the electronic patient systems that were pretty close to the NPfIT vision, while other hospitals were barely starting the journey from paper-based records. I identified over 30 different EPR systems in London. Some were as small as an individual research project and built by the researchers. At the other end there were full systems from the likes of Cerner. These systems were not interoperable.

Talking to clinicians and administrators showed some consequences of this approach.

First, agency staff, moving from hospital to hospital, was frequently faced with systems that they did not know and were untrained on. Many staff reported backlogs, in one case 3-6 months, in getting the medical records in order and high associated costs. Difficulties in tracking down the records once they were created were a frustration I found around the country in many health settings. Interestingly, even for those advanced EPR systems, the costs of the paper records were still high as no-one at that stage had gone paperless. The rough consensus was that £350m per annum was being spent on EPR out of £800m pa on patient records in total.

The vision if realised was in line with what many thought was needed even if there was scepticism of the capability to deliver. The wide variety in existing infrastructure across the LSP regions was a clear problem. A single system across a region would reduce some of the problems such as the agency staff example above. On the other hand, ripping up the systems of those who were well down the paths to EPR systems to fit the regional norm risked professional reaction at those institutions. As mentioned above, the lack of clarity on change management costs meant that some of the leading institutions could be faced with high costs for little benefit to staff or patients.

The NPfIT programme is often described as Top Down, and that this aspect is declared as the source of its problems.

It is claimed that giving local autonomy would be far more effective. It can however be argued that this was the position in 2002. Indeed, the centre defined standards for systems which the local bodies were required to adopt. Despite this, the evidence on the ground suggested that this approach led to fragmentation and inconsistency. To see the problems in terms of top down vs. bottom up is to miss some important learning to inform future major projects.

What was the hallmark of the successful implementations that I found was a combination of clear leadership, clinical engagement and good working between clinical and administrative staff.

Turning to the procurement process within NPfIT I would argue that this flawed approach did not reflect that proven approach from local level, in its national ambitions.

Functionality, user interface design, response times and security often bore little evidence of clinical engagement in defining requirements.

After OBS1, the programme became increasingly focussed on an IT shopping basket rather than health service outcomes. This enabled the procurement to run very quickly by public standards and tied suppliers to very tight contracts. What was lost was a governance model that bought clinical and administrative staff together to enhance patient services at good value for the public purse.

NPfIT in the end was much delayed and the centre of much investigation over its benefits and problems.

What I have argued above is that to blame these problems on a top down approach is to ignore the problems of the starting point.

I wanted to work on NPfIT because I believed that the vision of a National Information Infrastructure for Health as outlined by the Wanless review was a necessary component of a 21st century health service.

A belief that simply devolving the responsibility to the local level will fix the problem is to ignore the history of IT in the NHS. Fragmentation of IT itself created costs and problems for the health system.

The real lesson is that successful delivery will require governance arrangements at local and national levels that bring together clinical and administrative staff together around patient experience and outcomes.

To hold those responsible to account for delivery requires new budgetary mechanisms for long-term capital projects that help create greater transparency and allow for dependencies, gaps and changes of scope to be properly scrutinised.

So, I phoned a friend at a City institution for a call. The outline, far from scientific, is illustrative nonetheless.

I asked what it costs to run a million bank accounts. Now the medical record is far more complex than a bank account. For most people the number of transactions will be lower on the health record.

We came to a figure of a medical record account costing £20 per person per year, or £1.2bn for 60m records, or £12bn over 10 years. So, has the programme been hampered from the start by gross underestimation?

I’m not arguing here that £12bn would have been a good investment in EPR that is for wider debate.

I have been involved in a number of rounds of discussion about improving public sector procurement of IT since that talk in 1990. It doesn’t feel like 21 years of progress! I’m coming to the conclusion that it is the interaction of the long-term funding processes and procurement, not procurement itself, that lies at the heart of the problem.

With the enormous universal credit system development underway, do you feel confident that lessons from NPfIT/CFH have been learned?

I remember one of our consulted experts being asked: ‘What do you think will be the outcome of NPfIT?’. His answer was ‘Long and lingering disappointment’. I owe him a drink probably.

Here’s my forecast. In 2017 there will be a series of reports looking at public sector including health IT system over runs. They will illustrate that fragmented local procurement carries costs and should be centralised. Projects will have been started without a full understanding of the true costs. Requirements will have been agreed that do not recognise the real user needs. IT projects have not been aligned with the organisational strategy.

This is much longer than most of my posts. If you’ve got this far, thank you.

The optimist in me hopes I’m wrong, but when will we ever learn?

Comments (6)

Leave Comment
  • 1
    VectaFrank wrote on 25th Oct 2011

    Pretty much all true but large organisations always struggle with large systems. Senior management, especially in government, is several technical generations away from the toys that techies want to play with so can be bamboozled all too easily if they don't pay attention. Keep the toys small and technology preferably old; and budgets very tight.

    Report Comment

  • 2
    Phil wrote on 26th Oct 2011

    Once any system gets beyond a certain size it will almost certainly fail. To go for the 'all singing all dancing system' that was NPfit was asking for trouble. If you are going to eat an elephant - do it one small piece at a time. Don't try to invent a system which relies on the bleeding edge of technology - let others do this.

    Report Comment

  • 3
    Martin wrote on 28th Oct 2011

    There's a fundamental clash between a self-regulating scientific profession (medicine) and the ambitious, but unachievable requirement to manage IT in the NHS as if the latter were a single entity, which it most certainly is not. Medicine seeks to maintain the position it has always enjoyed in the Establishment where individuals and experts, rather than organisations, are the loci of influence. Managers and consumers demand information, structures and systems that are consistent and centralised. But the data that a truly national business needs to run things well cannot be delivered from or reconciled between disparate systems that reflect different methods of working within professional fiefdoms. This is the problem that any large body eventually faces: the need to address the conflict between talented federalism with its concomitant inefficiencies, and centralised control with its tendency to suppress innovation, marginalise professional expertise and diminish the influence of local power bases.

    Loosely controlled entities self-manage themselves towards the optimum balance of risk and usability with regard to their IT solutions. The difficulty of achieving local integration of NHS IT systems and an emphasis on improving the quality and utility of information at a local level for local commissioning will probably absorb the majority of the effort that goes into systems architecture and implementation over the next few years, reducing the amount that is available for unifying outputs and standardising data for national use. It is therefore unlikely that the majority of health systems will really generate the standardised outputs that the proponents of the local solutions advocate as the answer to the NPfIT’s national information plans. The situation will be made more difficult by the tendency of IT suppliers to follow the line of least resistance with regard to developing, delivering and realising the cash benefit of solutions. A federal IT market will result in federal solutions.

    Report Comment

  • 4
    Geoff Codd wrote on 28th Oct 2011

    Certainly all true and very depresing indeed. Those of you who have been involved for decades in IT stimulated change in business & Government will once again sigh with resignation at yet another NAO report says that 'lessons have been learned' - once again! - after a particularly dramatic failure of a prestige Government IT project.

    When will they realise that the management of change has not kept pace with technology innovation - indeed it now lags seriously behind - and that fundamental management and behavioural change is needed in order to address cultural disconnects that lie at the root of many failures.

    As an independent observer of the NHS story, it sems to me that there was already a damaging disconnect between clinitians and managers, which was further compounded by that between the IT 'professionals' and everyone else.

    Add to this cocktail the centralisation V devolved accountability tensions, that management seemed to think would be resolved by the new systems design process - but which clearly would only compound an already serious problem, and failure was inevitable.

    The resultant Government strategy for IT once more comes up with the hoary old wisdoms of Smaller Projects, more Smaller Suppliers etc. etc. - all of which is actually good stuff. However, none of that will bring about the transformation in IT exploitation performance that is so desperately needed. ( And please don't mention G-Cloud which by another name is simply sensible virtualisation of technology resources.)

    There are management initiatives at all levels - including at the top dare I say - that would yield short term benefit and at the same time lay the foundation for the much needed improvement in performance. There is however a lack of willingness to open the mind to the fact that everyone involved - including ministers - need to address the introduction and management of change in a new way that is in tune with the e-business age that we now find ourselves in.

    I wonder how many more 'lessons will have to be learned' before the penny will drop - and how many years will pass - before the real lesson is generally recognised. Perhaps we old stagers in this game of change need to band together so that the voice of experience can be heeded.

    Report Comment

  • 5
    Steve wrote on 28th Oct 2011

    I believe the government is already looking closely at the next ‘big’ NHS IT programme (Information Strategy). Will we ever learn? We could start stabilising the business strategy (not a small task in the NHS), work with user groups checking you have an accurate causal model of the current system(s) dynamics, mandate an emphasis on defining measurable benefits as the desired transformation outcomes, look to use the best SOA design principles (including federated identity) to enable you to work with the existing estate, run a realistic model office implementation (would still be a sizable implementation), check to see if it delivers, learn lots of lessons. Finally invest in the best people (in-house), who have a passion for delivering transformational change in the NHS. Repeat above until you arrive at your destination. Spread the wealth rapidly when ready. Definitely don’t centralise, run several concurrent massive procurements and implementations, appoint several different contractors and nail them to a long contract, when you know you will have a number of significant technology step changes in that period.

    Report Comment

  • 6
    David Hewett wrote on 28th Oct 2011

    Sadly this outcome was predictable from the start. I for one am glad it has been put to death.

    I put in the TDS systems in Winchester in 1986-92 and my friend and colleague Alan Jones carried on the continued operations after that. This was the first, and only one of 2, level 5 systems that have ever existed in the NHS. The other was Arrowe Park in Cheshire. We were beacon sites for the DoH in the early 90's. There was nothing in London that came anywhere near what these two sites had. We even cracked electronic prescribing and dispensing within the main record. The lessons we learned and published are very close to Chris's list. The method we used to implement, was bottom up. We used former clinical staff as business analysts and implemented the TDS software ourselves on an IBM network .

    My best decision, early in the implementation, was to dismiss Arthur Andersen ( contracted by the Regional Health Authority) who did the very first stage and appoint a local team. Then we contracted the mainframe service directly with IBM. As a Executive Member of the health authority, I had the necessary management clout, and as a doctor, the credibility to lead this project. NHS bureaucracy today prevents anyone from aquiring this sort of power and position any more.

    The system was put in on time and within budget, the only bit of the otherwise ill-fated Wessex programme that was, and ran for 20 years before being replaced by Cerner in 2007. When that change happened the functionality at Winchester went backwards 12 years. I retired a few days after the change very saddened by what had been done.
    This was the ill thought out scheme of an authoritarian government and a prime minister who by then believed he could walk on water.

    In my view it never stood a chance, but no one listened and particularly they ignored all the experience that had been accumulated. But then that is how it always is.

    Report Comment

Post a comment

About the author
Chris is a technology and policy futurologist. Chris has been in the IT industry since 1980. His roles have spanned Honeywell, ICL, HP, Microsoft and Capgemini. He is a Fellow of the BCS and a Fellow of the RSA.

See all posts by Chris Yapp

Search this blog

January 2018