However, I believe there are some important lessons that are still to be fully understood. I believe that these lessons are important, if future projects in health IT are to deliver real patient benefits and value for the public purse. Back in 1990, with my late colleague Hugh MacDonald of ICL, we gave a session in Whitehall to senior civil servants on the findings of the Sloan School of Management, MIT landmark study, ‘Management in the 1990s’. Our case was to distinguish the characteristics of success and failure in business-IT.
- IT projects were most likely to succeed where the project was owned by business, not seen as a technical discipline.
- Alignment between business strategy and IT was a key to realising business value.
- Engagement of users at an early stage and then throughout the project reduced project risk.
- The organisational costs, notably training and change management, were larger than the hardware and software costs measured over the business life-cycle and needed to be managed alongside the capital costs of IT.
So, does that sound familiar? Hence my title, ‘when will we ever learn?’
Back in 2002 I was involved in assessing the possibility of bidding for the NHS IT programme. To understand the scope and challenges of the proposed programme, various research projects were undertaken and external experts consulted to build up a picture of the background for a proposed consortium.
The data available at the time showed some inconsistencies and gaps which made for difficult planning for a future bidder.
In turn, this has led to various misunderstandings and complexity since then which have impacted on the programme, private sector suppliers and the NHS staff, leading to a widespread disappointment with the delivery and benefits realised by the programme.
The state of play in 2001-02 in NHS IT
Much has been written about the cost over runs and delays in the delivery of NPfIT/Connecting for Health.
In 2001-02 the conclusion I made for the existing expenditure on IT within the NHS was that the NHS was spending around £1bn per annum in the areas covered by NPfIT. This estimate came from the salaries of the IT staff employed by the NHS and adding up the health turnovers of companies engaged in selling IT products and services to the NHS. Figures from market researchers using other methods confirmed this figure to be a good starting assumption. Looking at IT expenditure in areas not covered by the NPfIT, a further figure of around £500m was suggested. These areas included dentistry, opticians and some GP-related IT.
This implied that the total NHS IT expenditure at £1.5bn. was running at 3-4 % of NHS expenditure. At that time, information intensive sectors of the economy typically would be investing around 3-5% of expenditure on IT, so the NHS was at the low-end but in range.
Given the rapid growth in total NHS expenditure during the decade, if NHS IT expenditure stayed at around the same percentage of total turnover, then expenditure by 2010 would be around £3-4bn per annum.
To kick start the programme, the Wanless review for the Treasury had suggested a ring-fenced sum of £2.3bn for three years.
The first bidding document OBS1 (output-based specification 1) outlined the scope for the NPfIT programme.
From the base work, it became possible to estimate what the full implications of implementation would be. The overall estimate I came to, was that OBS1 implied a rise from £1bn to £3bn per annum if fully implemented. OBS1 was much broader in its scope than that which was finally contracted for in terms of the spine and the local service providers, LSPs. Areas such as telemedicine and telecare were not carried forward to the initial contracts.
There were some serious areas where OBS1 and its successor document OBS2 were lacking in clarity. The costs of change management in implementing the national systems at a local level were outside the NPfIT scope. However, talking to the local bodies, they seemed unaware of their responsibilities in this regard and were not budgeting for them.
The initial contracts
When the contracts were signed they were for a total of £7.4bn over 7 years. This was when the first headlines appeared implying that NPfIT was already £5bn over budget. This figure was derived by assuming that the Wanless funding was the project budget. The difference actually arises from the difference between the 3 year and 7 year terms.
With the late delivery of the national programme, the overrun has been reported as anything from £5- £12bn over budget.
Using the OBS1 figures over 10 years would be approaching £30bn, so the programme could be described as over £25bn over budget, double the current claims.
On the other hand the projected £12bn cost to 2015 was close to the run rate of IT expenditure back in 2002. So it could be spun that there was no overrun, only late delivery. It would be possible to spin the claim that the 2002 situation was itself the scandal.
I’m not trying to defend any one position here. It is the certainty which various reports claim over having found the problem that I am sceptical about.
As with capital expenditure in defence, what the narrative above shows is that it is difficult to get to a consistent and complete set of figures to understand the long term impacts and commitments in programmes like NPfIT.
The publication of the full data via data.gov.uk is to be welcomed in the future, but building a model which is more robust than that described above for NPfIT/CfH will not be solved by the data alone. There is too much room for interpretation. Missing commitments, such as the change management costs outlined above, will not be surfaced by publishing data alone.
What is needed is for a model or template for major IT projects which outlines scope and external dependencies as well as life time commitments to major capital projects. Only in this way can the data be organised to understand the ‘big picture’ issues.
Top-down or bottom-up
Returning to 2002, it is instructive to look at how IT expenditure was organised and the implications for the operation of the service.
IT expenditure was very fragmented across the NHS. That was why it was so difficult to get a baseline figure for the expenditure across the service.
In terms of functionality, looking at London as an example there were some implementations of the electronic patient systems that were pretty close to the NPfIT vision, while other hospitals were barely starting the journey from paper-based records. I identified over 30 different EPR systems in London. Some were as small as an individual research project and built by the researchers. At the other end there were full systems from the likes of Cerner. These systems were not interoperable.
Talking to clinicians and administrators showed some consequences of this approach.
First, agency staff, moving from hospital to hospital, was frequently faced with systems that they did not know and were untrained on. Many staff reported backlogs, in one case 3-6 months, in getting the medical records in order and high associated costs. Difficulties in tracking down the records once they were created were a frustration I found around the country in many health settings. Interestingly, even for those advanced EPR systems, the costs of the paper records were still high as no-one at that stage had gone paperless. The rough consensus was that £350m per annum was being spent on EPR out of £800m pa on patient records in total.
The vision if realised was in line with what many thought was needed even if there was scepticism of the capability to deliver. The wide variety in existing infrastructure across the LSP regions was a clear problem. A single system across a region would reduce some of the problems such as the agency staff example above. On the other hand, ripping up the systems of those who were well down the paths to EPR systems to fit the regional norm risked professional reaction at those institutions. As mentioned above, the lack of clarity on change management costs meant that some of the leading institutions could be faced with high costs for little benefit to staff or patients.
The NPfIT programme is often described as Top Down, and that this aspect is declared as the source of its problems.
It is claimed that giving local autonomy would be far more effective. It can however be argued that this was the position in 2002. Indeed, the centre defined standards for systems which the local bodies were required to adopt. Despite this, the evidence on the ground suggested that this approach led to fragmentation and inconsistency. To see the problems in terms of top down vs. bottom up is to miss some important learning to inform future major projects.
What was the hallmark of the successful implementations that I found was a combination of clear leadership, clinical engagement and good working between clinical and administrative staff.
Turning to the procurement process within NPfIT I would argue that this flawed approach did not reflect that proven approach from local level, in its national ambitions.
Functionality, user interface design, response times and security often bore little evidence of clinical engagement in defining requirements.
After OBS1, the programme became increasingly focussed on an IT shopping basket rather than health service outcomes. This enabled the procurement to run very quickly by public standards and tied suppliers to very tight contracts. What was lost was a governance model that bought clinical and administrative staff together to enhance patient services at good value for the public purse.
NPfIT in the end was much delayed and the centre of much investigation over its benefits and problems.
What I have argued above is that to blame these problems on a top down approach is to ignore the problems of the starting point.
I wanted to work on NPfIT because I believed that the vision of a National Information Infrastructure for Health as outlined by the Wanless review was a necessary component of a 21st century health service.
A belief that simply devolving the responsibility to the local level will fix the problem is to ignore the history of IT in the NHS. Fragmentation of IT itself created costs and problems for the health system.
The real lesson is that successful delivery will require governance arrangements at local and national levels that bring together clinical and administrative staff together around patient experience and outcomes.
To hold those responsible to account for delivery requires new budgetary mechanisms for long-term capital projects that help create greater transparency and allow for dependencies, gaps and changes of scope to be properly scrutinised.
So, I phoned a friend at a City institution for a call. The outline, far from scientific, is illustrative nonetheless.
I asked what it costs to run a million bank accounts. Now the medical record is far more complex than a bank account. For most people the number of transactions will be lower on the health record.
We came to a figure of a medical record account costing £20 per person per year, or £1.2bn for 60m records, or £12bn over 10 years. So, has the programme been hampered from the start by gross underestimation?
I’m not arguing here that £12bn would have been a good investment in EPR that is for wider debate.
I have been involved in a number of rounds of discussion about improving public sector procurement of IT since that talk in 1990. It doesn’t feel like 21 years of progress! I’m coming to the conclusion that it is the interaction of the long-term funding processes and procurement, not procurement itself, that lies at the heart of the problem.
With the enormous universal credit system development underway, do you feel confident that lessons from NPfIT/CFH have been learned?
I remember one of our consulted experts being asked: ‘What do you think will be the outcome of NPfIT?’. His answer was ‘Long and lingering disappointment’. I owe him a drink probably.
Here’s my forecast. In 2017 there will be a series of reports looking at public sector including health IT system over runs. They will illustrate that fragmented local procurement carries costs and should be centralised. Projects will have been started without a full understanding of the true costs. Requirements will have been agreed that do not recognise the real user needs. IT projects have not been aligned with the organisational strategy.
This is much longer than most of my posts. If you’ve got this far, thank you.
The optimist in me hopes I’m wrong, but when will we ever learn?
