One of the main causes of overruns on technology projects is, not surprisingly, late discovery of issues with the technology and software itself. Adopting a more ‘solution delivery’ rather than ‘software delivery’ mindset helps project teams to engage in ongoing assurance and identify issues early - avoiding costly rework before it hits the project timescales.
Projects involving software are notorious for overruns and as project managers, we’re familiar with the signs: business stakeholders get pressured to accept a reduced scope, the project team gets pressured to deliver more rapidly and external stakeholders apply pressure to senior management and the project board. Meanwhile, the number of defects begins to rise and, in many cases, critical defects relating to the basic design or operation of the system are found mid-way or late in the development.
If we look at the technology related problems, quality issues are generally identified as defects and integration problems during the various testing phases of the project. There are ways to mitigate these issues and some agile approaches can improve quality. However, for the vast majority of software development projects there are still basic problems in how quality is built into software products.
Adding quality by discovering defects
In the traditional model, developers build code and pass it on to testers, who execute a series of pre-defined tests against it. This approach can only ever discover defects after the fact; hence, the traditional process of test and rework can only ‘add quality’ to the solution.
As project managers we build in rework budgets and allow time for testing to allow for the defects that will be discovered, however, the later a technical issue is discovered, the higher the cost to resolve it - this isn’t a linear relationship.
The groundwork for the most difficult to resolve defects is laid early on in requirements and design, those defects tend to be discovered late, during integration, performance or security testing - when many components are finished or close to completion.
Building quality in - ‘test’ everything
An alternative way to tackle the problem is to build quality in from the outset, seeking to prevent those expensive defects from getting into the design or requirements in the first place.
Planning for three cycles of testing is quite common; sadly discovering that re-tests and rework cause testing to ‘spill over’ into later test stages is quite common too.
In such a project, as rework is discovered quality assurance (QA) costs rise because much of the test team’s life is spent performing testing or re-testing code that has been reworked; the proportion of time testers spend focussing on new functionality or system behaviour can drop dramatically.
A better approach based on the principles that QA and test teams already apply is to treat all products delivered through the project lifecycle as testable (i.e. assurable) items.
Just like the code, documents and other artefacts are part of the overall solution. When done properly, improving quality by addressing technical and business risk factors in early lifecycle products is a high value-add activity.
The results from taking a more solution-based approach and assuring products earlier in the project can be dramatic. Figure 1 shows that the earlier a defect is introduced, the higher the ultimate cost of removing that defect e.g. removing defects introduced at the requirements stage can cost 100 times more than defects introduced during integration.
Starting assurance earlier in the lifecycle may mean that QA resource is engaged on projects for longer, but the reduction in workload towards the end of the project more than compensates for this (because fewer defects are discovered and re-tested).
Catching just one major issue early on could be enough to make the difference between on time and overrun.
There is some irony in the fact that one of the best ways to reduce your spend on testing effort is to do more testing, albeit of a different sort to that which most test and project managers are accustomed to.
We think about assurance as an ongoing activity, not something we do after a piece of software has been built. The fact remains that by assuring and testing products earlier in our projects, we can significantly increase the likelihood of avoiding technical surprises and thus bringing our projects in on time.
Making it happen - four steps for building quality into your projects
Step one: Mindset is critical. We often hear of the software development lifecycle (SDLC), which tends to focus the attention on the software. Instead, we should think in terms of a SOLUTION DELIVERY lifecycle, in which all project/programme artefacts form part of the solution and are subject to scrutiny by testers.
Step two: Identify the key artefacts produced during the solution delivery lifecycle, and define the business and technical risk based acceptance criteria for them.
Step three: Coach the testers in the review process that you follow - these can range from full blown Fagan type inspections that appeared in the late 1970’s to informal reviews. Exercise caution here: it is all too easy for this kind of assurance activity to turn into a box-ticking exercise and assurers need to have people skills and objectivity.
Step Four: Record the outcomes of inspections or reviews as you would for a software defect, assigning priorities to the defects; if using a defect tracking tool, record the fact that they were raised following an inspection and identifying the artefact that they were found in. Independence from the project is important to ensure objectivity and assurers should not normally be part of the project delivery team.
