Risk: The new language of e-Business testing

The web offers a fabulous opportunity for entrepreneurs and venture capitalists to stake a claim in the new territory - E-Business. Images of the Wild West, wagons rolling, gold digging and ferocious competition over territory give the right impression of a gold rush. But who knows where the gold is?

Pressure to deliver quickly, using new technology, inexperienced staff, into an untested market place and facing uncertain risks is overwhelming. Where does all this leave the tester?

In fast-moving environments, if the tester carps about lack of requirements, software stability or integration plans they will probably be trampled to death by the stampeding project team.

In high integrity environments (where the Internet has made little impact, thankfully), testers have earned the grudging respect of their peers because the risk of failure is unacceptable and testing helps to reduce or eliminate risk.

In most commercial IT environments however, testers are still second class citizens on the team. Is this perhaps because testers, too often, become anti-risk zealots? Could it be that testers don't acclimatise to risky projects because we all preach 'best practices'?

In all software projects, risks are taken. In one way, testing in high-integrity environments is easy. Every textbook process, method and technique must be used to achieve an explicit aim: to minimise risk.

It's a no-brainer. In fast-moving E-Business projects, risk taking is inevitable. Balancing testing against risk is essential because we never have the time to test everything. It's tough to get it 'right'. If we don't talk to the risk-takers in their language we'll never get the testing budget approved.

So, testers must become expert in risk. They must identify failure modes and translate these into consequences to the sponsors of the project. "If xxx fails (and it is likely, if we don't test), then the consequence to you, as sponsor is" In this way, testers, management, sponsors can reconcile the risks being taken to the testing time and effort.

How does this help the tester? Firstly, the decision to do more or less testing is arrived at by consensus (no longer will the tester lie awake at night thinking: "am I doing enough testing?").

Second, the decision is made consciously by those taking the risk. Third, it makes explicit the tests that will not be done - the case for doing more testing was self-evident, but was consciously overruled by management. Fourth, it makes the risks being taken by the project visible to all.

Using risk to prioritise tests means that testers can concentrate on designing effective tests to find faults and not worry about doing 'too little' testing.

What happens at the end of the test phase, when time has run out and there are outstanding incidents?

If every test case and incident can be traced back to a risk, the tester can say, "at this moment, here are the risks of release". The decision to release needn't be an uninformed guess. It can be based on an objective assessment of the residual risk.

Adopting a risk-based approach changes the definition of 'good' testing. Our testing is good if it provides evidence of the benefits delivered and of the current risk of release, at an acceptable timeframe.

Our testing is good if, at any time during the test phase, we know the status of benefits, and the risk of release. No longer need we wait a year after release before we know whether our testing is perfect (or not). Who cares, one year later anyway?

In a recent E-Business project, we identified 82 product risks of concern. Fewer than 10 had anything to do with functionality. In all E-Business projects, the issues of Non-Functional problems such as usability, browser configuration, performance, reliability, security seem to dominate people's concerns.

We used to think of software product risks in one dimension (functionality) and concentrate on that. The number and variety of the risks of E-Business projects forces us to take a new approach.

It could be said that in the early 1990s, the tester community began to emerge and gain a voice in the computer industry. Using voice, the language or risk will make testers effective and influential in the ambitious projects coming in the next millennium.

Paul Gerrard, Systeme Evolutif Ltd