Before the gloves come off

Dr Stephen Castell, chairman of Castell Consulting, looks at expert resolution of IT disputes via forensic systems analysis. 

Disputes on the increase 

Experienced practitioners in procurement, contracting, execution, management, and delivery of major IT systems agree that the quality and certainty of these processes are not improving: major IT project disasters continue to occur. This has led to a growth in IT disputes, project abandonment, contract termination and actual litigation.

From my fifteen years of experience working professionally as an independent forensic IT expert analyzing and giving opinion in major IT systems disputes and litigation,  I have culled valuable lessons to be learned in regard to the various IT dispute resolution methods available.

This has also resulted in the development of a range of effective techniques for independently assessing and reading the 'technical entrails' of failed, stalled, delayed, or generally troublesome IT projects.

This inquisitorial method, called forensic systems analysis (FSA), focuses on compliance testing of the software in dispute. 

Methods of IT dispute resolution 

Methods of IT systems dispute resolution (ITSDR) fall into one of seven broad categories, as shown in the table.

Methods of IT systems dispute resolution 

The parties almost always benefit by having early involvement of an experienced external, non-partisan, independent forensic IT expert.

Such an IT expert (or team, for large, complex systems projects) is capable of dispassionately and objectively examining all aspects of the project, and professionally employing appropriate FSA techniques in an unbiased manner.  

A recently-published Cutter Executive Report details components of FSA.

It also presents a case study, based on a fictional but highly realistic IT dispute scenario, that illustrates how FSA methods are applied to: identify the technical issues; arrive at an expert assessment; discover how issues relate to poor contract negotiation and project management; recognise early warning signs of IT project failure; and teach how to avoid IT disaster projects. 

A key technical issue in IT systems disputes 

A key issue in systems development contractual disputes is: What was the quality of the delivered software, and was it fit for purpose? To the IT expert, quality can only mean fitness for purpose in the sense of 'Is the delivered software compliant with its stated requirements?'  

The independent forensic IT expert called in to assist in an ITSDR is presented with large volumes of project documentation, including software testing records, typically in the form of testing incident reports (TIRs) numbering in the several hundreds to many thousands. 

Disputes then often boil down to a customer's allegation that the TIRs represent errors in the software that are evidence of serious non-compliance with the stated requirements, and are a material breach of the contract, entitling the customer to reject the software and terminate the contract.

The software vendor/developer responds that the TIRs do not constitute 'showstopper' faults, that any (alleged) faults arose principally as a result of the many changes in requirements specification made by the customer, that the software is materially compliant with all such requirements, and that the customer is not entitled to terminate. 

The independent forensic IT expert may address this key issue using in particular the EFLAT (Expert's Fault Log Analysis Task) FSA component: this is tied intimately into the key legal concepts of material breach and material defect.

During software development or customisation, software defects are routinely encountered and fixed, and there is generally nothing alarming about their occurrence.  

For rejection of software and termination of a software development contract, any alleged software fault must be assessed using a strict test as to whether or not it is truly a material defect – that is, whether it is of large consequential business effect; is impossible, or would take a long time, to fix; and is incapable of any practical workaround.

EFLAT constitutes a careful rerunning of acceptances tests, under expert observation, with each TIR assessed according to this material defect rule.  

Typical FSA findings – avoiding IT disasters and disputes 

The findings of such FSA investigations usually permit the forensic IT expert to arrive at an objective and compelling independent expert opinion on the key technical issues in dispute for any sizable IT systems development/implementation contract case.

One insight that often emerges is that FSA analyses could readily be produced during the project itself and used as objectively justifiable evidence for persuading the customer not to lose patience and terminate the contract.

This might save the project, resolve the dispute, and avoid costly litigation or ITSDR entirely.  

Given that the key to arriving at an expert opinion in complex software project/contract disputes is compliance testing of the software in dispute, the FSA methodology can provide strong, objective support in assisting with resolving and/or settling such disputes, whichever ITSDR method may be used.   

Better yet, applying FSA techniques during the conducting of a systems project, before it starts to head toward the IT failure rocks, can deliver illuminating and dramatic insights into the ongoing status of the software, contributing mightily to IT disaster avoidance.

Further reading 

'Forensic Systems Analysis: A Methodology for Assessment and Avoidance of IT Disasters and Disputes', Cutter Consortium Executive Report, Vol. 3, No. 2 (March 8, 2006), Enterprise Risk Management & Governance Advisory Service series:  

www.e-expertwitness.com 
www.castellconsulting.com

This article first appeared in July 2006 ITNOW.