Automation - Is it the right way to Y2K testing?

He also reinforces the need to perform a structured evaluation before purchase of an automated testing tool and focuses on the skills needed to make automation a success.

Many organisations are turning to automated testing tools to solve the problem of too much testing in too little time. Indeed tool vendors have experienced a sharp upturn in sales as organisations seek to squeeze as much testing in, between now and the year 2000.

On the face of it, year 2000 testing seems like an ideal project to utilise automated testing tools, stable code, little or no changes to functionality etc. In fact the time pressures of completing your YK2 compliance tests in time for the millennium makes the use of automated tools more attractive than ever.

There is a temptation to see automated testing as the key to solving the problem that virtually all organisations are facing with testing for year 2000 compliance i.e. too much testing but not enough resource. Often the fear of the millennium bug is used as justification for the budget to buy an automated tool.

It is true that the use of ATT can greatly improve your ability to perform the amount of testing required to ensure compliance. However, beware, do not let the need to increase testing capacity mask the realities of successful implementing an automated test tool. The same rules apply irrespective of whether you are testing functionality of a new system or for year 2000 compliance.

Take the time to conduct a proper evaluation, if possible this should be independent of any tool vendor. Remember tool vendors are in the business of selling software, so whatever the question their likely answer will always be "buy the tool". Whatever your technical environment or testing needs are, they will have a success story to match it.

Work out exactly what you need from an automated test tool. A structured evaluation can usually be done in 5-10 days, by which time you should have a good idea of:

• which areas of testing and on which systems would derive the most benefit from automation;
• what tool best suits your technical environment and testing needs;
• an understanding of what the resource requirements are in term of effort and skill profile.

If automation is going to play a key part in your Y2K strategy then this initial study could be the difference between compliance and non compliance.

Test automation often ends in expensive failure due to lack of analysis, preparation, and planning. But probably the single most important factor that is overlooked when selecting and implementing an automated testing tool; is the skill set of the people who will be using it.

All too often the assumption is made that you can take existing test team members and after a two-day course on the chosen test tool they will then be able to automate their manual tests. This simply is not true.

Recently at a client site I overheard a salesman from a well known test tool vendor explain how their tool was so easy to use that they had recently trained a secretary whose only previous IT experience was using word processors. He enthusiastically explained how she was now producing reusable automated test scripts - ummmm.

Anybody who has had any degree of success with automated test tools will tell you that to build robust automated test scripts requires a unique skill set. Remember "coders do not make good testers, and testers generally do not code".

Therefore what is required is a person with a good understanding of the testing process, but who also has an appreciation for programming. This does not mean that they have to be a competent programmer, more that they understand common programming techniques e.g. What objects or variables are, or how to programme a loop. This combination of testing know-how and programming appreciation gives a good platform for automation.

Many sites that I visit have chosen a test tool with the assumption that most tools do more or less the same. However after a very short time it is obvious that through lack of knowledge or time the evaluation stage has been skipped and the tool has been bought. Often the result is that the client reverts back to manual testing within 3 months.

Recently did a question and answer survey of clients who are using automation as part of their Y2K strategy, which produced mixed results. Below is the response from Ann-Marie Lamming of ICL Sorbus.

Q What automated tools have you used for Y2K testing?
A Ascent Logic and Mercury Interactive Tool Set.

Q On reflection do you think that using an automated tool has saved you time, or do you think that you could have achieved the same with manual testing?
A I don't think we could do it without, particularly as staff seem to be difficult to keep!

Q Will you continue to use the test tools after Y2K compliance?
A Yes, that was the justification for the investment.

Q Would you recommend other organisations to buy and use automated test tools for Y2K compliance testing?
A Yes, no reservations. Not just compliance, but regression generally.

Her answers confirm that using the right approach and understanding what your needs are before you purchase an automated testing tool can pay dividends.

John Reilly, Test Automation UK Ltd