One half of the author team behind our User Acceptance Testing book, Pauline, has more than 17 years’ experience of change management, testing and training. We wanted to know a little more about her thoughts when it comes to the UAT process.
What do you think are the biggest challenges when it comes to UAT?
There are lots of potential challenges, but the top one for me has to be maintaining the will to do UAT right when everyone is battle weary and the project is so close to the finish line (or way past the finish line and over budget).
The other is thinking that you can just ‘do’ UAT without some serious planning and preparation. It is key to involve the end users as much as possible in the creation of scenarios, use cases, scripts and in the selection of the test data - if you have BAs working in isolation, it is likely that the quality of the testing will be suboptimal and that the users will spend a lot of time in UAT pointing out the flaws in your scripts and data.
Test data is also a potential stumbling block and many projects I have worked on will aim to create a copy of the live data for testing without considering whether this will infringe the data protection act or will compromise sensitive business information. If it is essential that live data is used data masking will at least depersonalise your data while retaining its properties.
Is there such a thing as agile UAT?
Interesting question. Historically, when we've thought of UAT it's generally been in the context of the waterfall model, where at the end of successful system testing we need to make sure that what has been created meets the original requirements the business had. In Agile delivery methods such as Scrum, every iteration has 'stories' that have their own acceptance criteria. The product owner (or SME or end users) is then asked to accept each iteration / against those criteria. In this sense acceptance testing / UAT is happening all the time. There will almost always be a final phase of UAT required at the end of the project to make sure that processes can be carried out from start to finish without major incident, but the ongoing, continuous acceptance of working software by the business reduces the risk of major issues being found so late in the day.
What for you is the biggest myth about UAT?
I actually have a top three:
- That UAT has to use detailed step-by-step scripts
A useful addition to detailed scripts is the high level script that simply says ‘book a ticket’ or ‘create a standard contract’. Often the best feedback comes from these informal tests because it gives UA testers the chance to use the system the way any user might, without the constraints of a script and to use data they know is likely to be problematic. High level scripts are particularly useful if users were not involved (or not involved enough) in the UAT preparation. Some instruction may need to be given to users on how to record incidents arising from these tests accurately so that they may be replicated by the project/dev team. - That training for UAT is unnecessary
Training may seem like an undesirable undertaking if it teaches testers processes that they should be testing. Training is actually very beneficial to the UAT process and should teach UA testers: what UAT is, what they can expect from the UAT process, how to record incidents, what the project is trying to achieve and the known issues and their workarounds. - That UAT can be done without users
The question of whether you can carry out UAT without users is hotly debated on the internet as if it was a question of opinion such as ‘which is better; PC or Mac?’ Many people will argue that the end users have not been available and so UAT was carried out by BAs or by another third party. It is certainly possible to do some kind of acceptance testing without the user, but UAT is only UAT when it involves the user. The clue’s in the title really...
Our book had a great review from testing practitioner, Peter Morgan: ‘This book fills a gap on my bookshelf, a volume I wish I had years ago.’ What was the last book you put on yours?
I was so thrilled that Peter Morgan wrote the review he did. Brian and I tried to write the book that we wished we had had when we started out in UAT and from the positive feedback we’ve had to date I hope we’ve been successful.
I recently bought ‘don’t make me think’ by Steve Krug about website usability and it made me laugh out loud on nearly every other page. Very useful and very funny.
I also keep coming back to ‘Great demo!’ by Peter Cohan. I don’t do software demos any more but, like a lot of people, I’ve been to a few. Peter’s approach of showing the ‘ta-da!’ part of the demo first and then showing the audience how he got there really resonated with me and, I think, translates well for making IT training more interesting.
And finally, if you could give your younger self one piece of career advice, what would it be?
I don’t think I have any real regrets, because I think I’ve learned something useful from every role I’ve undertaken, but I do sometimes wish I had studied something more creative like graphic design. I have a secret passion for design and I spend a lot of my spare time creating slideshare presentations on topics that are relevant to my job - including one on quick ways to improve UAT! I also fancy myself as something of a programmer, but I don’t think I quite have the capacity for 2am code hacks.
Are there any other UAT challenges or myths that you would add to the above.
