His popular first book, Managing the Testing Process is now in its second edition, his latest book, Critical Testing Processes was published at the end of 2003.
Black is the president of the American Test Board, and president of the International Software Test Qualification Board.
Your career in software quality and testing is exemplary - you've written two bestselling books on testing and you've also recently been elected president of the International Software Testing Qualifications Board. Congratulations! How did it all start?
Thank you! This [ISTQB Presidency] is a big job…
I started as a programmer, and I remain not such a bad programmer. I can write decent code, but I find programming quite frustrating because, say, of my inability to write perfect programs, and I don't really enjoy debugging.
I did a systems administration and porting job for UCLA, and learnt a lot about hardware-specific issues in that job. I got started in testing in 1987 when I was hired by an independent test lab to develop UNIX test scripts.
I worked for a few years as I finished my degree. I worked afterwards for a company that had great technology but a bad business plan. I took another (excellent!) testing job with a company that had great technology, but once again a bad business plan…
I was out of work for the second time in two years, so I decided to try consulting, which would allow me to make my own business decisions. It was a major leap forward. I then had the chance to write my first book.
It was a huge endeavour, with challenges an order of magnitude larger than writing an article, but it was a huge differentiator for my career. I had the idea in 1997, the contract was signed in 1998, and the book published in 1999.
In 1997, I was just another guy providing contract test services, working with a number of associates. Writing a book inspired me to grow my business, and when it was published, it pushed it forward. All this made a big difference when we had to face recession…
Testing seems to be a very good market to be in nowadays. Gartner estimates the total size of the software quality market at $13bn, and IDC estimates the testing services market at $2.5bn.
These figures are huge when one thinks that, traditionally, testing has been an afterthought on many IT projects, with little budget allocated to it. Can you provide some insight into these figures? Is something 'big' happening?
One of the biggest things happening is India. There are a lot of Indian companies providing testing and other services.
Nowadays, there's a place for commodity testing services. Not all Indian companies provide high-end services, but there is also a place for such services, and I want to be part of that. Higher end services seem to be picking up.
Test automation tools take a big share of the total market size. What are your views on automation?
I recently had the opportunity to talk about advances and non-advances in quality and testing, and commercial automation tools [working at the graphical user interfacing (GUI)] fall under the non-advances category.
I'm personally disappointed with commercial tools, and this is a general rule. They seldom work as advertised. The non-traditional approach to automation, however, is promising.
There are some freeware and open-source building blocks available out there, and their potential looks great. By the way, I think GUI automation shouldn't be called the 'traditional approach' - I was doing UNIX-based script-driven automation way before Mercury Interactive existed…
You need to look at testing from different angles - and there is a way out of GUI automation. Automate at the GUI if you must, but consider lightweight scripting languages. GUI tools are over-used. They set test automation back by a good 10 years.
By focusing on GUI automation, you forget about command line and application program interface automation. We're just now rediscovering these forms of automation, but we're not building on a foundation.
Let's talk for a moment about 'context-driven testing', which seems to be the latest buzzword in the field. What is it about?
I won't try to define context-driven testing for people who claim it. I'm sceptical that it is truly something different, from my studies of it. To say you're context-aware is like saying 'I breathe oxygen', but what does context-driven mean?
Only morons close their eyes to the situation where they are. If context-driven is making decisions based on where I am, then we all are - or we are idiots.
So, the cynic in me says that context-driven is a mixture of new wine in old skins combined with an attempt to claim common sense for one side as opposed to all others.
Regarding the first point, there seems to be an approach to consulting that involves inventing something as opposed to building on top of something else.
I have the view that you're better off building on top of someone else's work. Regarding the second point, I don't feel I have a monopoly on common sense, but I don't feel that I am an idiot, and conversely I don't feel that others who disagree with me are idiots either.
I would like to see a lot more civility in the discussion about approaches to testing than some who call themselves context driven bring to the debate.
What about the context-driven view that there are no best practices, only good practices?
I utterly reject this! I'm glad you brought it up… There's a great discussion in a book by Caper Jones [about the meaning of the phrase 'best practice'].
He uses an analogy, writing that it is a best practice to treat particular types of infections with penicillin; however, for someone who has a penicillin allergy, it would be a reckless thing to do.
What Caper Jones means by best practice, and what I mean, is that a best practice is a damn good idea most of the time - but not under all conditions. Only morons would blindly apply the same approach under all conditions disregarding any additional factors…
I am fortunate to have worked in different countries with different people and different clients. There are, of course, differences - but a lot of common problems too. Best practices aim at tackling just that.
A risk for context-driven people is to become overly fascinated by the differences. I believe that one should intelligently tailor best practices to take the differences into account - this is being 'context-sensitive' maybe, not 'context-driven'.
Forrester recommends that test outsourcing be done for quality gains, not only cost savings. Is this really possible when testers and end users are not under the same roof, let alone thousands of miles apart?
I believe it's possible to have a combination of on-site and off-site services. Some parts of the testing process can be systematized, and it doesn't matter where those parts are done. Other parts such as risk analysis and planning involve building a consensus, and should be done face-to-face.
I believe there's a place in the testing world for people who don't have a deep expertise in testing - handling those tasks which can be systematized - and this is another difference I have with some in the context-driven school.
I was initially very sceptical about outsourcing. This was grounded in fear - the economic consequences on my personal life. But then I thought about it more dispassionately.
Capitalism is a machine to maximize return on investment; for example by cutting costs. The wage differences between China, India or other outsourcing countries and the US is so big that capitalism will drive this industry through, over, or around any obstacles.
If you stand back and thoughtfully enumerate the obstacles, you'll see they are all surmountable with patience and due care. I've been able to solve those outsourcing problems for past clients.
You can think of the wage differential as the locomotive of a train of change. As software and systems professional, each of us can choose to be on the train riding it to the next place in our careers in this field, standing in front of the train yelling stop, or walking away from the track to some other career in some other field.
What other recommendations would you give to enterprises contemplating test outsourcing?
Take the challenges seriously. Remember Fred Brooks's silver bullet metaphor: there are no easy solutions to hard problems.
Outsourcing has gotten ensnared in the same silver bullet hype as object-orientation, fourth-generation languages and reengineering. So, there's a hype issue, but companies that can separate the good ideas from all this hype, will do well.
I know outsourcing works - I've made it work for companies like Dell and Hitachi and Hewlett Packard. I solved the problems that came up; we surmounted the obstacles.
In your view, what countries or regions offer most benefits for offshore operations?
India had the advantage of language and legal system similarities with the US and the UK - but there are some intellectual property rights issues. China has enormous legal and intellectual property rights issues as well as the language problem.
I haven't visited Vietnam but I guess they would have the same legal problems as the Chinese. It would be interesting to see the level of fluency in English there.
South Africa could be a powerhouse, with English fluency, an English-derived legal system and low labour costs. Lots of countries could be powerful.
What about the 'sleeping giants' in South America and Africa?
Any country that has English as a native language has an easier time of it than one that doesn't. South America has to deal with that issue one way or another to become a major player, at least outside of their region and perhaps Spain and Portugal.
There are several outsourcing houses in Argentina, Colombia, and so on, with people speaking English, and it is not inconceivable that they become big players.
The potential is there. Reliable infrastructure is essential, however, to make it work: phone lines, airports, road and electricity all influence your ability to compete.
Before we wrap up, would you like to tell us about your forthcoming book, Effective and Efficient Software Testing?
In response to customer demand, I developed a course explaining the essentials of testing. It has grown over a decade, and it is very broad in scope. While the book's scope is not as broad, it is deep on topics important to test engineers.
The book is for them. There's a series of chapters explaining major topics, followed by a chapter with exercises and detailed answers. You can't be a good test engineer if you don't get your hands dirty by using the techniques to solve realistic testing problems.
The approach used makes it easy for anyone to understand the underlying concepts. While I value Boris Beizer's books [on test design], which are very good, they are very difficult to master for the average practitioner. There's also Lee Copeland's book, which I also like, but have heard people say does not go not deep enough.
My point is that there is room for a book on test design techniques that is broad and deep at the same time.
A final thought on software quality and testing - maybe your vision for what the future holds?
I'm working very hard with several professional bodies, including the BCS, ISEB, the American Software Testing Qualifications Board, and the International Software Testing Qualifications Board, to get a consensus on standards for professionalism at entry-, mid- and senior-levels. I'm trying to avoid fragmentation and I hope to establish universally-accepted standards of professionalism within five years.
Rex Black was interviewed by Youssef Bouguerra, BCS North London Branch secretary.
