Artificial Intelligence and Software Testing

Building systems you can trust

By (author) Rex Black, James Davenport, Joanna Olszewska, Jeremias Rößler, Adam Leon Smith, Jonathon Wright Edited by Adam Leon Smith

Publication date: 10 Mar 2022

WINNER: Independent Press Awards 2023 - Category: Technology

AI presents a new paradigm in software development, representing the biggest change to how we think about quality and testing in decades. Many of the well known issues around AI, such as bias, manifest themselves as quality management problems. This book, aimed at testing and quality management practitioners who want to understand more, covers trustworthiness of AI and the complexities of testing machine learning systems, before pivoting to how AI can be used itself in software test automation.

The authors of this book are industry experts across the fields of software testing and artificial intelligence.

Dimensions: 244x170mm

Print ISBN-13: 9781780175768

Ebook ISBN-13: 9781780175782

150 pages

Imprint: BCS, The Chartered Institute for IT

  1. Introduction
  2. AI Trustworthiness and Quality
  3. Quality and Bias
  4. Testing Machine Learning Systems
  5. AI-based Test Automation
  6. Ontologies for Software Testing
  7. Shifting Right into the Metaverse with Digital Twin Testing
A brilliant reference with a focus on introducing the reader to new AI ideas and challenges. The danger with AI and software testing is the mistaken belief that people understand it all. This book addresses this issue by opening the reader up to a rich source of references & useful concepts using use cases, models and references, to both stimulate and challenge the reader's own knowledge of this broad subject. Highly recommended.

Paul Mowat MBCS CITP, BCS SiGIST Social Media Secretary & Committee Member, Quality Test Engineer Director, Deloitte UK

AI-based systems conquer more and more areas of our daily life. People are concerned whether these systems are trustworthy. 'Artificial Intelligence and Software Testing' tackles this issue and provides an insight into AI quality and how it differs from conventional software quality, and where the difficulties and challenges are in testing machine learning systems. A great introduction into this topic and must read for all interested in building AI-based systems that you can trust.

Klaudia Dussa-Zieger, Chair GTB & Vice President ISTQB®, Head of ISTQB® Certified Tester AI Testing (CT-AI) taskforce

In the ever expansive and evolving virtual domain, the prominence of AI is becoming more and more prolific, and this evolution will not be without its challenges. This title provides an excellent resource into the potential dilemmas faced in this evolutionary field as the virtual, cognitive, and physical spaces become more interlinked with the dawn of the metaverse. The part that humans play in the growth, development and testing of AI is discussed. Supported by a wealth of experience, research, and evidence from the authors, the title provides a great introduction to and understanding of AI and software testing. Highly recommended for all with an interest in this area.

Jonathan Miles MBA BSc(Hons) FCMI, Head of Strategic Intelligence, Mimecast

Shift Right! A concept you won’t find in ‘The Seven Principles of Testing’. 'Artificial Intelligence and Software Testing' puts the principles into perspective. Not only does it explore early testing, but it also looks at the concept of exhaustive testing thoroughly and effectively. As a trainer of software testing I will definitely use all this book has to offer. Guiding the next generation of testers to question the intricacies of machine learning. A must for anyone in tech, not just software testers.

Rachel Hurley MBCS TAP.dip, Technical Trainer (Software Testing)

As the title describes, this book is a robust AI and ML testing exploration that also dives into the juxtaposition of the trustworthiness and bias in AI systems. It touches on the basis of ontologies and how to enable the considerable impact of testing and monitoring of AI-based systems. After reading this you would be able to answer an important challenge: how to determine that your AI system has been extensively tested?

Dina Dede, AI/ML and Cloud Architect Lead, UK

This book beautifully captures the game-changing complexity of artificial intelligence (AI) and the traditional discipline of software quality management. It is a comprehensive manual addressing the conundrum and tantalizing promise of both disciplines with good pace and a distinct future-present context. Forget waterfall and DevOps, we’re right shifting into OpsDev, AIOps and digital twins in the metaverse, so things are about to get a whole lot more interesting. Excellent effort, and a much-needed treatment of this topic by true experts.

Jude Umeh FBCS CITP, Senior Program Architect, Salesforce

'Artificial Intelligence and Software Testing' is a great read. The vast experience of the authors is evident as they comprehensively explain the challenges and benefits of not only applying AI to testing, but also testing the AI software itself. I found the insight into the shift-right approach and its application during the development of the test and trace application fascinating. A must read for any testing/QA professional plus any C-suite looking to rapidly increase their ROI on testing.

Anil Pande, Managing Partner, TestPro Consulting Ltd

This book is a very good introduction to using AI in software testing as well as testing AI systems, covering several relevant topics like societal risk, bias, ethical behavior, quality, trustworthiness, and the problems associated with AI/ML systems. I specifically liked the section that details on the problems associated with AI/ML systems. I would recommend this book to anyone who is starting their study on software testing vis-a-vis AI/ML systems.

Venkat Ramakrishnan, Software Quality Leader And Software Testing Technologist

'Artificial Intelligence and Software Testing' is a valuable resource for anyone curious in how to approach testing AI models as they expand into our daily lives. This is a clear, informative read which discusses within each chapter different testing challenges with AI software and advice on how to handle them effectively. I can highly recommend this to testers and students alike.

Katy Hannath BSc(Hons), MSc in Artificial Intelligence and Data Science student, & Quality Assurance Tester, VISR Dynamics

This book is an exceptionally practical resource which is a remarkable reference guide to understanding the foundations of AI & ML for anyone wishing to build a career in AI or define a test approach. It has a clear, direct, and concise explanation of AI, ML, ethics, ontology, quality, bias, challenges, test automation, and the significance of ‘shift-right’ testing. It offers thorough, data-driven and real-world examples that bring together the rich wealth of experience from these expert authors and authorities in this area.

Boby Jose BSc MBA MBCS, Author of BCS publication ‘Test Automation: A manager’s guide’

What an exciting and relevant publication! Beyond the positive game-changing societal benefits delivered by AI, it has proven equally disruptive to all aspects of software engineering including software testing. This book provides great insight into new build and test design techniques to augment our traditional thinking. An essential guide for technology leaders and test professionals alike, looking to understand how to approach the critical problem of building and testing today’s complex and often unpredictable AI systems.

Jack Mortassagne, Director at Cigniti Technologies and TMMI Accredited Assessor

This is a great book for those who want to gain more insight into how AI will affect the software testing profession. The writers introduce the challenges in AI in an easy-to-understand manner, while the case studies showcased are extremely interesting and contemporary, clearly exemplifying the topics presented. Brilliant read and highly recommended!

Dr Diana Hintea BEng(Hons) PhD SFHEA, Associate Head of School (School of Computing, Electronics and Mathematics), Coventry University

In a time the promised paradigm shift of Artificial intelligence is starting to have a real-world impact, this is a vitally important book. It explains the social, ethical, and technical concerns around AI in an easy to understand way, making a complex subject easily accessible. Everyone involved in IT is likely to be impacted by AI whether from a business, technical, ethical, or quality point of view and so this book will be an invaluable resource for everyone in IT. As a Testing and Quality specialist, this is going to have pride of place on my bookshelf as a practical, real-world reference for helping me navigate testing and quality in the emerging world of AI.

Bryan Jones MBCS, Director of Testing Practice, Sopra Steria Private Sector

This book is a must-read for anyone in software testing with responsibility for quality assuring AI technology that must engender public trust. With topics that feel both familiar and challenging, the authors confidently explore a range of subjects to broaden and deepen the reader’s understanding of the intersection of AI and testing.

Bronia Anderson-Kelly, IT change consultant, Sabiduria Ltd

This book addresses an often ignored but critically important aspect of AI implementation: how to ensure that AI's are producing, and continue to produce, acceptable output. As AI is non-deterministic and complex, and dataset quality can be highly variable, it is notoriously difficult to determine suitable test cases for modern systems. In this book, the authors provide practical methods and examples that can be used to ensure AI quality, and as such is an extremely useful resource for anyone implementing systems involving AI and machine learning.

Peter Brightwell MSc, Intelligent Automation Architect, NDL Software