As with all movements, there are different degrees of agile thinking, ranging from the clear life cycle and architecture-based thinking of dynamic systems development method (DSDM) Atern through to what is often perceived as the 'just focus and do it' (JFDI) approach of the extreme programming and other wings of the 'agilista' movement. David Piper lifts the lid off the agile world.
Whatever the perception of these different parts of the agile movement, there is an important common theme, expressed in the agile manifesto, which still underpins and unites agile thinking. These are summed up in four key statements of the manifesto:
- individuals and interactions over processes and tools;
- working software over comprehensive documentation;
- customer collaboration over contract negotiation;
- responding to change over following a plan.
Given the origins of the agile movement, these statements are simply an appropriate reaction to the constraints of overbearing development processes.
Challenges
It is certainly true that many organisations have met with success in adopting agile techniques and working practices. Such successes have often (not universally) been local in nature, where the characteristics of the work - scale, range and complexity - conspire with a talented group of developers to precisely meet the needs of those agile working practices - a round peg in a round hole.
However, adopters of agile thinking have a number of challenges to face if they are to successfully attack the scale, range and complexity of systems development that must be realised today.
Scale
Agile development relies substantially on the skills, abilities, commitment and experience of highly talented individuals. Can agile development be made scalable and resilient?
Range
Agile development works well in environments of uncertainty and rapid change where interactions and collaboration are the essence. Can agile development respond to different commercial situations and project types?
Complexity
Agile development works well when the scale of the development can be encompassed by a small team working in a collaborative, minimal environment. Can agile development respond effectively to complex, large-scale projects?
Redefining agility
Today's definition of agility is great in the 'round hole, round peg' situation. Most development organisations will have some round holes, but the chances are they will have many more holes of other shapes. Anyway, most organisations consider scalability and resilience to be important characteristics of their development capability - being agile is all very well, until the key team members leave...
Can we successfully facilitate a broader understanding of what agility is, so that its core principles can be applied and benefits realised in many more development situations? Such a redefinition of agility would have to take into account the needs of the organisation in addition to the needs of the team and the individual that are emphasised so strongly in the manifesto.
Today's agile manifesto stresses how teams can work very effectively with their stakeholders - but only in a situation where personal endeavour, skill and close collaboration are able to deliver the stakeholders needs. This is a statement of the key qualities of projects that can be successful if agile techniques are used. It is a (high-level) criterion for judging whether agile techniques are suitable for use on any project. Most importantly, by having a criterion at all, we can make the assessment a priori - before the project starts.
Project types
Can we identify such project qualities for other types of projects? Of course we can.
We may have to work with stakeholders who are widely geographically dispersed and are unwilling or unable to collaborate on a continuous basis with the project team. What will help our organisation to respond in an agile way in this situation? The team is going to have to adopt a much more formal approach to the management of requirements, working in more constrained ways to achieve a mutual understanding of what is needed. Management of change will also be important so that all stakeholders are kept consistently aware of change and have the opportunity to buy into that change.
Alternatively, the system to be delivered may be business-critical, with a hard deadline for delivery. Here characteristics such as the robustness of the requirements baseline, control of change, quality of engineering and coverage of testing will all be vital in the delivery of a system that meets its quality criteria. With a hard deadline to meet, prioritisation of requirements will be essential so that the team can actively manage the risk of late delivery. It is precisely in this type of project that some of the best lessons from the agile movement become important. The management of delivery risk by clear prioritisation, time-boxed working, and incremental implementation are vital techniques now widely used across the industry.
The important move we have made in each of these cases is to consider the characteristics of the project, its working environment, stakeholders and characteristics. Choices based on these characteristics determine how the project team is going to work to determine the processes used by the project. Another vital change in our thinking has been to move away from the idea that a single process can meet all development needs. In the agile arena the process will emphasise working with users, fluidity of plans and responding to change. In the more formal arena, the process will emphasise control of requirements, controlled engineering and so on.
With geographically dispersed stakeholders, the process will emphasise controlled working with the users, management of the set of requirements and change. In contrast, the engineering aspects of the process may still rely on agile-based techniques - there are no specific additional needs being imposed on the project.
Finally, to deliver the business-critical solution, not only will requirements have to carefully managed, but it is likely that a more rigorous approach to design and engineering will be adopted. Even here, agile techniques remain vital to help the project manage the risk of a hard deadline.
Project characteristics
So what are these project characteristics that are being considered when an organisation begins to think in this way?
In the CMMI model the phrase 'quality and process-performance objectives' is used. This term is introduced at maturity level 3, which focuses on how an organisation can begin to express its processes in such a way as to preserve the diversity of work undertaken.
Another key concept introduced at this maturity level is the organisation's set of standard processes (OSSP). A set of processes makes it absolutely explicit that a 'one size fits all' process is simply not a practical approach for any but the simplest organisations. It also helps to eliminate an ivory tower approach to the definition of processes. Where is an organisation going to get a set of processes from? From the formalisation and documentation of the real-world experiences of projects that have already been completed.
These two concepts combine to provide a very powerful vision of how organisations can work to meet the needs of the widest possible range of stakeholders. When a project starts, it must select, from the OSSP, the processes that best fit its set of characteristics, and maximise the chances of meeting its quality and process performance objectives. Beyond selecting processes, the project can tailor the process within defined limits, making it an even better fit for its needs - the project's defined process. The project team can plan to deliver their work in a way that is clearly aligned to their own defined process.
Higher levels of maturity
The concept of quality and process performance objectives becomes ever more important as the organisation strives for higher levels of maturity. CMMI uses maturity level 4 to introduce the concept of quantitative management of process performance. Rather than simply choosing processes from the OSSP, based on an ad hoc analysis of the project's needs, the processes are selected according to predefined and quantified quality and process performance objectives.
Such objectives may include the delivery rate, the time variance that is permitted around the delivery date, the latent defect density in the delivered solution and so on. Critically, the project team is being guided in the choice of process by the quantified needs of its stakeholders.
What happens if the organisation has no process that is capable of meeting the stakeholders' needs? This is addressed at maturity level 5 in the CMMI model. The organisation seeks to engineer new processes, techniques and tools if its existing processes cannot deliver the required quality and process performance objectives. This type of change is not just tinkering around the edges of existing processes; it is best represented by the innovation of wholesale change in the organisation.
Scaling agility
The current view of agility is local in the sense that it emphasises the importance of the team's ability to work as it sees fit - it does this without imposing limits on the freedom of the team. In disciplined organisations, with highly motivated and skilled individuals, this approach can work and deliver startling results. In other organisations, the result can be chaos.
From the organisation's viewpoint, this approach is neither scalable, nor resilient. Worse, it does not help the organisation to deliver to all its different sets of stakeholders with the same level of commitment and ability to deliver.
An organisational approach to agility can be seen as the organisation's ability to meet the needs of all of its stakeholders equally well. Such an organisation can certainly equal the ability of an 'agile' team to deliver - since it is highly likely that at least one group of stakeholders will focus on this type of delivery need. Equally, though, an organisational approach to agility will ensure that all groups of stakeholders can achieve their objectives. Not only this, if current processes are not capable of delivering effectively to a group of stakeholders, the organisation is equipped to understand the gap and to change through innovation.
The CMMI concept of quality and process performance objectives encapsulates this measurement of agility. It requires the existence of a set of standard process, avoiding the 'one size fits all' trap. Combined with the organisational and quantitative thinking that lie at the core of the higher levels of maturity, CMMI presents a uniquely powerful approach to achieving true organisational agility.
David Piper is a managing consultant at Lamri Ltd.
