There are two competing schools of thought in software engineering that define how to organise software projects: the waterfall model and agile methods. Both of these approaches can arguably trace their origins to the same academic paper.

This single, common source actually describes one of the two in unfavourable terms. Referring to this paper as the source of the model that it describes in this way might be considered somewhat Machiavellian, but perhaps not in the way that we might first expect. Luther Martin examines the origins and interpretations of software engineering.

The term 'Machiavellian' has come to mean behaviour that combines diabolical cunning with a ruthless disregard for morality, but this may be due to the fact that the Niccol˜ Machiavelli's irony was missed by many readers of The Prince.

Supporters of this interpretation note that all of Machiavelli's other writings supported a more republican view of politics, the fact that he had been imprisoned and tortured on the rack by the Medici family to whom he dedicated The Prince, and that Casare Borgia, whom Machiavelli cited as a role model in this book, was widely known as a fool and a failure. If this interpretation is correct, The Prince is probably one of the most widely misunderstood books ever written.

From our point of view, almost 500 years after Machiavelli wrote The Prince, it's impossible to know for sure whether or not he meant the book to be taken literally, but the fact that many people interpret it as a serious guide to politics shows that if he meant the book to be a satire, he was much too subtle. This shouldn't be too surprising, for even less subtle arguments can be misunderstood, and the origins of what's now known as the waterfall model of software development provides an interesting example of this.

In the waterfall model, the process proceeds in an orderly, structured way through a set of steps that take you from defining requirements through to a finished product. Proponents of this model argue that the big, up-front costs that ensure that the requirements and design are correct pay for themselves in terms of greatly reduced time and effort to fix any bugs that may occur. They note that a bug found at design time can be up to 200 times cheaper to fix than the same bug found after software is in use by customers.

An alternative are agile methods. Instead of moving through a single, carefully-considered set of steps, these techniques require more than one iteration through a set of much smaller steps. Agile methods assume that you cannot get requirements and design correct the first time because the requirements will always change over time, and that the development process for software should reflect this reality. Smaller steps let you adjust to changing requirements after each step, while a strict implementation of the waterfall model will unerringly move towards the same requirements that it started with, even if they are later proven to be inadequate.

Winston Royce summarised the lessons that he had learned in large government software projects in his 1970 paper Managing the Development of Large Software Systems. This paper is often cited as the basis for the waterfall model, but Royce's discussion of software development in this paper certainly doesn't suggest that the waterfall model is a good one that should be followed. Although Royce did describe what came to be known as the waterfall model, he also noted that it is 'risky and invites failure'. He also suggested that a more iterative process would be better, a process much like what we call agile methods today.

Thus Royce was even less subtle than Machiavelli - he was fairly clear in expressing his disapproval of what came to be known as the waterfall model. So we might wonder why the model that Royce described as being bad was widely adopted while the agile models, which Royce described as being good, were largely ignored for many years. And although cynics might be tempted to note a possible similarity between Machiavelli being tortured by the Medicis and Royce's work on large government contracts, it's unlikely that Royce's paper was ever interpreted as satire. So we need to find another reason to explain the adoption of the waterfall model.

One reason that the waterfall model was adopted soon after Royce wrote his famous paper may have been that it's much easier to understand and define than agile methods. It's relatively easy to write standards that define the use of waterfall-like methods, but doing the same for agile methods is very difficult. As early as 1974, for example, the US Navy had already used the waterfall model as the basis for their MIL-STD-1679 standard, 'Weapon System Software Development'. By 1995, this had evolved into the ISO/IEC 12207 standard 'Software Life Cycle Processes', keeping a strong bias towards the waterfall model as it did.

But while the more rigid waterfall process has found its way into several standards, the more complicated agile methods have yet to find the same level of acceptance by standards bodies, and this may be because agile methods are more difficult to understand and define than more rigid models are. It may even be the case that even though we can easily list general principles that they may follow, we still don't exactly know how to define agile methods, and this makes creating standards for them extremely difficult.

Many of the early discussions of software development models were based on the experiences in large government projects, where rigid, hierarchical structures tend to be preferred, and this probably led to a bias that favoured such structures in managing software projects.

Software project managers may also have just supported what they believed to be feasible, in the large government software projects, with which they were familiar, instead of promoting what they really thought was the best way to manage their projects. Even if they believed that agile models were better, it may have been the case that it wasn't practical to use these within the constraints of their projects, so that they may have accepted an alternative that was practical for them to use, hoping that it would at least reduce some of the problems that they faced.

The adoption of the waterfall model may also have been a reaction to the more chaotic processes that were common in the early days of software engineering. When these processes were seen not to be working well, a natural reaction may have been to try to impose a more rigid structure, and the waterfall model provided an easy way to formalise this additional structure. So the eventual return to agile models that we see today may have been caused by the realisation that rigid structures also had shortcomings. Because agile models are also not perfect, we shouldn't be too surprised if the pendulum swings back towards more structured approaches in the future. If the past is an accurate guide to this, we might expect this to start in the next 10 years or so.

So it may have been the case that software project managers didn't misinterpret Royce's first discussion of the waterfall model as satire. Instead, they may have adopted the waterfall model because it was relatively easy, fit within the constraints they were accustomed to and seemed to be an adequate solution to some of the problems that they faced at the time. The fact that Royce first described it in a less-than-positive way may have been ignored because of these perceived advantages. So early software engineers didn't develop the waterfall model from Royce's work - they probably developed it in spite of it.

Machiavelli's reaction to this would probably depend on his intent when he wrote The Prince. If he meant it to be taken literally, he would probably admire the audacity that early software engineers showed when they pointed to Royce's paper as the first description of the waterfall model. But if he meant The Prince to be interpreted as satire, he might be surprised by how writing that was even less subtle that his could be misinterpreted by its readers.

Luther Martin works for Voltage Security Inc.