Introduction
Back in Alan Turing's day computers were less about bits of metal, glass and silicon and more about humans, with a female edge, as most were referred to as 'she who computes'.
Nowadays, according to Bjarne Stroustrup (the designer and original implementer of C++), civilisation runs on software. It effects how we go to work, how we govern countries, communicate, how we eat, sleep and play.
In fact, there are approximately 15 million software professionals currently operating worldwide, 30 percent of which are 'code warriors'.
If each of these is writing between 5 and 10 thousand lines of new or modified code each year then roughly 33 billion lines of code are being written each year, which means that literally trillions of lines of code have been written cumulatively.
Hence, it can be safely suggested that software designers are amongst the world's most important individuals creating opportunities for others worldwide with their vibrant output.
The promise
Nobel Prize winner Lech Walesa once wrote 'You have riches and freedom here but I feel no sense of faith or direction. Why don't you use computers in search of love?'
What software designers do as an industry impacts on the human experience (industry and culture) and upon the human heart.
Software can amplify human intelligence but there are limits. Software can never replace human judgement and can not create knowledge in itself. Software is not just an intensely technical thing but a moral and ethical one too.
The limits
We can have amazing vision but turning it into reality is another thing. Not everything we want to build can be built and not everything that can be built should be built.
Software designers should not ignore the other implications of their work, whether that is political, economic, or environmental. From time to time designers should step outside the boundaries and think creatively and ethically.
The limits of software are regulated by:
- the laws of physics;
- the laws of software;
- the challenge of algorithms;
- the difficulty of distribution and concurrency;
- problems of design;
- importance of organisation;
- impact of economics;
- the influence of politics;
- the limits of human imagination.
The laws of physics
These include the pragmatic implications for distributed systems. Engineers need to consider:
- Relativistic effects (no such thing as absolute time)
- Quantum effects (theoretical as well as practical limits to information density)
- Thermodynamic effects – software is weightless, but containers for software are not, and computation itself produces heat; elements which are significant in astrophysics.
Software laws
- sometimes we can't do it, and reach a 'halting' problem;
- Gödel's theorem/highly non-compatible problem;
- sometimes we can't afford it;
- sometimes we just don't know.
The challenge of algorithms
- compression, photorealism rendering;
- speech recording;
- simulation;
- knowledge representation;
- intimate/massive parallelism.
Computer graphics can't do faces yet as this requires a truly kinetic model of human facial features. Mankind has evolved to develop cues on how to recognise real faces or at least how real faces should look, hence are not easily fooled.
The difficulty of distribution and concurrency
'A distributed system is one in which the failure of computing you didn't even know existed can render your computer unusable.' (Leslie Lamport).
No software is an island unto itself. Software integrates with other pieces of a system. The average software developer doesn't have, as a core competency, the ability to develop secure, concurrent and distributed systems.
Problems of design
- entire history of software engineering is that of the rise in levels of abstraction;
- the limitations of human understanding;
- building for resilience;
- sometimes worse is better;
- the discovery of patterns.
The problem is that designers don't know yet how to fully exploit their system's power/capacity. The halting factor often reveals a systems' inability to exploit capabilities. The best systems are plagiarised from prior working systems where new improves on old in a form of artificial evolution.
The entire history of software engineering is marked by the rise in levels of abstraction, greater understanding of the limits of human understanding, the building for resilience and the discovery of patterns. So many of our current systems are unpresidentated, meaning that there is no legacy from which we can learn.
Importance of organisation
One of the most important discoveries in recent years is that of patterns. It is by understanding these patterns that we can produce better software. Patterns also allow us to reuse, which in itself can be more efficient.
All meaningful development is formed by the resonance of activities that beat at different rhythms. Most are based on the activities of one developer. Vitally important are the social dynamics among a small set of developers and the dynamics of teams of teams.
Impact of economics
Performance = (complexity) (process) * (Team) * (Tools)
- Performance is represented by effort/time;
- Complexity by volume of human gene code;
- Process by methods, notations and maturity;
- Team by skill set, experience and motivation;
- Tools by software process automation.
Influence of politics
Success defined by a software development team is sometimes misaligned with success as defined by the management team. Frequently software is used as a strategic weapon, or as a pawn.
Managers often say 'the company needs to get a foothold in the marketplace', hence force their developers to rush things through with negative consequences.
Limits of human imagination
Software development has been, is, and will remain fundamentally hard. Imagination is our friend to be harnessed and used well, but there is nothing on the horizon which will make programming fundamentally easier. Software should be seen as our friend, but only if it is used well.
The beauty
There is significant beauty within software engineering. Software intense systems are perhaps the most intellectually complex artefacts created by humans even though the vast majority are invisible and in a way it should remain so.
The best software is simple, elegant and full of drama, manifest in cunning patterns that form their structures and command their behaviour. Systems are becoming more complex, and often try to overcome complexity with more complexity. A bad process itself can amplify complexity.
We must, therefore, be masters of that illusion of complexity. Developers should endeavour to produce simpler software, which itself increases productivity, or not write any new software at all – let it all progress through future reuse.
Software developers seem to love complexity, which often thrives because of the immature processes within companies. We need to take a step back more often and simplify things. In fact 10 percent of the time should be devoted to de-factoring and simplification.
Turing realised that there are limits to artificial intelligence and the human brain. He also observed the beauty of maths, in the under-pinning of code - a cold and austere type of beauty admittedly. He said 'Maths, rightly viewed, possesses not only truth but supreme beauty, a beauty cold and austere like that of a sculpture'.
Contradictions
The internet has changed the way that individuals communicate and businesses collaborate. However, there is a dark side to the web, namely terrorism, exploitation of children and fraud.
The web has provided unprecedented opportunities for social networking but simultaneously has created opportunities for crime. Such connectivity has also permitted real time and distributed access to information which can erode personal privacy.
Software has created new forms of artistic expression but piracy can disrupt this artistic endeavour. In fact it has helped to erode the rights of the artists.
Email and other software can increase the velocity of communication but threaten the preservation of history due to the state of our aging digital archives, or in some cases the lack thereof.
We could be faced with a whole generation of activity which could fade away due to being saved solely on digital media, which only has a limited lifetime before it deteriorates.
Software has also enabled and accelerated scientific research including human genomic research, but has also created new weapons and a new war - namely cyber-warfare.
Turing's own contradiction was that despite his work in crypto-analysis, which helped the war effort, he himself was oppressed by the society he saved because he was a gay man in an anti-gay world.
One of the greatest challenges the industry faces is the need to talk about its ideas and mistakes so that through greater openness we can all successfully move forward.
Software developers often 'grow up' in one domain without any knowledge of the others. Ideally, we should study hundreds of systems and extract patterns to learn from each of them.
Software timeline:
How we got here:
1910s: beginning of automation
1920s: beginning of expansion
1930s: beginning of dependence
1940s: beginning of von Newman machines
1950s: rise of the machines
1960s: death of the mainframe
1970s: rise of languages and methods
1980s: age of the PC
1990s: age of internet and new methods
2000s: retrenchment
The future
Current state:
We can never 'turn off' now – our systems are continuously evolving, are connected, distributed and concurrent. They need to be multilingual, multiplatform, autonomic and secure.
There is a need for greater transparency in order to improve the efficiency of future software. It is likely that our civilisation will be totally dependant on software within the next decade. Most systems are actually systems of systems where services and other messaging mechanisms dominate.
We need to build a sustainable communication network. Behind the preservation of classic software there is a need to have a museum of software, not just old PCs but the software that ran on them. There is a need to establish the source code for Microsoft within this museum for future generations.
There is a disconnect between how students are trained and what is really required in the real world. There is also a need for more abstraction, to study other developers' code so one can learn from their mistakes as well as one's own.
Additionally, team skills are under taught, hence there is a requirement to stretch the curriculum when it comes to training both at college and at work.
Future timeline:
2010 – age of transparency
2020 – total dependence
2030 – rise of the machines
The rise of the machines can be predicted by looking at probably the most mechanised nation on earth - Japan. This highly technological country has a shrinking population with an ever increasing dependency on machines and technology.
The aging population will become dependant on robots to assist them in their twilight years since many don't have children. Hence, they are a good indicator for our future as a technological culture.
Other predictions
Software will continue to expand human experience. As such Grady felt that it is a tremendous privilege to be a software professional, changing the way the world works and the way civilisation develops.
However, developers have an extreme responsibility to get things right otherwise the far reaching effects could be devastating. Software has the ability to influence government policies, but can also be a double edged sword.
Software developers, therefore, have a responsibility to consider the moral and ethical implications of the software they are involved in; much of which underpins modern society's infrastructure.
The science of systems engineering is evolving and will continue to do so, for example, the embedding of software into a machine's engine, e.g. ship building.
Will software engineering be replaced by software? The answer Grady surmised has to be 'yes', via compilers, which will change the nature of programming, but these could also put traditional workers out of work.
Grady himself felt that the industry had been fortunate not to have killed more people through the software it has produced and that the software industry has not serviced business or individuals well and he went on to apologise for it.
Final points for software improvement
- grow a system's architecture through the incremental and iterative release of testable executables;
- those things which change should be kept separate from those which have changed;
- to build simple systems requires injected energy to make it simple (the number of lines of code should decrease).
