The conventional computer-science view is that the scaling of systems is all about computational complexity; about whether the storage or communications required for a task grow more than linearly in the number of users. Over the past thirty years a good theory of that has developed, but it is as-yet nowhere near enough.
Prof Furber argues that the real limits to scale are usually elsewhere. Even where the data are manageable and the algorithms straightforward, things can fail because of the scaling properties of the social context, the economic model or the regulatory environment. This makes some automation projects much harder than they might seem.
Likewise, when it comes to safety and privacy, many of the attacks that are easy to do in the lab are rare in the wild, as they don't scale either. However, others surprise us: no-one in the intelligence community anticipated a leak on the Snowden scale.
In short, scaling is now a problem not of computer science but of systems engineering, economics, governance and much else. Information systems fail more often because of perverse incentives than because of poor design. Conceiving problems too narrowly makes failure likely, while good engineering will require ever more awareness of context. The implications for research, education and policy bear some thought.
Watch Professor Ross Anderson's 2016 Lovelace lecture
Questions and answers
More about Prof Ross Anderson
Ross Anderson is distinguished for his many contributions to building security engineering into a discipline. He was a pioneer of the peer-to-peer movement with his Eternity Service, a forerunner of freenet and gnutella. He was also one of the designers of prepayment electricity-meter mechanisms, used to electrify millions of homes worldwide; and of HomePlug AV, which carries broadband over power lines and is used as a wireless LAN extender.
In the world of finance, he documented many ways in which payment systems can fail. He pioneered the study of API security, which led to the redesign of most of the hardware security modules used to protect bank PINs. He also made important early contributions to the tamper-resistance of smartcards and the robustness of copyright-marking systems.
His highest-impact work has been to the economics of dependability and security. Ross is a Fellow of the Royal Society and the Royal Academy of Engineering, and wrote the standard textbook "Security engineering: a guide to building dependable distributed systems".