Simplification and integration in AI

Gerry Wolf explains how the simplicity and power theory of artificial intelligence may help solve problems in computing.

Much research in AI adopts a depth-first strategy: start with one area of AI, such as reasoning or computer vision, and then try to extend it to other areas.

By contrast, research developing the simplicity and power (SP) theory of intelligence has adopted a breadth-first strategy: try to develop a framework that has the potential to work well in several different areas of AI (perhaps with related fields), and then try to deepen it across all areas.

In the SP programme of research, the overall aim has been to simplify and integrate observations and concepts across AI, mainstream computing, mathematics, and human learning, perception and cognition. Despite the exceptionally wide scope of the research, the quest for simplification and integration has proved to be very successful.

The product of about 20 years’ research is the SP theory itself, expressed in a computer model with some mathematics, a verbal description of the theory, a book, and several peer-reviewed papers, which are mainly about potential applications of the theory.

The term ‘SP’ is short for simplicity and power, partly because those two words, for technical reasons, mean the same as information compression - something that is central to the workings of the SP system - and partly because they describe how a good theory should combine simplicity with descriptive or explanatory power.

Why should information compression be central to the workings of the SP system? This is mainly because there is a lot of evidence - too much to review in this short article - that much of human learning, perception and thinking is about the compression of information.

SP-multiple-alignment

A central idea in the SP system is the powerful concept of SP-multiple-alignment, borrowed and adapted from the concept of multiple sequence alignment in the branch of biochemistry that is concerned with the processing of information.

An example of that second kind of multiple alignment is shown in the figure (see below). Here, there are five DNA sequences laid alongside each other so that, with judicious ‘stretching’ of sequences in a computer, matching symbols are brought into line.

With some changes to the basic idea, the concept of SP-multiple-alignment has proved to be a means of representing several different kinds of knowledge. It also has strengths in several different aspects of intelligence, including pattern recognition, the processing of natural language, different forms of information retrieval, several different kinds of reasoning, planning and problem solving.

SP-multiple-alignment is also central in the unsupervised learning of new knowledge, meaning learning without the aid of a teacher or anything equivalent.

Because all these things flow from one relatively simple framework, there can be seamless integration of diverse kinds of knowledge and diverse aspects of intelligence, in any combination.

That kind of seamless integration appears to be essential in any system that aspires to human-like versatility and fluidity in intelligence. It seems likely that, without it, AI systems will struggle in the human master class of common sense knowledge and common sense reasoning, making use of varied kinds of knowledge, and varied aspects of intelligence, according to need.

DNA diagram

How the SP theory may help solve problems in computing

In keeping with the words attributed to psychologist Kurt Lewin: ‘there is nothing so practical as a good theory’, there is a cascade of potential applications that flow from the SP theory - a bit like hitting the jackpot on a fruit machine.

For reasons described in, including the versatility of the SP computer model across several aspects of AI, and in the representation of several different kinds of knowledge, the SP system provides a much firmer foundation than the main alternatives for the development of general, human-like AI.

One of the main alternatives to the SP system is the popular deep learning approach to AI. In that connection, section V of describes thirteen problems with deep learning and how, with the SP system, those problems may be overcome.

The SP system also provides a robust solution to a fourteenth problem with deep learning, a problem called ‘catastrophic forgetting’, meaning the way in which deep learning systems forget what they already know when they learn something new.

How the SP system may help to solve nine significant problems with big data is described in. Those problems include the problem of variety in the many different formats and formalisms for representing knowledge; the problem of volume in big data, which threatens to overwhelm even the most powerful computers; and the problem of visualisation of knowledge structures created by the system, and inferential processes in the system.

In connection with big data, the SP system provides a solution to two of the problems mentioned in volume 59, issue 3 of ITNOW:

The problem of ‘combining machine learning and reasoning’ (p. 17 of ITNOW). The SP system provides a solution because its overall design, with its focus on simplification and integration across several areas, provides for the smooth integration of machine learning and reasoning.
The problem of ‘how to combine sub-symbolic AI with symbolic artificial intelligence.’ (p. 17 on ITNOW). Here, the SP system provides a somewhat different kind of answer: that the distinction between ‘symbolic’ and ‘sub-symbolic’ AI is one of granularity. At ‘low’ or fine-grained levels, processing in the SP system will have a sub-symbolic flavour, while at ‘higher’ levels, the kinds of ‘symbolic’ objects and classes that people recognise will come into play.

Other potential areas of application for the SP system include: the development of human-like intelligence in autonomous robots; serving as a model for a ‘neural’ version of the SP theory called SP-neural; computer vision and the modelling of natural vision; serving as a database system with intelligence; assisting medical practitioners with medical diagnosis; and others described in.

Future directions

Things to be done in the future development of the SP system are described in6. If you would like to contribute to the development, please get in touch via jgw@cognitionresearch.org

With Dr Vasile Palade, of Coventry University, an aim now is to create a high-parallel SP machine, based on the SP theory and the SP computer model, and hosted on an existing high-performance computer. This is intended mainly as a vehicle for further research but it may eventually lead to the creation of an industrial-strength SP machine, perhaps with new kinds of hardware.

Since the foundations of the SP computer model are search mechanisms that are broadly similar to those in any of the existing search engines, an interesting possibility would be to create a version of the SP machine as a software virtual machine, founded on the high-parallel search mechanisms of an existing search engine. Assuming a global reach for the search engine, there is potential for the entire internet to become an AI!