Supercomputer created on the cheap

01/08/2013

De Montfort University teamWhile millions of pounds are being spent by some universities on buying supercomputers to handle intense levels of research a four-man team at De Montfort University (DMU) in Leicester has harnessed a university’s bank of computers to create a supercomputer - at a fraction of the cost being spent elsewhere.

In a nutshell, the team has basically developed a way of linking their high-spec computers to utilise their unused resources, especially at weekends and night-times.

Since spring 2012, a string of machines with more than 500 computational cores between them in DMU’s Creative Technology Studios have been linked into a cluster, labelled the Memenet.

Controlled by an external computer across the road in Gateway House in the university’s Centre for Computational Intelligence (CCI), the cluster has been working in the background creating algorithms (problem-solving programs) without affecting work being done by students.

‘All the problems of the world are optimisation problems’, explained Professor Ferrante Neri, from the CCI. ‘To optimise means to make a clever choice. Our role is to build up the intelligence of the machine so that the computer has to autonomously make choices - so it can behave in an intelligent way. It has to solve some optimisation problems.

‘Part of our project consists of letting the machine design the algorithms itself. The machine analyses the problem and makes the decision. In order to do this we need to run very extensive computational experiments - many algorithms, many times, on many problems.’

The professor goes on to say: ‘Other universities have been spending a high budget to purchase supercomputers. Our idea is that we have many little computers and we can distribute our optimisation algorithms over all the available computational cores.

‘At the same time, we do not use the full power of each machine because those machines are for the students to use. Our processes are going on in the background using only a little part of the capabilities of each machine.

‘We can do this without disturbing the students at all. Actually the students can continue their normal study day without even realising that our algorithms are running in the background.’

A seed is sown

The idea germinated from chats between Professor Neri, who arrived at DMU in spring 2012 from a university in Finland, and Dr Lorenzo Picinali, a senior lecturer in music technology who runs DMU’s Fused Media Lab, whose shared Italian origins had brought them together.

Because the audio, video and multimedia tasks done by Dr Picinali’s students sometimes need heavy and extensive processing, the computers in the Creative Technology Studios are much more than standard desktop machines. They typically cost about £2,500; have 8-16 core processors, high bandwidth RAM and multiple hard-drives.

But, they are only really used during weekday working hours, and only during term-times.

The beauty of the Memenet project, which runs on Kimeme, a software package developed by Professor Neri’s former PhD students in Finland, is it costs little more than the extra electricity used.

And a computational cluster built up in this way can potentially be expanded indefinitely, reaching thousands of cores, if needed, with no extra cost.

In comparison, last year, the universities of Warwick, Nottingham, Birmingham and Queen Mary (London) launched a joint project to set up a £3.5 million supercomputing hub available to small businesses to optimise process design. And earlier this year the University of Loughborough launched its own £1million supercomputer.

Since last spring, one of Professor Neri’s PhD students, Fabio Caraffini, also an Italian, has been the ‘pilot’ of the cluster, with the crucial role of administrating the cluster and taking care of most of the software issues. Jos Lange was also recruited from the beginning of this June through the university’s Frontrunner student employment scheme to measure the Memenet’s impact on students’ work.

Dr Picinali said: ‘We needed to test the impact of running our Memenet on a student’s standard work so we created six local tasks, of different degrees of system usage, and measured their impact.

‘For example, in week one, we ran the six tests without Memenet operating in the background. Then in week two we introduced low-usage tasks from the Memenet side, and so on. We found that when Memenet is using the network heavily, there are only a few situations when it might affect the local tasks being done by students.

‘But we can synchronise the use of Memenet with the course timetable to avoid those situations and, of course, we can run it overnight and during holidays when the students are not working.’

Professor Neri continues the thread: ‘It fits with many of the strategies at DMU, especially with the aim of using our existing resources better. It also fits with sustainable development. For example, the huge supercomputers have to be disposed of at the end of their lives, but the methods of cooling them use toxic materials. We don’t have those toxic materials.”

As well as the creation of a prototype supercomputer, the Memenet has already seen the publication of 20 research papers, with more planned to follow.

‘We hope we can show other universities what we have done and that they can have a supercomputer without spending £1million!’ said Professor Neri.

Sources