In the past few years we have seen the emergence of a new form of distributed computing called grid computing. This uses the internet as a communications backbone to enable the use of large numbers of internet-connected computers for collaboration in executing computing tasks to solve very large problems.
Popular examples include the BBC's climate change project, which is aimed at identifying important patterns of weather change over time, and the identification of new candidate drugs that have the right shape and chemical characteristics to block HIV protease.
In the academic community we are seeing further changes in the conduct of scientific research through the radical capabilities of the grid computing infrastructure.
The UK is a world leader in this field through the e-science programme. While these applications operate over what may be considered as very loosely coupled networks of computers, more tightly coupled configurations of computers are also used in industry for more accurate modelling of reservoirs and flows of gas and liquid through geological structures, or simulations of structures and flow dynamics in engineering design.
In business, we have seen the widespread adoption of large cluster configurations in the financial sector to support applications such as market simulation, portfolio valuation and derivative pricing.
The technologies that have enabled these breakthroughs are now becoming more mature and available from the mainstream IT hardware and software suppliers, and through the provision of open source software, such as that available from the Globus community.
So what are these technologies and what value do they bring to organisations in the public and private sector?
The initial value of grid computing lies in the increased utilisation of existing computing servers to provide more capacity, and once built, the flexible processing on demand across a wider range of equipment.
Ultimately, the aim is to provide a scalable, resilient computing infrastructure for meeting business needs. A properly architected computing infrastructure provides effective means for scaling resources, adding resources and managing resources dynamically to meet business needs.
Virtualisation
The key enabling concept here is virtualisation. This refers to an abstract description of a set of resources that can be accessed via some management middleware that takes care of resource allocation to meet the needs of the application, often in real time.
Grid middleware classically acquires and maintains: management information about applications and resources; execution management services for jobs submitted; data access and transfer management and security through authorisation, role and access privileges.
We are quite used to virtualisation techniques being applied for data storage networks through storage area networks (SAN) and in communication networks through examples such as virtual private networks (VPN). Why not one for computing services?
Grids take advantage of virtualisation to harness computer resources such as servers, mainframes and even PCs, to act as a single virtual computer. The grid middleware provides monitoring and control functions to enable this.
So what are the catches? To take advantage of a virtual computer, applications must not make assumptions or create dependencies on specific hardware capabilities inherent in the server on which the application is running.
Today, much of the value we deliver from IT is inherently linked to specific configurations of application and hardware, developed, tested and maintained in silos in the data centre. This approach has certain advantages when we have stable requirements and long-lived computer equipment.
Indeed, applications that are 20, 30 or more years old are not unusual in our organisations. But the price is paid in over capacity with utilisation at levels around 20 per cent on average and yet more popular applications may often run out of steam at critical moments in the day. How to solve this problem and ease the pain of the finance director?
Achieving the effective utilisation of computing capabilities can be best seen in the large scale utilisation of resources for applications which are able to run widely across the network, making use of available capacity wherever it may be. Our ability to migrate or replace legacy applications lies on the critical path for achieving this.
So, what's the plan for achieving this?
Virtual infrastructure
Those legacy applications which depend upon mainstream applications from the likes of Oracle, IBM, SAP and BEA may well have a future as these companies are forging ahead to service-enable these offerings. Replacing silo applications with services looks the best long term bet.
The capability to utilise a virtual infrastructure is implicit in this strategy. Therefore, this provides a potential migration path, although possibly expensive in terms of the inevitable re-engineering and testing procedures.
An alternative is to create islands within the computing infrastructure in which the existing applications are maintained and eventually replaced by new applications. Replacement is always a challenge in terms of cost and return, but with the advance of service-enabled applications perhaps a necessary move forwards.
Certainly, there may be some cost advantages to swapping out expensive-to-maintain legacy hardware and taking advantage of the commodity hardware available to expand the virtualisation infrastructure. We discussed virtualisation above as being a middleware delivered capability.
However, we are also seeing a trend to provide hardware virtualisation through the offerings of vendors such as VMWare and XenSource, which provide the capability of running different operating system instances on servers and multi-core systems.
This is designed to maximise the utilisation of increasingly powerful server configurations and represents another way of addressing the challenge of system diversity and maintaining legacy systems in a modern computing infrastructure.
Software licensing presents another challenge. The basic problem being that to transport an instance of an application around your computing infrastructure, you will require licences for the relevant applications wherever they may be executed.
The current business model assumes you purchase a license per processor but this may not scale in relation to the business value of occasionally running a task on a server. This is a current industry challenge and one of which the software suppliers are keenly aware.
There is some creativity in pricing around site licensing and transportable licences available, but perhaps there are better solutions in setting policies in the management middleware to limit the class of resources on which certain types of work are run. After all, you do tend to run database applications close to the databases.
So how do you make a start towards creating a grid computing infrastructure? Our recommendation is to follow industry best practice: standardise on a hardware and software platform; consolidate your existing hardware and applications towards this platform; automate the management of the infrastructure platform through management middleware.
This will provide the basis from which you can develop your virtual computing infrastructure for the future and enable you to contemplate the next step of delivering software as a service.
Ian Osborne is project manager for Intellect working on Grid Computing Now! a DTI funded Knowledge Transfer Project aimed at championing the use of grid computing technologies in public and private sectors.
