Service oriented architecture, 2007 style

Are SOAs producing viable commercial successes? The conclusion is a cautious yes says David Barnes MBCS CITP, with the next few years expected to see much more

On behalf of the BCS, David attended the IDC SOA conference (March 22, London). The conference included most of the big names in the European SOA arena including IBM, SAP, Sun Microsystems, plus participation by a number of organisations spanning the financial, energy, logistics and manufacturing sectors, describing their challenges and also returns on investment in SOA projects so far.

Main themes from the conference.

Hype?

It was universally acknowledged that SOA had been over-hyped in the industry. Early attempts at adoption in many cases had been ad hoc, technology-driven projects, often involving web services, but delivering little business value, ironically delivering 'legacy services' to the organisation.

Giles Nelson of Progress Software however made the point that this is paradoxical as that even given such scepticism surrounding SOA, it is forecast that over the next three years the total value of SOA projects within the industry worldwide is set to more than double (Source IDC: 2006).

Real successes

Real successes can be realised with SOA, but it is very important to select the right areas to focus on and where to start. A number of presenters stated that SOA adoption within an enterprise needs to follow a maturity model similar to the CMM entailing for many the need to make major cultural shifts in the working practices of the IT department.

In order to be successful it was suggested to start with small projects in business areas subject to strong market competition where agility and responsiveness to the market or industry sector are critical success factors.

It was mentioned in a number of presentations that it was almost a prerequisite for an SOA project to lie within an area where current and future business agility are key. Picking a business area not subject to these dynamics may result in an SOA project with a lower need for genuine reuse of services, delivering less value.

Don’t expect the return on an SOA project to come from initial cost savings; the return comes on the reuse of services in subsequent projects.

If your organisation has not committed to ongoing planned reuse of services, and made the corresponding changes to organisational structure to absorb this fundamental shift in the way IT and business work together, then you won't ever see the return from the project. Plan for and build around reuse.

Governance and leadership are critical

Probably the strongest common theme throughout the conference was that SOA is fundamentally about leadership. Delivery of SOA solutions at an enterprise level (enterprise SOA) requires strong and effective leadership, perhaps from a new breed of leader, someone with a strong track record in business but with a firm grip on projects with high degrees of IT involvement.

The need for a reference architecture comprising a 'substantial tome' was mentioned in several presentations indicating that success with SOA is not a trial and error affair.

Jan Duffy (IDC) stated that too many early SOA projects had been 'skunk works' projects, and that enterprise SOA has to be fully funded and fully backed from the CIO (or equivalent) of an organisation.

Jan emphasised the relationship between SOA and leadership with the quote '.... if your SOA fails then you need new leaders....'! Jeremy Lock of British Energy endorsed this with their personal experience that '.... only 30 per cent of SOA is about the development process, the remainder is about governance and management, both pre-development and operations....'

It's (still) not just about the technology

All presenters agreed (some vendors through gritted teeth) that SOA is not equivalent to any single technology or mixture of technologies. Many projects had been built on web services only to encounter serious maintenance issues after delivery (often around performance and scalability).

Steve Elliot of Sun Microsystems pressed this point with a thorough enumeration of the many ways that services can be implemented and explained the fallacy of the 'Highlander Principle' - there can be only one!

In other words there are many ways to deliver services to the enterprise not just one, and think first about things like '.... identity and interoperability....' before deciding on the appropriate technologies to realise this.

Steve also referred to the Sun/Microsoft TAC technical advisory committee set up in 2004, again stressing the importance of not trying to deliver SOA through any particular technology silo, and the need for real collaboration between vendors on interoperability issues.

Big bang and fail

It was repeatedly stated by a number of presenters that SOA implementation could never succeed in a 'big bang' style of delivery. Incremental adoption is key.

It's IT but not quite as we've known it

It has long been said that SOA requires a transformation in the relationship between IT and business. This message came across loud and clear within the conference.

E2E presented their approach of utilising model driven integration (MDI) via a UML 'virtual machine' to allow the execution of UML models (describing business services) without any need to write source code. Impressive claims were made of reducing development cycles from months down to weeks via this approach.

Agile methods were also advocated as being highly suitable for SOA projects as long as the organisation concerned is prepared to undertake potentially radical changes in its IT development model.

Phil Head of BEA talked of their experiences of a major SOA project for Zurich Insurance where production standard code was being delivered every six weeks in a six-month overall project deadline. Clearly this is a very different way of working for many organisations.

Beware of the data spaghetti

Giles Nelson (Progress) was not alone in flagging up the important issue of what was referred to as 'data spaghetti'. By this what was meant was that the integration of disparate systems and applications via a business services model inevitably requires standardisation of the meaning of data (data semantics) within services spanning multiple (often legacy) applications and technology platforms.

Ironically, if this is not well thought out and is done by a number of point to point data transformations then the end result can end up being another hardwired architecture, hardwired at the next level up, by fragmented silos of data transformation. This hampers the ongoing reuse of the newly developed services from a very early stage in the service life cycle.

More 'architecture' less 'service orientation'?

Laurence Wilkes is somewhat of an SOA authority, being a director and a principal consultant with the highly respected CBDI Forum (now Everware CBDI). Laurence delivered one of the more demanding presentations of the day and said that he felt that to date there had been plenty of service orientation but not enough architecture as far as SOA was concerned.

By this he meant that organisations were not applying sufficient, rigorous architectural principles to their SOA, sometimes resulting in silo'd legacy services. An analogy was also drawn that '.... SOA is a complex change management problem .... where a .... formal approach was required with a high correlation with quality initiatives.'

In other words the breadth and depth of what is required to deliver success Enterprise SOA may seem to be being underestimated currently. Again this harks back to organisational and SOA maturity levels?

Legacy is well within scope of SOA

John Billman of Microfocus delivered a presentation that would be of interest to a many IT people. Starting from a staggering statistic that legacy COBOL applications worldwide represent an investment of $5 trillion and 200bn lines of code, John went on to explain that this investment in the application landscape is ripe for exposure as services and for reuse without rewriting any code.

Microfocus have developed a number of products that can assist with the automatation, discovery and exposure of services within legacy COBOL applications, interoperating with Java / J2EE, .Net, and Websphere across Unix, Wintel and Linux platforms. This means that the challenge of how to integrate legacy applications within an SOA environment need not necessarily be the obstacle it may be perceived to be.

Conclusions

IDC are well known and respected for the standard of their market analysis and consultancy on a global scale. The conference lived up to their reputation and it was a credit to the way the event was arranged that many of the presenters were from different industry sectors, sharing their SOA successes and challenges, and not just vendors.

Where successes had been achieved it made for compelling listening and it was refreshing to hear that a 'technology neutral' approach had often been adopted, with priority having been placed on organisational culture-shift and leadership first.

Soapbox

In closing - it is the author's opinion that organisational maturity will play a key part in the successful delivery of enterprise SOA, and in turn the national infrastructure surrounding a organisation will (sometimes dramatically) help or hinder this. Laurence Wilkes mentioned that Denmark has a national infrastructure for SOA and are world leaders in e-government.

In the UK will have to look seriously at where we lie in relation to this. A recent World Economic Forum (WEF) report shows we have dropped from 9th to 10th place in the Global Technology Networked Readiness Index (NRI), with (guess who) Denmark moving to first place.

Note, the EIU e-readiness rating has similar findings.

Whilst it is not being suggested that we all migrate to Scandinavia, it is clear that the UK still has some way to go in matching this national level of investment in infrastructure to support business, something Intellect and the BCS have been stressing for over a decade.

Until then real enterprise SOA federated between distinct, distributed business service providers may be somewhat of a pipe dream - fans of Web 2.0 may be have different views, but that’s another story.

Conference participants.

Financial sector: