SOA: Is everyone talking the same language?
ISSUE 15.3, Nov 05
David Chappell,
Sonic Software
A key attribute of a Service Oriented Architecture (SOA) is interoperability but will vested interests reduce the benefits?
SOA looks in danger of becoming bogged down by uncertainty and vested interests. If we assume that it is not just more consultancy and supplier driven hype, can it really be harnessed to bring about a transformation of processes and ageing architectures? One of the hurdles could well be interoperability, with different camps looking to take the lead but with a seeming lack of coordination. Given that the whole concept is to do with components that can coexist with each other and can be sourced from different parties, such a failing could be highly debilitating.
For a start, is everyone talking about the same thing? As Adobe’s Duane Nickull, chairman of the Oasis SOA-RM technical committee, recently observed: ‘The term SOA is used in an increasing number of contexts with differing - and even conflicting - meanings’. It is ever more difficult to find an application software supplier that does not claim to be heading towards SOA; more and more banks are saying the same thing (Kaupthing Bank, Standard Bank, Wachovia and others have been highlighted in recent editions of IBS). Swift is talking about SOA in regard to its CIO/COO Reach strategy (see article 'CIO/COO Reach - Swift extends a helping hand'). However, much of the debate is bogged down by dubious definitions, with the SOA label being used freely for all sorts of initiatives. The fact that a monolithic application has browser support does not mean it is SOA compliant. The fact that a core banking system is being translated from legacy code to Java is similarly not SOA per se, even if it is a first step on the route.
SOA in its purest sense is centred on loosely coupled components which support generic services and are based on web technology. Those components should be much more flexible when it comes to reuse and can be combined to create new business functions both within and across enterprises. They should embody best practices and should enhance the ability to outsource and extend processes to business partners. SOA is also meant to embrace legacy systems, allowing these to be wrapped so that they can coexist in a SOA world. The next step after wrapping is to gradually replace them, through piecemeal deconstruction, with functions gradually moving to the new components. The generic nature of the components means they are intended to traverse silos and departments, thereby facilitating the breakdown of barriers which only exist for historical, technical and antiquated organisational reasons. Lance Hill, VP, product and solution marketing at Webmethods, says: ‘Banking for years and years has realised that silos are very bad - SOA is a great opportunity to finally break down those barriers.’ It should allow IT to provide much better support to the business users, he says.
Another advantage of SOA for the finance industry could be to do with compliance, says Steve Craggs, a highly experienced consultant in the middleware space and European chairman of the Integration Consortium. SOA should allow ‘much better visibility of what is actually happening in the operating systems’, he says. ‘It can give a much more business oriented picture.’ For governance and other forms of compliance, such as Mifid, this could be a valuable attribute.
With these sorts of promises, there is a requirement to better understand the definitions and issues. In particular, if SOA really can live up to the claims, then there is the need for the technology side of banks to be able to articulate this to the business users.
Models and Blueprints
One grouping that has added to the debate of late is headed by Sonic Software. A subsidiary of Progress Software, Sonic is among those touting SOA enabling technology. This is in the form of a message oriented Enterprise Service Bus (ESB) which is positioned as the communications backbone for building and delivering SOA components. Bearing Point (KPMG Consulting, as was) is a key partner. It has a SOA Roadmap, with a delivery methodology and framework. Of late, these two have teamed up with a couple of other suppliers to unveil a ‘SOA Maturity Model’. The other partners are AmberPoint, which describes itself as ‘the industry’s leading provider of SOA visibility, management and security solutions’, and Systinet, which focuses on SOA governance and lifecycle management.
David Chappell, who has the grand title at Sonic of VP and chief technology evangelist, describes the SOA Maturity Model as a ‘framework for discussion, to help an organisation understand where it is with regards the adoption of SOA’. He feels the subject is ‘very nebulous’ and ‘in need of demystifying’ so that the value proposition can be articulated to other parts of the business. The model is made up of five levels, comprising initial services; architected services; business and collaborative services; measured business services; and optimised business services. Next to these are the criteria, as defined by the partners, for SOA adoption.
The SOA Maturity Model has gained a fair amount of publicity in the couple of months since it hit the streets, perhaps because it has stemmed from commercial organisations who have done a decent marketing job, including roadshows in the US and beyond. However, Oasis itself formed a SOA Reference Model (RM) technical committee in May, ‘to create a standard nomenclature for SOA’. Adobe’s Nickull said the model would be useful for the entire industry, ‘to preserve a common layer of understanding’. Oasis has been around since 1993 and is a non-profit-making consortium focused on the development, convergence and adoption of business standards. In the SOA arena, it is seeking to define infrastructure standards for web services as well as implementation standards for specific communities and industries. As well as the Reference Model, it also has Adoption Blueprints. There does seem to be a healthy degree of pragmatism about the Blueprints, so that Miko Matsumura, VP of marketing at Infravio Software and head of the Blueprint initiative, has said that the aim is not to be dogmatic and sit in judgement on whether or not something is SOA at a philosophical level.
SOA Adoption
What of the uptake of web services and SOA? Sonic’s Chappell says a lot of organisations are at the lowest levels, focused on initial services or architected services, with this typically on a department basis, rather than company-wide. His colleague, Mark Palmer, VP, complex event processing, at Progress Real-Time Division (Objectstore, as was) says major sell-side institutions are mostly still at level one. ‘Optimisation is usually done at a localised level.’ This might be for a single trading desk, rather than across asset classes.
Webmethods has also been looking of late at the uptake of SOA. It polled customers across the globe to ask about web services adoption and volumes, SOA strategy maturity and SOA implementations, gaining 480 responses. Hill points out that financial services is Webmethods largest vertical. Over 80 per cent of respondents said they were already deploying web services; within Webmethods implementations, over half were conducting in excess of 10,000 web services transactions per month, with six per cent handling more than one million per month. Hill cites Bank of America as one of the largest proponents of web services, running millions of web services transactions for CRM. While government institutions in the US have moved very fast with web services and SOA, he says, ‘financial services is probably a close second’.
However, when it comes to SOA, most respondents to the Webmethods survey did not feel that they were too advanced. They were asked - on a scale of one to ten (where one was not moving forward with a SOA strategy) - to define how far they were along a SOA strategy. The most common mark was ‘three’ (17.5 per cent), then ‘five’ (15 per cent), ‘two’ (13.1 per cent) and then ‘four’ (12.6 per cent). The main drivers were cited as reuse of services (20.4 per cent), lower integration costs (17.6 per cent), faster delivery of projects (13.4 per cent) and lower application development costs (13.4 per cent).
In terms of the application areas, this was pretty evenly spread, says Hill. ‘We were a bit surprised how strong the uptake was for mission-critical systems starting to be built using web services and SOA.’ Recent research by AMR found that 60 per cent of SOA deployments are currently internal, 32 per cent are customer-facing, 21 per cent are supplier-facing, and four per cent fall into ‘other’ categories. However, while 40 per cent of adopters intend to keep their SOA efforts internal, 40 per cent intend to extend SOA to customers and 15 per cent to suppliers.
Potential Pitfalls
Although pragmatism is a good thing, careful thought and control needs to go into the introduction of SOA and web services, otherwise the potential benefits will not arise. As SOA is all about business services, it must straddle the business and technical worlds. The business service itself, what it does, what it achieves, must be laid down by the business. Craggs is currently working on a paper entitled ‘Web services anarchy’. Strong discipline and governance are key, he argues. Part of the challenge is at the definition stage. If a company is really going to achieve reuse, then the components have to be designed correctly from the outset, with input from across the enterprise. He gives the example of a ‘Get Customer Details’ web service which is not designed to capture the customer’s email address. At some stage, a user will need the missing capability and might take the component and come up with a tailored version, which then takes on a life of its own. Before long, the company has lost control of its components, doesn’t know what is going on and doesn’t know who is using what. One question that an organisation should ask, he believes, is ‘how would we kill a web service?’ If this is not straightforward, then there is probably a lack of control.
A lack of governance means that benefits can tail off. An organisation might start to see returns, says Craggs, and enthusiasm grows, then all of a sudden, the users wonder why the productivity curve has started to head downwards. Webmethods’ survey backs up the governance issue, with some clear inhibitors flagged by respondents. Top of the list is general knowledge about SOA within the enterprise (16.9 per cent), quantifying the ROI around SOA (15.4 per cent), governing SOA standards within the enterprise (12 per cent), complexity of the SOA environment (11.5 per cent), and supportability and availability of applications based on SOA (11.5 per cent). In any SOA environment, there will be the need for all of the ‘plumbing’. Craggs has come up with a ‘SOA Ecosystem’. One essential piece, which is meant to bring the necessary control, is a registry, which is used at both development and run-time. It defines the available services and manages their use. There are a number of specialists in this space, including Systinet (one user in the financial services space is Société Générale), Webmethods, and Infravio. Although not a prerequisite, an ESB is one of the usual enablers for SOA. Most models will have such a layer, with services interfaces back into core systems. The ESB ties together and manages the environment. Understandably, Sonic’s Chappell suggests that an ESB is not your every-day J2EE server or middleware. A lot of the offerings are good for high throughput and a publish and subscribe model, he says, but only within a closed network. Today’s requirements are for much wider availability, across geographical and organisational boundaries. While you do not need to have an ESB, says Craggs, the problem with SOA is that there is ‘tonnes of XML flying about’. This is ‘very verbose’ so is expensive to pass around and process. It is also fairly difficult to secure. He points to IBM’s recent acquisition of DataPower as significant. DataPower specialises in XML network infrastructure, with its XG3 platform intended to protect against XML security vulnerability and, in the words of the supplier, ‘to deliver wirespeed performance without sacrificing security or flexibility’. IBM has also announced Websphere ESB, for availability before the end of this year, with this intended to bring enterprise-wide and external SOA integration capabilities.
Webmethods’ Hill has tried to pinpoint the factors for success among the disparate approaches to web services and SOA. First, the most successful customers are those that tackle a business problem rather than an IT problem. Benefits should span both areas. Research from AMR backed up the view that customers taking a business process-centric approach have the greatest chance of success. There also needs to be a ‘very early understanding of the complexity of the environment’. Failure to do so is one of the things that destroys reuse. The support issues also need to be understood. Hill likens it to when companies moved from mainframes to client-server computing - there were compelling reasons to do so but the resultant complexity had to be considered and managed. He feels that many in the marketplace are pitching SOA as ‘very easy, a quick win’ and something that everyone should do. ‘Our view is that it is an evolution of what companies have been doing already.’
Different Camps
Most heavyweights and specialists in this space are members of Oasis. The ‘Foundational Sponsors’ are BEA, supply chain solution specialist, Innodata Isogen, SAP and Sun. However, if this suggests harmony, then that is probably misleading. Sonic’s Chappell is critical of SAP and Oracle’s SOA efforts. ‘Both SAP and Oracle are taking a fundamentally wrong approach to providing a backbone for SOA.’ With SAP’s Netweaver-centric approach, he feels it is little more than putting a J2EE application server in front of SAP’s applications to ‘wrap ABAP code and provide a web services veneer’. He refers to Oracle’s Fusion Middleware as ‘Confusion Middleware’. Palmer accuses both of having too closed an approach, so aiming for ‘SOA Oracle’ or ‘SOA SAP’. ‘That is not the reality of the integration problems.’
Of course, both SAP and Oracle would counter such arguments, pointing to their efforts to decouple services within their ERP systems and provide these on a true SOA basis, with Netweaver and Fusion Middleware respectively as the platforms for hosting and distributing these, just as IBM would position Websphere for this role.
Webmethods’ Hill says: ‘Certainly [large vendors] are not interested in exposing their application portfolios so that they make it easier for users to migrate to competitors’. However, those closed environments are a market opportunity for suppliers with enabling technology, he feels. The ‘plumbing’ will always need to sit in front of disparate applications to provide a single view and the required transparency, pulling services and information into the infrastructure. He points out that even the largest SAP customers are likely to have more third party software within their IT functions than they have from this one supplier.
From the perspective of standards, Craggs feels that at least everyone seems to basically agree on what constitutes a SOA (or ‘an SOA’ depending on whether or not you pronounce the acronym). There are also emerging web services standards. For instance, Oasis has its Web Services Reliable Messaging standards. It is possible to have SOA without web services, ‘but they happen to go together pretty well’, says Craggs. However, it is ancillary areas, such as governance, where there is no agreement, he feels. ‘Any vendor can draw a picture of an ecosystem and, strangely, there will be boxes for all of the things that it has and none for those it does not.’ The definitions behind the boxes will also fit a vendor’s own offerings. For instance, Craggs is ‘staggered’ by the number of definitions of an ESB, with vendors ‘trying to match the definitions with what they actually have’. This is where he feels a consortium can help, by bringing a consensus of opinion to the debate.
So much of the SOA debate is still muddied by dubious definitions and supplier hype. Maturity Models, Ecosystems, standards and Adoption Blueprints might clear up some of the issues or, if there are too many of them, they could just add another level of confusion. Either way, no doubt the discussions will continue to rage as soon as you drill down below the model, with suppliers at odds with each other about the finer details. There is a real danger that SOA will be held back as a result.
