Temenos T24: T24's Coming of Age?
ISSUE 14.10, July/Aug 05
George Koukis,
Temenos
Temenos’ T24 looks in better shape now than one year ago but it has been a difficult year. IBS reports from the Temenos Client Forum in Monaco.
Temenos’ T24 successor to Globus looks considerably more stable and workable today than it did twelve months ago. This is not to say that the route has been easy, nor that all of the issues have been ironed out. However, there are users live on both the Oracle and jBase versions, so it is now possible to better understand the challenges, strengths and weaknesses of the offering.
Temenos’ Client Forum in Monaco last month was a lively affair as always. Whatever accusations can be levelled at the company, being dull is not one of them. This time, the usual bullishness was mixed with considerably more humility and apology than has been seen in the past. The T24 pilot customers have had a pretty torrid time, but those that have persevered have been able to go live, albeit often with initial phases rather than the full solution. At the same time, there is still one crucial aspect to T24 - centralised back office support - that does not look proven to date.
Temenos is claiming 40 T24 recruits, with 25 or so live. A couple have taken the DB2 version, 15 have opted for the Oracle version, and 22 have gone for Temenos’ own database, jBase. It has tended to be the existing customers that have taken the latter route. There have been two key issues: performance and quality. The Oracle version in particular has run slowly in most instances, so there has been a lot of tuning needed. That tuning is feasible, but it is not straightforward. As the first Oracle taker, Saudi Hollandi Bank (see 'Case Study: Saudi Hollandi Bank' below) found, the Close of Business within T24 could suddenly go from three hours to ten, twelve or even over 25 hours if inquiries, updates and reports were not carefully thought out. The Close of Business routine is intended to replace the traditional end-of-day batch processing, and is meant to be able to run independently of the rest of the system, so should not impact availability.
In terms of those T24 challenges, Temenos’ partners are in a good position to comment. Switzerland-based Sofgen is a 130-person specialist and has worked on a couple of T24 implementations, including Bank Leumi. Sofgen’s business development manager, Rolf Senn, suggested that, in terms of database, ‘jBase fits Globus’. The differences are relatively few, so a user who understands one would understand the other. He feels this will influence a lot of upgrade customers. There are also licence cost savings in taking jBase rather than Oracle, as well as savings on infrastructure (hardware etc) and resources.
With regards timescales, Temenos has suggested eight weeks for an upgrade, but Sofgen director, Stefan Gsell, feels this is more likely to be the time needed for the physical upgrade steps, with all of the testing and tuning to follow. Where there has been a lot of customisation, the timeframes will probably be extended. The process should be honed over time. In the early cases, there was a fair amount of bug fixing along the way, said Senn. For T24 upgrades, Sofgen advocates an on-site/off-site approach (along the lines being proposed by Temenos itself), suggesting that Globus should be upgraded through each release to R13, then a migration of the Universe database to jBase, DB2 or Oracle, and then the shift to T24 itself, followed by performance testing and tuning.
Sofgen’s observation about bug fixing brings us to problem number two. The quality of the early releases was poor. Temenos’ banking services vice president, Adrian Hadley, put his hands up to this, admitting the quality of code was a big issue. He claimed that maintenance issues have been reduced by almost 60 per cent in the last six months. The company is moving to one release per year and promises to further tighten the release procedures. There will be less customisation and an emphasis on a ‘model bank’ version of T24. For both performance and quality, Release 5 of T24 was a marked improve-ment over R4, so this is a tangible sign that Temenos is rectifying the problems. In terms of performance, R5 particularly brought multi-threading capabilities.
Temenos chairman, George Koukis, promised never to put users through such problems again. Those problems had been ‘a consistent theme from most clients’. Temenos undertook a recent customer satisfaction survey and the messages were clear, in case there had been any doubt before. The company will now introduce a ‘customer satisfaction index’ against which account managers will be incentivised.
There are other measures being taken. In what could be construed something of a u-turn from a year or so back, Temenos is going to step up its product-related services. It will ‘take more control of implementations’, said CEO, Andreas Andreades. Upgrade services will be productised, so Temenos will take a customer’s specific version, carry out off-site upgrades, testing and tuning, and will then deliver it back for final on-site testing. It is clear that the company lays much of the blame for recent failed and problem implementations on third parties. It recently introduced an accreditation service for partners and will pursue this, so that there will be a set of approved third parties. Temenos has also set up advisory boards for retail and private banking; another was suggested for Islamic banking.
One part of T24 which is not proven within those initial 25 or so live sites is centralised processing, along the lines of what was envisaged within the failed ING project (IBS, February 2005). A few banks use Globus for centralisation, including Dresdner Lateinamerika, Allied Irish Bank and EFG Bank (the latter as it has pursued an acquisition strategy over the last couple of years). A number of multi-site Globus users are apparently considering replacing outlying systems with a centralised T24 (Mashreq Bank is believed to be one) and Hadley hinted that a couple of projects are under way which include this aspect, although he declined to name them. Even users that do not have T24 in multiple sites might make use of the Close of Business capabilities, with Saudi Hollandi talking about running different Close of Businesses for its different units supported by T24.
Some of the T24 live sites have already been mentioned. In addition, Schroders in Zurich has cut over, moving from R12.2 of Globus to R4 of T24 with jBase and Sun Solaris. This bank was one of the first two to sign, along with Hatton National Bank in Sri Lanka (IBS, July/August 2004). Sacombank in Vietnam, after an eight month project, became the first to go live on R5, opting for Oracle. Bank of Botswana has gone live with the jBase version to replace Bankmaster. Bank Leumi in Switzerland also went live with the jBase version, to replace Midas. Alubaf International Bank in Tunis has gone live with jBase and IBM AIX after a six month project. And Credit Suisse in Singapore has also done so, on R4.
As mentioned, by no means all of these are live with all elements. Sacombank, for instance, is only live in a first branch, albeit one of its main 20 branches (which, out of its network of 105 branches, contribute 80 per cent of its business). It has gone live with the GL, MIS, collateral/limit management, and loans and deposits, plus an ‘All in One’ account module, dubbed AZ. The bank was the first with the latter component and there were some bugs but no major problems, said Sacombank’s CEO, Le Tan Loc. There were bugs within R5 as a whole, but Temenos mobilised to solve these, he said. Many of the glitches during the implementation came from data conversion problems, he added. The bank tried to restrict itself to essential customisation for the Vietnamese market. Cut-over happened on time a couple of months ago, with this managed by the bank itself, with Temenos providing technical assistance.
Work has started at Sacombank on the roll-out to its other main branches and it will also implement treasury in July, then derivatives and securities in the third quarter. It has jBase at present but will move to Oracle 10g in the third quarter. Le Tan Loc said there had been no performance problems with jBase but the bank wants to take advantage of the grid computing aspects of 10g. At present, the bank’s Close of Business is taking around an hour and a half, with 50,000 customers and 4000 transactions per day.
Hatton National Bank (HNB) in Sri Lanka has made significant progress with T24, compared with its trials and tribulations of a year ago (IBS, July/August 2004). At that stage, it had signed as one of the first two takers for T24 primarily because of the problems it was experiencing with the end-of-day batch processing within Globus. T24 is now live to support 44 customer centres and work is about to start on the roll-out to the rest of the bank’s 150 outlets, with the aim of completing this within one year. It has R4 of T24 with jBase. This is working fine, said the bank’s deputy general manager and head of IT, Chandima Hemachandra, and is much faster.
In early June, HNB went live with the non-stop portion of the system, so that the Close of Business can be run in the background, providing 24x7 capabilities. This is also working well, says Hemachandra, and includes interfaces to the bank’s ATM switch. He believes it makes HNB the only bank in the country with on-line, real-time 24x7 banking. There are a few remaining issues with T24, but these are being rectified, he said. There are also certain features that are still being tested. And it is using the client-server version, not the thin client. Its recent experiences mean it can press on with the rest of the project with confidence, he concluded.
Temenos has bitten off a lot and was not making too much play on new developments. However, the Oracle 10g version of T24 is on the way, with tuning for this being carried out in San Francisco. T24 can be demonstrated in Temenos’ Geneva labs on 10g on a clustered configuration with MQ Series. Temenos’ chief technology officer, Andre Loustau, said at present this is running faster than jBase. Support is also on the way - for the first time - for SQL Server, with availability set for the second quarter of 2006, albeit with this dependant on performance tests. Ji Sun Commercial Bank in Taiwan is implementing T24 on a Microsoft platform, with User Acceptance Testing due to start shortly. It is a possible first taker for the SQL Server option when this becomes available. Temenos and Microsoft have recently extended their partnership. The Linux version is up and running, with Blue Nile Mashreq Bank in Sudan having successfully moved from SCO Unix. Temenos’ Indian partner, Thesys, worked on this three month project and claimed there were no glitches, with the bank taking the jBase version.
Temenos has not always had success running its system on other platforms, with the AS/400/iSeries version being a particular problem. Of the first two takers, China Merchants Bank has apparently given up waiting (it went live with another version of Globus a couple of years ago) but, according to Andreades, the Vatican Bank is due to enter User Acceptance Testing shortly (despite the fact that this bank was known to have looked at other, non-Temenos options). In fact, despite the problems, Turkish Bank has also signed, for London, Cyprus and Istanbul. Reassurances had been gained from IBM, said bank board member, Hakan Börtecene, and it is initially replacing an in-house developed AS/400-based system with T24 in London. Cut-over is scheduled for October.
Questions were asked about a migration to Java (Temenos’ top-end retail banking system, Corebanking, is going this route) but Loustau said this is merely being watched with regards T24. Andreades said it has come up in discussions when Temenos has been bidding for new business but has not been an issue for the existing users, which seems a reasonable argument. In terms of functionality, an asset management module is under development and there has been a lot of work in the area of syndicated loans over the last couple of years within an over-running project at Caja Madrid.
Koukis admitted that Temenos had ‘made some mistakes’ - it had tried to do too much too quickly in terms of both technology and functionality. Not all users have battled on (RZB turning back to Misys by taking MidasPlus for global processing would seem to be one to give up - IBS, June 2005) but many seem willing to give Temenos considerable rope because they recognise that their system is always moving forward. The easiest way to avoid making mistakes is by doing nothing, an accusation that cannot be levelled at Temenos. Its users might sometimes wish for a quieter life, but that is not the Temenos way.
T24 Partners
On the one hand, Temenos is saying that it will ramp up its own product-related services; on the other, it is saying it will work closely with a set of accredited suppliers. In fact, this does not seem to be contradictory, because everyone agrees that the impending workload from T24 implementations and upgrades means it will be all hands to the pumps. Some of the partners, such as Switzerland-based Sofgen and Greece-based Informer Financial, are long-standing. Bishop Cavanagh and LogicaCMG are also active, and an Indian company, Thesys Technologies, has emerged since 1999 (in particular, it supports Amex Bank’s seven Globus sites, with Hong Kong being implemented at present). It started by supporting Temenos’ partner at that time, EDS.The independent specialists have all been set up by early Globus developers and all seem to be thriving. With 300+ Globus users looking to upgrade to T24 over the next few years, there is no doubt that the demand for resources will outweigh supply. Sofgen has recently set up a centre in Bucharest to train new Globus/T24 experts; Thesys already does this, claiming that none of its 60 or so Temenos specialists have been ‘poached’ from Temenos or elsewhere. Thesys also claims to be the only accredited Temenos developer, so carries out work on behalf of the supplier.
At the Client Forum, Unisystems was signed as a new partner, to act as both a distributor and service provider for Greece, south-east and central Europe, and north Africa. Temenos contacted Unisystems, said the latter’s financial sector business manager, Apostolos Tsoubris. Rather oddly, Unisystems is a part shareholder in Financial Technologies, which has its own back office system, bMaster, as well as a consumer lending offering based on this, eLoans Consumer Finance. Unisystem’ senior account manager, Alex Alexandrakis, said there was no conflict because bMaster uses different technology (Microsoft) and is focused on different areas (it is currently used solely in Greece and a few surrounding countries).
The Unisystems tie-up also seemed slightly strange as Temenos already has a well entrenched Greek partner in Informer. The latter has 400 people, of which half are in the Temenos unit. There was a falling out between the two companies a while ago, but this was resolved (IBS, December 2004). Indeed, with the blessing of Temenos, a couple of senior managers moved over to the partner, and Temenos has subcontracted a fair amount of work to Informer. The Greek company has a range of systems such
as internet banking and credit card management that are meant to complement Temenos’ offerings, although would also seem to overlap in one or two instances (United Bulgarian Bank uses them), and claims to have worked on over 30 Globus/T24 implementations (for T24, it has worked of late on two projects in Egypt, for instance, at Suez Canal Bank and CIB). Informer is emphasising its expertise in comparison to that of Unisystems. ‘We’ve done it 30 times,’ said deputy CEO, Andrew Ruffell (formerly head of partnerships, then Russia, at Temenos), ‘they haven’t. We’ve got the products, they haven’t.’
Case Study: Saudi Hollandi Bank
As the first to go live with T24 on oracle, how has this bank fared?
Saudi Hollandi Bank embarked on a search for a core banking system five years ago, with clear goals in mind. It had almost 100 systems, with each business line having its own. This was to support its diverse business (it does more or less everything, including Islamic banking, with the exception of private banking) across its 40 branches. The bank’s CIO, Hakam Abu-Zarour, says it basically needed to be in a better position to compete in its home market. The previous infrastructure was very expensive to maintain and modify, and the bank wanted a ‘360 degree view’ of its customers. It also wanted full risk management and comprehensive MIS and profitability analysis. The technology needed to be scalable, open, secure, and 24x7.
In fact, Globus was one of the existing systems, having been taken in 1999 to support only local and international share trading activities, so with the focus on the securities module. After a full selection, the bank opted for the Temenos system as its core platform, with the project starting in January 2003 and with an expected duration of twelve to 18 months. The bank decided it would keep only those systems which provided functionality that was not supported by Globus, so the final architecture means interfaces between the Temenos back office and a separate credit card management system, insurance front-end, call centre/IVR platform, ATM/POS, a payment system (Sterci’s Stelink) and a treasury front-end (Trema’s Finance Kit).
The project over-ran, in part due to security issues, with Temenos not able or willing to send staff to Saudi Arabia at different times during the project, resulting in large portions of the work being relocated to Chennai and Dubai. The disruption meant the bank headed for the T24 version of Globus, as this emerged over the course of the project, opting for the Oracle version (9i, with IBM’s AIX). There was also a shift to a phased introduction, with core customer accounts and share trading going live in January 2005. Payments followed in mid-June, with contracts and the general ledger to come in late July. The branch roll-out has just started. This phased implementation has been synchronised using in-house developed middleware, on top of IBM’s MQ Series, which Abu-Zarour describes as a ‘thick layer’ handling the differences in transaction management between the old systems and T24. There is a lot of ‘non-critical’ functionality to follow once the GL is bedded in, and the bank will also take the Islamic banking module which has been fleshed out of late with Al-Bilad Bank (Saudi Hollandi has nine Islamic products at present, of which six are covered by the T24 module; the other three are not critical, he says).
The bank intends to take advantage of the global processing capabilities of T24, so will seek to run the Close of Business batch processing for different businesses at different times, with this feasible in T24 due to the ability to run this without impacting the availability of the system.
One unpleasant surprise has been the hardware requirements. The bank originally envisaged relatively small IBM machines but has ended with two IBM p690s, with Oracle clustered across both and with six CPUs on each machine for the database and six on each for T24. One machine supports the share trading part of T24; the other modules reside on the other machine. In fairness, says Abu-Zarour, the original hardware sizing was done two and a half years ago when the bank had 5500 share trades per day, as opposed to today’s 25,000, and had 20-25,000 retail transactions, compared with today’s 50-55,000.
A number of lessons have been learned. First, says Abu-Zarour, banks should stick to the ‘model bank’ as offered by the system - the moment you start to move away from this, complexities arise, he says. Reports and inquiries should be optimised, as they can have a major impact on performance, whether on-line or at Close of Business. The Oracle version was initially very slow but a lot of work has been done on this, with more needed. The Close of Business now takes around three hours but, in one extreme case during testing, a change to the customer charge table saw this become 25 to 28 hours. Users should plan to spend a fair amount of time tuning the Close of Business, he says. ‘Our biggest issue is always the Close of Business and how long it takes - it can go from three hours to ten hours to twelve hours.’ By way of mitigation, he points out that, as the bank was the first client for the Oracle version of T24, it was a new area even for Temenos’ resources. And some problems were to do with customisations. For instance, the bank added some reports to the Close of Business but these adversely affected the performance, so they were removed. And on-line performance has been ‘excellent’ (at present, it has 55 users on the system, with this to become 600 when the project is complete). For payments, the bank did a benchmark with 60,00o actual transactions. The first time this was done, the Close of Business took ten hours; after tuning, it was down to three.
The choice of Oracle, as opposed to jBase, came down to the bank’s overall decision to standardise on the former database. Reasons included openness and the availability of resources. Abu-Zarour points out that T24 uses Oracle in a certain way which is not fully relational, so still has the table structure of the Universe world [the previous database] to a large degree. The bank has gone with the desktop front-end, not the browser, with Abu-Zarour describing the desktop as ‘the most stable part’. The browser was not available when the bank made its original selection and is likely to be added later.
The pioneer nature of the bank hopefully means that others will have less of a challenge, with no doubt Temenos itself learning from sites such as this. A relatively high number of bug fixes were needed and the tuning and hardware sizing were clearly significant parts of the project. But while it has been a long slog, the originally hoped for benefits are starting to be realised, with more to follow as the roll-out continues.
