TSB meltdown...

[Benny]

Hmmm now pulled in IBM to sort it out, does IBM still have onshore fire firefighting capability?

Getting Offshore bods involved would seem difficult

IBM has spent the past three years dumping overheads in the West, and building the workforce in offshore locations including India. In the Global Technology Services division, for example, IBM employs just one-fifth of the workforce onshore, one fifth near-shore and 60 per cent offshore.

[OwlHoot]

TSB boss Paul Pester says bank is on its knees over IT chaos - BBC News

he has said its the "service provider" at fault... so outsourced to somebody

Can't they just roll back to their old system, to give themselves time to work out what the hell went wrong?!

(Says he, trying not to sound too pitifully naïve! )

[Win]

I was guessing Diligenta

Having worked in Lloyds IT (only for a few days) a few years ago, and noticing that most of the building were Indian nationals (or ex Indian nationals), I expected quality like this

It really is a miracle that any IT system works in this country. So many of them around !

[CoolCat]

It really is a miracle that any IT system works in this country. So many of them around !

yes the quality of delivery has crash dived, and the quality of senior IT management is pretty dire in this country too, many having done nothing in their career but sacking Brits and replacing them with cheap Indian nationals from the outsourcers and claiming that as some great cost saving victory

[tazdevil]

yes the quality of delivery has crash dived, and the quality of senior IT management is pretty dire in this country too, many having done nothing in their career but sacking Brits and replacing them with cheap Indian nationals from the outsourcers and claiming that as some great cost saving victory

And nit picking about dotted i's and crossed t's. I recently raised a licence price by £1K which caused the client IT manager to go into a tail spin about lack of contracts, T&C's and so forth such was his ire Subsequently the admin cost far exceeded the price increase and this was a client I've worked with for more than 10 years and this was the first price increase in that time Cost saving

[Mordac]

Have you got a link on el reg?

I worked at Avanade for many years and I really doubt they have the scale to manage this kind of project. Also, they are purely MS stack solution providers, so probably not them.

Since when did that matter?

[motoukenin]

Probably Autosys job definition error that caused it to go wrong like RBS

How these guys dont do rehearsals and roll back I just dont know.

Whats to bet they dont have a proper dev uat and prod env?!

Problem is they can't roll back when some of the accounts are working and some are not, if customers update the account with transactions then rolling back puts you in the same boat, thats the whole problem with migrations on a mass scale they are a one shot goal as you transfer and then shut down the old account.

There was a lot of testing done in early stages but testing with a few hundred accounts is different to a few hundred thousand, another reason they could not roll back you can only tackle the problem when its there.


When I realised they were using TIBCO middleware then was not surprised, in my experience the product is difficult to scale and tune correctly as it hides all the underlying problems, would bet thats where a lot of the problems will be found.

I am sure TSB will get this working its become more of a PR fix now than a technical one.

[SueEllen]

Interesting - linky

Just before the bank’s services crumpled, software engineers and Banco Sabadell, TSB’s Spanish owner, were toasting their own efforts with champagne and claiming a job well done. The comments posted below the photo read: “Hell of a team!” and “Champions!” However, the warning signs that a catastrophe of this magnitude might happen were apparent a full year earlier.

When TSB split from Lloyds Banking Group (LBG), a move forced by the EU as a condition of its taxpayer bailout in 2008, a clone of the original group’s computer system was created and rented to TSB for £100m a year.

That banking system was a “bodge of many old systems for TSB, BOS, Halifax, Cheltenham and Gloucester and others” that had resulted from the “nightmare” integration of HBOS with Lloyds as a result of the banking crisis, according to one insider who had extensive access to and intimate knowledge of LBG and TSB’s internal systems over a prolonged period.

“The idea with the IT was to create a mirror copy of the sprawling LBG merged systems and use this to service the much smaller TSB bank. It seemed a bad fit for a smaller bank to inherit all the problems of a bloated mess to service far fewer customers,” the insider said.

Under this arrangement, LBG held all the cards. It controlled the system and offered it as a costly service to TSB when it was spun off from Lloyds in September 2013.

[cojak]

Can't they just roll back to their old system, to give themselves time to work out what the hell went wrong?!

(Says he, trying not to sound too pitifully naïve! )

Many no longer have rollback plans.

This plan was obviously using the ‘tulip or bust’ approach.

[SueEllen]

Many no longer have rollback plans.

This plan was obviously using the tulip or bust’ approach.

But it worked when Sabadell integrated accounts from small Spanish banks into their system, so why shouldn't work with TSB accounts from the fecked up system cloned from Lloyds Banking Group?