We'd like to remind Forumites to please avoid political debate on the Forum... Read More »
📨 Have you signed up to the Forum's new Email Digest yet? Get a selection of trending threads sent straight to your inbox daily, weekly or monthly!
Santander upgrade gone wrong?
Options
Comments
-
You know shop workers don't tend to make decisions about how their organisations will transact, don't you?surreysaver said:
As a shop, you should have a system in place to work around the situation. How did shops function with credit cards before everything was online?briskbeats said:
No - shop worker. Had customers shouted at us when Lloyds’ systems went down and another bank went down earlier this year. They blamed us even though the problem was with the bank NOT uscolsten said:
So are you Santander IT?briskbeats said:
Me and my colleaguescolsten said:
Who is us?briskbeats said:
^^ This is why it’s important to have a backup.greyteam1959 said:Same here............
Always like to have an alternative bank account & credit / debit card.
Hate when a bank is having problems as customers blame US for it!
What a ridiculous thing to say. Sadly the sort of crap retail workers get thrown at them day in, day out.
4 -
Well said. The 'expert's who pop out from under their stones when things like this happen have no clue about the complexity of such systems, the planning that goes into upgrades, or the fall-back procedures required. Things go wrong and, as far as I can see, the problem was dealt with one way or another. A few hours outage is certainly inconvenient but it's hardly the end of the world.IvanOpinion said:
That depends on the nature of what has gone wrong. Several years back we hit a problem which meant it took us almost 80 hours to get a stable system back online. The press and social media was rife with BS about software updates, incompetence etc. etc. and 101 "experts" providing simple solutions and pointing the finger of blame - but not one was even close to guessing what had actually happened.tempus_fugit said:This sort of thing happening so often suggests that they didn't have a proper backout plan or it hadn't been properly tested. Unfortunately these things happen, but they should always be able to put everything back if necessary, it shouldn't all be down for this long.
The problem was a set of unpredictable conditions. For commercial reasons I can't go into great detail but ... despite all the sensors, our primary site had suffered undetected damage that brought the servers down over a period of about 1-2 weeks (as best as we can guess). This meant they were still syncing with our DR site (several hundred miles away) using an industry standard best practice process that had been tested many times in the past, but (unknown to us) the data contained corruptions. In fact we had only audited and tested our fail over process 6 weeks previously and it worked perfectly. When the primary site went down it went down big time in a way that did not trigger DR to kick in. We spent the first few hours not realising the problem and trying to get the primary site up and running, eventually we sent someone to the primary site itself (120mile round trip) and realised the scale of the problem (there was no way it would be recoverable or brought back up for many weeks).
We then manually fired up the DR site and it was online within the hour - but reports started flooding in of data problems so we had to take it back down. Realising the problem we restored from a backup (a process that takes several hours - and took us 3 goes before we found a stable data set) but the regulator would not allow us to bring the site back up until we had done a load of additional actions and checks. The board did what boards do best, ranted, raved and demanded hourly updates (which meant we were basically a man down just to provide them with their updates). Our entire team did not leave the office for those 80 hours with us often seen sleeping in our chairs or under desks as we waited for various processes to complete. On the plus side it made us all valuable resources in the market and targets for head hunters.
If it wasn't for 24/7 rolling news or the Twitterati, 99% of the population wouldn't even have known about this 'problem'.
Storm in a teacup.4 -
Looks like you've answered your own rant therecolsten said:
You should not assume that your requirements are the same for the other 14 million Santander customers. There have been plenty of completely credible reports from people stranded at petrol stations, unable to buy food or pay for their train/underground ticket or their taxi etc etc. Those people have a right to compensation, and they will no doubt keep the Santander complaints team busy for some time.mr_accountant said:Right shall I be the first, I don't need to use santander today nor have I been inconvenienced but which social media platform can I put my boring story so that the world knows and so help me claim my compensation because its my birth right?
Instead of bragging with an I‘m alright Jack attitude, it would be constructive to remind people of the importance of always having alternative payment means. This can be anything from carrying cash to having one or more other current accounts with a positive balance to one or more credit cards. Goes (almost) without saying that one should have at least one each of a VISA and a Mastercard, in case one of them has a problem (as has happened in the past).
As aside, having my main current account with Santander: I won‘t be making any compensation claim myself as I was not materially inconvenienced by this outage. I will also not close my Santander account.
Yes, it SHOULD go without saying that people should have more than one means of paying for stuff, but I guess some people just like risking it. Their choice, their responsibility.
4 -
I haven't ranted. I merely took exception with a post that tried to ridicule those who were unfortunate enough to be left without a means to pay for essentials.Mickey666 said:
Looks like you've answered your own rant therecolsten said:
You should not assume that your requirements are the same for the other 14 million Santander customers. There have been plenty of completely credible reports from people stranded at petrol stations, unable to buy food or pay for their train/underground ticket or their taxi etc etc. Those people have a right to compensation, and they will no doubt keep the Santander complaints team busy for some time.mr_accountant said:Right shall I be the first, I don't need to use santander today nor have I been inconvenienced but which social media platform can I put my boring story so that the world knows and so help me claim my compensation because its my birth right?
Instead of bragging with an I‘m alright Jack attitude, it would be constructive to remind people of the importance of always having alternative payment means. This can be anything from carrying cash to having one or more other current accounts with a positive balance to one or more credit cards. Goes (almost) without saying that one should have at least one each of a VISA and a Mastercard, in case one of them has a problem (as has happened in the past).
As aside, having my main current account with Santander: I won‘t be making any compensation claim myself as I was not materially inconvenienced by this outage. I will also not close my Santander account.
Yes, it SHOULD go without saying that people should have more than one means of paying for stuff, but I guess some people just like risking it. Their choice, their responsibility.
It's not about 99% or 100% of the population (most of whom probably still don't know, nor have a need to know) but about those Santander customers who were adversely affected by the outage. Whilst nobody would argue that it takes some time and effort to fix the systems one way or another, your dismissive remarks simply show that you are another I'm-alright-Jack.Mickey666 said:
Well said. The 'expert's who pop out from under their stones when things like this happen have no clue about the complexity of such systems, the planning that goes into upgrades, or the fall-back procedures required. Things go wrong and, as far as I can see, the problem was dealt with one way or another. A few hours outage is certainly inconvenient but it's hardly the end of the world.IvanOpinion said:
That depends on the nature of what has gone wrong. Several years back we hit a problem which meant it took us almost 80 hours to get a stable system back online. The press and social media was rife with BS about software updates, incompetence etc. etc. and 101 "experts" providing simple solutions and pointing the finger of blame - but not one was even close to guessing what had actually happened.tempus_fugit said:This sort of thing happening so often suggests that they didn't have a proper backout plan or it hadn't been properly tested. Unfortunately these things happen, but they should always be able to put everything back if necessary, it shouldn't all be down for this long.
The problem was a set of unpredictable conditions. For commercial reasons I can't go into great detail but ... despite all the sensors, our primary site had suffered undetected damage that brought the servers down over a period of about 1-2 weeks (as best as we can guess). This meant they were still syncing with our DR site (several hundred miles away) using an industry standard best practice process that had been tested many times in the past, but (unknown to us) the data contained corruptions. In fact we had only audited and tested our fail over process 6 weeks previously and it worked perfectly. When the primary site went down it went down big time in a way that did not trigger DR to kick in. We spent the first few hours not realising the problem and trying to get the primary site up and running, eventually we sent someone to the primary site itself (120mile round trip) and realised the scale of the problem (there was no way it would be recoverable or brought back up for many weeks).
We then manually fired up the DR site and it was online within the hour - but reports started flooding in of data problems so we had to take it back down. Realising the problem we restored from a backup (a process that takes several hours - and took us 3 goes before we found a stable data set) but the regulator would not allow us to bring the site back up until we had done a load of additional actions and checks. The board did what boards do best, ranted, raved and demanded hourly updates (which meant we were basically a man down just to provide them with their updates). Our entire team did not leave the office for those 80 hours with us often seen sleeping in our chairs or under desks as we waited for various processes to complete. On the plus side it made us all valuable resources in the market and targets for head hunters.
If it wasn't for 24/7 rolling news or the Twitterati, 99% of the population wouldn't even have known about this 'problem'.
Storm in a teacup.0 -
We still have these flatbed card machines. Only dug out when needed. One episode happened three years ago - when I was on holiday, so missed this. As internet went down around the area. So could not pay with card, buy phone vouchers, buy 3rd party gift vouchers. The ATMs in the area affected were down. My teammates said it was a nightmare. Customers were yelling at them asking which is the nearest ATM which was working. How on earth were my teammates supposed to know this when the internet went down 40 mins previously? As had no idea how far the problem was.RG2015 said:
Before credit card data was online, there was a machine that ran a roller or press over the embossed card to take an imprint of the details. These are of course long gone. The shop would have a backup system, either cash or cheques. It is not the fault of the shop. The bank is clearly to blame for providing systems that are not robust.surreysaver said:
As a shop, you should have a system in place to work around the situation. How did shops function with credit cards before everything was online?briskbeats said:
No - shop worker. Had customers shouted at us when Lloyds’ systems went down and another bank went down earlier this year. They blamed us even though the problem was with the bank NOT uscolsten said:
So are you Santander IT?briskbeats said:
Me and my colleaguescolsten said:
Who is us?briskbeats said:
^^ This is why it’s important to have a backup.greyteam1959 said:Same here............
Always like to have an alternative bank account & credit / debit card.
Hate when a bank is having problems as customers blame US for it!
The bank should have a backup and most prudent customers would also have a backup but I cannot see what other backup a shop could have.
It's best not to bank with one bank just in case things like this happened. I am with Santander myself for current accounts. Have a Halifax credit card and a Nationwide bank account with £70 in. That reminds me, must get a new PIN for Nationwide.2 -
With this, you write the numbers manually on the slip! My first job in the late 90s was working in a now longer exist high street chain. Had a customer from abroad with a non embossed card. Had to fill in the slip manually.IanManc said:
I agree with you. And of course more and more cards these days aren't embossed and are flat, so the old manual machines wouldn't work with them even if a shop still had them.RG2015 said:
Before credit card data was online, there was a machine that ran a roller or press over the embossed card to take an imprint of the details. These are of course long gone. The shop would have a backup system, either cash or cheques. It is not the fault of the shop. The bank is clearly to blame for providing systems that are not robust.surreysaver said:
As a shop, you should have a system in place to work around the situation. How did shops function with credit cards before everything was online?briskbeats said:
No - shop worker. Had customers shouted at us when Lloyds’ systems went down and another bank went down earlier this year. They blamed us even though the problem was with the bank NOT uscolsten said:
So are you Santander IT?briskbeats said:
Me and my colleaguescolsten said:
Who is us?briskbeats said:
^^ This is why it’s important to have a backup.greyteam1959 said:Same here............
Always like to have an alternative bank account & credit / debit card.
Hate when a bank is having problems as customers blame US for it!
The bank should have a backup and most prudent customers would also have a backup but I cannot see what other backup a shop could have.4 -
A few years ago, I was caught up in a total power failure one morning. I had just filled up in a busy petrol station when everything went off - lights, the petrol pumps and their displays, the tills (no cash!), the CCTV cameras, the lot. Staff scrambled and found the card swipe machines but nobody knew how much people owed as the pumps were out. Those customers who had their tanks partially or fully filled were asked for an honest guess about the amount of petrol/diesel they have had, and most, incl. myself, signed a swipe machine slip. One guy drove off in a huff, after a tirade of mindless abuse for the staff, without paying anything. I was lucky that I had already filled up - goodness knows what those waiting for petrol did, as this was the only petrol station in town. The power in the entire town was off for several more hours, as the problem was caused by a substation in which something had blown up. I can only imagine the havoc in the town's shops and homes.briskbeats said:
We still have these flatbed card machines. Only dug out when needed. One episode happened three years ago - when I was on holiday, so missed this. As internet went down around the area. So could not pay with card, buy phone vouchers, buy 3rd party gift vouchers. The ATMs in the area affected were down. My teammates said it was a nightmare. Customers were yelling at them asking which is the nearest ATM which was working. How on earth were my teammates supposed to know this when the internet went down 40 mins previously? As had no idea how far the problem was.RG2015 said:
Before credit card data was online, there was a machine that ran a roller or press over the embossed card to take an imprint of the details. These are of course long gone. The shop would have a backup system, either cash or cheques. It is not the fault of the shop. The bank is clearly to blame for providing systems that are not robust.surreysaver said:
As a shop, you should have a system in place to work around the situation. How did shops function with credit cards before everything was online?briskbeats said:
No - shop worker. Had customers shouted at us when Lloyds’ systems went down and another bank went down earlier this year. They blamed us even though the problem was with the bank NOT uscolsten said:
So are you Santander IT?briskbeats said:
Me and my colleaguescolsten said:
Who is us?briskbeats said:
^^ This is why it’s important to have a backup.greyteam1959 said:Same here............
Always like to have an alternative bank account & credit / debit card.
Hate when a bank is having problems as customers blame US for it!
The bank should have a backup and most prudent customers would also have a backup but I cannot see what other backup a shop could have.
It's best not to bank with one bank just in case things like this happened. I am with Santander myself for current accounts. Have a Halifax credit card and a Nationwide bank account with £70 in. That reminds me, must get a new PIN for Nationwide.3 -
Sorry to disappoint your uninformed assumption but I was actually affected as I had a rather large invoice to pay by bank transfer and wasn't able to. I suppose I could have paid it from a different bank account but since I already had the payment details set up in my Santander account I took the easier option of simply waiting until today instead.colsten said:
I haven't ranted. I merely took exception with a post that tried to ridicule those who were unfortunate enough to be left without a means to pay for essentials.Mickey666 said:
Looks like you've answered your own rant therecolsten said:
You should not assume that your requirements are the same for the other 14 million Santander customers. There have been plenty of completely credible reports from people stranded at petrol stations, unable to buy food or pay for their train/underground ticket or their taxi etc etc. Those people have a right to compensation, and they will no doubt keep the Santander complaints team busy for some time.mr_accountant said:Right shall I be the first, I don't need to use santander today nor have I been inconvenienced but which social media platform can I put my boring story so that the world knows and so help me claim my compensation because its my birth right?
Instead of bragging with an I‘m alright Jack attitude, it would be constructive to remind people of the importance of always having alternative payment means. This can be anything from carrying cash to having one or more other current accounts with a positive balance to one or more credit cards. Goes (almost) without saying that one should have at least one each of a VISA and a Mastercard, in case one of them has a problem (as has happened in the past).
As aside, having my main current account with Santander: I won‘t be making any compensation claim myself as I was not materially inconvenienced by this outage. I will also not close my Santander account.
Yes, it SHOULD go without saying that people should have more than one means of paying for stuff, but I guess some people just like risking it. Their choice, their responsibility.
It's not about 99% or 100% of the population (most of whom probably still don't know, nor have a need to know) but about those Santander customers who were adversely affected by the outage. Whilst nobody would argue that it takes some time and effort to fix the systems one way or another, your dismissive remarks simply show that you are another I'm-alright-Jack.Mickey666 said:
Well said. The 'expert's who pop out from under their stones when things like this happen have no clue about the complexity of such systems, the planning that goes into upgrades, or the fall-back procedures required. Things go wrong and, as far as I can see, the problem was dealt with one way or another. A few hours outage is certainly inconvenient but it's hardly the end of the world.IvanOpinion said:
That depends on the nature of what has gone wrong. Several years back we hit a problem which meant it took us almost 80 hours to get a stable system back online. The press and social media was rife with BS about software updates, incompetence etc. etc. and 101 "experts" providing simple solutions and pointing the finger of blame - but not one was even close to guessing what had actually happened.tempus_fugit said:This sort of thing happening so often suggests that they didn't have a proper backout plan or it hadn't been properly tested. Unfortunately these things happen, but they should always be able to put everything back if necessary, it shouldn't all be down for this long.
The problem was a set of unpredictable conditions. For commercial reasons I can't go into great detail but ... despite all the sensors, our primary site had suffered undetected damage that brought the servers down over a period of about 1-2 weeks (as best as we can guess). This meant they were still syncing with our DR site (several hundred miles away) using an industry standard best practice process that had been tested many times in the past, but (unknown to us) the data contained corruptions. In fact we had only audited and tested our fail over process 6 weeks previously and it worked perfectly. When the primary site went down it went down big time in a way that did not trigger DR to kick in. We spent the first few hours not realising the problem and trying to get the primary site up and running, eventually we sent someone to the primary site itself (120mile round trip) and realised the scale of the problem (there was no way it would be recoverable or brought back up for many weeks).
We then manually fired up the DR site and it was online within the hour - but reports started flooding in of data problems so we had to take it back down. Realising the problem we restored from a backup (a process that takes several hours - and took us 3 goes before we found a stable data set) but the regulator would not allow us to bring the site back up until we had done a load of additional actions and checks. The board did what boards do best, ranted, raved and demanded hourly updates (which meant we were basically a man down just to provide them with their updates). Our entire team did not leave the office for those 80 hours with us often seen sleeping in our chairs or under desks as we waited for various processes to complete. On the plus side it made us all valuable resources in the market and targets for head hunters.
If it wasn't for 24/7 rolling news or the Twitterati, 99% of the population wouldn't even have known about this 'problem'.
Storm in a teacup.
I guess that makes me an 'alright-Jack' in your eyes when the reality is that I could see the situation for what it was - a temporary glitch - and didn't feel the need to hit the panic button or leap aboard the compensation bandwagon. Takes all sorts I suppose
As for being 'adversely affected' by a few hours of bank outage, I suppose it depends on your definition of 'adverse' but my suspicion is that we're talking about first-world problems here. Like I said, storm in a teacup. 2 -
Only just heard about this from BBCgreyteam1959 said:Same here............
Always like to have an alternative bank account & credit / debit card.
https://www.bbc.co.uk/news/uk-57131476
I'm amazed how many people rely on just one bank card for their life. I know it's unusual to have the number of accounts many people here have but to rely on a single card for everything seems crazy. At least with a credit card from a different bank you aren't reliant on that platform. One guy interviewed for that article doesn't seem to realise the irony of what he was saying - "I work in IT so I know about backups & upgrades" - yet relies on a single card for payments! Is it just people don't think they can be trusted with a credit card?
If you only have one card/account, is there a reason why you don't have another?Remember the saying: if it looks too good to be true it almost certainly is.5 -
Anecdotally, I think there is some truth in that. But wary of drawing conclusions on the wider population.jimjames said:
Is it just people don't think they can be trusted with a credit card?If you only have one card/account, is there a reason why you don't have another?
I tried to find some research findings on this but what I found was fairly old: https://www.fca.org.uk/firms/consumer-credit-research-cards/consumer-use
As of 2013, 90% of households with income more than £50k have a credit card, but only 33% of households with income less than £10k. It's presumably the case that these lower income households are simply unable to access credit cards. And as a result the impact of a banking failure on these, already vulnerable, households could - as Colsten suggests upthread - be really severe.
5
Confirm your email address to Create Threads and Reply
Categories
- All Categories
- 351K Banking & Borrowing
- 253.1K Reduce Debt & Boost Income
- 453.6K Spending & Discounts
- 244K Work, Benefits & Business
- 599K Mortgages, Homes & Bills
- 177K Life & Family
- 257.4K Travel & Transport
- 1.5M Hobbies & Leisure
- 16.1K Discuss & Feedback
- 37.6K Read-Only Boards