Ownership of bank statement data?

redpola

So, I need to take my finances in hand. One part of this is to download ALL my statement data to manipulate in a suitable budgeting/analysis tool.

To my astonishment, LloydsTsb has an atrocious web interface to do this; the easiest way is to download 25 transactions at a time, resulting in hundreds of files; and if I export the data in (the maximum of) three month blocks the data is limited to 150 transactions per file meaning I have to split the period I'm exporting further! Pure pain at every level.

I'm a software engineer, so let me tell you, an interface which says "Enter start date" "enter end date" and "export data" is nearly trivial to implement. So LloydsTsb have deliberately made it horrendous to export a reasonable amount of data.

I've contacted them to ask if they can just send me my data for a nominal fee. This was roundly refused as impossible. Well, I can tell you that if a web page can see a subsection of my data, that data jolly well exists and can be recovered!

I'm still pursuing my request for some helpfulness and common sense from my bank, since from a technical point of view my request is trivial. The only obstacles are unhelpful people inventing reasons why this can't be achieved.

Ultimately I'm assuming I can rely on the data protection act and at some point insist they send me anything and everything they store electronically about me for some nominal fee.

But then that begs the question: what is the status of my bank statement data? I assume as its uniquely relevant and personal to me, I own it and it is automatically part of the data drop mentioned above. But something is nagging and I'm fully expecting the bank to be obstinate about this.

Has anyone here had any success with obtaining lots of old statement data, and should I be preparing for the bank refuting that I own these data?

Thanks,

Neil.

pmduk

I've not had to try this but see absolutely no reason why a subject access request under the data protection act would not be successful. The fee for this is £10.

redpola

pmduk wrote: »

I've not had to try this but see absolutely no reason why a subject access request under the data protection act would not be successful. The fee for this is £10.

The reason I expect this to not be successful is because the charge for a reprinted statement is already astronomical, so even though this costs them a pittance, why not argue that they own that data and make the customers pay through the nose for it?

Of course, I am rather cynical. Jaded, perhaps.

Thanks for the response.

Neil.

dtaylor84

redpola wrote: »

The reason I expect this to not be successful is because the charge for a reprinted statement is already astronomical, so even though this costs them a pittance, why not argue that they own that data and make the customers pay through the nose for it?

They are legally required to give you the data in response to a SAR, for a maximum of £10.

However, it will most likely be in hardcopy form.

getmore4less

If you use something like MSmoney and repeating transactions you can do a years worth of data very quickly manualy.

To do a decent analysis you need to go through and catagortise the payments anyway. Some may even require split catagories for full analysis.

Doing through the data can be an eye opener if you are trying to get a handle on things.

Having a rant about the way banks do stuff will fall on deaf ears.

Do a SAR, scan(OCR) the pages and write your own filter

MPH80

As others have said - yes your data will be available as part of a SAR - and yes - you'll get charged a tenner and yes - it'll come in hard copy.

I'd have thought the simpler way (since you are a software engineer) is to write a simple scraper in your language of choice to systematically extract each group of 25 entries from their site.

By the time you've put the sar request together, waited for them to action it and received it back, scanned it, OCR-ed it, corrected it etc. It'd be quicker to put together a quick scrape in Perl.

redpola

Yup I'm coming to the conclusion that writing a script to do this - though it may contain bugs and go mental on their website (the nature of software) - is going to be my best option, before an SAR and then OCR (what year is this?!).

I'm a little hesitant though because I can't find an existing script - which leads me to wonder whether they're a little hmm.... litigious about that kind of thing.

Having said that, LoveMoney scrapes LloydsTsb quite successfully (and also doesn't do exports!) so maybe it's not such a big deal.

Many thanks all.

Neil.

pmduk

You could try ANZ Moneymanager, although I am not sure how far back they can retrieve records.

The reason I expect this to not be successful is because the charge for a reprinted statement is already astronomical, so even though this costs them a pittance, why not argue that they own that data and make the customers pay through the nose for it?

The only reason for the disparity of costs is that the SAR is set legally. The cost of erisuring a statement is at the banks' discretion.

JuicyJesus

I'm a software engineer, so let me tell you, an interface which says "Enter start date" "enter end date" and "export data" is nearly trivial to implement.

This is easy to say from the perspective of someone who does not work on the software that runs ledgers for one of Britain's biggest banks.

Yes, any monkey with 5 minutes and a knowledge of PHP can do that. It's interfacing it with actual, critical production systems that might well date back decades and may not even store data (which WILL go back decades) in an acceptable format for doing such things quickly. Although I too have no personal familiarity with such systems, I would imagine it would be a little more work than just running an SQL query.

pjread

There's an obvious reason for limiting the number of records retrieved: managing load...

Yes, the code to allow it is trivial, and also possibly simple enough to get someone in an awful lot of trouble for authorising - can you imagine what 100k people hitting the 'get all my data for the last 5 years' button could do to the database engine(s) at the back?

Load management by restricting users ability to generate load isn't really a new concept. Slightly less trivial but 'probably acceptable' to you or I might be "if you need all data for this range, tick this box. the data will be available in approximately 24hrs", followed by deferring the lookups to an 'idle time'. But can you imagine the customer service requirements of trying to explain why this is necessary to the 95%+ of the banks customers who wouldn't need this and this would just confuse?

redpola

JuicyJesus wrote: »

I would imagine it would be a little more work than just running an SQL query.

A noble rebuke, but for the fact that I can, with lots of clicking and paying-attention-to-the-screen, get that data by hand. With a few lines of extra code in the JavaScript of their web platform they could

Take my dates
Automatically finagle them into the dumb old-format-that-works
Simulate me doing it by pushing the mouse around
Return me the full results by concatenating all that under-the-hood magic.
End user me is non-the-wiser but happy

I'd also argue that anyone with the intention of building reliable future-proof (yes I know that term is meaningless- take it as a statement of intent) software wants to blow the cobwebs out of their legacy systems with challenging code. If the reliability of those legacy systems is critical, there are plenty of ways to manage the risk of exposing bugs in them and/or those defects actually shipping.

Neil.