Continuos payment authority not cancelled by supplier

Scpacker

Ok. First thing first, I'm a wally who doesn't check his bank statement properly and has paid the price for it literally. AOL now owned by TalkTalk have been taking a payment from my account for ten years, yes ten years, for an Internet service that I cancelled with them ten years ago. They won't cancel the service now because they say they haven't got a record of me in their system now (an old account, old system). They told me to stop the payment with my bank which I have by having my card stopped and a new one issued, which will cause their continuos payment authority to fail next time they try to take payment. This will then generate a letter from them to me informing me of that fact, which I then need to take up with customer services again. Problem... I don't live at that address anymore, to which they said I need to get that letter from the old address otherwise I can't progress it further. Good thing I know the people who live there now.

Question: how many years back can I claim the monies paid to them in error, theirs not mine? From PDF copies of bank statements from my banking online, the last two years four months the total paid is in excess of £700.

Any advice?

Thanks
Steve