Higher Lending Charge - Charged twice

swiftshell

When we first took out a mortgage in Jan 05 we were charged a HLC because we only paid 3% deposit, we added this charge onto our mortgage. With in the key facts of this mortgage it states it is portable - Does this mean that what we have already paid on this we dont have to pay again?
We decided to move to a bigger house in Sept 06 and because we were tied into a Tracker mortgage for 3 years we were advised by Halifax that we should just take out another mortgage with them for the additional amount we needed and another HLC charge. This meant having two different mortgages a Tracker and the second mortgage a Fixed. These mortgages ended at different times so did not run along side by side.
We knew we would have to pay a HLC but when we took the second mortgage they charged us HLC on the Total amount of what we have borrowed from them and not just on the additional mortgage that we have took out. We have now paid an HLC on the first mortgage twice. Why did they not just charge us on the second mortgage that we took out?
Can they do this or is there somewhere I can write to as we have paid nearly £4k in HLC, on a £141k mortgage.