Not wishing to rain on your parade, but unless you can afford to live on part time income, I don't see this is going to work very well. You're not going to be able to find work for 35-40 hours per week every week that will seamlessly fit together. Therefore in order to ensure you have enough to live on, at the times you are working, you'll need to charge more to cover the times when you're not. Simply, if you can only find work for half the time, you have to charge double the hourly rate to stand still. Factor in your costs for your holiday pay (which your employer is paying at the moment), sickness cover, NI, pension, increased travel costs as you spread yourself further afield trying to find work, etc etc all of which you'll be having to charge your clients for, and it gets even more expensive. That then takes you into the territory of the sorts of rates businesses pay agency staff, who can send along somebody else if the original person goes sick, can't do the job well enough etc etc.
And how are you going to find your next assignment whilst working in your current one? Your current customer isn't going to be happy for you to be out visiting your next potential customers in their time.
I know a couple of ladies who clubbed together to form a business doing this sort of thing so they could cover assignments between them. I think they found it very hard going, one of them gave up and went back to their previous employer, the other I think continues on her own but is only part time. They spent quite a lot of time and money doing business networking to spread the word about their business.
Self employment isn't as easy as it may appear, especially when what you are selling is solely your own time.