would you b giving up your other work? of not then i can't see it being a problem, you'd only run into problems if you were seen to be employed by that company, make sure you're still invoicing them as usual and doing your own tax etc,
if you work for that company for more than 2 yrs with no other clients you might run into problems with IR35 etc but otherwise u should b ok
thsi is how i understand it anyway, others may be able to give you more advice
