Legal - Working for employer's clients

Hi all

I am thinking of going freelance. I have been working at a design agency for a few years and just wondered if anyone knows the legality of working for your previous employer's clients? I have signed a contract to say I wont work with their clients for something like a year after I leave - Does anyone know how powerful these agreements are in the following scenario?

1. I leave my job, go freelance
2. One of my company's clients contacts me for work
3. I accept

Any help would be very much appreciated!!