All well and good, but if the client has severed business dealings with the agency their handcuff clause has no value, unless - and only unless - the agency can offer him immediate comparable work with another client on the same terms to the OP and the agency. Otherwise they have no business to protect and the handcuff can be ignored.
Comment