Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!
It might not harm your chances, as long as you have a good rapour with the interviewer. However, if an opportunity arises in the company again in the future, I'm sure the agency would be keen to bite my hand off, assuming they were honest about the feedback they provided from the client.
I send invites to all useful business contacts I meet, including interviewers (even if I don't get the gig). Downside - what downside. If they really don't like you, they won't accept. Big deal, and as for future chances, who knows.