Yes SVN is pretty good, free, and has good GUI tools like Tortoise SVN to make it easy to use. IIRC from a previous client, CVS is all file based, so if you say commit 10 files as part of one change you get 10 commits with different version numbers for each, which then makes tracking changes as a whole much more difficult. And I think what Ministrone says about renaming or moving files is right too. SVN is really CVS done properly, so there's no reason to go with CVS.
GIT sounded quite interesting, but seems to be based around large distributed teams which might not be quite so good for a smaller team. It seemed to me that the actual day to day working would be more difficult, and would require much more management. But I don't speak with any experience of using it.