I would say go with the .204 just because it has a little more range than the Hornet. And before I get yelled at for saying that, what I mean is the bullet shoots flatter and gives a little more predictable flight and you have a better selection of bullets with the .204 IMO.
The Hornet is my wife's favorite rifle to shoot because of the very very light recoil, low muzzle blast, and she generally only shoots within 100 yards. But when we set up soda cans at 300+ yards, she always wants the .204.
Obviously the .204 has very little recoil to begin with, but my wife seems to think it makes a difference shooting the Hornet the rest of the time.
Another plus to the .204 is brass. Locally we seem to get Hornet brass for a few months, then it disappears, then it comes back a few months later.... and the .204 brass is always available when I need it