I don't think that it is a huge risk personally as long as you are careful with the anti-virus you run. It's not the best of things, but honestly I only got my first firewall about 14 months ago and my machine ran fine before then. I have no qualms turning it off again.

But I would also think that you can just tell the firewall to allow that site.