I voted for 3+ seconds. If you look at GTmetrix's page load speed results, some sites that load in one second get lower scores than others that load in 2 seconds. Once you get up to 3 seconds, something isn't right. Of course, this depends on what type of site this is. Forums typically load in 4 to 5 seconds. Rackspace.com loads in 7 seconds, but scores well on PageSpeed scores. I'm actually not so quick to move on if a site loads slow, of course unless I can fix lunch while waiting.