You can also manipulate things in your favor.
You can write your robot.txt files to specify only certain folders that are inaccessible.
This will cut down on a lot of bandwidth while still allowing your site to be spidered.