Cheap is a relative term depending on what you get. You can find VPS's out there for $15, but there's nothing installed on it. You can find cPanel installed for $20 and up, and you can find Managed VPS machines from $25 and up.

Then depending what it is you're doing on the site your requirements for disk space and memory will change, and that can increase your fees too.

Check out the Offers and Request section - I'm sure someone is posting special discounted rates in there.