The really low-cost VPS are availiable at gazzin.com - this company has many plans to choose from and to all appearances is quite reliable provider.

Also, ******.com, which has alredy been suggested here provides cheap deals, but they are only windows related. Actually it doesn't matter, unless linux is more prefferred.