I may be slightly biased but you might want to consider a VPS instead of a dedicated server. They generally are more reliable than a low end dedicated server since they run on high end servers with redundant power and hardware RAID etc.

A VPS can also be upgraded with a few clicks so your server can grow in power with your business with no downtime and minimal cost.

If you application is very disk write intensive however, a VPS can lag behind and you'll need a dedicated server.

The bottom line is to know your requirements.