If you are not familier with server administration, make sure that you get a MANAGED server. Self managed servers can be cheap, but you need certain amount of skills to manage a dedi.

Also, be sure about what you need : OS/Control panel etc. Learn the basic and requirements before going to a host.

Also look for fast support, good network and overall good reputation.