You can attack this question from a variety of angles, IMHO.
One of the common statements I see in the hosting industry is that downtime hurts. The problem is, most folks think that they should experience no downtime when they pay a couple dollars a month. They simply expect too much for too little; yet they are the very first to rail against the host for the downtime. The fact is that in the hosting industry, you get what you pay for. If you pay $4-$5/month, you'll get web hosting worth exactly that. That $4-$5 month is barely enough to cover hardware and support costs; much less the other expenses of a company. The resulting function of that operation is that customers get cheap hardware, cheap support, and ultimately cheap service.
It's not uncommon to hear folks pine away about losing "hundreds" or "thousands" of dollars when their site is down for 10-15 minutes; yet they refuse to pay any more than five bucks for a host. If your website is really that mission critical, your budget and service provider choice will reflect that. Business is certainly all about the $, more explicitly the budget and frugality, but even the shrewdest, most-miserly business person recognizes that some corners just cannot be cut.
In the long run, I think it settles down to what you want as the customer. Would you prefer a less-capable support team and poorer quality hardware for a lower price; or, would you prefer higher-end hardware and accompanying knowledgeable support for a larger hit to the wallet?
If it is a personal or hobby site, I think the former will be the common answer. If it is a business website, the latter should be the standard answer. Again, though, it depends upon whom you talk to.
In the end, I'd easily answer that I'd rather pay more for a better provider. I learned that lesson many years ago now.