I’d recommend both, especially if your servers are likely to be ddos/dos attacked and susceptible to other similar threats as hardware firewalls come into their own with this kind of threat. As a rule of thumb think of them as a front line defence as more often than not without reconfiguration they do little more than scan, route and drop packets.
Software firewalls are very similar but protect individual machines and are necessary if your sever utilises FTP to help ensure and also remove any threats that may have slipped past. Though many data centred already utilise network level (hardware) firewalls.
A more indepth but by no means a complete 101 could be
http://www.windowsnetworking.com/art..._Firewall.html