I purchased a unmanaged Virtual Private Server(VPS) service from a company and I need to setup the mail server on it. It is running CentOS 5. Now my question is, to send and receive mail can I just install a SMTP software such as postfix? Or do I need to also install a POP3 server software. (Disclaimer: I am far far away from being a linux expert). My VPS company needs to give them an MX record information. So I am thinking I just install SMTP on this server and give them the IP address of the server for MX record(the server is primarily a web server). Please advise.

Favor ask web hosting company to help you. Or pay them for it. I think in this way you will save time, not money.

First, do you actually need to run mail on this server, and if so do you need to send and/or receive mail? Or is this VPS just intended to be a web server?

If email for your domain is handled elsewhere and you don't need to send mail from the VPS, you can avoid setting up any mail server locally. This will simplify your configuration as well as reduce potential security issues.

If you need to send mail from this machine, for example you have web pages that send email but not receive it, installation of an SMTP server like Postfix or Exim will be sufficient.

If you need to send and receive email, you'll need an SMTP server plus a POP or IMAP server. However, in this case you might want to look at having someone else handle this if your Linux skills aren't too strong - there are a lot of pieces to get setup, especially if you're looking to do spam filtering as well. There are dedicated email offerings from hosting companies, or even GMail for your Domain would work for this.

Get cPanel...

With a question like that, the best answer is to either go managed or, in the very least, find a control panel. Seriously.

If you have issue installing it, it's going to be even worse troubleshooting it when it doesn't work properly.