Can't Stop Intense SPAM Attack

Question

Hello,
As of the past day or two, I've started encountering so major SPAM attacks to one of our multiple servers. We have a mail server on the domain, but this one isn't it (as its only used for QTSS streaming and some web hosting).
Basically, I first noticed it when I got two seperate emails from postmasters saying to fix the problem, and then noticed "smtp" showing up on "top" continuously and draining the CPU. So far, I've tried turning the web server off and on, turning the mail server off and on, keeping the mail server on with SMTP with closed relay, and nothing has worked. Eventually, I had to set up a firewall to block port 25 incoming and outgoing which has seemed to solve the problem (but there are continuing entries related to MX records in the system.log every minute).
I noticed that before this started, there were tons of 404's lookin for formmail.cgi and some php mail client, and then since there's been no odd web traffic at all. I've also checked /tmp and /var/tmp, and nothing out of the ordinary either. Otherwise, this is what i've been seeing in system.log:
With firewall off:
Dec 31 03:35:35 xserve2 lmtpunix[15146]: warning: unable to post message for user: colin, mail is not enabled for this user
With firewall off and making the colin account:
Dec 31 03:36:21 xserve2 postfix/smtpd[13569]: warning: 218.248.240.72: hostname nda-svr-mta-out-01.bsnl.net.in verification failed: Host not found
Dec 31 03:36:22 xserve2 postfix/qmgr[13546]: warning: premature end-of-input on private/smtp socket while reading input attribute name
Dec 31 03:36:22 xserve2 postfix/qmgr[13546]: warning: private/smtp socket: malformed response
Dec 31 03:36:22 xserve2 postfix/qmgr[13546]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description
With firewall on:
Dec 31 14:06:51 xserve2 postfix/smtp[22534]: warning: valid_hostname: empty hostname
Dec 31 14:06:51 xserve2 postfix/smtp[22534]: warning: malformed domain name in resource data of MX record for altavista.net:
Dec 31 14:07:03 xserve2 postfix/smtp[22232]: warning: valid_hostname: empty hostname
Dec 31 14:07:03 xserve2 postfix/smtp[22232]: warning: malformed domain name in resource data of MX record for ayhoo.com:
If anyone could help, I'd most MOST grateful, as we do use the php mail command on one of our big sites hosted on that server and obviously that whole system is down right now...
----- EDIT -----
It seems as though the smtp is still coming up even with port 25 firewall on.
The firewall looks like its blocking a TON of packets, but the smtp is still utilizing over 80% of my CPU.

Answer 1

Hey MrHoffman,
Again, thanks for all the help with this... its very appreciated over here. It's a major university website as well, so last thing we want is to be blacklisted.
I had a look through ipfw.log just now and can see the incoming blocking from late last night through today... some sample lines:
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 194.106.141.85:20782 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 62.41.48.171:42706 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 65.54.246.103:7820 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 85.31.177.235:55054 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 194.109.127.153:3302 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 129.15.3.86:34962 204.194.30.246:25 in via en1
Dec 30 23:38:38 xserve2 ipfw: 65534 Deny TCP 217.174.202.185:15703 204.194.30.246:25 in via en1
Now in addition to that, last night when incoming 25 was still open and then I locked it, all the ipfw denials were to port 25. Now... today, there are more ports as well during the big spikes of ipfw problems today.. here's a set:
25, 135 -139, 445, 1026, 6050, 33438, 49423-4
Needless to say, looking at all of the entires, is that the IPs are all over the map (and definitely not on our 204.194.30.x or 128.2.103.x) so it does indeed look like it's coming from the outside. The good news is that right now, all smtp processes and outgoing problems have stopped completely, and the only thing happening is this once in a while spike of incoming packets like the logs above. It starts out of nowhere, keeps going for around 15-45 min, and then just stops, but at least nothing's coming in. For the time being just in case, i've locked off port 25 outgoing as well, since I did conclude that when this started happening, I discovered that the all the outgoing spam was coming out of postfix, given the millions (yes millions) of emails in que in /var/spool/postfix. But as of now looking at the network stats using cacti and some other methods, we've found that the past few days, our outgoing bandwidth was between 200Kb/s to 2Mb/s sustained during the ordeal, but it's now down to under 20Kb/s patches, mainly from me doing admining.
As of right now, SMTP incoming is unchecked and authentication required is set in the mail pane in server admin, and the "accept SMTP relays from only..." checkbox is checked with only 127.0.0.1/32 listed, but this doesn't seem to help at all the second I open up incoming port 25... so I would think that the relay is closed, but it just doesn't seem to help... unless I'm missing something?
With that said, I'd totally be for doing a basic firewall!
I use one here at home (a netgear FVS114), although I have cable internet here, and wasn't sure if I would be able to just use the same kind of router-esque firewall like that at school and just run the LAN network port into the WAN and connect the LAN ports to our server room switch?
Meanwhile, I did get computing services involved mid afternoon, and someone's actually there today and the ball seems to be rolling. I have them cc'd on the board post so hopefully they'll gain some insight from all the great help on here so far.
Thanks again!