Exchange 2003 SMTP resolves to incorrect MX record for Outbound emails

Question

Starting October 2007, we have encountered numerous incidents where users complaint that they can not send emails to randomly with Relayed Denied error. 
First we thought the problem is with the receipient servers, but after reviewing the SMTP log intensively, I have noticed that the SMTP service on Exchange Server 2003 SP2 tries to send out emails to wrong IP Addresses (instead of sending out to listed IP of MX record, it tries to send to primary domain IP address).
To test further, we have changed the DNS server of the Exchange Server to public DNS service provided by the ISPs and still received same result.
If the user tries to send the email again (from the bounced back message), it some times goes through.
Has anyone know what the problem is?
Have checked the server configuration, DNS configuration, ran all updates, and yet still same result.  Is one of the Microsoft Updates causing this issue?

Answer 1

Here is the official explanation i got from MS, sounds like it is doing what they expect it to do.
Problem Description
When Exchange 2003 sends e-mail to the Internet using DNS to resolve external domain names a DNS query is done for the remote domain's MX records. In the event that the initial MX query returns a "server failure", Exchange 2003 will fall back to a DNS A record query. If this query is, in turn, successful and there is an A record for the domain in question that happens to be listening on port 25, Exchange will make a connection to that server. However, if this server does not relay, or accept e-mail for the intended domain a NDR will be generated with a 5.7.1, or
5.5.0 error message.
There are several domains that have A records that point to IP addresses that accept connections on port 25, but do not relay for the same name space:
Eg: Mindspring.com; Earthlink.net
Resolution
Resolve the DNS resolution problems that are causing occasional MX record failures.
Examples:
1. The Exchange server has multiple default gateways set on a multi-homed server.
This is not a recommended configuration and can cause a number of unexpected network problems. If the internal DNS server the internal NIC points to cannot resolve external DNS queries, this problem can occur.
Action: remove the default gateway from one of the NICs.
This doesn’t applies to us, as we don’t use Internal DNS
2. If the Exchange server points to multiple DNS servers on TCP/IP properties, or on the SMTP Virtual Server properties, verify that each of these DNS servers will return MX records properly. If one of the DNS servers does not return DNS records consistently, rectify this problem.
Action: remove this DNS server from the external DNS tab.
Workaround:
Bypass DNS by creating a SMTP connector with address space of <domain.com> and forward to smart host as the IP address the MX/A points to.
Exchange Query Explained
1. If Exchange finds the MX record(s) for a remote domain (through DNS query) it will not fall back to an A record query in the event the MX record (more precisely, the A record the MX record points to) is temporarily not responding on port 25. Instead, Exchange will try another MX record if one exists. If no other MX records exist and none are responding on port 25, the queue will go into a retry state for this remote domain. When that retry state has expired, Exchange will again attempt to resolve MX records for this domain, and so on.
2. Exchange will fall back to an A record query for the remote domain in the event that no MX record can be found. This is not to say that the MX record does not exist - only that when Exchange issued a DNS query for the remote domain the DNS response was "server failure", which basically means no MX was found. So, the MX record may indeed exist, but Exchange simply could not find it through DNS resolution.
3. Exchange then queries the A record and this happens to return a result. If the A record is listening on port 25, but does not accept mail for <domain.com> then we will get the NDR with 550 5.7.1 unable to relay (or some variation of this NDR error code) when Exchange connects and attempts to send the e-mail.
So, the intermittent nature of the problem is caused by a failure to find the MX record and a success in finding A record. This can happen for a variety of reasons, but it boils down to flakey DNS resolution.
For example:
a. A bad DNS server listed as a forwader on the DNS server.
b. A bad DNS server on the TCP/IP properties.
c. A bad DNS server on the SMTP VS (external DNS servers)
d. Multiple default gateways on Exchange (ie. multiple NICs on different networks, each NIC having a default GW)
A netmon on Exchange will show the failed MX lookup and successful A record lookup; however, it may not be conclusive unless the "bad" DNS server is listed in the netmon trace. For example, if Exchange points to internal DNS server(s) for name resolution and these DNS servers service these DNS queries by Exchange, you may need a netmon on the DNS server(s) as well to determine which DNS server/Forwarder/etc is at fault.
Regards,
Shahul Hameed Dasthagir
Support Engineer | Enterprise Communications Support