Send Connector, Removing one Source Server

Question

I have one send connector. I have two exchange servers not in HA, when I attempt to remove one of the source servers, email backs up in the que and I get this error
451.4.4.0 Primary target IP address responded with 451.5.7.3 cannot achieve exchange server authentication
I then have to add the other server back for email to go out again. Both servers are listed as source servers on the send connector. But leaving the one I want, its not routing email through that server. Do I have to restart any of the services after making
this change?

Answer 1

Here is the output
PS] C:\Users\administrator.PHDMI\Desktop>Get-SendConnector | FL
AddressSpaces                : {SMTP:*;1}
AuthenticationCredential     :
Comment                      :
ConnectedDomains             : {}
ConnectionInactivityTimeOut  : 00:10:00
DNSRoutingEnabled            : False
DomainSecureEnabled          : False
Enabled                      : True
ErrorPolicies                : Default
ForceHELO                    : False
Fqdn                         : owa.xxx.com
HomeMTA                      : Microsoft MTA
HomeMtaServerId              : xxxEXCH
Identity                     : XXX
IgnoreSTARTTLS               : False
IsScopedConnector            : False
IsSmtpConnector              : True
LinkedReceiveConnector       :
MaxMessageSize               : 10 MB (10,485,760 bytes)
Name                         : xxx
Port                         : 25
ProtocolLoggingLevel         : None
RequireOorg                  : False
RequireTLS                   : False
SmartHostAuthMechanism       : None
SmartHosts                   : {outbounds6.obsmtp.com}
SmartHostsString             : outbounds6.obsmtp.com
SmtpMaxMessagesPerConnection : 20
SourceIPAddress              : 0.0.0.0
SourceRoutingGroup           : Exchange Routing Group (DWBGZMFD01QNBJR)
SourceTransportServers       : {xxxEXCH0, xxxEXCH}
TlsAuthLevel                 :
TlsDomain                    :
UseExternalDNSServersEnabled : False
I added the xxx to remove our company name. I am trying to remove xxxEXCH0, but when I do email gets stuck in the que for outgoing. Now I didnt restart any services after removing EXCH0 as a source server.