Cannot find a token authenticator for the 'System.IdentityModel.Tokens.X509SecurityToken' token type. Tokens of that type cannot be accepted according to current security settings.

Question

i am using a custom binding in the BTS Adapter with the following elements (similar to TransportWithMessageCredential with both the client and the server certs)
 encoding (soap11)
 https transport
Security : CertificateOverTransport
Problem: the request is sent successfully, but when i receive the response in BizTalk i get the following error
System.ServiceModel.Security.MessageSecurityException: An unsecured or incorrectly secured fault was received from the other party. ,after turning on tracing in the WCF Trace the following error is present "Tokens of that type
cannot be accepted according to current security settings. "
Solutions tried
1) Changed the security to MutualCertificate , this time request also fails with the following error message  The remote endpoint did not provide a domain name system (DNS) claim and therefore did not satisfied DNS identity 'xxxx.com'.
This may be caused by lack of DNS or CN name in the remote endpoint X.509 certificate's distinguished name.
Binding configuration
 <behaviors>
      <endpointBehaviors>
        <behavior name="EndpointBehavior">
          <clientCredentials>
            <clientCertificate findValue="XXXXXXXXXXXXXXX" x509FindType="FindByThumbprint" />
            <serviceCertificate>
              <defaultCertificate findValue="XXXXXXXXXXXX" storeName="TrustedPeople" x509FindType="FindByThumbprint" />
              <authentication certificateValidationMode="None" revocationMode="NoCheck" />
            </serviceCertificate>
          </clientCredentials>
        </behavior>
      </endpointBehaviors>
      <serviceBehaviors>
        <behavior name="ServiceBehavior" />
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <customBinding>
        <clear />
        <binding name="XXXXXXXXX">
          <textMessageEncoding messageVersion="Soap11" />
          <security allowSerializedSigningTokenOnReply="true" authenticationMode="CertificateOverTransport" requireDerivedKeys="false" securityHeaderLayout="Lax" messageSecurityVersion="WSSecurity10WSTrust13WSSecureConversation13WSSecurityPolicy12BasicSecurityProfile10"
requireSecurityContextCancellation="false">
            <secureConversationBootstrap />
          </security>
          <httpsTransport />
        </binding>
      </customBinding>
    </bindings>
Thanks -Madhu

Answer 1

Please refer to the similar discussion:
http://social.msdn.microsoft.com/Forums/en-US/6a3d38ee-30ca-43fb-b906-6e95808df69d/cannot-find-a-token-authenticator-for-the-systemidentitymodeltokensx509securitytoken-token?forum=wcf