RPC over HTTP trouble Exchange 2013/2007 coexistence, 2013 RPCProxy cannot ping GC.

Question

I currently have an Exchange 2013/2007 coexistence scenario which gives me trouble with the RPC over HTTP part with users with a 2007 mailbox. the MS RCA website performs 2 tests with the MAPI address book endpoints, once against a 2007 mailbox server and
once a against a GC/DC, the last one fails. The logs are from our test domain, but the exact same happens in the production domain. running CU8 (recently upgraded, but problem was exactly the same with CU7)
Testing the MAPI Address Book endpoint on the Exchange server.
The address book endpoint was tested successfully.
Additional Details
Elapsed Time: 7872 ms.
Test Steps
Attempting to ping the MAPI Address Book endpoint with identity: exmb11.domain.test:6004.
The endpoint was pinged successfully.
Additional Details
The endpoint responded in 156 ms.
Elapsed Time: 4153 ms.
Testing the MAPI Address Book endpoint on the Exchange server.
An error occurred while testing the address book endpoint.
Additional Details
Elapsed Time: 3079 ms.
Test Steps
Attempting to ping the MAPI Address Book endpoint with identity: tdc01421.domain.test:6004.
The attempt to ping the endpoint failed.
 <label for="testSelectWizard_ctl12_ctl06_ctl02_ctl09_ctl00_tmmArrow">Tell
me more about this issue and how to resolve it</label>
Additional Details
The RPC_S_SERVER_UNAVAILABLE error (0x6ba) was thrown by the RPC Runtime process.
Elapsed Time: 3079 ms.
texmb11 = ex2007 mailbox
texfr11 = ex2007 CAS
tdc01421 = DC/GC
texch31 = 2013 multirole
other symptoms :
Browser test
https://texch31.domain.test/rpc/rpcproxy.dll?texmb11:6004
--> 503 (which is correct)
https://texch31.domain.test/rpc/rpcproxy.dll?tdc01421:6004
--> 404.0 Not Found
RPCPing
rpcping -t ncacn_http -s texmb11 -o RpcProxy=texch31.domain.test -P "user,dom,*" -I "user,dom,*" -H 2 -u 9 -a connect -F 3 -v 3 -e 6004
Success
rpcping -t ncacn_http -s tdc01421 -o RpcProxy=texch31.domain.test -P "user,dom,*" -I "user,dom,*" -H 2 -u 9 -a connect -F 3 -v 3 -e 6004
Fails!
so the Ex2013 RPCProxy doesn't proxy to the DC/GC, but the RPCProxy to an ex2007 mailbox server works fine, so the authentication methods configured are correct i'd say..
the same tests using the ex2007CAS server as RPC proxy all succeed!:
MS RCA is all green
https://texfr11.domain.test/rpc/rpcproxy.dll?texmb11:6004 
--> 503
https://texfr11.domain.test/rpc/rpcproxy.dll?tdc01421:6004 
--> 503 
rpcping -t ncacn_http -s texmb11 -o RpcProxy=texfr11.domain.test -P "user,dom,*" -I "user,dom,*" -H 2 -u 9 -a connect -F 3 -v 3 -e 6004
Success
rpcping -t ncacn_http -s tdc01421 -o RpcProxy=texfr11.domain.test -P "user,dom,*" -I "user,dom,*" -H 2 -u 9 -a connect -F 3 -v 3 -e 6004
Success!!
Logs
all tries against the 2013 CAS server generate '404' log entries in several logs int the Logging directory, the most explicit being the one in Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\RpcHttp : HttpProxy_xxxxxx.LOG
2015-04-30T19:58:00.153Z,895cdf07-f2eb-4beb-b787-da02ba11b0c2,15,0,1076,0,,RpcHttp,webmail.domain.test,/rpc/rpcproxy.dll,,Basic,true,DOM\user,,,MSRPC,10.10.142.132,TEXCH31,404,,MailboxGuidWithDomainNotFound,RPC_IN_DATA,,,,,,,,,4,,,,1,,,0,,0,,0,0,,0,2,0,,,,,,,,,1,1,0,,1,,2,2,,?TDC01422.domain.test:6004,,BeginRequest=2015-04-30T19:58:00.153Z;CorrelationID=<empty>;ProxyState-Run=None;ProxyState-Complete=CalculateBackEnd;EndRequest=2015-04-30T19:58:00.153Z;,HttpProxyException=Microsoft.Exchange.HttpProxy.HttpProxyException:
RPC server name passed in by client could not be resolved: TDC01422.domain.test    at Microsoft.Exchange.HttpProxy.RpcHttpProxyRequestHandler.ResolveToDefaultAnchorMailbox(String originalRpcServerName  String reason)    at
Microsoft.Exchange.HttpProxy.RpcHttpProxyRequestHandler.ResolveAnchorMailbox()    at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.InternalBeginCalculateTargetBackEnd(AnchorMailbox& anchorMailbox)    at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<BeginCalculateTargetBackEnd>b__3b();
the error suggests the name could not be resolved. but nslookup works fine. i can ping the tdc01421 correctly form texch31, it returns me the ipv4 address. i can telnet to port 6004 from texch31 server to tdc01421 giving me the correct 'ncacn_http' answer..
i look at the 'ValidPorts' and ValidPorts_Autoconfig_Exchange reg keys and filled them with the same i have on the 2007cas servers (all ex servers, all dc/gc's, all of them with netbios & fqdn ports 6001,6002 & 6004..) but also no success.
im out of ideas by now..

Answer 1

Hello togehter, 
if anyone is using the Interim Update (Hotfix) "Exchange2013-KB2997209_2997847-x64-en.msp" please be aware of the following bug in this hotfix: 
NOTE: Another issue that we have seen only from some E2013 On-Premises customers who installed
fixes for any of the above E2013 CU6 related issues is that “Using OWA, users cannot create new messages or reply to existing messages” – These are mostly installer issues where OWA files are not copying correctly in the new version directory. Ideally the
content of these two folders, i.e. “V15\ClientAccess\Owa\prem\15.0.995.31” and “V15\ClientAccess\Owa\prem\15.0.995.29” should be identical, now if for any reason, they are not then it can break the OWA experience like described above. Take these steps if you
run into related issues:
-Backup the contents of “15.0.995.31” folder to a different folder/directory
-Copy the files and folders from “15.0.995.29” to “15.0.995.31” folder – this should fix the issue …
Best, 
Martin