An authentication error has occured (Code: 0x607)

Question

Hi all,
This one is driving me NUTS! The problem itself is when I go to connect to a session host using a web access server I get the error in the title.  This is only happening to some of my session hosts and not all.  I have compared them and can't find
a single difference.  I also cant find anything useful in the event logs about this.  Below is my setup.
A full RDS environment using all Windows Server 2012 Data Center.  Nothing 2008 R2.  All Clean installs.
I have 6 servers a VM's split evenly between 2 ESXi 5.1 Hosts.
1. MP-RDP-CB1.inucoda.net (Connection Broker 1)
2. MP-RDP-CB2.inucoda.net (Connection Broker 2)
3. MP-RDP-GW1.inucoda.net (Gateway Server 1)
4. MP-RDP-GW2.inucoda.net (Gateway Server 2)
5. MP-RDP-WA1.inucoda.net (Web Access Server 1)
6. MP-RDP-WA2.inucoda.net (Web Access Server 2)
inucoda.net is an network that is the Domain that all servers are joined to via 2 Domain Controllers splits between each ESXi Host.
My outside domain that you can get to from the web is ucoda.net
The connection brokers have all servers used including session hosts added to the server pool and are configured in HA mode. They use a SQL Server 2012 Fail-over cluster that is on a separate set of VMs for their database and the DNS is configured as round
robin. MP-RDP-CB.inucoda.net.  There are two entries of this each with one of the two IPs of the CB1 and CB2 servers.
On each CB server there is a RDS License server role installed with CALs installed and activated/registered. Both LIC servers have been added to the RDS deployment properties.
The GW servers each have the NLB role installed with an extra network adepter for NLB use. There is a DNS name of MP-RDP-GW.inucoda.net that points to the NLB IP of the GW Cluster.  Also both GW servers were added to the GW Server Farm part of the the
GW properties.  
The WA servers are also in a NLB Cluster with an extra adapter and a DNS of MP-RDP-WA.inucoda.net pointing to the NLB IP.
Up steam from our inside Windows Domain at our ISP level there is a DNS entry of MP-RDP-WA.ucdoa.net and it points to the NLB IP of the WA NLB Cluster.  (This is not a public IP, we require you be on our VPN to be able to access the IP).
For certificates we have a Comodo issued wildcard of *.ucoda.net with the corresponding Comodo Root Trust and Intermediate Certs. We also have a wildcard *.inucoda.net created by our inside CA.
The *.inucoda.net cert is used for the CB SSO, CB Publishing, and GW while the *.ucoda.net cert is used for the WA.
All session hosts have been configured to use the *.inucoda.net for their RDP sessions.
I can confirm that the *ucoda.net cert is used for the WA part and all other parts are reporting the *inucoda.net, all with no errors or warnings.
For each session collection only one session host is used with no apps, (just RDP).  Security is set to only use NLA, SSL 1.0, High.
On each session host I have verified that the *inucoda and *ucoda certs are installed and the internal CA and Comodo CA/Intermediate CA is installed in the correct stores.  I have also verified that COM Security has the domain\TS Web Access group set
with full perms for the Access and Launch/Activation. Also for WMI  Root\CMIV2\TermicalServcies Security has the domain\Ts Web Access group set with full perms. Lastly each group/user that has access to RDS is listed in the Remote Desktop users.
I've checked that both WA servers are listed in the TS Web Access group.
The GW servers RAS/RAP policies are set to be pretty open for testing with using any port, any network resource, and Domain Users and Domain Admins listed.
I have been trying to connect with Windows 8 and Windows 7 clients as the domain\administrator account.  Some of my session hosts connect fine and other don't .  It's always the same ones that connect and don't connect.  I can't find any difference 
between the.   I've also blown away my entire RDS and started over with just a 3 server single node model with no NLB or RR DNS and the same exact error happens on certain servers.  I have sense gone back to the 6 server setup described here
and again the same error on the same session hosts.
I have also tried Negotiate and RDS Compatible and disabling NLA only for security.  No change.  Now here is the interesting part. If I remove GW servers from RDS by just saying not to use them (not actually uninstalling them or anything), all
session hosts connect just fine every time.  When I first did my RDS setup I got he same error with code 0x607 for every connection attempt and found i had to set the RAS/RAP to use any network resource instead of Domain Computers.  However, it is
currently set like that and some still don't connect.   So it works with out the GW servers just fine.  It also works without them in the 6 node setup as well as the 3 node setup. 
I don't want to use it without the GW servers because since I am using all inside subnets with a VPN I have to add the CB IP/Name to my host file or it will not resolve and give an error about reaching the Connection Broker. Because I want to use a HA setup
this is no good as there are two servers for it.  That's why I use the NLB IP of the WA and publish it with outside DNS with our ISP. 
Any ideas at all??
Thanks,
Chris

Answer 1

Hi All,
I'm the original poster and if you have been following this I was never fully able to get things working.  Sometimes it would just work and other times it would just fail with the 607 error.  I have finally got it all working
for over a week now with multiple systems using it!  Below is a rather large explanation of what I had to do and what I learned about RDP.   I've included links to guides that helped a lot. 
First a small recap of my environment.
Using all windows server 2012.
Using two Gateways, Connection Brokers, and Web Access servers.
Two domain names, ucoda.net for external connection via web to web access servers and inucoda.net to inside windows domain that all servers are members of.
No external client systems are domain members, all just workstations.
Using two wildcard *.domain certs for both domain names.
External wildcard cert is from Comodo CA and internal wildcard cert is from my internal CA.
Now for how I setup the RDS environment.
I used this guide for setting up high availability of the connection brokers. 
http://blogs.msdn.com/b/rds/archive/2012/06/27/rd-connection-broker-high-availability-in-windows-server-2012.aspx
I used a back end SQL Server 2012 that was configured in a two node failover cluster for maximum HA.  As you can see by the guide it uses round robin DNS for load balancing the two CBs and does not require any hardware or software NLB.  
For both the two gateways and web access servers you need to use some kind of NLB.  You can use the MS NLB to create a virtual Cluster IP and set a DNS record for you gateway and web access name to point to that cluster IP. HOWEVER!  If you are
in a virtualized vmware environment as I am then you have some other things to do.  I can not comment as to Hyper-V setups, only vmware on ESXi-5.1.  If you use MS NLB then you must use it in Multicast mode and not Unicast. You must also setup static
ARPs on your Layer 3 router/firewall and Layer 2 switches.  The static ARP should match the NLB cluster IPs to the NLB Cluster MAC address.  Below are the guides for a Cisco Cat switch and ASA firewall.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006525
http://www.cisco.com/en/US/docs/security/asa/asa84/asdm64/configuration_guide/mode_fw.html#wp1224694 see adding a static arp section.
Now in the end I still was not fully happy with MS NLB as it is not Layer 7 aware and can only check the network health.  So I ditched my MS NLB for a linux solution. HAProxy.  It is a software NLB that is Layer 7 aware and easy to setup.
I used two Ubuntu Server 12 VMs with 1 GB RAM, 8GB HDD, and 1 vCPU each.  I also used Keepalived to setup virtual cluster IPs for HAProxy to use with failover.  so the HAProxy NLB is in high availability mode as well.
Setup HAProxy on Ubuntu
http://www.networkinghowtos.com/howto/compile-haproxy-from-source-on-ubuntu/
Configure HAProxy and Keepalved
http://leowadsworth.com/blog/2012/02/21/high-availability-load-balanced-web-servers-using-ubuntu-10-04-haproxy-keepalived-apache/ Skip the install part and see just the config parts for HAProxy and Keepalived. 
Now once NLB is done and you have DNS pointing to it you need to add both Gateways to a Gateway Web Farm.  Not required for the Web Access Servers only the Gateways.  for the Web Access server you only need NLB with a common DNS.  
Setup Gateway Farm
http://technet.microsoft.com/en-us/library/cc732370.aspx
Also as my client systems are not a part of the domain and have different subnets I needed to set the gateway RAP and CAP to allow users to connetc to any network resource.
Now that the main configuration was done and running I had to fix/fine tune/and mess with a bunch of other things!
There should be a domain user group account called TS Web Access Computers.  It should be populated with the Web Access server computers.  However in my deployment it was empty! great.  However, I also found other documentation that states
it should be populated with the Gateway servers.  So for me I added both Gateways, Web Access, and Connection Broker Servers.  I figured it can't hurt.
Now this group account needs to be added to COM security and WMI security for terminal services.  Below is a guide for both of these. I applied this configuration to every single system including all session hosts. 
http://technet.microsoft.com/en-us/library/ee891251%28v=ws.10%29.aspx
Now something interesting. Most of my systems were all server 2012 but a few were 2008R2 that had been upgraded in place to 2012.  For these systems the above config is till needed but you will find on the local systems user groups a TS Web Access Computers
group.  This is not in the local groups for 2012 but got merged over from 2008 R2.  So for it I also added the domain\TS Web Access Computers group to the local TS Web Access Computers group and added the local one to COM and WMI security as well.
Further into local user groups. On all systems in the deployment there is a local RDS Management Server group and it should have both Connection Broker servers listed.  I found this to be true on all my session hosts but
on the Connection Brokers them self they only have their own server listed but not the other connection broker server.  I added both to each.  I also found a few of my systems had a third ? SID account listed that was no longer was a real
account in the domain.  I removed it. Possibly from how many freaking times I had to re do my setup. 
Now on the Connection Brokers local group accounts there is a RDS Remote Access Servers group.  It should have all the Gateway and Web Access Server listed here.  In my setup I found only the Web Access Servers were listed and no Gateways.
 GREAT! This only needs to be populated on the Connection Broker Servers. There is also a RDS End Point Servers group and it should have every Session Host server listed.  Again only needed on the Connection Brokers.
That concludes user accounts/groups.
Now onto the fun land of Certs!
Something you need to make sure works is Revocation Checks!!!!!! It needs to pass from both the external client systems and internal server systems.  I had two certs used.  I used my *ucoda.net (external) for my Web Access Server Deployment and
my *inucoda.net (Internal) for The Gateway and both Connection Broker parts. 
My external was issued by Comodo so it passed rev checks just fine.  While my internal was issued from my internal CA and needed some work.  For the internal servers it could pass a rev check fine as it used the LDAP path in the CRL CDP
part of the cert.  However my clients are external and not part of the domain.  So it can't use LDAP.  To check rev checks I used:
certutil -f –urlfetch -verify <your_certificate>.cer
You can download it for Windows 7 and 8 systems from:
http://www.microsoft.com/en-us/download/details.aspx?id=7887 win 7 
http://www.microsoft.com/en-us/download/details.aspx?id=28972 win 8
To get it to pass on my client systems I had to add a CRL CDP http point that they could access instead of the LDAP point. In short on you internal CA you need to add a CRL that uses the FILE path to publish rev lists to a file share.  The file share
is located on a server that has IIS and public access.  You then create a virtual directory with read rights to the that share in IIS and add a CRL HTTP point using the external FQDN of public web server for the CRL site.  Below is a guide
to do all of this.
http://blogs.technet.com/b/configmgrteam/archive/2009/05/01/how-to-publish-the-crl-on-a-separate-web-server.aspx
Now once this is done you need to re generate a new cert and apply it to your RDS environment so it has the updated CRL CDP.
Now after this I was able to pass using certutil tool.  But! wait there's more!  When I tried to connect to a server using normal RDP (not the full web access and gateway deployment), just direct to the end server I still got the warning about
a rev check fail! I just didn't get it!  After a ton of researching it appears that RDP will only use LDAP and OCSP CDPs and not HTTP.  Great!  So while it passes the rev check from the tool it still fails for RDP. 
So next was to add a OCSP CDP and Online Responder.  I chose to add the Online Responder role to my public web server where I had just added the HTTP CRL CDP.  Below are a few guides about setting this up and configuring your CA
to use it.
http://www.windowsitpro.com/content1/topic/online-certificate-status-protocol-ocsp-in-windows-server-2008-and-vista--103523/catpath/security
http://blogs.technet.com/b/askds/archive/2009/06/24/implementing-an-ocsp-responder-part-i-introducing-ocsp.aspx
http://www.sysads.co.uk/2012/10/install-and-configure-ca-online-responder-ad-cs-part3/
I fond all helpful.  Now here comes a part that drove me NUTS!.  All these guides show that after installing the Online Responder role it automatically adds a ocsp webapp to IIS!  This is to be the CDP point you add to
the CA. THIS IS NOT TRUE FOR 2012!  It does not add the IIS config what so ever.  Luckily I manged to find this:
certutil -vocsproot  
You need to run that command on the web server where you installed the Online Responder role.  It will add the IIS config and app pool!
Now once this is all done and tested you need to re issue the cert again so it has the new OCSP CDP in it and install it in RDS deployment.
Finally after this I received no rev check errors for RDP!
Some more things on certs.
For all my servers I installed the internal and external cert to their computer personal store and made sure the corresponding root and intermediate root certs were installed in the correct stores.  I also did this on my external client systems.  Be
sure to add your internal CA's root cert to the trusted root store of you client systems or again the certs generated from it will not pass fully as the client system will not know to trust the CA that issued the cert.
Now you also need to install a cert for each session host to use for RDP.  I really recommend wildcards as it much easier to just use a *domain cert for RDS deployment and install it on each session host for RDP than to have unique ones for
each session host.  You use to be able to easily add a RDP cert in 2008R2 to a session host.  This is now gone in 2012.  So to do it you need to use the power shell.  Below is  guide on how to do this.
http://blog.skadefro.dk/2012/08/windows-server-2012-server-8-remote.html
Now I also used a little utility to help check that my certs were installed on each server correctly. I found on a few of my servers where one of my certs was missing the private key or had other problems.  This free tool from DigiCert can help and
can also be used to test certs for rev checks.
https://www.digicert.com/util/
Lastly there is the issue of what RDP version you are using.  For me my systems they are all server 2012.  I found the only way to get SSO to fully work without a 2nd login prompt was to update all my Windows 7 RDP clients to the latest RDP. 
http://blogs.msdn.com/b/rds/archive/2012/10/23/rdp-8-0-update-for-windows-7-sp1-released-to-web.aspx
Well after all that I was able to access every RDSH in my environment without a single error!  It has been a ridiculously long and pain full journey.  I think MS needs to do more work and documentation of  2012
RDS as it's changed so much, needs a better way to issue session host certs for RDP instead of just the power shell, and needs more documentation and clarity on RDP rev checks.   I hope this helps others and if anyone wants to see what my configs
look like for HAProxy if they decide to use it feel free to ask.
Thanks and Good Luck!
Chris