Claims debacle (error) with Term Store: "Could not retrieve a valid windows identity" for all sites in a particular web app.

Question

When I pull up the Term store in CA or any MySite collection, it works.
When I do so in any other site collection (HNSCs, incidentally), It doesn't return any term stores.
My ULS log immediately before and after the "/_vti_bin/taxonomyinternalservice.json/CheckPermission" POST on termstore .aspx triggers the WCF call:
Claims Authentication af30y Verbose Claims Windows Sign-In: Successfully signed-in the the user 'contoso\domainUser' for request url 'https://sp13-root-prd.contoso.com/_vti_bin/taxonomyinternalservice.json/CheckPermission'.
Claims Authentication af30q Verbose Updating header 'LOGON_USER' with value '0#.w|contoso\domainUser' for the request url 'https://sp13-root-prd.contoso.com/_vti_bin/taxonomyinternalservice.json/CheckPermission'.
Authentication Authorization agb9s Medium Non-OAuth request. IsAuthenticated=True, UserIdentityName=0#.w|contoso\domainUser, ClaimsCount=77
Logging Correlation Data xmnv Medium Site=/
Topology e5mc Medium WcfSendRequest: RemoteAddress: 'http://CONTOSOFE3:32843/00e6d55691824965ac223f1d1cfae6d2/MetadataWebService.svc' Channel: 'Microsoft.SharePoint.Taxonomy.IMetadataWebServiceApplication' Action: 'http://schemas.microsoft.com/sharepoint/taxonomy/soap/IDataAccessReadOnly/GetChanges2' MessageId: 'urn:uuid:590e916c-c89a-4f89-9819-a82c97fabcaa'
Claims Authentication bz7l Medium SPSecurityContext: Could not retrieve a valid windows identity for username 'contoso\domainUser' with UPN '[email protected]'. UPN is required when Kerberos constrained delegation is used. Exception: System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: WTS0003: The caller is not authorized to access the service. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.UnauthorizedAccessException: WTS0003: The caller is not authorized to access the service. at Microsoft.IdentityModel.WindowsTokenService.CallerSecurity.CheckCaller(WindowsIdentity callerIdentity) at Microsoft.IdentityModel.WindowsTokenService.S4UServiceContract.PerformLogon(Func`1 logonOperation, Int32 pid) at SyncInvokeUpnLogon(Object , Object[] , Object[] ) at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc) at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet))..
Claims Authentication g220 Unexpected No windows identity for contoso\domainUser.
The "The caller is not authorized to access the service." message seems pertinent.
Both web apps are using only NTLM auth.
The url for both web apps ends in the same contoso.com domain. 
I get the same errors no matter what account I use, including the install account.
Things I've tried:
Deleting and building a new HNSC root web app and site. Error happens in all sites in all web apps except the PBSC hosting MySites.
Giving the root site app pool identity full control of the metadata service app (even though the MySite identitiy doesn't have it)
Giving the root site app pool identity full permissions on the metadata service app.
Comparing database and web app config permissions between dev (where everything works perfectly) and prod (where it does not).
Made sure IIS auth settings on both sites are identical
Both sites are using the same SSL certificate (though the call to the web service appears to be http)
Reprovisioned the metadata service app with a new database and new app pool identity.
Made sure C2WT is running. Tried it with the service stopped as well.
Web.configs are identical between working and non-working apps.
I'm stumped but still Googling. I'm hoping to avoid having to call Micrososft. Any help would be appreciated!
UPDATE:
Interestingly, when I restored the web application from backup (via CA), I ended up with 3 identical "Windows Authentication" authentication providers assigned to the problem web app. Since there was more than one, I was directed to the provider-chooser
page when visiting the site. Upon choosing 1 of the 3, I was authenticated, and *poof*, no more authentication errors and the term store loaded term sets as expected.
Of course, 3 providers was not an ideal state, so I grabbed the one that worked (#1) via get-spauthenticationprovider, and assigned it to the web app via set-spwebapplication, and my problem returned.
I am currently updating the farm to SP1 from June 2013 CU. Fingers crossed.
Update:
The update to SP1 went smoothly, but did not resolve the issue. Also related (I believe) are the random authentication errors when trying to upload images to some libraries, and 401-errors on the accessdenied.aspx page itself.
Update:
The problem is resolved, seemingly after making 4 changes. I'm trying to narrow down which change was the cure, if any:
I installed SP1 on all 6 servers, rebooted and upgraded. This appeared to have no effect.
Removed an old login from SQL that no longer existed in AD because of this ULS error:
System.Runtime.InteropServices.COMException: The user or group contoso\svc_xxxxxxxxx' is unknown., StackTrace:    at Microsoft.SharePoint.Utilities.SPUtility.GetFullNameFromLoginEx(String loginName, Boolean&
bIsDL)
This login was the identity of the application pool that used to run the web app in question.
This login was the schema owner of a schema named after itself on every SharePoint database so I changed the schema owner to dbo but left the schema attached.
The problem may have surfaced initially when the app pool identity was changed in CA, but went unnoticed?
Note that the web app had been deleted and recreated many times with a new identity and pool to no avail, but the URL remained the same throughout each attempted fix. Relevant?
Grasping at straws, I changed the app pool identity for this web app to the same one that runs the MySite web app pool as per this only slightly related problem: http://www.planetsharepoint.org/m/preview.php?id=372&rid=34764&author=Vlad+Catrinescu
I changed the authentication method from NTLM to Negotiate.
I am rolling back #3 and #4 to see if the issue resurfaces.
Update:
It doesn't appear to have been the NTLM/Negotiate setting. Web app is currently set to NTLM and all is well. No strange accessdenies, and term Store is still manageable from all sites.
Update: Sorry for the delay. I am administering 6 farms these days. Will update as soon as the final phase of rollbacks happens.
I think I can. I think I can.

Answer 1

maybe that web app was accidentally created with classic auth?
here's an example of how to create claims based, with classic, and then "doing 2013" claims
#Create the example web application, as mentioned above, either with gui, and pick later, or
New-SPWebApplication-ApplicationPool$applicationPool-ApplicationPoolAccount$serviceAcct-Name$WebApp-Port
5050
-databaseName$contentDB-securesocketslayer
#If doing for 2013
New-SPWebApplication-ApplicationPool$applicationPool-ApplicationPoolAccount$serviceAcct-Name$WebApp-Port
5050
-AuthenticationProvider(new-spauthenticationprovider)
-databaseName$contentDB-secureSocketsLayer