Office Web Apps - "Could not find trace string in ULS logs" unhealthy?

Question

I have reviewed everything I could find on unhealthy WAC clusters as my problem seems unrelated to certificate or missing components.  I've already digested
http://www.wictorwilen.se/office-web-apps-server-2013---machines-are-always-reported-as-unhealthy (Thanks Wictor).
The particular configuration is an Office Web Apps 2013 ([X-OfficeVersion, 15.0.4551.1005]), running on top of Windows Server 2012, configured for http access (SSL offloaded NLB cluster) and finally linked to Exchange 2013, Lync 2013 and SharePoint
2013.  Everything works as expected from client side after setting IIS ARR to handle all reverse proxy bits.
FarmOU                            :
InternalURL                       : https://officeapps.fqdn/
ExternalURL                       : https://officeapps.fqdn/
AllowHTTP                         : True
SSLOffloaded                      : True
CertificateName                   :
EditingEnabled                    : True
LogLocation                       : C:\ProgramData\Microsoft\OfficeWebApps\Data\Logs\ULS
LogRetentionInDays                : 7
LogVerbosity                      : Unexpected
Proxy                             :
CacheLocation                     : C:\ProgramData\Microsoft\OfficeWebApps\Working\d
MaxMemoryCacheSizeInMB            : 75
DocumentInfoCacheSize             : 5000
CacheSizeInGB                     : 15
ClipartEnabled                    : False
TranslationEnabled                : False
MaxTranslationCharacterCount      : 125000
TranslationServiceAppId           :
TranslationServiceAddress         :
RenderingLocalCacheLocation       : C:\ProgramData\Microsoft\OfficeWebApps\Working\waccache
RecycleActiveProcessCount         : 5
AllowCEIP                         : False
ExcelRequestDurationMax           : 300
ExcelSessionTimeout               : 450
ExcelWorkbookSizeMax              : 50
ExcelPrivateBytesMax              : -1
ExcelConnectionLifetime           : 1800
ExcelExternalDataCacheLifetime    : 300
ExcelAllowExternalData            : True
ExcelWarnOnDataRefresh            : True
OpenFromUrlEnabled                : False
OpenFromUncEnabled                : True
OpenFromUrlThrottlingEnabled      : True
PicturePasteDisabled              : True
RemovePersonalInformationFromLogs : False
AllowHttpSecureStoreConnections   : False
Machines                          : {WAC15PD-02, WAC15PD-01}
The problem however is an incessant logging on the WAC cluster nodes of event 1204,2204 followed almost immediately by 1004,2004.  This repeats every 4min or so...
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider
Name="Office Web Apps Monitoring" />
  <EventID
Qualifiers="0">1204</EventID>
  <Level>2</Level>
  <Task>1</Task>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated
SystemTime="2014-02-04T20:49:37.000000000Z" />
  <EventRecordID>3043246</EventRecordID>
  <Channel>Microsoft Office Web Apps</Channel>
  <Computer>wac15pd-01.fqdn</Computer>
  <Security
/>
  </System>
- <EventData>
  <Data><?xml version="1.0" encoding="utf-16"?> <HealthReport xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <HealthMessage>UlsControllerWatchdog reported status for UlsController in category 'Verify Trace Logging'. Reported status: Could not find trace string in ULS logs in C:\ProgramData\Microsoft\OfficeWebApps\Data\Logs\ULS.</HealthMessage>
<ComponentOwner>ServicesInfrastructure</ComponentOwner>
</HealthReport></Data>
  </EventData>
 </Event>
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider
Name="Office Web Apps Monitoring" />
  <EventID
Qualifiers="0">2204</EventID>
  <Level>2</Level>
  <Task>1</Task>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated
SystemTime="2014-02-04T20:49:37.000000000Z" />
  <EventRecordID>3043247</EventRecordID>
  <Channel>Microsoft Office Web Apps</Channel>
  <Computer>wac15pd-01.fqdn</Computer>
  <Security
/>
  </System>
- <EventData>
  <Data><?xml version="1.0" encoding="utf-16"?> <HealthReport xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <HealthMessage>UlsControllerWatchdog reported status for UlsController in category 'Verify Trace Logging'. Reported status: Could not find trace string in ULS logs in
C:\ProgramData\Microsoft\OfficeWebApps\Data\Logs\ULS.</HealthMessage> <ComponentOwner>ServicesInfrastructure</ComponentOwner>
</HealthReport></Data>
  </EventData>
  </Event>
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider
Name="Office Web Apps Monitoring" />
  <EventID
Qualifiers="0">1004</EventID>
  <Level>2</Level>
  <Task>10002</Task>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated
SystemTime="2014-02-04T20:49:39.000000000Z" />
  <EventRecordID>3043266</EventRecordID>
  <Channel>Microsoft Office Web Apps</Channel>
  <Computer>wac15pd-01.fqdn</Computer>
  <Security
/>
  </System>
- <EventData>
  <Data><?xml version="1.0" encoding="utf-16"?> <HealthReport xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <HealthMessage>AgentManagerWatchdog reported status for
AgentManagerWatchdog in category 'Recent Watchdog Reports'. Reported status: Machine health is Unhealthy</HealthMessage> </HealthReport></Data>
  </EventData>
 </Event>
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider
Name="Office Web Apps Monitoring" />
  <EventID
Qualifiers="0">2004</EventID>
  <Level>2</Level>
  <Task>10002</Task>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated
SystemTime="2014-02-04T20:49:39.000000000Z" />
  <EventRecordID>3043267</EventRecordID>
  <Channel>Microsoft Office Web Apps</Channel>
  <Computer>wac15pd-01.fqdn</Computer>
  <Security
/>
  </System>
- <EventData>
  <Data><?xml version="1.0" encoding="utf-16"?> <HealthReport xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <HealthMessage>AgentManagerWatchdog reported status for
AgentManagerWatchdog in category 'Recent Watchdog Reports'. Reported status: Machine health is Unhealthy</HealthMessage> </HealthReport></Data>
  </EventData>
  </Event>
Further exploration of ULS log files (C:\ProgramData\Microsoft\OfficeWebApps\Data\Logs\ULS) did not yield particularly much, except the following;
02/04/2014 20:48:04.48  UlsControllerWatchdog.exe (0x1244)       0x0F60 Services Infrastructure        Uls Controller Watchdog        ajbam Assert 
 We're about to trace a string for category MsoSpUlsControllerWatchdog at level Info and we expect to find in the log later, but it appears that the category has been throttled. We will never be able to find the string and this watchdog will always fail.
StackTrace:   at Microsoft.Office.Web.UlsControllerWatchdog.Program.CheckServiceInstance(ServiceInstance serviceInstance)     at Microsoft.Office.Web.Common.WatchdogHelperThreadManager.GetHealthResults(WatchdogExecutionContext
context, ServiceInstance si)     at Microsoft.Office.Web.Common.WatchdogHelperThreadManager.WatchingThreadMethod(Object o)     at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback
callback, Object state, Boolean preserveSyncCtx)     at System.Threading.ExecutionContext.Ru... 345fbec5-e958-4f1f-bf56-d65c1c0d472a
02/04/2014 20:48:04.48* UlsControllerWatchdog.exe (0x1244)       0x0F60 Services Infrastructure        Uls Controller Watchdog        ajbam Assert 
 ...n(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)     at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()    
at System.Threading.ThreadPoolWorkQueue.Dispatch()   345fbec5-e958-4f1f-bf56-d65c1c0d472a
02/04/2014 20:48:05.52  UlsControllerWatchdog.exe (0x1244)       0x0F60 Services Infrastructure        Services Infrastructure Health adhog Unexpected Health report
by UlsControllerWatchdog: Agent: UlsController, eventId: 1204, eventType: Error, categoryId: 1, eventMessage: <?xml version="1.0" encoding="utf-16"?>  <HealthReport xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">    <HealthMessage>UlsControllerWatchdog reported status for UlsController in category 'Verify Trace Logging'. Reported
status: Could not find trace string in ULS logs in C:\ProgramData\Microsoft\OfficeWebApps\Data\Logs\ULS.</HealthMessage>    <ComponentOwner>ServicesInfrastructure</ComponentOwner>  </HealthReport> 345fbec5-e958-4f1f-bf56-d65c1c0d472a
02/04/2014 20:48:05.52  UlsControllerWatchdog.exe (0x1244)       0x0F60 Services Infrastructure        Services Infrastructure Health adhoh Unexpected Health report
by UlsControllerWatchdog (persistent): Agent: UlsController, eventId: 2204, eventType: Error, categoryId: 1 345fbec5-e958-4f1f-bf56-d65c1c0d472a
I suspect these might be related, but can't seem to find any logical explanation why this should cause the Get-OfficeWebAppsMachine to report HealthStatus of Unhealthy.  If related, is there a way to disable this check or remove throttling in a safe
way?  Alternatively if this is some coding issue (I've not found any other blog/QA dealing with this particularly) it would be nice to get confirmation of this and potentially a fix/solution.
Any help would be greatly appreciated. Thank you!

Answer 1

Hi ChristiaanB,
You get this ULS error because you change the log verbosity of the OWA farm. I wrote an article for this on my blog : OWA unhealthy uls issue
Regards,
Wes