SCOM 2012 SP1 UR4 management servers grey state

Question

Hi,
My SCOM environment is made up of the below :-
SCOM 2012 SP1 UR4.
3 SCOM Management Servers all on Windows 2008 R2 SP1.
Shared SQL 2008 cluster with 2 Windows nodes also on same OS.
Just recently all our SCOM management servers have been flipping in and out from grey to green state.  Gateways/agents all look ok as showing green.  Alerting from agents appears normal as can see lots of them in console.
Have flushed the health state cache folder on all 3 SCOM MS's and still the same issue.
Appreciate any help on this one.

Answer 1

Event id: 7011 - Was your server recently patched (Installed by any automatic updates) ?
IS SCCM Configured in your MS? If Yes disable and check?
Is Windows update service running ? Stop if for one or two days and check if this issue still appears
Reference threads:
http://social.technet.microsoft.com/Forums/en-US/b86e5a3d-0c2e-4d5e-9d3d-905da91fc982/scom-2012-event-id-7011-service-control-manager-error-when-fep-definition-updates-apply?forum=configmanagersecurity
http://stefanroth.net/2012/09/26/scom-2012-event-id-7011-service-control-manager-error/
Solution also available in: http://technet.microsoft.com/en-us/library/cc756319(v=ws.10).aspx
===========================================
For Event id 20026 - 
1. Does your Operationsmanager database have enough space ? Check that first.
What is you DB size ?
How much is the free space left ?
2. Was there any resent change in the SCOM Action accoutn password ? Or has the password expired. Try re entering the SCOM Action password by re directing your self to Administration tab --> Run as Config -- > Accounts --> SCOM Action account.
The description would be - This is the user account under which all rules run by default on the agent.
Right click and go to properties and re enter the account name and password there and check.
Refer the below screen shot
Check this article as well:
http://social.technet.microsoft.com/Forums/systemcenter/en-US/102d443c-db0e-4bf2-b0d6-31d7f9328537/all-agents-greyed-out-multiple-event-errors-with-ids-20026-20028?forum=operationsmanagergeneral
============================================
Event id : 11904 - As per my knowledge appears due to incorrect Alrting string display name selected in any Rule or monitor.
Also the description what you pasted in Event id : 11904 says Microsoft.SystemCenter.HealthService.ActionAccountConfigured.Error as highlighted below.
Suggest to re enter the action account password and tell the results.
Also is the Healthservice on the MS running using System account or Domain account ?
=================================================================
Description : The Microsoft Operations Manager Expression filter Module failed to query the delivered item, item was dropped.
Property Expression: Reachability/State
Error : 0XC00EE22
One or more workflows were affected by this. Workflow
name: Microsoft.SystemCenter.HealthService.ActionAccountConfigured.Error
Gautam.75801