Unix Log Monitoring regular expression not picking up alerts

Question

Hi,
We are moving our unix monitoring to SCOM 2012 SP1 rollup 4.
What I have got working is indvidual alert logging of Unix Log alerts by exporting the MP and changing the <IndividualAlerts> value to true and removing the suppression xml section then reimporting the MP.
What I am trying to do is use the regular expression to peform the suppression of specific event (such as event codes).
The expression is:
((?i:warning)(?!(.*1222)|(.*1001)))
ie Search the log for "warning" (not case sensitive) then check if events 1222 or 1001 exist if so return no match, if they dont exist then return true. 
I use the built in test function in SCOM when creating the rule and the tests come back as expected but when I inject test lines into the unix log, no alerts get generted.
I suspect it could be the syntax not being accepted on the system (its running RedHat 6 )
I have tested this with regex tools and works.
When I try and test it on the server i get:
[root@bld02 ~]# grep ((?i:Warning)(?!(.*1222)|(.*1001))) /var/log/messages
-bash: !: event not found
[root@bld02 ~]# tail /var/log/messages
Nov 13 15:07:26 bld02 root: SCOM Test Warning Event ID 1001 Round 18
Nov 13 15:07:29 bld02 root: SCOM Test Warning Event ID 1000 Round 18
Nov 13 15:07:35 bld02 root: SCOM Test Warning Event ID 1002 Round 18
So I am expecting 2 alerts to be generated.
SCOM tests to show expression working:
Test 1 Matching
Test 2 to exclude
Need some help with this, Thankyou in advance :)

Answer 1

Hello,
Here's an example of modifying the MP to exclude particular events.  Firstly, I created a log file rule using the MP template that is fairly inclusive - matching the string Warning (with either a lower or upper case W).
I then exported the MP, and modified the rule.  I set the IndividualAlerts = true and removed the AlertSuppression element, so that every matched line will fire a unique alert.  You don't have to remove the AlertSuppression, but you should use
Individual alerts so that the exclusion logic doesn't exclude concurrent events that you actually want to match.
Implementing the exclusion logic involves the addition of a System.ExpressionFilter definition in the rule. This will use a conditional evaluation of the //row element of the data item.  Here's an example of a dataitem matching an individual row:
<DataItem type="System.Event.Data"time="2013-11-15T10:33:14.8839662-08:00"sourceHealthServiceId="667FF365-70DD-6607-5B66-F9F95253B29F">
<EventOriginId>{86AB962D-2F44-29FD-A909-B99FF6FEB2C5}</EventOriginId>
<PublisherId>{EC7EA4B1-0EA5-7E8E-701F-82FEF3367BC4}</PublisherId>
<PublisherName>WSManEventProvider</PublisherName>
<EventSourceName>WSManEventProvider</EventSourceName>
<Channel>WSManEventProvider</Channel>
<LoggingComputer/>
<EventNumber>0</EventNumber>
<EventCategory>3</EventCategory>
<EventLevel>0</EventLevel>
<UserName/>
<RawDescription>Detected Entry: warning 1002</RawDescription>
<CollectDescription Type="Boolean">true</CollectDescription>
<EventData>
<DataItem type="SCXLogProviderDataSourceData"time="2013-11-15T10:33:14.8839662-08:00"sourceHealthServiceId="667FF365-70DD-6607-5B66-F9F95253B29F">
<SCXLogProviderDataSourceData>
<row>warning 1002</row>
</SCXLogProviderDataSourceData>
</DataItem>
</EventData>
<EventDisplayNumber>0</EventDisplayNumber>
<EventDescription>Detected Entry: warning 1002</EventDescription>
</DataItem>
Here is the rule in the MP XML.  The <ConditionDetection>...</ConditionDetection> content was what I added to do the exclusion filtering:
<Rule ID="LogFileTemplate_66b86eaded094c309ffd2631b8367a32.Alert" Enabled="false" Target="Unix!Microsoft.Unix.Computer" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100">
<Category>EventCollection</Category>
<DataSources>
<DataSource ID="EventDS" TypeID="Unix!Microsoft.Unix.SCXLog.VarPriv.DataSource">
<Host>$Target/Property[Type="Unix!Microsoft.Unix.Computer"]/PrincipalName$</Host>
<LogFile>/tmp/test</LogFile>
<UserName>$RunAs[Name="Unix!Microsoft.Unix.ActionAccount"]/UserName$</UserName>
<Password>$RunAs[Name="Unix!Microsoft.Unix.ActionAccount"]/Password$</Password>
<RegExpFilter>warning</RegExpFilter>
<IndividualAlerts>true</IndividualAlerts>
</DataSource>
</DataSources>
<ConditionDetection TypeID="System!System.ExpressionFilter" ID="Filter">
<Expression>
<RegExExpression>
<ValueExpression>
<XPathQuery Type="String">//row</XPathQuery>
</ValueExpression>
<Operator>DoesNotContainSubstring</Operator>
<Pattern>1001</Pattern>
</RegExExpression>
</Expression>
</ConditionDetection>
<WriteActions>
<WriteAction ID="GenerateAlert" TypeID="Health!System.Health.GenerateAlert">
<Priority>1</Priority>
<Severity>2</Severity>
<AlertName>Log File Alert: ExclusionExample</AlertName>
<AlertDescription>$Data/EventDescription$</AlertDescription>
</WriteAction>
</WriteActions>
</Rule>
I traced this with the Workflow Analyzer as I tested, which shows the logic being applied.  Here is the exclusion happening:
Here's more info on the definition of an ExpressionFilter:
http://msdn.microsoft.com/en-us/library/ee692979.aspx
And more information on Regular Expressions in MPs:
http://support.microsoft.com/kb/2702651/en-us
You can also have multiple Expressions in the ExpressionFilter joined by OR or AND operators.
Also, if you are comfortable with the MP authoring, you can just skip the step of creating the rules in the MP template and just author your own MP with the VSAE tool:
http://social.technet.microsoft.com/wiki/contents/articles/18085.scom-2012-authoring-unixlinux-log-file-monitoring-rules.aspx
www.operatingquadrant.com