Ciscoworks feed parsing issue

Question

Sentinel 6.1 will not parse my feed from Ciscoworks. All network syslog
data is collected by ciscoworks and stored in a file syslog.log. This
file can be pulled by the collector manager as a file source or pushed
via Snare to the collector mgr.. the data cannot be parsed using either
method using standard Cisco Collector (Cisco IOS Router xx Cisco Switch
and Router 6.1r2).
Is there a working collector or do I have to build one?
eisensee
eisensee's Profile: http://forums.novell.com/member.php?userid=98444
View this thread: http://forums.novell.com/showthread.php?t=425629

Answer 1

OK, hmm...
1) At issue is more likely the Collector version, not so much the
platform version. The old Collectors made an attempt at correcting some
weird syslog formats, but the feature ended up being too confusing and
error prone for us to support. The new Collectors only support proper
RFC-compliant syslog, for a number of reasons which we can get into if
you like.
2) I may be wrong, but I believe a customer told me that latter-day
versions of CiscoWorks introduced a syslog forwarding option. I poked
around on Cisco for a bit, but there are hundreds of specific products
with the 'CiscoWorks' label so I don't know what applies.
3) OK, so here's the deal. RFC-compliant syslog message are constructed
as:
MMM DD HH:MM:SS hostid message
The original message from your switch should look something like:
Nov 11 15:30:17 172.253.248.33
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:
It actually looks like (guessing a bit, here:
Nov 11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:
Which indicates that a couple numbers and another timestamp are being
injected into the "message" portion. May or may not be an issue, if the
Collector handles it that way - review the Collector doc for proper
configuration details.
But then your Epilog gets a hold of the message, and inserts *another*
header:
Nov 11 15:30:54 10.101.251.4 sm-ciscowks.smad2.savemart.com\t\t0\tNov
11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:
This violates RFC3164 in a number of ways, namely:
- it's not supposed to modify the original message IN ANY WAY if it's
already a proper syslog message (of course, Epilog may assume it's NOT a
syslog message, since it's in a file)
- It's using a fully-qualified hostname in its header, which is NOT
VALID
- There are tab characters after the header, which aren't proper syslog
characters
If I couldn't configure Epilog to not be stupid, what I would do is
create a little 'custom.js' script, and define my customerPreparse()
method to strip off the entire ugly Epilog header, something like:
Record.prototype.customPreparse = function() {
this.s_RXBufferString =
this.s_RXBufferString.substr(this.s_RXBufferString .lastIndexOf("\t"));
(You may need to do the same thing to rec.s_Body as well, and note that
I haven't tested this code at all!).
Then just follow the normal process to inject custom.js into your
Collectors, set the Execution Mode to 'custom', and you'll be up and
running.
NOTE: the major thing that Epilog is breaking here, however, is not the
Collector - the Syslog Connector also does some minimal parsing of the
input and will automatically create Event Source nodes based on the
syslog header. The syslog header is supposed to list the hostid of the
ORIGINAL event source as it's second element (after the timestamp), and
by injecting the ciscoworks device hostid, Epilog breaks that (in our
parlance, that's the Reporter, not the Observer).
If you look directly at the file on the CiscoWorks device, what does it
look like? Everything after the \t\t0\t in the sample you sent?
You might ditch Epilog entirely and use something like tail -F or
'netcat' instead, which won't add silly stuff to the messages.
DCorlette
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=425629