ODM Apriori and Anomaly Detection Questions

i have 2 questions the 1st concerning the Apriori algorithm does it
support timestamps and dates ??
if yes then how to use it :$ (is there any sample ?)
the other questiong regarding the Anomaly Detection when ever i try
building the model i get wiered errors and when i searched for the
last error i found online "Reported as a bug" its error ORA:00600 but
most of the times i get the error ORA-40109: inconsistent logical data
record

see Re: Some Questions regarding Apriori algorithm and anomaly detection for a reply

Similar Messages

Global Correlation and Anomaly detection drop messages?

We've implemented an SSP-40 and were wondering if there were event messages for Global Correlation or Anomaly detection drops. We seem to only have signature event messages.
Dennis

Sure. Here is an example:
evIdsAlert: eventId=1332748411090083862 severity=informational vendor=Cisco alarmTraits=32768
originator:
   hostId: sensorName
   appName: sensorApp
   appInstanceId: 19247
time: 2012/03/27 15:12:41 2012/03/27 15:12:41 UTC
signature: description=ICMP Echo Request id=2004 created=20001127 type=other version=S592
   subsigId: 0
interfaceGroup: vs0
vlan: 1104
participants:
   attacker:
     addr: locality=OUT A.B.C.3
   target:
     addr: locality=OUT A.B.C.2
     os: idSource=unknown relevance=relevant type=unknown
actions:
   deniedPacket: true
riskRatingValue: attackRelevanceRating=relevant targetValueRating=medium 95
threatRatingValue: 60
interface: ge2_0
protocol: icmp
globalCorrelation:
   globalCorrelationScore: -9.2
   globalCorrelationRiskDelta: 60
   globalCorrelationModifiedRiskRating: true
   globalCorrelationDenyPacket: true
   globalCorrelationDenyAttacker: false
   globalCorrelationOtherOverrides: false
   globalCorrelationAuditMode: false
Alternatively, you can see the stats using:
sensor# show statistics analysis-engine | be Malicious
MaliciousSiteDenyHitCounts
A.B.C.D/16 = 1
MaliciousSiteDenyHitCountsAUDIT
Regards,
Sawan Gupta

ODM ANOMALY DETECTION PROBLEM !!!!

i'm trying to create an anomaly detection model, but i encounter this error: ORA-40101: Data Mining System Error ORA-00600: internal error code, arguments:
[KGHALO2], [0x0], [], [], [], [], [], []
can any 1 help me with this please ?

see Re: Some Questions regarding Apriori algorithm and anomaly detection for request for more information.
--Marcos

Startup guide for oracle data mining for anomaly detection

hi
well i have setup oralce 10g for data mining. ihae also downlaoded and nstalled demo prog.
now im wondering how to start to develop my own model.... basically my idea is to use anomaly detection tecnuique for network traffic.
i want ot scann network packets and mine them for anomaly. do i have to create profiles for that and if yes how?????
A BIG DILEMMA... ANY ONE CAN PLS GUIDE, ILL APPRECIATE
CHEERS
ninja

Ninja,
You may also want to take a look at this thread in the forum:
Re: Some Questions regarding Apriori algorithm and anomaly detection
It has some discussion that might help.
-Marcos

Anomaly detection using ODM

I was asked the following question:
"My question is very simply, we are doing a monitoring system for a
website that helps the admin to mine on specific data (using ODM to
produce Web mining) so we want to apply the anomaly detection. We dont
know what we should do and what we should produce as a results."
A couple of suggestions come to mind:
1) For an overall discussion of intrusion detection in general using the Oracle RDBMS as an analytical platform the following paper might be useful:
http://www.oracle.com/technology/products/bi/odm/pdf/odm_based_intrusion_detection_paper_1205.pdf
2) A couple of things to think about and do:
(a) Define what is the "mining case", that is, the "object that defines what is the concept you want to mine. For example, in web mine you may want to detect anomalous session activity. This can be defined over the whole activity of a session or over time windows. In the first case each session will define a mining case (it will be a row in the training data). In the second case each section will generate many mining cases, one per time window. Let's assume for sake of discussion that the goal is to identify anomalous session activity. Then the training data will consist of the session activities (e.g., clicks, pages visited, and/or information from forms; or more generally, http requests). There will be one row per session in the training data. If we know beforehand that some of those sessions where intrusion or anomalous in some sense we can also capture this data as a target for supervised modeling.
(b) Decide what modeling to do. Two types of modeling can be performed (see the paper above for examples):
(i) Supervised modeling - case there are examples of anomalous cases as well as normal cases
This can be done by building a classifier on the training data. It is also possible to measure the quality of the classifier on a held aside sample.
(ii) Unsupervised modeling - this should be done as well even if we can create a supervised model
Unsupervised approaches don't provide a measure that indicates how good the model is at predicting anomalous events. These models are better at ranking cases by how anomalous the model believe they are.
Two common unsupervised techniques for anomaly detection are: Clustering and One-Class SVM. The latter is considered a state-of-art in many problem domains and it is the one implemented by ODM. ODM also has clustering but it does not return distance of a row to the center of cluster. This information is necessary for using it clustering for anomaly detection. If one wants to use clustering, the Oracle Data Mining blog has a post that can help compute distance from rows to centroids:
http://oracledmt.blogspot.com/2006/07/finding-most-typical-record-in-group.html
It is important to note that the method described in the post doesn't support nested column attributes.
When building unsupervised models, only the data for normal cases should be used to training the models. The unsupervised models can be seen as defining what is normal. It will recognize that something is anomalous when it does not match the definition of normality learned by the model.
(c) Use ODMR to help with modeling
(d) As new session information is gathered it is possible to score in real-time the session to detect anomalous behavior. One should score both supervised (if information was available) and unsupervised models to detect anomalous behavior. See the above paper for some discussion on this.
The supervised model will indicate if a case is anomalous or not based on known types of anomalous behavior. One should use ROC tuning in ODMR to find a good operating point for the model. This is necessary because the number of anomalous cases is usually small compared to normal ones.
The unsupervised model (one-class SVM) will provide a ranking. The higher the probability of belonging to class 1 the more normal. A 0.5 probability for class 1 indicates the boundary between normal and not normal. In reality it marks a boundary where normality dominates. There can be some anomalous cases with probability higher than 0.5 and some normal cases with probabilities less than 0.5. If working in batch mode we can rank the probabilities in ascending order and select the first K rows for investigation.
--Marcos

A suggestion to speed up the process: provide more information about your data (e.g., schema) and how you are invoking the algorithm (GUI, API, settings). Case you are using the APIs, have you tried the sample programs for anomaly detection?
Regarding the Apriori algorithm it does not support timestamps and dates columns. In fact, none of the algorithms in ODM does (see the documentation for Oracle Data Mining for the supported column data types). the dbms_predictive_analytics package does. Are you trying to do sequential association rules or just trying to do plain association rules using data from a date column? ODM does not support the former. The latter can be done by converting the date column to a VARCHAR or NUMBER column.
--Marcos

MFP Anomaly Detected Access Points are moving from one wlc to another and vice versa

Hi together,
a customer has lost some Access Points to another WLC with 7.2 and then they come back after 15 minutes to the origin WLC with 7.5
Attached the messages
MFP Protection is configured as optional
152
Wed Nov 27 05:33:26 2013
MFP Anomaly Detected - 1 Not encrypted event(s) found as   violated by the radio 58:bf:ea:0f:67:4a and detected by the dot11 interface   at slot 1 of AP 58:bf:ea:0f:67:40 in 300 seconds when observing . Client's   last source mac 70:11:24:e4:43:0f
153
Wed Nov 27 05:31:40 2013
AP Disassociated. Base Radio MAC:88:43:e1:56:91:d0
154
Wed Nov 27 05:31:40 2013
AP's Interface:0(802.11b) Operation State Down: Base Radio   MAC:88:43:e1:56:91:d0 Cause=New Discovery Status:NA
155
Wed Nov 27 05:31:33 2013
AP Disassociated. Base Radio MAC:58:bf:ea:0f:73:d0
156
Wed Nov 27 05:31:33 2013
AP's Interface:1(802.11a) Operation State Down: Base Radio   MAC:58:bf:ea:0f:73:d0 Cause=New Discovery Status:NA
157
Wed Nov 27 05:31:33 2013
AP's Interface:0(802.11b) Operation State Down: Base Radio   MAC:58:bf:ea:0f:73:d0 Cause=New Discovery Status:NA
158
Wed Nov 27 05:31:28 2013
AP Disassociated. Base Radio MAC:58:bf:ea:0f:fc:20
159
Wed Nov 27 05:31:28 2013
AP's Interface:1(802.11a) Operation State Down: Base Radio   MAC:58:bf:ea:0f:fc:20 Cause=New Discovery Status:NA
160
Wed Nov 27 05:31:28 2013
AP's Interface:0(802.11b) Operation State Down: Base Radio   MAC:58:bf:ea:0f:fc:20 Cause=New Discovery Status:NA
161
Wed Nov 27 05:31:17 2013
AP Disassociated. Base Radio MAC:b4:e9:b0:e4:02:20
162
Wed Nov 27 05:31:17 2013
AP's Interface:1(802.11a) Operation State Down: Base Radio   MAC:b4:e9:b0:e4:02:20 Cause=New Discovery Status:NA
163
Wed Nov 27 05:31:17 2013
AP's Interface:0(802.11b) Operation State Down: Base Radio   MAC:b4:e9:b0:e4:02:20 Cause=New Discovery Status:NA
164
Wed Nov 27 05:31:15 2013
AP Disassociated. Base Radio MAC:a4:18:75:eb:da:b0
165
Wed Nov 27 05:31:15 2013
AP's Interface:1(802.11a) Operation State Down: Base Radio   MAC:a4:18:75:eb:da:b0 Cause=New Discovery Status:NA
166
Wed Nov 27 05:31:15 2013
AP's Interface:0(802.11b) Operation State Down: Base Radio   MAC:a4:18:75:eb:da:b0 Cause=New Discovery Status:NA
167
Wed Nov 27 05:28:26 2013
MFP Anomaly Detected - 35 Not encrypted event(s) found as   violated by the radio d8:24:bd:2f:df:6f and detected by the dot11 interface   at slot 1 of AP d8:24:bd:2f:df:60 in 300 seconds when observing Deauth.   Client's last source mac 00:23:14:a7:e3:54
168
Wed Nov 27 05:23:26 2013
MFP Anomaly Detected - 23 Not encrypted event(s) found as   violated by the radio f8:4f:57:a5:40:b2 and detected by the dot11 interface   at slot 0 of AP f8:4f:57:a5:40:b0 in 300 seconds when observing . Client's   last source mac 44:4c:0c:ba:27:77
Don´t know at the moment how to handle it.
Regards
Alex

Hi lAlex,
Disable Client MFP under WLAN advanced tab & see if this still occur
Regards
Rasika
**** Pls rate all useful responses *****

How to test anomaly detection in IPS6 ?

Hi!
Does anybody have experience with AD in IPS6? I tried to test it today with 3 nmap sessions each scanning 100 different IPs. I saw the Sig 13003-0 (single scanner) fired:
signature: description=AD - External TCP Scanner id=13003 version=S262
alertDetails: . adExtraData: numDestIps=150; currentThreshold=150; destPort=80
The scanner threshold was indeed set to 150:
S1# sh ad-knowledge-base vs0 thresholds current
External Zone
TCP Services
Default
Scanner Threshold
User Configuration = 150
Threshold Histogram - User Configuration
Low = 10
Medium = 3
High = 1
UDP Services
This is ok. The problem is that the Sig 13003-1 (warm) didn't fire, however the number of scanned IPs was very high:
S1# sh statistics anomaly-detection vs0
Statistics for Virtual Sensor vs0
Attack in progress
Detection - ON
Learning - OFF
Next KB rotation at 10:00:00 MSK Fri Dec 28 2007
Internal Zone
TCP Protocol
UDP Protocol
Other Protocol
External Zone
TCP Protocol
Service 80
Source IP: 10.0.1.1 Num Dest IP: 280
Questions:
- what does Low/Medium/High exactly mean in threshold histogram?
- how does the sensor detect worms? When the Sig 13003-1 fires? What sequence of events should happen?
- how can I test it?

The sensor constantly watches for scanners on each port.
There are 3 categories of scanners:
Low scanners - scanners that are only scanning a low number of hosts.
Medium scanners - scanners that are scanning a medium number of hosts
High scanners - scanners that are scanning a high number of hosts
NOTE: I can't remember for sure how many hosts must be scanned for it to be a "Low" number of hosts, or "Medium" or "High". But it may be something like 5 hosts scanned is a "Low" scanner, 20 for Medium and 100 for High. Once again I am not sure of those numbers.
Also be aware that the number of hosts scanned is not the Total numner of hosts scanned, but is instead the number of hosts scanned THAT did not respond.
If you connect to 100 web servers and all web servers respond then it does not count that as a scan. If you try to connect to 100 web servers and 92 respond, then for the 8 that don't respond you would be categorized as a Low scanner.
But just because a scanner is counted in a category does not mean an alert will be generated.
There are 2 types of alerts (subsig 0 alerts, and subsig 1 alerts)
Subsig 0 alerts are for a scanner that is scanning enough hosts that you want an alert for it even when no worm has been declared.
This is the "scanner Threshold / User Configuration = 150" that you see in the "show ad-knowledge-base vs0 thresholds current" output.
If a scanner scans more than 150 hosts then a specific alert is generated even though no worm has been declared.
Any scanners scanning less than 150 hosts are still categorized but do not have alerts generated for them when no worm has been declared.
The subsig 1 alerts are for when a Worm has been declared.
Here is how a worm gets declared:
The Thesholds for Low, Medium, and High that you see in "show ad-knowledge-base vs0 thresholds current" is the number of active scanners in each category that are allowed to normally be seen on your network (this is the number of scanners that will be seen on your network even when there are no worms).
A worm gets declared when the number of scanners in any one of the 3 catgeories goes above the threshold for that category.
Let's take for example Medium=3 as the threshold for port 21. And let's assume it takes a scan of 20 hosts to be categorized as a Medium scanner.
This means normally you could have up to 3 scanners on your network where each scanner is scanning 20 or more non-responding hosts on port 21.
(Maybe these are 3 network administrators periodically checking to see which machines have port 21 open)
Suddenly you have 5 scanners that start scanning on port 21 and each of the 5 winds up with 20 or more non-responding hosts.
That 5 has broken the threshold of 3, and a worm is declared. Now any Medium Category scanner on port 21 will begin being declared a scanner under a worm condition (subsig 1).
So for your testing.
Instead of running a scan of 100 hosts from just one machine, I would recommend you scan the same 100 hosts from 2 or 3 machines (NOTE: Only need to scan a single port across those 100 hosts).
Scanning 100 hosts should get them categorized as High scanners. And having 3 High Scanners should push it over the threshold of 1.
BUT keep in mind that it needs to be 100 hosts not responding on the scanned port.
Then you will also want to try it with fewer hosts being scanned (like say 25), but with say 5 machines running nmap doing the scanning.

IPS anomaly detection knowledge base

Hello
i have configured my IPS device anomaly detection policy for learning accept mode for 48 hours.
and after finishing learning i see knowledge base file which is only 88 bytes in size. Is this normal ?

Depending on your network complexity, you may want to have anomaly detection in learning accept mode for longer than the default 24 hours Yes the knowledge base will replace with new knowledge base.
Regards
Rajeswar

Anomaly Detection Internal Zones

Hello,
I have specified my corporate full IP subnet in internal zone, but i have not configured any TCP or UDP port for any destination also i have kept the default thresholds,
Is it necessary to configure destination port for the TCP and UDP protocol.???????????????
Thanks

Hello,
Uptill now i m not facing any issues with IPS but i want IPS to monitor all the ports for the Internal zone so this is the reason i m asking that while configuring the Internal zone we have to mentioned specific port of tcp and udp for anomaly detection.
If i m not specifying any port than what does it monitor?? is it this incomplete configuration OR it monitors all the ports (1-65535)
Thanks

Anomaly Detection

Guys,
I need to create my KB because the current is very very old ( 09:59:59 GMT-06:00 Tue Sep 22 2009 ) When I try to save it manually with the command
anomaly-detection vs0 save MYKB I get an error that says: Attack in progress
I need to create a new KB and load it because the Rotate methot is not working since the last KB is very old. I thisk it's not working because there is an attack ALWAYS.
Can I save a load a KB file manually even if there is an attack in progress?
If not, How can I fix my problem
Thanks,
DiegoCR CCSP

Hi Diego
You can fix this by:
Turn the anomaly detection off (operational-mode inactive)
Erase/copy/load the files you need and start the anomaly detection or preferably put the sensor in learning accept mode (operational-mode learn) and wait for 24 hours.
Br
Johan Kellerman

Anomaly Detection not detecting host machines (learned OS)

I have an ASA5540X firewall with the internal (software based) IPS module. The module has the up-to-date signatures and seems to be running correctly. However, after enabling anomaly detection (ad0), and specifying the internal zones, I don't see any "Learned OS" in IME
My settings are pretty basic for the sensor
access-list ips_traffic extended permit ip any any
access-list ips_traffic extended permit udp any any
class-map ips_class
match access-list ips_traffic
policy-map global_policy
class ips_class
ips inline fail-open
not sure why it isn't learning the OSs

Learned OS maps—OS maps observed by the sensor through the fingerprinting of TCP packets with the SYN control bit set. Learned OS maps are local to the virtual sensor that sees the traffic.
can you verify the OS finger printing from
sensor# show os-identification learned
Enable passive-traffic-analysis {enabled | disabled}

Anomaly Detection syntax/options

I want to configure anomaly detection on my IPS, but was a little unclear on the syntax for the zones.
Looks like I can configure the internal/service zone as
172.25.13.1-172.25.13.254,172.25.20.1-172.25.13.254
What if I want to make a very general internal zone (because I have a lot of subnets). Would I do something like this?
172.25.1.1-172.25.255.255
I want to define pretty mcuh everything in 172.25.0.0 /16 as internal, but not sure about the syntax here

Anomaly Detection Zones
By subdividing the network into zones, you can achieve a lower false negative rate. A zone is a set of destination IP addresses. There are three zones, each with its own thresholds: internal, illegal, and external.
The external zone is the default zone with the default Internet range of 0.0.0.0-255.255.255.255. By default, the internal and illegal zones contain no IP addresses. Packets that do not match the set of IP addresses in the internal or illegal zone are handled by the external zone.
We recommend that you configure the internal zone with the IP address range of your internal network. If you configure it in this way, the internal zone is all the traffic that comes to your IP address range, and the external zone is all the traffic that goes to the Internet.
You can configure the illegal zone with IP address ranges that should never be seen in normal traffic, for example, unallocated IP addresses or part of your internal IP address range that is unoccupied. An illegal zone can be very helpful for accurate detection, because we do not expect any legal traffic to reach this zone. This allows very low thresholds, which in turn can lead to very quick worm virus detection.

MFP Anomaly Detected - WLC-4402-25-K9 - 5.0.148.0

From time to time I see messages like the one below in the Trap logs of a WLC-4402-25-K9 running 5.0.148.0:
MFP Anomaly Detected - 1 Invalid MIC event(s) found as violated by the radio <offending-MAC> and detected by the dot11 interface at slot 0 of AP <reporting-MAC> in 300 seconds when observing Deauthentication frames. Client's last source mac <client-MAC>
Is my WLC misconfigured or is this a (known) bug in 5.0.148.0?
Trond.

There are some known issues in this area (mainly cosmetic) but it might also be an indication of an attack. You'd have to track this down with a packet capture to see if this is a false positive or not. From the MIB, the description of the event that triggers this message is:
"bcastDeauthenticationFrameRcvd - The Access Point detected a broadcast deauthentication frame. Broadcast
deauthentication frames are rejected by CCXv5 compliant
devices."
More info in: CISCO-LWAPP-TC-MIB.my

My ipod wont let me buy apps etc... keeps saying this is the first time this device has been used and to sign in and answer security questions. I have had this account for years but cant remember the answer to the security questions. How can i fix it?

My iPod touch wont let me buy anything, i've beem using this account for a couple of years and now it says that this is the first ime this id has been used on my device... it's not.... and to sign in and answer security questions. i cant remember the answers to the questions. How can i fix this without making a new account and losing all my stuff???

From a Kappy post
The Three Best Alternatives for Security Questions and Rescue Mail
1. Use Apple's Express Lane.
Go to https://expresslane.apple.com ; click 'See all products and services' at the
bottom of the page. In the next page click 'More Products and Services, then
'Apple ID'. In the next page select 'Other Apple ID Topics' then 'Forgotten Apple
ID security questions' and click 'Continue'. Please be patient waiting for the return
phone call. It will come in time depending on how heavily the servers are being hit.
2. Call Apple Support in your country: Customer Service: Contact Apple support.
3. Rescue email address and how to reset Apple ID security questions.
A substitute for using the security questions is to use 2-step verification:
Two-step verification FAQ Get answers to frequently asked questions about two-step verification for Apple ID.

My Mac is doing strange things: delaying between users, mouse flickering and bouncing, gets stuck with rainbow wheel flickering. I ran the appel hardware test and it detected an error: 4MOT/4/40000003:HDD-1233 Does anybody know what that means? HELP!

My imac is doing strange things:
-delaying between users: when closing session it goes to blue, then takes a while to appear users signin box, and then wont recognize mouse command to enter until a couple of minutes later... then everything seems alright until....
-it gets stuck between things, the rainbow wheel appears and its just delays there forever....
-and every now and then the mouse starts flickering and bouncing wildly onscreen.
I ran the appel hardware test and it detected an error:
4MOT/4/40000003:HDD-1233
Does anybody know what that means? HELP!

WZZZ answered about where to get iStat. And do check the SMART status. If it is an overheating problem due to a fan or logic board problem, your hard drive is possibly cooking itself to death. If so it isn't a faulty hard drive even though the hard drive might fail. So assuming it's a temperature problem, even if you are able to repair things on the disk with software, that is working on symptoms, not causes. I could be wrong however.
RE: AppleCare: Your iMac came with one year of AppleCare (Apple's warranty program), but within the first year you can buy 2 more years. You have to extend by the one year anniversary of purchase of the computer. Your 10,1 is too old to still be in the first year, and since you asked what it was, I'm sure you don't have it. Bottom line meaning is that whatever this problem turns out to be, you'll have to pay for it. Unless there is something like this. It is for 2011 iMacs with certain Seagate drives. You can put in your serial number for fun, but it looks like yours is too old. Lastly, some people have had Apple help them anyway if it is just out of warranty, but many have not. Your machine is one of these. Type in 10,1 in the search box. Is there an Apple Store near you? Just b/c it's out of warranty doesn't mean you shouldn't have it looked at by Apple. But no one here can say at all what Apple will or will not do.
Hope you get it taken care of!

ODM Apriori and Anomaly Detection Questions

Similar Messages

Maybe you are looking for