Startup guide for oracle data mining for anomaly detection

hi
well i have setup oralce 10g for data mining. ihae also downlaoded and nstalled demo prog.
now im wondering how to start to develop my own model.... basically my idea is to use anomaly detection tecnuique for network traffic.
i want ot scann network packets and mine them for anomaly. do i have to create profiles for that and if yes how?????
A BIG DILEMMA... ANY ONE CAN PLS GUIDE, ILL APPRECIATE
CHEERS
ninja

Ninja,
You may also want to take a look at this thread in the forum:
Re: Some Questions regarding Apriori algorithm and anomaly detection
It has some discussion that might help.
-Marcos

Similar Messages

From where to download the The Oracle Data Mining sample programs for DB11g

In the Administrator's Guide to Oracle Data Mining 11g. One of the steps is to install the the Oracle Data Mining sample programs, which, according to the author, is included in the Oracle Database Companion. The author did not specify from where to download neither the sample programs nor the database companion CD/DVD!
I would be grateful if you could provide me with the URL that has the required download.
Regards,

You should be able to download ODM DB11g sample programs from the following OTN link:
http://www.oracle.com/technology/software/products/database/oracle11g/111060_win32soft.html
Choose 'Oracle Database 11g Examples (formerly Companion)'

How to use Data Mining for a blind DBA?

Hi, I'm a blind DBA and I need some input and some suggestions from the forum on how to access the Data Mining feature.
This feature is becoming more and more an essential part of my daily tasks, so I need a way to use it.
Database: Oracle 11.1.0.7.0
OS: Windows XP Pro and Linux
Adaptive Technology:
JAWS Version 11.0.1467 by Freedom Scientific BLV Group, LLC
JAVA Access Bridge by SUN Microsystems
I know there is a Windows program called Data Miner, unfortunately my JAWS speech synthesizer does not work with the interface and I need an alternative. I
read Oracle's documentation on the Data Miner API., but it is short on examples, details , and step by step instructions. Does anyone have a more
comprehensive set of examples on how to use the Data Mining API. from start to finish, specifically for market basket analysis? Any help would be
appreciated.
Thanks.

Hi,
I am not very knowledgeable about Oracle Data Miner, the graphical user interface to Oracle Data Mining, but I can respond to your question about API documentation.
You may find the source files for the Data Mining sample programs to be very helpful. They provide many examples. After installing Oracle Database, install Oracle Database Examples. This separate installation process installs the sample programs in the DEMO directory under Oracle Home. Instructions for installing and configuring the sample programs are provided in Oracle Data Mining Administrator's Guide. Here is the link to the relevant chapter in the 11.1 Administrator's Guide.
http://download.oracle.com/docs/cd/B28359_01/datamine.111/b28130/sampleprogs.htm#BABEHFEB
As for the API documentation, the best place to start is probably the Data Mining Application Developer's Guide. Here is the link to the 11.1 Application Developer's Guide.
http://download.oracle.com/docs/cd/B28359_01/datamine.111/b28131/toc.htm
The Data Mining PL/SQL API for creating data mining models is implemented in the DBMS_DATA_MINING package. The syntax of Oracle PL/SQL packages is documented in PL/SQL Packages and Types Reference. Here is the link to the chapter in the 11.1 version of that manual.
http://download.oracle.com/docs/cd/B28359_01/appdev.111/b28419/d_datmin.htm#i1062150
The Data Mining functions for scoring (applying mining models to new data) are implemented as SQL functions. The syntax of Oracle SQL is documented in SQL Language Reference. Here is the link to the Data Mining SQL functions in the 11.1 version of that manual.
http://download.oracle.com/docs/cd/B28359_01/server.111/b28286/functions001.htm#sthref953
Hope this helps,
Kathy
Edited by: ktaylor on Nov 10, 2010 8:14 AM

Using Oracle Data Miner for Future Sales Prediction

Hi ,
I have one question on predicting the sale for next 5 years along with month the data as follows.
YEAR Month Total_Sale
2009    1          88187
2009     2         87654
2009     3         87656
2008     1         10000
2008     2          30000
2008     3          40000
How to do this in Oracle data Miner ie the prediction of sale for next years like 2014,2015 etc.?
The expected output
is as follows.
2014      1       2000
2014       2      3000
2014       3      9000
2014      4       234
2015       1      2344
2015       2      4000
and so on for all the 12 months and year.
CREATE TABLE "SALE_GROWTH"
   ( "YEAR" NUMBER(*,0),
"MONTH" NUMBER(*,0),
"TOTAL_SALE" NUMBER
REM INSERTING into SALE_GROWTH
SET DEFINE OFF;
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,1,881725);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,2,1036585);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,3,1406252);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,4,550700);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,5,985413);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,6,727485);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2009,8,228480);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2008,9,699);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2008,10,446428);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2008,11,975335);
Insert into SALE_GROWTH (YEAR,MONTH,TOTAL_SALE) values (2008,12,4853690);
The above is the historical data.
If i use oracle sql slope , am i going get the same sale numbers as the oracle data miner will predict.
Please write.
I appreciate your help.
Thanks,
HS

Hi,
For a background on how you might be able to use ODM for sales forecasting, see Marcos Campos blog on timeseries (link below).
Thanks, Mark
Oracle Data Mining and Analytics: Time Series Forecasting Series

Microsoft OLE DB Provider for Oracle: Data type is not supported.

I got the error:
Microsoft OLE DB Provider for Oracle: Data type is not supported.
Shortly after upgrading from Oracle 8 to Oracle 9. I was advised to download more up to date oracle drivers, but I was wondering if there was a way to tell what version of the 'OLE DB Provider for Oracle' is already at. Is there a command I can use via SQL Plus or something?

I have found Microsoft ODBC for Oracle to be more stable than the Microsoft OLEDB for Oracle driver. I have also found both Microsoft ODBC and OLEDB drivers to be more stable than the drivers from Oracle.
You could always get the latest MDAC (Microsoft Data Access Components) from Microsoft's MSDN Download site and then get the ODAC (Oracle Data Access Components) from Oracle's OTN Download site. ODAC requires MDAC. And ODAC has the latest drivers.
I suppose it would help to have the latest patches for your Oracle client software too. Maybe Oracle MetaLink would have these?
It may even help to have the latest service pack for Visual Studio 6 (Visual C++ 6 and Visual Basic 6) too.

Someone has nomenclature (naming conventions) for oracle data integrator?

Someone has nomenclature (naming conventions) for oracle data integrator?

You should really move this question over to the Oracle Data Integrator forum(Data Integrator for a more expedient response.

Date for Oracle Enterprise Pack for Eclipse?

Is there a date for Oracle Enterprise Pack for Eclipse for Eclipse 3.4?

There isn't any activation or a Oracle specific perspective. If the installation was successful you should see the following features:
- In Server view, New > Server
>> "BEA Systems, Inc." node with a list of WLS version support (Older version)
>> "Oracle" node, with "Oracle WebLogic Server v10.3" support
- Switch to Database Development perspective
>> right click on Databases > New - You should see "Oracle Database Connection"
- Help > Help Contents - "Oracle Enterprise pack for Eclipse" contains the documentation for all the features.

Oracle Data Mining - How to use PREDICTION function with a regression model

I've been searching this site for Data Mining Q&A specifically related to prediction function and I wasn't able to find something useful on this topic. So I hope that posting it as a new thread will get useful answers for a beginner in oracle data mining.
So here is my issue with prediction function:
Given a table with 17 weeks of sales for a given product, I would like to do a forecast to predict the sales for the week 18th.
For that let's start preparing the necessary objects and data:
CREATE TABLE T_SALES
PURCHASE_WEEK DATE,
WEEK NUMBER,
SALES NUMBER
SET DEFINE OFF;
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('11/27/2010 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 1, 55488);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('12/04/2010 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 2, 78336);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('12/11/2010 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 3, 77248);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('12/18/2010 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 4, 106624);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('12/25/2010 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 5, 104448);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('01/01/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 6, 90304);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('01/08/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 7, 44608);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('01/15/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 8, 95744);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('01/22/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 9, 129472);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('01/29/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 10, 110976);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('02/05/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 11, 139264);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('02/12/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 12, 87040);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('02/19/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 13, 47872);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('02/26/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 14, 120768);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('03/05/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 15, 98463.65);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('03/12/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 16, 67455.84);
Insert into T_SALES
(PURCHASE_WEEK, WEEK, SALES)
Values
(TO_DATE('3/19/2011 23:59:59', 'MM/DD/YYYY HH24:MI:SS'), 17, 100095.66);
COMMIT;
There are a lot of linear regression models and approaches for sales forecast out on the market, however I will focus on what oracle 11g offers i.e. package SYS.DBMS_DATA_MINING to create a model using regression as mining function and then, once the model is created, to apply prediction function on the model.
Therefore I'll have to go through few steps:
i) normalization of data
CREATE OR REPLACE VIEW t_sales_norm AS
SELECT week,
sales,
(sales - 91423.95)/27238.3693126778 sales_norm
FROM t_sales;
whereas the numerical values are the mean and the standard deviation:
select avg(sales) from t_sales;
91423.95
select stddev(sales) from t_sales;
27238.3693126778
ii) auto-correlation. For the sake of simplicity, I will safely assume that there is no auto-correlation (no repetitive pattern in sales among the weeks). Therefore to define the lag data I will consider the whole set:
CREATE OR REPLACE VIEW t_sales_lag AS
SELECT a.*
FROM (SELECT week,
sales,
LAG(sales_norm, 1) OVER (ORDER BY week) L1,
LAG(sales_norm, 2) OVER (ORDER BY week) L2,
LAG(sales_norm, 3) OVER (ORDER BY week) L3,
LAG(sales_norm, 4) OVER (ORDER BY week) L4,
LAG(sales_norm, 5) OVER (ORDER BY week) L5,
LAG(sales_norm, 6) OVER (ORDER BY week) L6,
LAG(sales_norm, 7) OVER (ORDER BY week) L7,
LAG(sales_norm, 8) OVER (ORDER BY week) L8,
LAG(sales_norm, 9) OVER (ORDER BY week) L9,
LAG(sales_norm, 10) OVER (ORDER BY week) L10,
LAG(sales_norm, 11) OVER (ORDER BY week) L11,
LAG(sales_norm, 12) OVER (ORDER BY week) L12,
LAG(sales_norm, 13) OVER (ORDER BY week) L13,
LAG(sales_norm, 14) OVER (ORDER BY week) L14,
LAG(sales_norm, 15) OVER (ORDER BY week) L15,
LAG(sales_norm, 16) OVER (ORDER BY week) L16,
LAG(sales_norm, 17) OVER (ORDER BY week) L17
FROM t_sales_norm) a;
iii) choosing the training data. Again, I will choose the whole set of 17 weeks, as for this discussion in not relevant how big should be the set of training data.
CREATE OR REPLACE VIEW t_sales_train AS
SELECT week, sales,
L1, L2, L3, L4, L5, L6, L7, L8, L9, L10,
L11, L12, L13, L14, L15, L16, L17
FROM t_sales_lag a
WHERE week >= 1 AND week <= 17;
iv) build the model
-- exec SYS.DBMS_DATA_MINING.DROP_MODEL('t_SVM');
BEGIN
sys.DBMS_DATA_MINING.CREATE_MODEL( model_name => 't_SVM',
mining_function => dbms_data_mining.regression,
data_table_name => 't_sales_train',
case_id_column_name => 'week',
target_column_name => 'sales');
END;
v) finally, where I am confused is applying the prediction function against this model and making sense of the results.
On a search on Google I found 2 ways of applying this function to my case.
One way is the following:
SELECT week, sales,
PREDICTION(t_SVM USING
LAG(sales,1) OVER (ORDER BY week) as l1,
LAG(sales,2) OVER (ORDER BY week) as l2,
LAG(sales,3) OVER (ORDER BY week) as l3,
LAG(sales,4) OVER (ORDER BY week) as l4,
LAG(sales,5) OVER (ORDER BY week) as l5,
LAG(sales,6) OVER (ORDER BY week) as l6,
LAG(sales,7) OVER (ORDER BY week) as l7,
LAG(sales,8) OVER (ORDER BY week) as l8,
LAG(sales,9) OVER (ORDER BY week) as l9,
LAG(sales,10) OVER (ORDER BY week) as l10,
LAG(sales,11) OVER (ORDER BY week) as l11,
LAG(sales,12) OVER (ORDER BY week) as l12,
LAG(sales,13) OVER (ORDER BY week) as l13,
LAG(sales,14) OVER (ORDER BY week) as l14,
LAG(sales,15) OVER (ORDER BY week) as l15,
LAG(sales,16) OVER (ORDER BY week) as l16,
LAG(sales,17) OVER (ORDER BY week) as l17
) pred
FROM t_sales a;
WEEK, SALES, PREDICTION
1, 55488, 68861.084076412
2, 78336, 104816.995823913
3, 77248, 104816.995823913
4, 106624, 104816.995823913
As you can see for the first row there is a value of 68861.084 and for the rest of 16 values is always one and the same 104816.995.
Question: where is my week 18 prediction ? or maybe I should say which one is it ?
Another way of using prediction even more confusing is against the lag table:
SELECT week, sales,
PREDICTION(t_svm USING a.*) pred
FROM t_sales_lag a;
WEEK, SALES, PREDICTION
1, 55488, 68861.084076412
2, 78336, 75512.3642096908
3, 77248, 85711.5003385927
4, 106624, 98160.5009687461
Each row out of 17, its own 'prediction' result.
Same question: which one is my week 18th prediction ?
Thank you very much for all help that you can provide on this matter.
It is as always highly appreciated.
Serge F.

Kindly let me know how to give input to predict the values for example script to create model is as follows
drop table data_4svm
drop table svm_settings
begin
dbms_data_mining.drop_model('MODEL_SVMR1');
CREATE TABLE data_4svm (
id NUMBER,
a NUMBER,
b NUMBER
INSERT INTO data_4svm VALUES (1,0,0);
INSERT INTO data_4svm VALUES (2,1,1);
INSERT INTO data_4svm VALUES (3,2,4);
INSERT INTO data_4svm VALUES (4,3,9);
commit;
--setting table
CREATE TABLE svm_settings
setting_name VARCHAR2(30),
setting_value VARCHAR2(30)
--settings
BEGIN
INSERT INTO svm_settings (setting_name, setting_value) VALUES
(dbms_data_mining.algo_name, dbms_data_mining.algo_support_vector_machines);
INSERT INTO svm_settings (setting_name, setting_value) VALUES
(dbms_data_mining.svms_kernel_function, dbms_data_mining.svms_linear);
INSERT INTO svm_settings (setting_name, setting_value) VALUES
(dbms_data_mining.svms_active_learning, dbms_data_mining.svms_al_enable);
COMMIT;
END;
--create model
BEGIN
DBMS_DATA_MINING.CREATE_MODEL(
model_name => 'Model_SVMR1',
mining_function => dbms_data_mining.regression,
data_table_name => 'data_4svm',
case_id_column_name => 'ID',
target_column_name => 'B',
settings_table_name => 'svm_settings');
END;
--to show the out put
select class, attribute_name, attribute_value, coefficient
from table(dbms_data_mining.get_model_details_svm('MODEL_SVMR1')) a, table(a.attribute_set) b
order by abs(coefficient) desc
-- to get predicted values (Q1)
SELECT PREDICTION(MODEL_SVMR1 USING *
) pred
FROM data_4svm a;
Here i am not sure how to predict B values . Please suggest the proper usage . Moreover In GUI (.NET windows form ) how user can give input and system can respond using the Q1

Is oracle data mining support to other databases to work datamining

Dear CB,
I am using oracle data mining and i have some doubts please clarify my doubts
1) Is ODM talk to other database or not ,like can we use ODM to prepare data mining work using other database
2) Is ODM supports social analytics
Thanks inadvance
thanks & regards
Suresh

Suresh,
1) Is ODM talk to other database or not ,like can we use ODM to prepare data mining work using other database
Yes, you can use Oracle Data Mining to talk to other databases, but ODM will need to "have" the data inside the Oracle Database during model build and model apply. You can use DB links to pull/push data to/from other Oracle and non-Oracle DBs, but all the data mining work and data transformations occur in-DB by our Oracle design. You can perform data prep, data transformation, build models and then compute dm predictions inside Oracle DB and then publish ODM results to any other dashboard/Q&R tool that can make a SQL call to Oracle to query results or ask ODM predictive model to make a real-time prediction based on current input data.
2) Is ODM supports social analytics
Depends on what you mean, but probably Yes. For example, we can mine unstructured data e.g. Twitter feeds and get 80% accurate Sentiment analysis. See http://www.google.com/url?sa=t&rct=j&q=mining%20twitter%20data%20clasification%20stanford&source=web&cd=1&ved=0CCMQFjAA&url=http%3A%2F%2Fwww.stanford.edu%2F~alecmgo%2Fpapers%2FTwitterDistantSupervision09.pdf&ei=Pk3VTsPOFYaIsQLknuiGDg&usg=AFQjCNGSErmPAa-n6kc_gVCCdxMRMKTeOw paper for a published tech paper that describes an approach that we have successfully replicated in-DB using ODM's text mining capabilities & Oracle Text. Add additional structured data, e.g. # purchases, $amount of purchases over time, etc. and you can have better Sentiment analysis or other types of predictive models.
Also, ODM can perform e.g. churn analysis and include as input to the model the "friends & family" usage, activities, and demographics as enriched input data to mine. We mine Star Schemas so we can pull together a 360 degree view of customer include "social" type data e.g. # links from a friend, etc. Broad topic.... hope this helps.
cb

Oracle data mining module is implemented in database?

Hello.
I want to know where the ODM module is implemented.
Is it implemented inside the database?
Or, is it implemented as an extension module of the database?
In this case, ODM module can be plugged into the database and it can use its functionalities...

Oracle Data Mining is an integrated component of the database engine itself. It is better not to think of it as a data mining module, but rather think of Oracle Data Mining as the Oracle database's built-in machine learning capabilities.
If you take a look at the PREDICTION function, you will see that this is a built-in SQL function. This is not an extension, not a UDF, not a pl/sql function, etc. It can execute at the storage tier of Exadata as part of smart scan. It is similar in the way it works to a function like the built-in substr, but applies a mathematical model instead. It leverages underling database functionality that is used by many parts of the database, such as memory management, shared cursors, etc. The data flows to this function just as it does to the others built-in SQL functions.
Some of the model training routines will craft SQL, just as is done by other database components. For example, statistics collection from dbms_stats also crafts SQL to assist in its work. However, the implementation of the mining algorithm is still C code that is part of the database codebase leveraging other underlying, specialized database functionality.
You could not separate Oracle Data Mining and attach it to something else - it is built as part of the Oracle database.
Ari

Can SQL*Loader be used for Oracle html pages, for e.g Customer, Tasks etc

Hi,
Can SQL*Loader be used for Oracle html pages, for e.g Customer, Tasks etc.
Reason: We have a job that creates leads, tasks in 11i, but r12 is not working as expected, an SR is open with Oracle for a long time and no solution so far. I am thinking of using SQL*loader to populate the data, but these are html pages. Will SQL* Loader work?
12.1.4 / 11g db
Appreciate your inputs.
Thanks,
K

Can SQL*Loader be used for Oracle html pages, for e.g Customer, Tasks etc. It should work.
Reason: We have a job that creates leads, tasks in 11i, but r12 is not working as expected, What is the issue you have with R12?
an SR is open with Oracle for a long time and no solution so far. I am thinking of using SQL*loader to populate the data, but these are html pages. Will SQL* Loader work?What was Oracle feedback? Did they say it is certified and can be used?
Thanks,
Hussein

Oracle Data Mining

Hello,
Does anyone know if Oracle 9i Data Mining is compatible with the Oracle 8i Database? Do I have to upgrade the DB to 9i to use the DM software?

Oracle Data Mining is available only in 9i. So you must upgrade to from 8i to 9i to get the data mining functionality.
Note: Data Mining is an option to the 9i Enterprise Edition.
-Sunil

Is oracle lite free or is there any license for Oracle Lite 10g for product

Is oracle lite free or is there any license for Oracle Lite 10g for production use ??
-Thanks-

Please note that Oracle Lite is no longer being actively developed, and no further releases are planned. The recommended small footprint database from Oracle is Berkeley DB.
http://www.oracle.com/technetwork/products/berkeleydb/overview/index-085366.html
Eric

Sample programs in Oracle Data Mining 10.1

I have installed Oracle Data Mining 10.1.
But I can't find the sample programs located in \dm\demo.
How can I find these sample programs??
Help me. Thx.

I think they are installed during the installation of the Companion Cd of Oracle 10g.
Simon

What time the Oracle Data Mining 10g ?

What time the Oracle Data Mining 10g ?

If you are referring to the 10.1 release of the ODM user interface, that should be available in beta form very shortly(a couple weeks) with the final release very soon after that.

Startup guide for oracle data mining for anomaly detection

Similar Messages

Maybe you are looking for