Help to do data mining and transformation...

I have a specific task to accomplish and I am wondering if Oracle Data Mining is the correct tool to use, and if not what possibly might be. Here is brief description:
I have a table with about 500 millions rows of data per day, transactional internet traffic data. It contains about 20 columns/dimensions. The requirement is to transform this flat data into a new table that contains (as one column each) each unique variation of those dimension values recorded.
So for example, if we have 3 dimensions of say gender, age and zip code we would determine each unique combination of those found in the actual data and write out x number of columns to identify them and store a count value for each one. The count will just tell us how many of that combination was found in the data, and the end result will be of course an aggregated table for fast querying on all observed dimensions.
For performance reasons we want to pass through the data only only.
We tried cubes but this takes too long (because it also tries to build out all the non-observed combinations), and we know we could try a code approach but fear this may take too long also. The problem is more of a performance one of course, with that many rows and possible combinations to consider.
Any ideas?
Thanks in advance.

After doing some research I realize what I need is a cube, but one that does not contain every single dimension combination but only those that actually exist (to speed up the creation time and reduce storage space). Is this something Oracle supports? Anyone?

Similar Messages

Help XSLT Data Mapper and Transformations

Hi guys,
I need help in oracle ESB (XSLT Data Mapper and Transformations). I need to use the XSLT Data Mapper and Transformations using Response XML to Request XML.
Thanks
Vyas

The concept is the same as BPEL. Without going into too much detail have a look at the folowing tutorial.
http://download.oracle.com/docs/cd/B31017_01/integrate.1013/b28212/buildendtoend.htm#BEICEFJD
Near the bottom they show how to do transformations.
cheers
James

Selective data load and transformations

Hi,
Can youu2019ll pls clarify me this
1.Selective data load and transformations can be done in
    A.     Data package
    B.     Source system
    C.     Routine
    D.     Transformation Library-formulas
    E.     BI7 rule details
    F.     Anywhere else?
If above is correct what is the order in performance wise
2.Can anyone tell me why not all the fields are not appear in the data package data selection tab even though many include in datasource and data target.
Tks in advance
Suneth

Hi Wijey,
1.If you are talking about selective data load, you need to write a ABAP Program in the infopackage for the field for which you want to select. Otherway is to write a start routine in the transformations and delete all the records which you do not want. In the second method, you get all the data but delete unwanted data so that you process only the required data. Performancewise, you need to observe. If the selection logic is complicated and taks a lot of time, the second option is better.You try both and decide yourself as to which is better.
2. Only the fields that are marked as available for selection in the DS are available as selection in the data package. That is how the system is.
Thanks and Regards
Subray Hegde

Difference between OLAP, Data Mining and Data Warehousing

Dear Sirs,
I am new to the above topics, but I know oracle DBA very well. I would like to move into the above field. Hence, can anyone tell me the basic oracle softwaare used for OLAP, Data Mining and Data warehousing and also in brief the difference between these three.
I would be great helpfull to me.
Thanks & Regards,
Manoj Mathew

Hi Manoj Mathew,
Check these links to what Oracle has to say about its own software specific for these topics:
DataMining (tool is Oracle Data Miner): http://www.oracle.com/technology/products/bi/odm/index.html
Datawarehousing (tool is OWB): http://www.oracle.com/technology/products/warehouse/index.html
OLAP (tool = Analytic Workspace Manager): http://www.oracle.com/technology/products/bi/olap/olap.html
Good luck, Patrick

Error with excel data mining and windows 8.1

Hi I found this thread online which is exactly the same issue I am having. Has this problem been resolved?
"I installed SQL Server 2012 SP1 Data Mining add-in for Excel 2013 64 bit on Windows 8.1 64 bit Enterprise. I then ran "Getting Started" to complete the add-in setup and connect to the SSAS DB. After I opened the "Sample
Excel Data" file to test I noticed the "Table Analysis" tab under "TABLE TOOLS" appeared on the ribbon blank (without the text "ANALYZE" on the tab). When I clicked on that tab I was able to see the "Table Analysis Tools"
buttons but I keep getting the "Object reference not set to an instance of an object" error message when I try to run any one of them and exiting out of Excel disables the data mining add-in. Is this a known issue and if yes is there a
hotfix available? Thank you."
I've also download the follwing hotfix which hasn't helped
http://hotfixv4.microsoft.com/SQL%20Server%202012/sp1/SQLServer2012_SP1_CU7_2894115_11_0_3393_/11.0.3393.0/free/470544_intl_x64_zip.exe
Any assistance would be appreciated!
Concatch

Hi Concatch,
Which edition of Excel 2013 are you using? The supported editions of Office supported by the SQL Server 2012 SP1 Data Mining Add-ins for Microsoft Office are as follows:
Professional
Professional Plus
Ultimate
Enterprise
Besides, make sure the cube is hosted in a SSAS 2012 instance.
Regards,
Mike Yin
TechNet Community Support

How to use standard Search Input Help (return 2 data code and desc) ?

Hi,
Please advise how to use standard Input Help provided by SAP and return 2 data (code and description) ? because the standard Input Help only return the code only ?
Thank You and Best Regards
Fernand

>
Saket Abhyankar wrote:
> Hi,
>
> I think you need to go for 'Search Help Exit' or OVS.
>
> Regards,
>
> Saket.
That is not true that this is the only way. The standard Data Dictory based search help can return more than one value as long as there are multiple exporting values defined in the search help, the search help is attached to a Data Dictionary Structure, and this same data dictionary structure is used as the source of the context node. You can read more about this in the online help:
http://help.sap.com/saphelp_nw70ehp1/helpdata/en/47/9f724642314aabe10000000a42189b/frameset.htm
The Note section under Integration:
If an input help structure is stored in a field in the ABAP Dictionary, and if you want to use the field mapping of search help parameters stored in the ABAP Dictionary as the field name for the structure for your Web Dynpro input help, then map your context nodes to this structure. This ensures that all components of the structure are available dynamically at runtime as attributes of the node.
If the context node is not mapped to the structure, the data element's input help can be used if there is one.

Master Data cleansing and transformation from non-SAP source systems

Hi all,
Our client (Media)wants to cleanse and transform his master data from non-SAP source system to be uploaded into BW (no R/3 yet). If anybody has a document regarding this topic that i could use, i will appreciate if u sent it to me.
thanks.

Hi,
https://websmp203.sap-ag.de/~sapidb/011000358700001965262003
https://websmp203.sap-ag.de/~sapidb/011000358700006591612001
https://websmp203.sap-ag.de/~sapidb/011000358700001971392004
https://websmp203.sap-ag.de/~form/sapnet?_SHORTKEY=01100035870000471477&_OBJECT=011000358700008927932002E
/manfred

Need help regarding Simple Data Client and Simple Data Server VIs

Hi everyone.
I have a simple objective. I just want to test the 2 example VIs, "Simple Data Client" and "Simple Data Server" between 2 computers. I just want to check whether is this working between the 2 computers.
What I have done for now is that I changed the "address", from "localhost" in the "Simple Data Client.vi" to the IP address of the computer running the "Simple Data Server". I runned the "Simple Data Server" VI in one of the computers first followed by the "Simple Data Client" in the other computer. Nothing is received and the client just timed out.
Can anyone please help me troubleshoot and tell me what are the possible problems for this? Are there any wires connections between 2 computers that I am missing or any other configurations I have to make before I can successfully do this?
Thanks.
Regards,
Jonathan

Hi Lee.P.
I understand that. I was just feeling frustrated about the project not working. Sincere apologies from me.
I was wrong about the error number. It is not Error 60. It is Error 59 - The network is down, unreachable, or has been reset.. Yes, I have tried changing the port numbers at the 2 computers when trying to send/receive.
Could anything else be the problem?
Regards,
Jonathan

Need help in fetching data - 0CALDAY and 0FISCPER

Hi BW Gurus,
Have a doubt in creating a query in a multi provider..
The multi provider has three cubes.
Two of them have data at Calday level, while another has data at fiscper level..
fiscper is not maintained in the cubes having data at calday level ..
I have to create a query on the multiprovider pulling data from all three cubes .. And the data to be restricted on the basis of fisper level ..
Could anyone suggest me how to go about it ..
Regards.

Hi saugata,
Both the selection will work.
But I was wrong when I said Fiscal period will be easy.
Because getting the date from fiscal period will be quite tricky in the case your user selection is range for Fiscal period As after user enters a fisacl period value you will always have to get the last date of the month for that fiscal period and which will require you to write the code to get last date of month as 30 or 31 . Also you will have to take care of the leap year in the case of feb to get 29
So that will requier you to write the complex code.
Better is you should go for the calendar day range for the user input as it will just require you get the fiscal period which can be just done by offset and concatenation
01.01.2006 can easily be converted to 001.2006
and 12.02.2006 can be converted 002.2006
There is one more solution
Check if your Cube contains 0CALMONTH/YEAR in your Cube where you have 0CALDAY or you can bring just by mappping your day with it.And check whether your fisacl variant is K4 coming up from R/3 it is always compounded with 0FISCYR then you can restrict your data with 0CALMONTH/YEAR only. and the conversion of 0CALMONTH/YEAR is pretty simple.
All these are options upon which you can work
I would say you should go for calmonth year as it will be a simple routine.
Hope it helps you understand better.
Thanks

Help for : Recover datas (Photos and Video) afert ...

Hello
Do someone know how to recover my data ?, I already tried recovery file tools without success.
I make a hard reset, I was sure that hard reset don't delete Mass Storage.
But it's not the case, I lost all my photos, and my videos. For me the built in storage was exactly a same behavior as a external SdCard.
Thank you for your help.

Unfortunately it cannot be done, that's why it is always adviseable to backup/sync your phone on a regular basis ! Sorry
If I have helped at all, a click on the White Star is always appreciated :
you can also help others by marking 'accept as solution'

Help: Problem with data types and classes

Hi there,
I'm currently struggling with getting a class to read in data from a file and from that select the correct constructor to use... Basically I read a line in from a file as follows:
<String Make>:<String model>:<String registrationNumber>:<String engineSize>
e.g.
Ford:Focus:HG42EG:1800
There is an Interface called Car, of which 'Ford' is an implementation along with 'Vauxhall', 'Chrysler', etc...
From this I wish to construct a new instance of a Ford called Focus with input parameters (RegNumber registration,int engineSize) i.e.
Car Focus = new Ford(HG4 2EG,1800);So far I can convert the engineSize String into an integer and can construct the RegNumber (a class of mine) but can only get it to construct a Ford by using a switch, i.e:
switch (modelKey){
case 'f': new Ford(); break;
case 'v': new Vauxhall(); break;
case 't': new Toyota(); break;
etc...
Now I do get this to work correctly but I'm concerned that it is not OOP therefore is the way I can get code to convert the String 'Ford' into Car 'Ford'.???
Many thanks in advance for any advice you can throw my way,
Leebo

What is this anyway???
If Car is an interface you can't create instances of it....No, if Car is an interface, his code should be (depending on what the RegNum constructor looks like:
Car Focus = new Ford(new RegNum("HG42EG"),1800);It's the same as:
List someList = new ArrayList();However, he won't be able to use a word read from the file as a variable name "Focus". He'll need a Map:
Map myCarMap = new HashMap();
myCarMap.put("Focus", new Ford(new RegNum("HG42EG"), 1800));(Could be defined as "Map<String, Car>" if he wants to use generics.)

Oracle Data Mining Workshop

Oracle is providing a free 1 day on-site technical workshop designed to help customers and prospects move forward with the data mining technologies. The workshop consists of the following:
1. Understanding of data mining and how it compares to other analytical techniques, the data mining process and data mining in the 9i database.
2. Taking a business problem and showing how to turn it into a data mining problem.
3. Based on the data mining problem, what are the best methodologies, techniques and algorithms to use. The pros and cons of using each.
4. What types of data are need for the problem and how the data should be transformed and stored in the database.
5. How to optimize and tune the 9i database for development and production data mining.
This workshop is given by Richard Solari who is Oracle's Technical Director of Data Mining Services. For a complete agenda and to schedule your workshop, please contact Richard at [email protected]

Its primary purpose is to help in the development of data mining applications. You can as you explain utilize it as well as an analytical tool. We expect that any mining application will require analysts and developers to work together in building mining applications.

Is oracle data mining support to other databases to work datamining

Dear CB,
I am using oracle data mining and i have some doubts please clarify my doubts
1) Is ODM talk to other database or not ,like can we use ODM to prepare data mining work using other database
2) Is ODM supports social analytics
Thanks inadvance
thanks & regards
Suresh

Suresh,
1) Is ODM talk to other database or not ,like can we use ODM to prepare data mining work using other database
Yes, you can use Oracle Data Mining to talk to other databases, but ODM will need to "have" the data inside the Oracle Database during model build and model apply. You can use DB links to pull/push data to/from other Oracle and non-Oracle DBs, but all the data mining work and data transformations occur in-DB by our Oracle design. You can perform data prep, data transformation, build models and then compute dm predictions inside Oracle DB and then publish ODM results to any other dashboard/Q&R tool that can make a SQL call to Oracle to query results or ask ODM predictive model to make a real-time prediction based on current input data.
2) Is ODM supports social analytics
Depends on what you mean, but probably Yes. For example, we can mine unstructured data e.g. Twitter feeds and get 80% accurate Sentiment analysis. See http://www.google.com/url?sa=t&rct=j&q=mining%20twitter%20data%20clasification%20stanford&source=web&cd=1&ved=0CCMQFjAA&url=http%3A%2F%2Fwww.stanford.edu%2F~alecmgo%2Fpapers%2FTwitterDistantSupervision09.pdf&ei=Pk3VTsPOFYaIsQLknuiGDg&usg=AFQjCNGSErmPAa-n6kc_gVCCdxMRMKTeOw paper for a published tech paper that describes an approach that we have successfully replicated in-DB using ODM's text mining capabilities & Oracle Text. Add additional structured data, e.g. # purchases, $amount of purchases over time, etc. and you can have better Sentiment analysis or other types of predictive models.
Also, ODM can perform e.g. churn analysis and include as input to the model the "friends & family" usage, activities, and demographics as enriched input data to mine. We mine Star Schemas so we can pull together a 360 degree view of customer include "social" type data e.g. # links from a friend, etc. Broad topic.... hope this helps.
cb

Difference between Data staging and Dimension Table ?

Difference between Data staging and Dimension Table ?

Data Staging:
Data extraction and transformation is done here.
Meaning that, if we have source data in flat file, we extract it and load into staging tables, we take care of nulls, we change datetime format etc.. and after such cleansing/transformation at then end, load it to Dim/Fact tables
Pros: Makes process simpler and easy and also we can keep track of data as we have data in staging
Cons: Staging tables need space hence need memory space
Dimension Table:
tables which describes/stores the attribute about specific objects
Below is star schema which has dimension storing information related to Product, Customer etc..
-Vaibhav Chaudhari

How to install data mining server?

when i build classification model,i found error,i don't
know how to install data mining server
please help me thanks

The data mining feature comes with the 9i/10g Enterprise database edition. If you have installed the Enterprise edition, you got the server. Look for the DM directory in your database installation for more information.

Help to do data mining and transformation...

Similar Messages

Maybe you are looking for