Best way to load old historical records into sc2 type 2 dimesnion

Hi,
We have scd type 2 diemsnion and have history dating form early 2012 in there but wish to add extra older historical episodes of recoprds for 2011 into it.
What is the best way to do this?
Thoughts?
Considering some pl/sql but 5 levels in the dimesnion.
Can't reload as need existing records due to integirty with facts.
Thnaks

Are you using OWB's dimension operator and it's SCD functionality?
Cheers
David

Similar Messages

Best way to insert millions of records into the table

Hi,
Performance point of view, I am looking for the suggestion to choose best way to insert millions of records into the table.
Also guide me How to implement in easier way to make better performance.
Thanks,
Orahar.

Orahar wrote:
Its Distributed data. No. of clients and N no. of Transaction data fetching from the database based on the different conditions and insert into another transaction table which is like batch process.Sounds contradictory.
If the source data is already in the database, it is centralised.
In that case you ideally do not want the overhead of shipping that data to a client, the client processing it, and the client shipping the results back to the database to be stored (inserted).
It is must faster and more scalable for the client to instruct the database (via a stored proc or package) what to do, and that code (running on the database) to process the data.
For a stored proc, the same principle applies. It is faster for it to instruct the SQL engine what to do (via an INSERT..SELECT statement), then pulling the data from the SQL engine using a cursor fetch loop, and then pushing that data again to the SQL engine using an insert statement.
An INSERT..SELECT can also be done as a direct path insert. This introduces some limitations, but is faster than a normal insert.
If the data processing is too complex for an INSERT..SELECT, then pulling the data into PL/SQL, processing it there, and pushing it back into the database is the next best option. This should be done using bulk processing though in order to optimise the data transfer process between the PL/SQL and SQL engines.
Other performance considerations are the constraints on the insert table, the triggers, the indexes and so on. Make sure that data integrity is guaranteed (e.g. via PKs and FKs), and optimal (e.g. FKs should be indexes on the referenced table). Using triggers - well, that may not be the best approach (like for exampling using a trigger to assign a sequence value when it can be faster done in the insert SQL itself). Personally, I avoid using triggers - I rather have that code residing in a PL/SQL API for manipulating data in that table.
The type of table also plays a role. Make sure that the decision about the table structure, hashed, indexed, partitioned, etc, is the optimal one for the data structure that is to reside in that table.

What's the best way to load FieldPoint measurement data into PI System?

I am finding the best way to load data collected by NI Field Point (FP2220) into the PI system of our power plant.
I found pieces of information about FieldPoint OPC server in NI.com. Not sure if it comes with Field Point Hardware, sold by NI as a separate product or it is actually non-standard NI products. Anyway, I know that there exists a thing called FieldPoint OPC server.
The PI system I mentioned has a OPC client software called PI-OPC interface. It is able to communicate with standard OPC DA server. If that FieldPoint OPC server is a standard OPC DA server provide data collected by Field Point complying to OPC standard, than that's perfect.
Anyone familar with PI system and NI product, please help if the above is going to work or if there is a better way to put Fieldpoint data into PI.

Hi Eric,
This information really helpful, thanks. Regarding to the NI OPC server for NI FieldPoint, I have the other query.
In my setup, there are two sets of FieldPoint located in two different locations on my ethernet network. They are going to be controlled by a single PC. If I am going to connect both my FieldPoint sets with OPC standard, how many NI OPC server for FieldPoint do I need to connect to? Are there two NI OPC servers each serves one FieldPoint set? Or there is only NI OPC server which serves both FieldPoint sets?
I am concerning about the number of NI OPC server instances running, because the number of OPC client license I need to purchase depends on how many OPC server I need to connect to. If one NI OPC server serves both my FieldPoint sets, I only need to buy one OPC client license; otherwise, I need to purchase two. In the future, I am going to have another two sets of FieldPoint sets, so the answer of my query determines how many OPC clients I need to purchase eventually - One or four. A huge price difference.
Looking forward to your reply.
Regards,
Roger

I burned a playlist to a data CD using itunes 11 but what is best way to get it import it into another computer with itunes running?

I burned a playlist to a data CD using itunes 11 but what is best way to get it import it into another computer with itunes running? I get an "X" when trying to copy to the "automatically add to itunes" folder on the 2nd computer.

ok... so i i just made sure all my music is all together in 1 folder and i am going to trash my old music folder that is in the itunes folder... and follow your directions - let itunes start w/new library and bring my music in. Well, last little hurdle/question. I noticed that the place where my itunes library is - on an external drive doesnt have much space left on it. The place where i gathered all my music together in order to start fresh - is a different external drive and has LOTS of free space on it.
SO... i now would like to just keep all that freshly compiled clean music right where it is but in advanced preferences I want to point to that new folder on the drive that has so much more space. So... should i delete the old music then go into itunes with a blank library and change the folder in preferences - quit and go back in and then follow your steps to loading my music? If i change the setting of where itunes looks for the music - that doesn't wipe out the old stuff... it just mixes it all up together right? I dont want that!!
i promise this is the last question - i would love to fix this problem tonight and just be DONE!
Thanks in advance for your patience and knowledge!!
Lisa

Best Way to Load Data in Hash Partition

Hi,
I have partitioning by Hash on a Large Table of 5 TB. We have to load Data say more than 500GB daily on that table from ETL.
What is the best way to Load data into that Big Table which has hash Partition .
Regards
Sahil Soni

Do you have any specific requirements to match records to lookup tables or it just a straight load - that is an insert?
Do you have any specific performance requirements?
The easiest and fastest way to load data into Oracle is via external file and parallel query/parallel insert. Remember that parallel DML is not enabled by default and you have to do so via alter session command. You can leverage multiple CPU cores and direct path operation to perform the load.
Assuming your database is on a linux/unix server - you could NFS load the file if it is on a remote system, but then you will be most likely limited by network transfer speed.

Loading multiple physical records into a logical record

Hello,
I'm not sure if this is the right place to post this thread.
I have to import data from a fixed length positioned text file into a oracle table using sql loader.
My sample input file (which has 3 columns) looks like:
Col1 Col2 Col3
1 A abcdefgh
1 A ijklmnop
1 A pqrstuv
1 B abcdefgh
1 B ijklmn
2 A abcdefgh
3 C hello
3 C world
The above text file should be loaded into the table as:
Col1 Col2 Col3
1 A abcdefghijklmnpqrstuv
1 B abcdefghijklmn
2 A abcdefgh
3 C helloworld
My question: Is there a way tht i can use the logic of loading multiple physical records into a logical record in my oracle tables. Please suggest.
Thanks in advance.

Hi,
user1049091 wrote:
Kulash,
Thanks for your reply.
The order of the concatenated strings is important as the whole text is split into several physical records in the flat file and has to be combined into 1 record in Oracle table.
My scenario is we get these fixed length input files from mainframes on a daily basis and this data needs to be loaded into a oracle table for reporting purpose. It needs to be automated.
Am still confused whether to use external table or a staging table using sql loader. Please advise with more clarity as am a beginner in sql loader. Thanks.I still think an external table would be better.
You can create the external table like this:
CREATE TABLE     fubar_external
(      col1     NUMBER (2)
,      col2     VARCHAR2 (2)
,      col3     VARCHAR2 (50)
ORGANIZATION EXTERNAL
(       TYPE             ORACLE_LOADER
     DEFAULT DIRECTORY XYZ_DIR
     ACCESS PARAMETERS (
                             RECORDS DELIMITED BY     NEWLINE
                      FIELDS (   col1        POSITION (1:2)
                                  ,   col2        POSITION (3:4)
                           ,   col3        POSITION (5:54)
     LOCATION        ('fubar.txt')
);where XYZ_DIR is the Oracle Directory on the database server's file system, and fubar.txt is the name of the file on that directory. Every day, when you get new data, just overwrite fubar.txt. Whenever you query the table, Oracle will read the file that's currently on that directory. You don't have to drop and re-create the table every day.
Note that the way you specify the columns is similar to how you do it in SQL*Loader, but the SEQUENCE generator doesn't work in external files; use ROWNUM instead.
Do you need to populate a table with the concatenated col3's, or do you just need to display them in a query?
Either way, you can reference the external table the same way you would reference a regular, internal table.

Best way to load initial TimesTen database

I have a customer that wants to use TimesTen as a pure in-memory database. This IMDB has about 65 tables some having data upwards of 6 million rows. What is the best way to load this data? There is no cache-connect option being used. I am thinking insert is the only option here. Are there any other options?
thansk

You can also use the TimesTen ttbulkcp command line utility, this tool is similar to SQL*Loader except it handles both import and export of data.
For example, the following command loads the rows listed in file foo.dump into a table called foo in database mydb, placing any error messages into the file foo.err.
ttbulkcp -i -e foo.err dsn=mydb foo foo.dump
For more information on the ttbulkcp utility you can refer to the Oracle TimesTen API Reference Guide.

What's The Best Way to Load a Replacement iPod Touch?

I just received a replacement iPod Touch 3G for my daughter. Her iTunes library for the Touch that we returned to Apple is on her MacBook Pro. What is the best way to load the new Touch? She plans on using the same name for it. Should I restore from a previous backup or just plug it in and let in sync? Do I need to set anything on iTunes for disk management?
Thank you,
Bruce

If she wants it like her old one the previous backup, if there is anything she wants to change though, now would be a perfect time to just resync it and start over. It really depends on what she wants.

Best way to load a local XML

What's the best way to load a local XML file into a Datagrid
without using the HTTP service?
Thanks

Use Class.getResource() and Class.getResourceAsStream,
when you eventually jar it up, just put the same
hierarchy in the jar and they can be found.
BufferedInputStream in = new
BufferedInputStream(Class.getResourceAsStream("foo/bar/
yimg.jpg"));for example.
That should work out of the Jar too.Thanks v. much - got it all going some days ago but found it all fell apart when I ran the code within the jar. However I used the similar ClassLoader routines and found that if you first get a uri for the resource then it is a fully-qualified file uri. This works ok when running directly against files on disk, but once they are in a .jar it all turns to custard, giving you "java.lang.IllegalArgumentException: URI is not hierarchical".
I followed your suggestion to use Class.getResourceAsStream and it now works perfectly - thanks again!

Oracle 10.1, Whats the best way to load XML in database?

Hi All,
I am a typical Oracle developer. I know some Java and some XML technologies, but not an expert.
I will be receiving XML files from some system, which will be,
- of reasonable size like 2 to 15 MBs
- of reasonable complexity, like the root element have children, grand-children and great-grand-children, with attributes and all
- Every day it needs to be loaded to Oracle database, in relational format
- we need not update the XML data, only put the XML data in relational table
My questions are,
- With Oracle 10.1, XML DB, what is the best way to load this XML file to relational Oracle tables ?
- What can be the steps in this process ?
- In the documentation, I was lost and was not able to decide anything concrete
- If I write a pure Java program with SAX API and load the data to Oracle database in same program, is it a good idea?
- Is there any pure Oracle based way to do this?
- If I want to avoid using DOM parser, as it will need more JAVA_POOL_SIZE, what can be the way ?
Please help.
Thanks

Many customer solve this problem by registering an XML Schema that corresponds to their XML and then creating relational views over the XML that allow them to access the content in a relational manner. They then use insert as select operations on the relational views to transfer data from the XML into relational tables where necessary. There are a large number of threads in this forum with detailed examples of how this can be done. Most of the customers who have adopted this approach have discovered that this is the least complex approach in terms of code that to be developed / maintained and offeres acceptable performance.

I used a usb turntable to import old vinyl records into iTunes. Then with the cloud and iTunes match I put them on another Mac. The songs come up in the music list but they are greyed out. How do I get them recognized.

I used a usb turntable to import old vinyl records into iTunes. Then with the cloud and iTunes match I put them on another Mac. The songs come up in the music list but they are greyed out. How do I get them recognized. The symbol states the songs were downloaded from iCloud. I would like to get them to play so that I can make a playlist, and burn a CD.

Try:
- Reset the iOS device. Nothing will be lost
Reset iOS device: Hold down the On/Off button and the Home button at the same time for at
least ten seconds, until the Apple logo appears.
- Unsync all music and resync
- Reset all settings
Go to Settings > General > Reset and tap Reset All Settings.
All your preferences and settings are reset. Information (such as contacts and calendars) and media (such as songs and videos) aren’t affected.
- Restore from backup. See:
iOS: How to back up
- Restore to factory settings/new iOS device.

What is the best way to merge a file content into log file

What is the best way to merge a file content into log file.
In worst case, I will read the file line by line as string, then use
logger.info(lineString)to output to log file.
However, is there better way to do this?
The eventual log file will be something like:
log message 1
log message 2
content from file line 1
content from file line 2
content from file line 3
log message 3
log message 4Thanks

John618 wrote:
Thank you and let me explain:
1. What do you mean by better?
I would like to see better performance. read line by line and log each line as string can be slow. Did you measure this and determine that it is actually a problem for your application? Or are you guessing?
Regardless of what you do you are still going to need to read the file.
>
2.The only better way I can think of is not having to do it, but I assume you have a very good reason to want to do this.
Yes, I have to do it beacuse the requirement is to have that file content be part of logging.
Any idea?How is it supposed to be part of it? For example which of the following is better?
        File AAA - contents
                   First Line
                   Second Line XXX
        Log 1
                2009-03-27 DEBUG: Random preceding line
                2009-03-27 DEBUG: First Line
                2009-03-27 DEBUG: Second Line XXX
                2009-03-27 DEBUG: Random following line
        Log 2
                2009-03-27 DEBUG: Random preceding line
                2009-03-27 DEBUG: ----- File: AAA -------------
                First Line
                Second Line XXX
                2009-03-27 DEBUG: Random following lineBoth of the above have some advantages and disadvantages.
The first in a mult-threaded app can end up with intermittent log entries in between lines, so having log lines with thread ids becomes important.
The first can be created by reading one line at a time and posting one at a time.
The second can be created by reading the entire file as a single string and then posting using a single log statement.

Best way to load configuration

Hi All
I've developed an application and I want to give my client the possiblity to change some configuration data (paths, languaje, etc).
Which is the best way to store this kind of data and wich is the best way to load it ?
I don't want a simple book or manual reference. I'll appreciate words of experience wisdom.. :)
Thanks in advance
<jl>

It used to be through java.util.Properties, but
I've found the java.util.preferences classes to
be almost too easy to use - pretty sweet stuff! (And
that should be enough hints for you right there...)I'll read preferences tips in API reference.
Thanks.
<jl>

Best way to load data from External data to ADC

Hi Guy,
I am new in BAM. I currently working for a existing system that have all it data in existing system, which i only have the view access to the database/table.
so i just wonder, what is the best way to load the data so that it can populate to the active data cache?
thanks.

Hello,
Use the EDS.For reference http://docs.oracle.com/cd/E14571_01/integration.1111/e10224/bam_extl_data_sources.htm
Regards
Siva Sankar

Best way to load "Action Essentials"

Kinda new with using Macs....need to know the BEST way to load Action Essentials?

well I was really over-thinking it! ha
Thanks so much Rick!
On Fri, Sep 12, 2014 at 12:24 PM, Rick Gerard <[email protected]>

Best way to load old historical records into sc2 type 2 dimesnion

Similar Messages

Maybe you are looking for