How can data load time can be optimized using routine

Hi,
Data laod is taking too much time . How can data load time can be optimized using routine
Regards,
Vivek

Hi Vivek,
If you have code wirtten, please try to post the same, so that i cna help you in optimising it and give it you...
General Guideleines.
1. Dont use select statement inside the loop.
2. where possibel try to use Read statement with Binary Search Option and avoid using loop statement.
3. Try to avaoid joins if you are using any, isntead use For alll entries.
4. Try When u use for all entries try to make sure that your internal tbale is not initial.
5. Try to Sort and delete adjacent duplicates from internal tables wherever needed.
6. Use clear statement whererver required.
Regards,
Nanda.S

Similar Messages

How to tune data loading time in BSO using 14 rules files ?

Hello there,
I'm using Hyperion-Essbase-Admin-Services v11.1.1.2 and the BSO Option.
In a nightly process using MAXL i load new data into one Essbase-cube.
In this nightly update process 14 account-members are updated by running 14 rules files one after another.
These rules files connect 14 times by sql-connection to the same oracle database and the same table.
I use this procedure because i cannot load 2 or more data fields using one rules file.
It takes a long time to load up 14 accounts one after other.
Now my Question: How can I minimise this data loading time ?
This is what I found on Oracle Homepage:
What's New
Oracle Essbase V.11.1.1 Release Highlights
Parallel SQL Data Loads- Supports up to 8 rules files via temporary load buffers.
In an Older Thread John said:
As it is version 11 why not use parallel sql loading, you can specify up to 8 load rules to load data in parallel.
Example:
import database AsoSamp.Sample data
connect as TBC identified by 'password'
using multiple rules_file 'rule1','rule2'
to load_buffer_block starting with buffer_id 100
on error write to "error.txt";
But this is for ASO Option only.
Can I use it in my MAXL also for BSO ?? Is there a sample ?
What else is possible to tune up nightly update time ??
Thanks in advance for every tip,
Zeljko

Thanks a lot for your support. I’m just a little confused.
I will use an example to illustrate my problem a bit more clearly.
This is the basic table, in my case a view, which is queried by all 14 rules files:
column1 --- column2 --- column3 --- column4 --- ... ---column n
dim 1 --- dim 2 --- dim 3 --- data1 --- data2 --- data3 --- ... --- data 14
Region -- ID --- Product --- sales --- cogs ---- discounts --- ... --- amount
West --- D1 --- Coffee --- 11001 --- 1,322 --- 10789 --- ... --- 548
West --- D2 --- Tea10 --- 12011 --- 1,325 --- 10548 --- ... --- 589
West --- S1 --- Tea10 --- 14115 --- 1,699 --- 10145 --- ... --- 852
West --- C3 --- Tea10 --- 21053 --- 1,588 --- 10998 --- ... --- 981
East ---- S2 --- Coffee --- 15563 --- 1,458 --- 10991 --- ... --- 876
East ---- D1 --- Tea10 --- 15894 --- 1,664 --- 11615 --- ... --- 156
East ---- D3 --- Coffee --- 19689 --- 1,989 --- 15615 --- ... --- 986
East ---- C1 --- Coffee --- 18897 --- 1,988 --- 11898 --- ... --- 256
East ---- C3 --- Tea10 --- 11699 --- 1,328 --- 12156 --- ... --- 9896
Following 3 out of 14 (load-) rules files to load the data columns into the cube:
Rules File1:
dim 1 --- dim 2 --- dim 3 --- sales --- ignore --- ignore --- ... --- ignore
Rules File2:
dim 1 --- dim 2 --- dim 3 --- ignore --- cogs --- ignore --- ... --- ignore
Rules File14:
dim 1 --- dim 2 --- dim 3 --- ignore --- ignore --- ignore --- ... --- amount
Is the upper table design what GlennS mentioned as a "Data" column concept which only allows a single numeric data value ?
In this case I cant tag two or more columns as “Data fields”. I just can tag one column as “Data field”. Other data fields I have to tag as “ignore fields during data load”. Otherwise, when I validate the rules file, an Error occurs “only one field can contain the Data Field attribute”.
Or may I skip this error massage and just try to tag all 14 fields as “Data fields” and “load data” ?
Please advise.
Am I right that the other way is to reconstruct the table/view (and the rules files) like follows to load all of the data in one pass:
dim 0 --- dim 1 --- dim 2 --- dim 3 --- data
Account --- Region -- ID --- Product --- data
sales --- West --- D1 --- Coffee --- 11001
sales --- West --- D2 --- Tea10 --- 12011
sales --- West --- S1 --- Tea10 --- 14115
sales --- West --- C3 --- Tea10 --- 21053
sales --- East ---- S2 --- Coffee --- 15563
sales --- East ---- D1 --- Tea10 --- 15894
sales --- East ---- D3 --- Coffee --- 19689
sales --- East ---- C1 --- Coffee --- 18897
sales --- East ---- C3 --- Tea10 --- 11699
cogs --- West --- D1 --- Coffee --- 1,322
cogs --- West --- D2 --- Tea10 --- 1,325
cogs --- West --- S1 --- Tea10 --- 1,699
cogs --- West --- C3 --- Tea10 --- 1,588
cogs --- East ---- S2 --- Coffee --- 1,458
cogs --- East ---- D1 --- Tea10 --- 1,664
cogs --- East ---- D3 --- Coffee --- 1,989
cogs --- East ---- C1 --- Coffee --- 1,988
cogs --- East ---- C3 --- Tea10 --- 1,328
discounts --- West --- D1 --- Coffee --- 10789
discounts --- West --- D2 --- Tea10 --- 10548
discounts --- West --- S1 --- Tea10 --- 10145
discounts --- West --- C3 --- Tea10 --- 10998
discounts --- East ---- S2 --- Coffee --- 10991
discounts --- East ---- D1 --- Tea10 --- 11615
discounts --- East ---- D3 --- Coffee --- 15615
discounts --- East ---- C1 --- Coffee --- 11898
discounts --- East ---- C3 --- Tea10 --- 12156
amount --- West --- D1 --- Coffee --- 548
amount --- West --- D2 --- Tea10 --- 589
amount --- West --- S1 --- Tea10 --- 852
amount --- West --- C3 --- Tea10 --- 981
amount --- East ---- S2 --- Coffee --- 876
amount --- East ---- D1 --- Tea10 --- 156
amount --- East ---- D3 --- Coffee --- 986
amount --- East ---- C1 --- Coffee --- 256
amount --- East ---- C3 --- Tea10 --- 9896
And the third way is to adjust the essbase.cfg parameters DLTHREADSPREPARE and DLTHREADSWRITE (and DLSINGLETHREADPERSTAGE)
I just want to be sure that I understand your suggestions.
Many thanks for awesome help,
Zeljko

How small Standard Cycle time can be defined in OEE standard?

Hello,
There's OEE standard definition in MII and it allows to define standard cycle time.
I'd like to know how small Standard Cycle time can be defined in OEE standard?
In my customer case, their product is very small and SFC size will be over than 10,000.
(Please imagine screw.)
This screw's required process time at resource A is 0.72 seconds.
Then, they defined the standard cycle time as 0.0002 hour.
0.72 / 3600 = 0.0002
Does OEE report allows such small standard cycle time?
If not, how we can define OEE standard and use OEE report functionality?
Best Regards,
Takahiro Uesugi

Hi Nick,
Main question here is: does business want to keep those members as separate members or it wants to merge them.
If they want to keep them as separate then you have to come up with a better conversion procedure.
If they want to merge them then you have to find out how to find another member to merge with.
You should also implement same procedure when you are going to load Transaction data.
Gersh

Mac Pro early 2008 2 x 2.8GHz Quad Core 10.7.5, replaced 4 x 500ghds with 4 x 3Tb and Raid card Raid5 them OK but can not load time machine back up as it does not see any of the 4 drives, neither does disc utilities, any thoughts?

On my Mac Pro, early 2008 2 x 2.8GHz Quad Core 10.7.5, replaced 4 x 500ghds with 4 x 3Tb. The Raid card Raid 5ed them OK but can not load time machine back up as it does not see any of the 4 drives, neither does disc utilities, any thoughts?

You don't see drives when using a hardware RAID only the volumes.
WD Green are not suitable for hardware RAID (or software RAID for that matter).
USB2 and TimeMachine are a disaster waiting to happen. I've used SATA (internal) and eSATA (using SATA PCIe cards) trouble-free.
I can't say I begin to understand your use or "choosing the install drive for the backup"
Install what?
You don't load TimeMachine. Maybe seems minor but that is not how to describe the behavior of software.
You read FAQ and How To http://www.apple.com/support/timemachine
Time Machine’s Gory Details:
https://www.apple.com/support/timemachine/
TimeMachine 101
https://support.apple.com/kb/HT1427
Lion Recovry & TimeMachine
http://www.apple.com/support/lion/installrecovery/
http://www.apple.com/support/lion/
How To Restore Your System
http://pondini.org/TM/14.htmlMac OS X v10.7 Lion
Pondini's Blog: Time Machine - Troubleshooting -- B5. Would you like to inherit (or re-use) the backup . . . ?

How to find Data load time ?

Hi,
Where do i look for the Total data load time ? from data source to PSA ?

Hi Honar,
1) Goto monitor of IP, in header tab you can find the runtime of IP.
2) So you are loading data from source to BW.In IP header tab copy the req number, goto the source systems from which data is loading.
goto SM37 give the request number with BI as pre-fix, you will find the total run time of job with job log.
Hope this helps.

HT1595 The date and time can't be set? What to do?

The date and time can't be set?

I'm having the same issue today on several of our Apple TVs. I am the Director of Technology in a school district and starting this morning I am getting reports that Apple TVs are not working and upon investigation they all sit at setting date and time. All network setting are properly configured. The firewall is set to allow port 123 both TCP and UDP from all internal clients. It occurs both over WiFi and Ethernet.
Is the Apple Time Server experiencing issues?

N82 date and time can't be saved.

Hi all,
Just wana share my frustration with Nokia Care Centre, Im from Singapore and recently just purchase a Nokia N82,but after 1 day of usage,found out the date and time can't be saved into the phone,have to manually set it everytime i power on my phone, brought it to nokia service centre, was told there is no 1 to 1 replacement for newly purchased product. The service centre can't help,promised to fixed it in 3 days then on the 3rd day,they told me they still can't fix the problem and have to wait for another 5 days. Imaging buying a faulty product, and can't change a new one and have to wait for 8 days for repair...The service is really a disappointment, called Nokia Careline, they can't help either...i really regretted buying a Nokia phone, as a consumer i have no say in things, i get push around by Nokia staffs, any bright ideas who can really help me in Nokia? Its such a big and reputed worldwide company but yet the service provided by the customer care centre is really dismal. If their company CEO sees what i have seen in their customer service centre, i think he would have quit. Anyone have any advice on this?

Did it was ever fixed?

Data Load Times, etc.

Hello Forums:
I saw this website:
http://www.s-w-h.com/index.php?language=en
And found that a website like this loaded very quickly, and
seemed to fit almost all resolution.
I was wondering how did they make it fit dynamically? And how
did they manage to reduce load times so significantly?
I kinda wondered if it's possible to develop a massive forum
or online community thing using purely Flash. Would that be
possible? What if there were lots of user interaction, can the load
times be significantly cut down?

Does it load all the data while connected with 5GHz frequency… What is the IP address that you are getting from the Router? You can check the IP address in the following manner:
If you wish to find out your own IP address, you should run the same ms-dos prompt by clicking Start -> All Programs -> Accessories -> Command Prompt. A black pop up box should come up, type: "ipconfig /all" … There check the IP address and Default Gateway under LAN….
# Open up the browser and on the address bar type “IP address of Default Gateway” that will open up the Router setup page…
# In that page itself make the MTU size to 1400 from Auto….
# Make the following Channel Settings of 2.4 GHz you can make Channel Width to 20 MHz only and Channel to 6, 9, 11.. Click Save Settings...
Then check the connectivity…

How to delete the data loaded into MySQL target table using Scripts

Hi Experts
I created a Job with a validation transformation. If the Validation was failed the data passed the validation will be loaded into Pass table and the data failed will be loaded into failed table.
My requirement was if the data was loaded into Failed database table then i have to delete the data loaded into the Passed table using Script.
But in the script i have written the code as
sql('database','delete from <tablename>');
but as it is an SQL Query execution it is rising exception for the query.
How can i delete the data loaded into MySQL Target table using scripts.
Please guide me for this error
Thanks in Advance
PrasannaKumar

Hi Dirk Venken
I got the Solution, the mistake i did was the query is not correct regarding MySQL.
sql('MySQL', 'truncate world.customer_salesfact_details')
error query
sql('MySQL', 'delete table world.customer_salesfact_details')
Thanks for your concern
PrasannaKumar

Data Load time

I have 12 files which are similar in size. When they were loaded on one server, the load time is pretty consistent. For example, it takes about 100 seconds for loading each of the 12 files. On the other server, however, the data load time increases dramatically when loading later files, i.e., 100s, 120s, 180s....The essbase version are the same on both servers (window 2000). What could cause this problem? Is this something to do with server settings or essbase settings? Thanks for your response.Lin

Hi,Can you please specify the version of Essbase server and development tool (app manager, admin services)?Have you checked the dense/sparce configuration?Grofaty

Can I access time capsule file without using imac

can I access time capsule file without using imac

From a time machine backup? No. If you mean just a regular file stored on there? Not directly. There are a couple of apps out there where you can indirectly get to a file but it's a pain. If you have need of accessing a file both from the mac and the iPad then I suggest DropBox. You get 2GB free and it's a good way to share information between machines and devices.

How erase data on Time Machine?

How erase date on Time Machine?

To erase everything on the Time Capsule hard drive.......
Open Macintosh HD > Applications > Utilities > AirPort Utility
Click Manual Setup
Click the Disk icon
Click the Disk tab just below the icon
Click Erase
The "Quick Erase" option will only take a moment or two

My new apple tv says that the date and time can't be set

My new apple tv say that it cannot set the date and time

I'm having the same issue today on several of our Apple TVs. I am the Director of Technology in a school district and starting this morning I am getting reports that Apple TVs are not working and upon investigation they all sit at setting date and time. All network setting are properly configured. The firewall is set to allow port 123 both TCP and UDP from all internal clients. It occurs both over WiFi and Ethernet.
Is the Apple Time Server experiencing issues?

How to decrease loading time for 0Mat_Plant

Hi Experts,
I am currently looking to decrease loading time of 0Mat_Plant. I have gone thru performance material available and checked following things listed below :
Plz note following:
1) Standard delta is avail but it has been enhanced and there are Z fields. So if i use delta, them I might be missing delta for Field.
2)Can we switch OFF number range SID buffering to avoid memory allocation on daily basis ...somehow through ABAP program?
3) BUffering can be switched OFF, but there is delta load also alonwith full load. So performance would decrease for delta records. But no of records for Full load is 188000 and for delta is just 1700 around. SO will this helps in decreasing overall time? Any idea?
Load is taking around 90 minutes in my system (Full Load)..
I m not clear abt data package size and if it will help me or not..
What more can we do??
Any help would be appreciated and points would be assigned......
Thanks in advance
Gaurav
Message was edited by:
Gaurav

Hi Nagesh,
Here are my replies;
1) Loading time is only for 0mat_plant..
2) R/3 job finishes quickly....no prob seems to be with this..
3) I m not clear about "locked trfc..but there are no errors in trfc...and there doe snot seem to be IDOC clogging problem...
4) Not sure what yu mean by trfc "released"....
I checked R/3 JOb log and its pasted below ( though for other datasouce)....
I want to know wht does it implies.....in general i hve few questions listed below after the log..::
BEGIN BW_BTE_CALL_BW204020_E 13.514
END BW_BTE_CALL_BW204020_E 13.514
BEGIN EXIT_SAPLRSAP_002 13.514
Datasource:0MATERIAL_ATTR, In: 13.514 , Out: 13.514
END EXIT_SAPLRSAP_002 13.514
Asynchronous send of data package 1 in task 0002 (1 parallel tasks)
IDOC: Info IDoc 2, IDoc No. 442405, Duration 00:00:00
IDoc: Start = 26.06.2007 19:34:06, End = 26.06.2007 19:34:06
tRFC: Data Package = 1, TID = 0A01A001012F4681A2A7362D, Duration = 00:0
tRFC: Start = 26.06.2007 19:35:03, End = 26.06.2007 19:35:04
BEGIN BW_BTE_CALL_BW204020_E 0
END BW_BTE_CALL_BW204020_E 0
BEGIN EXIT_SAPLRSAP_002 0
Datasource:0MATERIAL_ATTR, In: 0 , Out: 0
END EXIT_SAPLRSAP_002 0
Asynchronous transmission of info IDoc 3 in task 0003 (0 parallel tasks
IDOC: Info IDoc 3, IDoc No. 442406, Duration 00:00:00
IDoc: Start = 26.06.2007 19:35:04, End = 26.06.2007 19:35:04
Synchronized transmission of info IDoc 4 (0 parallel tasks)
IDOC: Info IDoc 4, IDoc No. 442407, Duration 00:00:00
IDoc: Start = 26.06.2007 19:35:04, End = 26.06.2007 19:35:04
Job finished
I want to confrm my understanding abt extraction process when infopackage gets triggerred...
Infopackge gets triggered....->R/3 job gets created/triggered....-> Info iDoc regarding request sent to BW..-> Application document(??)) posted..(( posted to what???.... passed to Trfc??....and then to psa which is seen in monitor??))...
Like wise after successfull passing of datapackage to BW ( not posting to pSA) , next datapackage is sent..... and so on...
What is TRfc?? .as far I know its function modules created to pass data to bW thru RFC call...(??)...
IF you can give me clear picture of whts happening behind the scenes, it would be great.. or helpful links would do....it would be help me to enjoy wht you ae saying and how to tune more...
THanks for your patience and help...
Already assigned points to you..
PS: I hve turned OFF buffering for master data records greater than 100, 000 and saved approx 25% for each master data records....
Regards
Gaurav

Data load times

Hi,
I have a question regarding data loads. We have a process cahin which includes 3 ods and cube.
Basically ODS A gets loaded from R/3 and the from ODS A it then loads into 2 other ODS B, ODS C and CUBE A.
So when I went to monitor screen of this load ODS A-> ODS B,ODS C,CUBE A. The total time shows as 24 minutes.
We have some other steps in process chain where ODS B-> ODS C, ODS C- CUBE 1.
When I go the monitor screen of these data loads the total time the tortal time for data loads show as 40 minutes.
I *am suprised because the total run time for the chain itself is 40 minutes where in the chain it incclude data extraction form R/3 and ODS's Activations...indexex....
*Can anybody throw me some light?
Thank you all
Edited by: amrutha pal on Sep 30, 2008 4:23 PM

Hi All,
I am not asking like which steps needed to be included in which chain.
My question is when look at the process chain run time it says the total time is equal to 40 minutes and when you go RSMO to check the time taken for data load from ODS----> 3 other data targets it is showing 40 minutes.
The process chain also includes ods activation buliding indexex,extracting data from R/3.
So what are times we see when you click on a step in process chain and displaying messages and what is time you see in RSMO.
Let's take a example:
In Process chain A- there is step LOAD DATA- from ODS A----> ODS B,ODS C,Cube A.
When I right click on the display messages for successful load it shows all the messages like
Job started.....
Job ended.....
The total time here it shows 15 minutes.
when I go to RSMO for same step it shows 30 mintues....
I am confused....
Please help me???

How can data load time can be optimized using routine

Similar Messages

Maybe you are looking for