EIS data load slow

Greetings all,
Windows Server 2003 SP2. SQL Server 2000 SP3. Hyperion 9.3.1.
I am running an EIS data load, and the load time is progressively longer depending on the number of records loaded.
For example, when doing a data load of 1,000,000 records, it takes 2 minutes to load the first 500,000 records but an additional 10 minutes to load the next 500,000 records.
Is there any reason for this, and can I do anything about it?
Regards,
Paul.

Since non-Streaming mode is the only mode that EIS leverages to load data, it's not surprising that- Data loads are slow.
If you want to continue using EIS for data loads, I believe- All you can do is- To refine/tune your User-defined SQL.
You've mentioned about just few million records. However, in the volume-intensive environments where records are in the order of hundreds/thousands of millions, the majority would think of using EIS only for Metadata load & deferring data load to the flat file data extracts, which is way faster. At the same time, you'd have to consider if the extraction time required is not too much that it defeats the very purpose.
- Natesh

Similar Messages

EIS data load via rules file

Hi,
PLease let me know the process to control the EIS data load into essbase using a rules file. I did not find the option in EIS. please help me.
Thanks,
pr

In EIS
1) You have to define the Logical OLAP Model connecting to the relational source.It defines the joins between fact table and dimension tables.
2) Based on the OLAP Model You have to create meta outline which defines the rules for loading members and data into essbase.

EIS Data load

Hi all,
I am bit confused about loading data via EIS. We have two process one is to load data via rule file hisdld created by EIS and second is using user defined sql query. Are both these methods independent of each other or not?
When i say independent i mean if we make some changes in user defined sql and load data are those changes first refelected in the rule file and then data load process occurs using this rule file or data load occurs just via sql quey we run
It would be great if somebody could clarify the same.
Thanks in advance!!

Thanks for the reply.
Yes i am talking about sql query within EIS so if i understood it right by your post if i make changes to that user defined sql code(within EIS ) it will make the changes in the data load rule file created by EIS i.e "hisdld". So If we talk of a data load process within EIS the User defined sql is used to create data load rule "hisdld" and hence they are dependant on each other.
Edited by: user10859098 on Jan 29, 2009 10:54 AM
Edited by: user10859098 on Jan 29, 2009 10:54 AM
Edited by: user10859098 on Jan 29, 2009 10:56 AM

EIS data load problem

Hi,
I use user defined sql to do a data load using EIS 7x
When i run the sql in toad just to see the data I see 70000 rows
But when I use the same sql in the userdefined sql and try to load data...I dont know why EIS says that it loaded around 69000 rows and also I see no rejections...
I even made some sql changes to find out what are the records that are not being loaded..and I see some rows when I run a sql in toad and if i use the same in EIS its not loading those rows which I can see in toad for the same sql ...
This is very strange.. Can any one help me with this,..??

Glenn-
I dont think there is anything unique about the missing rows..
Actually the a part of code has been added to the previously working view(which i use to load data) to bring in some additional data...
I took that part and tried to load data...and I see no data being loaded or rejected..
It just says that records loaded "0"..rejected 0...
but the same part brings in the required additional data... when exectued in toad..
and about the excel sheet lock and send..I did that a week and as you said to my surprise it loads everything....
This was the test bu which I figured out that EIS is not even able to load a part of data...and I found the exact part of data by going through it closely..
So I think this is something to do with sql interface..
And I did not understand the last lines in your post...
/////I know when I do regular Sql interface some data types don;t load and I have to convert them to varchar tto get them to work. If the Excel file and Flat file loads work, look at that//////
what did convert to varchar???
Excel load works fine..So I think its something similar to what you mentioned in your post...
Can you please explain it ...

EIS data load issue (04/24).

Hi,
I am trying to load data using EIS 7.1 to an ASO cube. The log says load was done successfully and that aggregation failed. However, there is no data at the zero level after the load (which could be why aggregation failed). It was running successfully earlier. I changed a dimension from Multiple Hierarchy to Dynamic...not sure if this caused the problem. What are the potential causes? Please help.
Thanks.

Hi,
Try this:
1. Backup model and metaoutlines
2. Open model, delete connection (and dimension for which You are having a problem)
3. Save model (open again and add the dimension)
4. Open one of Your metaoutlines and do the same with this dimension
5. save it and make test
Please let me know if it works fine (I have such a problem and usualy it was the right solution :) )
regards
Karol

Data loading slow in BI server

hi all,
We have installed our BI production server recentlly. while loading data BI functional consultants complain of poor speed. The process which took only few seconds in development and quality system is taking several minutes in the production system even though the production server is a complete dedicated server for the BI production with 32GB RAM. being a basis person Please help us in identifying the issue in the production server which is causing the speed to be so slow in the prod server as the prod server would be used by users who wouldnt like to wait for so long. and what can i do to improve the speed of the servers.
thanks,
Priya

loading from PSA takes always less time.
However, is quite generic a time load issue, certainly the production system will have more data so it is not always possible to use development and test as reference.
Anyway, i think you should focus on the tuning (using the SBIW)
For example
on R3 you have to customize the dat aload.
on BW for datamart.
So, in R3
maximal package size in kB (default 10,000 KB)
maximum package size in number of records (Maximum memory consumption per data package is around 2Max. lines1000
Byte)
Frequency of Info IDocs. Default: 1. Recommended: 5 - 10
Max proc. (= maximum number of dialog workprocesses which were
allocated for this dataload for processing extracted data in BW)
in BW
values recomended
Recommended values:
max size: 20,000 50,000 (kbyte)
max lines: 50,000 100,000 (Data lines)
frequency: 10
max proc.: 2 5
there are a lot of parameters to set, but i suggest you to proceed stepbystep.
ciao

EIS data load rejects

Is there a way to find out if there were any dataload rejects after the EIS dataload when running using scripts?
If I use the EIS console and run the load then I can see the number of records loaded and rejected, but is there a way to find out about this info if we are running the load using EIS scripts?

Hi,
I think Donald has already mentioned the answer in his above post, although indirectly.
All you've to do is-
(i) Before your cube refresh starts, delete all the folders(corresponding to the cube you're loading) from loadinfo folder
(ii) After your load completes, let your batch script check if any new folder is created for the cube you're loading. If yes, there're errors/rejections. Else, your load is successful!
- Natesh

EIS data load issue

I have a dimension in the metaoutline as given below heirrachyORG ~ -> State ~ -> Division ~ -> Agency ~ -> Agent ~ I am able to build the heirrarchy in this order , but the data that I load needs a diffirent filter on each dimension level for example : For state load data where type = 5For division load data where type = 4For agency load data where type = 3For Agent load data where type = 2 When I put a filter on each level it uses the 'AND' operator which I do not want. Please advice how this can be accomplished in EISThanksande

Load the member data in four different passes and set the appropriate filter for each pass. The filter on the first pass would be where type = 5, on the second pass where type = 4, and so on.

Data Load slow on some days

Dear SDN Team,
I have a load into ODS which sometimes takes 5 minutes to finish but on some days takes >40 minutes. (Though number of records are same)
Problem is not in R/3 but in BW itself.
In the details tab I notice the following:
The time stamp at 'Tranfer (Idocs and TRFC) shows 5:05
The time stamp at 'processing datapacket' shows 5:50
Kindly help me as this is really annoying.
Best Regards,
Krishna.
Edited by: Siegfried Szameitat on Oct 29, 2008 12:39 PM
I deleted your offer of points as it is against the rules.

Hi,
1)You can check with just a click on the last step before the first data package in the details tab.
This will create a displa of time below
once you click on the step it will give you a time and if you click on a step below it will give you another time in the same box.
This is how I determin the time elapsed between different steps in the manage tab.
2)To find the job search for job begning with BI* and user ALEREMOTE.
You will get a long list just try to see the tart time for both and he one which matches with the start of the job in BW and may be slightly after that.
You will have to go inside job log of each job to see the data source and the one whch contains your data source is the job you want.
Thanks
Ajeet

4.2.3/.4 Data load wizard - slow when loading large files

Hi,
I am using the data load wizard to load csv files into an existing table. It works fine with small files up to a few thousand rows. When loading 20k rows or more the loading process becomes very slow. The table has a single numeric column for primary key.
The primary key is declared at "shared components" -> logic -> "data load tables" and is recognized as "pk(number)" with "case sensitve" set to "No".
While loading data, these configuration leads to the execution of the following query for each row:
select 1 from "KLAUS"."PD_IF_CSV_ROW" where upper("PK") = upper(:uk_1)
which can be found in the v$sql view while loading.
It makes the loading process slow, because of the upper function no index can be used.
It seems that the setting of "case sensitive" is not evaluated.
Dropping the numeric index for the primary key and using a function based index does not help.
Explain plan shows an implicit "to_char" conversion:
UPPER(TO_CHAR(PK)=UPPER(:UK_1)
This is missing in the query but maybe it is necessary for the function based index to work.
Please provide a solution or workaround for the data load wizard to work with large files in an acceptable amount of time.
Best regards
Klaus

Nevertheless, a bulk loading process is what I really like to have as part of the wizard.
If all of the CSV files are identical:
use the Excel2Collection plugin ( - Process Type Plugin - EXCEL2COLLECTIONS )
create a VIEW on the collection (makes it easier elsewhere)
create a procedure (in a Package) to bulk process it.
The most important thing is to have, somewhere in the Package (ie your code that is not part of APEX), information that clearly states which columns in the Collection map to which columns in the table, view, and the variables (APEX_APPLICATION.g_fxx()) used for Tabular Forms.
MK

EIS 6.2 Data load

We are upgrading to EIS 6.2. I've set up an outline that mirrors our production outline and can perform a member load without errors. The data load also runs without errors, the problem is no data gets loaded. I have cut and paste the SQL query the EIS generates from the log file and used it the query the Oracle database. It returns valid data. Has anyone had this data load issue with EIS 6.2. I checked the log file after the data load and I cannot find any errors.

have you tried putting that working SQL statement in the user defined sql and using it for the data load? what do the loadinfo/dataload.txt files say (if anything)

EIS to Essbase data load process

I have two user defined queries in a metaoutline. When i perform dataload operation from EIS, the Essbase application log says that dataload updated [] cells and Data Load Elapsed Time [] seconds but the EIS dataload process is still running. It seems that Essbase is loading the data only for the first query and completing dataload while EIS is still running second query.Has anyone else experienced this problem? Essbase and EIS both are 6.5.0.

In EIS
1) You have to define the Logical OLAP Model connecting to the relational source.It defines the joins between fact table and dimension tables.
2) Based on the OLAP Model You have to create meta outline which defines the rules for loading members and data into essbase.

In EIS at data loading Few Records are getting rejected

Hello,
I m loading the data in Essbase through EIS.I am using user defined SQL for data loading.
In log this this is showing that 22000 records are loaded and 564 records rejected.
Can any Expert let me know why these records are getting rejected.
Thx
Shivendra

If you look on your EIS server, under hyperion\eis there is a logfiles directory. Under it are folders for every load. In these folders are the error files. If you look at the file, the first column shows you the member that got rejected. You can then look to see if it is in your outline. My guess is your custom sql is bringing back rows that are not in the cube

All data load records rejected by EIS

I have a very strange behavior with EIS:
We used to successfully build an Essbase cube (V6.5.3) from a SQL 2000 database using EIS (V6.5.4).
A few weeks ago we doubled the size of a dimension (from about 13000 members to 27000) by adding a new branch exclusively based on shared members (no increase in the data).
Since then, most of the time (but not all the time !) the data load rejected absolutely all the records.
I can certify that the records are OK. When we try to reduce this dimension to smaller parts, each individual section can be built and loaded successfully. But when loading everything, EIS reject all the data records.
We were able to do the full load only once or twice just after having restarted EIS...
Did anybody already faced such problem ? Are you aware of a limit in the number of members EIS can handle in one dimension ?
Thanks for any tips or workaround about this issue that becomes very critical for us.
Best regards,
Stephane

i faced some similar problem AIS 9.3.0.1
once check the rul file for data load in essbase.EIS automatically creates one rul file for data loading.

Automate EIS dim builds and data loads

I want to automate the dimesion builds and data loads from my ETL tool (DTS). I have not been able to find anything about scripting EIS automation in the documentation. Is there any?

what you can do is create go into EIS metadata outline and create a member load and data load script. Do this by selecting the Outline menu item, then select member load. click next, on this screen, select only save load script. Click the button "Save scripts" to give it a name. click finish. repeat for the dataload script. (If you are using ASO cubes, you must use separate scripts, you can't do both in one script)Then create a batch file to run the member load and data loads. In DTS, use an execute process task to run the batch file

EIS data load slow

Similar Messages

Maybe you are looking for