How to reduce long extraction time - Generic extractor based on view
Hello. The previous thread about this error was closed without any answer.
I'm in the same configuration.
"Using a generic extractor (Delta enabled on confirmation date) based on view to load data from ECC6 to BI.
Now my issue is that the delta extraction is running for a long time (around 1 hr) even if the data volume is very small
While checking the job overview in source system i saw that the job is hanging at the below step for almost 1 hr."
1 LUWs confirmed and 1 LUWs to be deleted with function module MF RSC2_QOUT_CONFIRM_DATA.
I am not understanding why this is happening.
Here is the log of the job:
30.05.2011 03:58:11 Job started
30.05.2011 03:58:11 Step 001 started (program SBIE0001, variant &0000000166864, user ID ALEREMOTE)
30.05.2011 03:58:11 Asynchronous transmission of info IDoc 2 in task 0001 (0 parallel tasks)
30.05.2011 03:58:11 DATASOURCE = 0CO_OM_WBS_6
30.05.2011 03:58:11 *************************************************************************
30.05.2011 03:58:11 * Current Values for Selected Profile Parameters *
30.05.2011 03:58:11 *************************************************************************
30.05.2011 03:58:11 * abap/heap_area_nondia......... 4000000000 *
30.05.2011 03:58:11 * abap/heap_area_total.......... 8000000000 *
30.05.2011 03:58:11 * abap/heaplimit................ 100000000 *
30.05.2011 03:58:11 * zcsa/installed_languages...... DEFS *
30.05.2011 03:58:11 * zcsa/system_language.......... E *
30.05.2011 03:58:11 * ztta/max_memreq_MB............ 2047 *
30.05.2011 03:58:11 * ztta/roll_area................ 6500000 *
30.05.2011 03:58:11 * ztta/roll_extension........... 2000000000 *
30.05.2011 03:58:11 *************************************************************************
30.05.2011 03:58:11 1 LUWs confirmed and 1 LUWs to be deleted with function module RSC2_QOUT_CONFIRM_DATA
30.05.2011 05:02:53 Call customer enhancement BW_BTE_CALL_BW204010_E (BTE) with 171 records
30.05.2011 05:02:53 Result of customer enhancement: 171 records
30.05.2011 05:02:53 Call customer enhancement EXIT_SAPLRSAP_001 (CMOD) with 171 records
30.05.2011 05:02:53 Result of customer enhancement: 171 records
30.05.2011 05:02:53 Asynchronous send of data package 1 in task 0002 (1 parallel tasks)
30.05.2011 05:02:53 IDOC: Info IDoc 2, IDoc No. 4667050, Duration 00:00:00
30.05.2011 05:02:53 IDoc: Start = 30.05.2011 03:58:11, End = 30.05.2011 03:58:11
30.05.2011 05:02:53 tRFC: Data Package = 1, TID = AC11082D38B44DE308DD028A, Duration = 00:00:00, ARFCSTATE = RECORDED
30.05.2011 05:02:53 tRFC: Start = 30.05.2011 05:02:53, End = 30.05.2011 05:02:53
30.05.2011 05:02:53 Altogether, 0 records were filtered out through selection conditions
30.05.2011 05:02:53 Asynchronous transmission of info IDoc 3 in task 0003 (0 parallel tasks)
30.05.2011 05:02:53 IDOC: Info IDoc 3, IDoc No. 4667051, Duration 00:00:00
30.05.2011 05:02:53 IDoc: Start = 30.05.2011 05:02:53, End = 30.05.2011 05:02:53
30.05.2011 05:02:53 Synchronized transmission of info IDoc 4 (0 parallel tasks)
30.05.2011 05:02:53 IDOC: Info IDoc 4, IDoc No. 4667052, Duration 00:00:00
30.05.2011 05:02:53 IDoc: Start = 30.05.2011 05:02:53, End = 30.05.2011 05:02:53
30.05.2011 05:02:53 Job finished
Thanks for your help.
Yann
Parth Kulkarni,
following the note and the other thread, I've check the index 4 of the table COEP.
Here is what I got:
Index ID COEP - 4
Short text MANDT/TIMESTMP/OBJNR, Index for Delta read method (CO-PA)
Last changed SAP 04.01.2010
Status Active Saved
Does not exist in the database
DB index nme
Not defined as DB index in the ABAP Dictionary
Is the fact that it doesn't exist in the database a problem ?
Is the fact that it is active is good enough ?
By the way thanks for your speedy reply !
Yann
Edited by: Yann GOFFIN on May 30, 2011 1:35 PM
Similar Messages
-
How to reduce smartform activation time
Hi, All
i create a new smartform and just activate without filling any value in form , smartfrom activated within 1 second properly
now again when i activate same smartform it take nearly 20-30 minutes for activation .
Does anyone know how to reduce this activation time of smartformHello,
strange problem.
I think you have contact your basis collegues and maybe is helpful to look this transaction : STATTRACE
Regards
roberto -
How to reduce dso activation time
hi
can any body explain me how to reduce dso activation time.I am having a dso with 5 crore records and it is taking 7 to 8 hrs for activation.Hi,
Try this.
T.code RSCUSTA2 . Here you can change the no.of records and no.of processes to use for activation job with this you can increase your ODS activation process, here the no.of processes can be increased based on no.of processes available in your server, check in SM50 or SM51.
ODS Query Performance
Thanks,
JituK -
Create generic extractor based on 2 tables
hey BI gurus,
I have to create a generic extractor based on these 3 tables:
eban - i need all fields
vttk - i need only 4 fields i.e
route,
tknum,
datbg,
daten
ekbe - budat.
now the problem is thr is no common field between vttk & eban AND vttk & ekbe. but thr is a common field between eban & ekbe.so i can create a view only on these 2 tables but i want 3 tables
how do i proceed???Hi,
Sometimes, if your application or requirement makes it necessary, you have to create your own function module to extract the data from whatever sap system to bw.
Therefore you normally make a copy of the standard fm 'RSAX_BIW_GET_DATA_SIMPLE'. To make this new function work for you, you have to realize the following.
1.) the fm gets called by the extraction api for at least 2 times. First time is for initialization and from the 2nd time on it is for data extraction.
2.) right after the last data package (somehow you need to make sure that it is the last one) you have to raise the exception no_more_data.
Refer this links for more Info
SAP BI Generic Extraction Using a Function Module
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33
SAP Network Blog: Generic Extraction via Function Module
/people/siegfried.szameitat/blog/2005/09/29/generic-extraction-via-function-module
Take the help of an ABAPer if your are new to this process
Regards
Karthik
Assigning points is the way to say thanks -
How to reduce the fetch time of this sql?
Here is the SQL, three-table join and joining conditions are:
ms_ets_cntrl.ims_cntrt_oid=ims_alctn.ims_alctn_oid
ims_alctn.ims_trde_oid=ims_trde.ims_trde_oid
SELECT 'MCH' Type, ims_ets_cntrl.STTS tp_stts, count(*) Count FROM ims_ets_cntrl where ims_ets_cntrl.ims_cntrt_oid in
(select ims_alctn.ims_alctn_oid FROM ims_alctn, ( select ims_trde.ims_trde_oid from ims_trde
where ( IMS_TRDE.IMS_TRDE_RCPT_DTTM >= TO_DATE('10/29/2009 00:00', 'MM/DD/YYYY HH24:MI') AND IMS_TRDE.IMS_TRDE_RCPT_DTTM <= (TO_DATE('11/5/2009 23:59', 'MM/DD/YYYY HH24:MI')) )
AND (IMS_TRDE.GRS_TRX_TYPE IN ('INJECTION','WITHDRAWAL','PAYMENT') OR IMS_TRDE.SSC_INVST_TYPE = 'FC' AND 1=1 and IMS_TRDE.SERVICE_TYPE='FS' )) TRDE
where IMS_ALCTN.IMS_TRDE_OID=TRDE.IMS_TRDE_OID) and ims_ets_cntrl.outbnd_dest = 'ETD' group by ims_ets_cntrl.STTSOptimizer and related parameter info:
SQL> show parameter optimizer
NAME TYPE VALUE
optimizer_dynamic_sampling integer 1
optimizer_features_enable string 9.2.0
optimizer_index_caching integer 0
optimizer_index_cost_adj integer 100
optimizer_max_permutations integer 2000
optimizer_mode string CHOOSE
SQL>select pname, pval1, pval2 from sys.aux_stats$ where sname='SYSSTATS_INFO';
DSTART 11-16-2009 10:23
DSTOP 11-16-2009 10:23
FLAGS 1
STATUS NOWORKLOADHere is autotrace output:
Execution Plan
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 SORT (GROUP BY)
2 1 VIEW
3 2 SORT (UNIQUE)
4 3 TABLE ACCESS (BY INDEX ROWID) OF 'IMS_ETS_CNTRL'
5 4 NESTED LOOPS
6 5 NESTED LOOPS
7 6 TABLE ACCESS (BY INDEX ROWID) OF 'IMS_TRDE'
8 7 INDEX (RANGE SCAN) OF 'IMS_TRDE_INDX4' (NON- UNIQUE)
9 6 TABLE ACCESS (BY INDEX ROWID) OF 'IMS_ALCTN'
10 9 INDEX (RANGE SCAN) OF 'IMS_ALCTN_INDX1' (NON -UNIQUE)
11 5 INDEX (RANGE SCAN) OF 'IMS_ETS_CNTRL_INDX1' (NON -UNIQUE)
Statistics
0 recursive calls
0 db block gets
244608 consistent gets
58856 physical reads
0 redo size
497 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1 rows processedHere is TKPROF output:
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 4.85 129.72 53863 244608 0 1
total 4 4.85 129.72 53863 244608 0 1
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 63
Rows Row Source Operation
1 SORT GROUP BY
12972 VIEW
12972 SORT UNIQUE
12972 TABLE ACCESS BY INDEX ROWID IMS_ETS_CNTRL
46236 NESTED LOOPS
19134 NESTED LOOPS
19744 TABLE ACCESS BY INDEX ROWID IMS_TRDE
176922 INDEX RANGE SCAN IMS_TRDE_INDX4 (object id 34099)
19134 TABLE ACCESS BY INDEX ROWID IMS_ALCTN
19134 INDEX RANGE SCAN IMS_ALCTN_INDX1 (object id 34094)
27101 INDEX RANGE SCAN IMS_ETS_CNTRL_INDX1 (object id 34101)
********************************************************************************Explain plan output:
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows | Bytes | Cost |
| 0 | SELECT STATEMENT | | | | |
| 1 | SORT GROUP BY | | | | |
| 2 | VIEW | | | | |
| 3 | SORT UNIQUE | | | | |
|* 4 | TABLE ACCESS BY INDEX ROWID | IMS_ETS_CNTRL | | | |
| 5 | NESTED LOOPS | | | | |
| 6 | NESTED LOOPS | | | | |
|* 7 | TABLE ACCESS BY INDEX ROWID| IMS_TRDE | | | |
|* 8 | INDEX RANGE SCAN | IMS_TRDE_INDX4 | | | |
| 9 | TABLE ACCESS BY INDEX ROWID| IMS_ALCTN | | | |
|* 10 | INDEX RANGE SCAN | IMS_ALCTN_INDX1 | | | |
|* 11 | INDEX RANGE SCAN | IMS_ETS_CNTRL_INDX1 | | | |
Predicate Information (identified by operation id):
4 - filter("IMS_ETS_CNTRL"."OUTBND_DEST"='ETD')
7 - filter("IMS_TRDE"."GRS_TRX_TYPE"='INJECTION' OR "IMS_TRDE"."GRS_TRX_TYPE"='WITHD
RAWAL' OR "IMS_TRDE"."GRS_TRX_TYPE"='PAYMENT' OR "IMS_TRDE"."SSC_INVST_TY
PE"='FC' AND "IMS_TRDE"."SERVICE_TYPE"='FS')
8 - access("IMS_TRDE"."IMS_TRDE_RCPT_DTTM">=TO_DATE('2009-10-29 00:00:00', 'yyyy-mm-
dd hh24:mi:ss') AND "IMS_TRDE"."IMS_TRDE_RCPT_DTTM"<=TO_DATE('2009-11-05
23:59:00', 'yyyy-mm-dd hh24:mi:ss')
10 - access("IMS_ALCTN"."IMS_TRDE_OID"="IMS_TRDE"."IMS_TRDE_OID")
11 - access("IMS_ETS_CNTRL"."IMS_CNTRT_OID"="IMS_ALCTN"."IMS_ALCTN_OID")
Note: rule based optimizationCould you please help tune this sql?
How can I reduce the elapsed time? How can I reduce query read?
If there is any other info that you need, please let me know!
thank you very much!What exactly is this logic meant to do?
AND (ims_trde.grs_trx_type IN ('INJECTION', 'WITHDRAWAL', 'PAYMENT')
OR ims_trde.ssc_invst_type = 'FC'
AND ims_trde.service_type = 'FS')is that really:
AND (ims_trde.grs_trx_type IN ('INJECTION', 'WITHDRAWAL', 'PAYMENT')
OR ims_trde.ssc_invst_type = 'FC')
AND ims_trde.service_type = 'FS'or is it maybe:
AND (ims_trde.grs_trx_type IN ('INJECTION', 'WITHDRAWAL', 'PAYMENT')
OR (ims_trde.ssc_invst_type = 'FC'
AND ims_trde.service_type = 'FS'))? -
How to Generate Information Errors in Generic Extractor Function Module
Hi, In my Generic Extractor logic I am reaching out to an additional table to get a field. I would like to produce an informational error or red error in the process monitor if I'm unable to get the field. Is this possible? How can I generate these messages in the Extractor Function Module?
Thanks!Hi,
Please go through the below doc. It explanins each and every step of creation of generic extractor through Function module.
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33?quicklink=index&overridelayout=true
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c062a3a8-f44c-2c10-ccb8-9b88fbdcb008?quicklink=index&overridelayout=true
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/30aefb04-7043-2c10-8e92-941536eebc79?quicklink=index&overridelayout=true
Regards,
mahesh -
How to reduce the cloning time if using cold backup?
Hi,
We are using EBS r12 (12.0.6) with database 10.2.0.3 in Linux Redhat 4 32-bit envoirnment.
Our datbase size is around 480 GB and we are facing the issues to provide the clone to my consultants for meet the target timelines in given time.
(Source:)---- Production Dell R900 Server machine having 32-GB RAM and 8 Quard Core CPU's.
(Target:)---- Clone Dell 2950 Server machine having 16-GB RAM and 4-Quard Core CPU's.
Currectly we are taking cold backup like:
1: Auto shutdown EBS r12 database nighthy at 12:00 AM daily and compress the backup using tar utility in linux. This process takes 6:00 hours aprox:
2: After that we move compressed file to clone machine and then uncompressed it, and this process takes 5:00 hours aprox.
3: And performe standard cloning steps.
Questions:
1: How to reduce time of this backup process?
3: Is there any other way to reduce the cloning process.
2: What type of backup oracle recommended to their customers for this type of process?
Thanks.1: How to reduce time of this backup process?Without using third party tools, it might be hard to tune the timing of compressing/uncompressing the file.
Have you tried to use scp command? This would help if your network throughput is acceptable.
3: Is there any other way to reduce the cloning process.Since the main issue you have with the copy, then you might copy the files remotely from the source to the target, or using any other storage/backup tools (like file system snapshot).
2: What type of backup oracle recommended to their customers for this type of process?Oracle does not recommend any type of backup as the tools used are not Oracle products.
Thanks,
Hussein -
How to reduce Oracle Cluster Time Synchronization time to 0
Hi Guys,
How to reduce Offset (in msec) of Oracle Cluster Time Synchronization time to 0 ???
is there any command or is it the only solution to wait to become 0 slowly...
[root@caslive bin]# ./crsctl check ctss
CRS-4701: The Cluster Time Synchronization Service is in Active mode.
CRS-4702: Offset (in msec): -1300
regards,
manish
Email ID: [email protected]Hi,
1.From DB02--> detail analysis menu you can take out top 50 tables & indexes. you can mark the size of the index should be generally less than the table.If its more or very much similar size of table, you can rebuild it using SE14. This can free some space.
or else you may use brspace to do this.
http://help.sap.com/saphelp_nw70/helpdata/EN/58/6bec38c9fa7e44b7f2163905863575/frameset.htm
In case of table this option is risky as it may result in data loss.
2. Earlywatch alert gives the top 20 degenrated index. you can check that which also gives a factor 'storage quality'.
3. Run report SAP_DROP_TMPTABLES. It removes temporary database objects. ( we do this in our BW system)
Hope this helps
Thanks
Sushil -
How to do Delta upload using Generic Extractor built on Function Module?
Hello Guys,
I have never created a Generic Extractor using a Function Module. I wanted to know how can we support Delta mechanism if we create Generic extractor using Function Module.
Regards,
AbhishekHi Abishek,
Please check the standard Function Module RSAX_BIW_GET_DATA for your delta mechnism.
Hope it will help you.
Thanks,
Chandra -
How to reduce Post-Processing time in oracle applications R12?
Hi All,
I ran a XMLP concurrent program at 16-APR-2015 11:23:52 . It's completed at 16-APR-2015 11:30:41. So it took 6 minute 49 seconds. I check the query in sql developer, it's executing less than two minutes. When I check the log file of that program shows the following info...
Beginning post-processing of request 22226009 on node DEV at 16-APR-2015 11:24:43.
Post-processing of request 22226009 completed at 16-APR-2015 11:30:41.
So what is the reason?, and how to reduce the time?HTML is a simple format (Text).
Excel is more complicated and requires more time to generate. The same applies for Word and PDF output.
Maybe the processing power and memory of the server are not high and this does not help generate the output faster.
I have a workaround for you, generate an XML file (which should finish faster) and open it in Excel. -
Incredibly Long Extraction Times
I have been trying to insert a series of PowerPoint slides, converted to jpegs into a taped lecture using a new MacBook Pro 15" with the latest Leopard OS and dual processor...nothing else running but IMovieHD from ILife 08
When I paste the image at the playhead, it is set to extract the lecturer's audio, but I am experiencing really long render times for the extraction. Clips of virtually any length take about 8 minutes to extract and if the image straddles two different clips, I have to sit through two renders which can take 15-20 minutes. Even if the first portion of the straddle covers only a few frames of the first clip, it still takes at least 8 minutes to render that portion and then it does a second render...again of around 8 minutes.
For a project of any length with lots of inserts like this, having to wait this length of time for every render is agonizing. I can't believe it actually should be taking so long to complete this task. Am I doing something wrong?Yes, If what you want is a slide show, then the easiest way is to put all the slides, in order, in an iPhoto album.
Then drag the whole iPhoto album into a new project. If you want to set a default KBE/crop, transition type and length, etc. It is easiest to do this in project preferences before you drag in the album. You can add music to the slideshow and do further edits.
The comment about dragging a jpeg to a video clip applies if you are trying to keep the main "voice track" from the video playing while superimposing jpegs over the video footage.
Also, if you need no video footage, and the video is only being used as a voice track, you would probably be better off creating the slide show, and then exporting it to the media browser. Then use Garageband to record the voice track.
Message was edited by: AppleMan1958
Message was edited by: AppleMan1958 -
How to reduced instance recovery time
Hello Friends how are you all. I hope you all will be fine. friends I want to reduced the instance recovery time but i don't know which views or parameters which could helpful for me. Please tell me the way how can i reduced the instance recovery time.
Thanks & Best WishesHi,
Reduced instance recovery time
check ur DB size, is it connected many users.
my advice is do not set the reduced instance recovery time, it may be decreased run-time peformance.
the following procedure
-) Frequent checkpoints
-) reduce instance recovery time
-) set FAST_START_MTTR_TARGET parameter
-) size the Online redo log files
-) Implent manual checkpoints
-) reduce log_buffer size
-) Decrease run-time performance -
How to check at what time the extractor has picked records from r3 tables
Hi experts ,
If we want to know exactly at what time the Extractor has picked up the records from r/3 tables .
or if we want to know the time stamp of extractor picking the records from r3 tables after r3 entries
Regards ,
Subash BalakrishnanHi,
The following are few function modules which will give you the information you need based upon the area you are working in.
SD Billing: LOG_CONTENT_BILLING
Delivery: LOG_CONTENT_DELIVERY
Purchasing: LOG_CONTENT_PURCHASING etc...
See if the above FMs help you in any way... -
How to reduce iCal event time span?
How do I set default time span of an event to 1 hour in iCal? I'm using Version 5.0.2 (1571) and the current default is set to something like 6 or 8 hours and it's driving me nuts! A related aggravation is that an event can unintentionally end up spanning 2 days! It used to work just fine in previous versions -- why did they "fix" this and how do I make the current version be truly useful like the old version?
Hi,
I guess you are adding events in Month view. From iCal's help file about entering events in Month view.
Add a new event in Month view by double-clicking within the day you want the event to appear. Enter a name and time duration for the event in the event’s title field, and then press Return.
iCal moves the event to the correct time and sets it to the correct duration. For example, you can double-click within a day in Month view, enter “Movie 7-9pm,” and then press Return.
If you don’t enter a time duration, iCal sets the event’s duration to 1 hour.
If you don’t enter any time information, iCal makes the event an all-day event.
Add a start time in the event title to make a timed 1 hour event. The time part of the title is stripped out.
Best wishes
John M -
How to reduce project load time?
I am currently working on a large project (25+ Mb, thousands of vis) with something on the order of 100 classes. Opening the project is painfully slow. It usually takes 7-8 minutes. I'd like to figure out a way to reduce the load time.
What is LV's behavior regarding loading project file vis and libraries (lvlib, lvclass, etc.)?
Are all items in the project automatically loaded when the project is opened?
Are items not in the project but used as sub-vis automatically loaded?
Are dynamically linked items not included in the project automatically opened? (I imagine not.)
Are dynamically linked items that are included in the project automatically opened?
Thanks,
DaveHello,
I have talked to multiple people about your issue and we honestly believe that you you are experiencing is expected behavior based on the size of the project and number of VIs used. Creating a new project will sometimes help when a project file has a corruption within it. Since you are experiencing the same behavior on multiple computers, any difference in load time will be a function of the speed, RAM, etc. of that particular computer as compared to your machine. In saying that the files are technically loaded, I am saying that their locations are known so that when they are called, they can be loaded into memory as needed. It does not have to load any portion of the VI other than the location on disk. I would not expect either of those methods to reduce your project load time. Switching to dynamic linking only changes the order in which LabVIEW searches for the files. As I stated before, files added to the project are defined by the name and location on the disk. You can think of the project file as an XML file with different pieces of information about the files. If the file cannot be found, it will try to dynamically link the files by searching for them on the disk. When this happens, it goes through a seris of locations where the files might be located. When the project is loaded, it looks for the files that are listed in the project file first. I hope this helps.
-Zach
Certified LabVIEW Developer
Maybe you are looking for
-
Smart VIew 11.1.2.1 - Error Message "Invalid Parameter" in a pop up box
Message leaves a bit to be desired.
-
When I disconnect from my PC, my nano shuts off and won't come back on. I have tried holding down both the On/Off and the Home buttons -- but nothing...
-
Hello, thank you in advance for taking the time. I have 2 Probook 4530s, one is fully functional, with cosmetic damage (and an nonworking optical drive). The other is missing HD and ram, with a dysfunctional MB, but pristine as far as aesthetics. My
-
Unable to launch adobe story app, because adobe air installer fails
Hi, I am having problems launching the adobe story app. Everytime I try to open it it tries to update and it then launches the adobe air installer which always fails and I get the following message: Sorry, an error has occurred. The application could
-
Can I Replace My OS Drive With A SSD?
I currently have 4 drives in my Mac Pro Tower and would like to replace them all (including the OS drive) with SSD's. Can I? TIA