Which Parallel processing is faster?

There are various ways in which parallel processing can be implemented e.g.
1. Call function STARTING NEW TASK which uses dialog work process.
2. Using background RFC (trfc) .. call function IN BACKGROUND TASK AS SEPARATE UNIT
3. Using submit via jobs
I want to know which technique is fastest and why?

soadyp wrote:
The throughput of the various workprocess is NOT identical.
> I was a little surprised to discover this. Is seems the DISP+WORK behaves differently under different workprocess types.
> Clearly a background process does need/ doesnt support dialog processing and the CLASSIC PBO/PAI process.
>
> All I can say, is TEST it on your kernel.
>
> We have done testing on this since every millisecond is import to us.
>
> Dialog processes are sometimes twice as slow as baclground tasks depending on what is happening.
> Im talking abap the ABAP execution time here.Not DB, RFC, external call , PURE ABAP execution times.
>
> DIALOG was simply slower than Background processes.
>
> TRY it: Build a report. SUBMIT report in background.
> Run the same abap in dialog.
> Include GET Runtime statements to measure the execution time microseconds.
> fill it with PERFORM X using Y , CALL FUNCTION B or CALL CLASS->Method_M
> set the clock resolution high, and measure the total execution time.
>
> When running HUGE interfaces, processing 10 of millions of steps every day, this is of genuine consideration.
>
> ALSO NOTE:
> The cost of open JOB submit via job close should also be measured.
> If your packets are too small, then background speed is lost to the overhead of submission.
>
> The new Background RFC functions should also be tested with the SAME CODE.
>
> happy testing
> Phil
Dialog might be slower only due to the GUI communication or difference in ABAP source code. In some standard SAP applications there is a different processing depending on SY-BATCH flag.
Technically, as it was already mentioned several time above, both work processes (dialog and batch) are pretty identical (with slight differences in memory allocation).
So please don't confuse the community.

Similar Messages

Parallel processing in background

Hi All,
I am processing 1 million of records in background, which takes approximately around 10 hrs. I wanted to reduce the time to less than 1 hr and tried using parallel processing. But the tasks run in Dialog workprocesses and giving abap short dumps due to time out.
Is there any other solutions using that i can reduce total processing time.
Please note that i cannot split. I am getting 1 million records from a select query and after processing all those records in SAP, I am sending to XI and XI will post in legacy system.
Please note that all other performance tunings done.
Thanks,
Rajesh.

Hi Rajesh,
Refer sample code for Parallel Processing:
By doing this your processing time will be highly optimized.
Go thru the description given in the code at each level.
This code Checks available WORK PROCESSes and assigns data in packets for processing. This way you save a lot of time esp when data is in Millions.
Hope it helps.
REPORT PARAJOB.
Data declarations
DATA: GROUP LIKE RZLLITAB-CLASSNAME VALUE ' ',
"Parallel processing group.
"SPACE = group default (all
"servers)
WP_AVAILABLE TYPE I, "Number of dialog work processes
"available for parallel processing
"(free work processes)
WP_TOTAL TYPE I, "Total number of dialog work
"processes in the group
MSG(80) VALUE SPACE, "Container for error message in
"case of remote RFC exception.
INFO LIKE RFCSI, C, "Message text
JOBS TYPE I VALUE 10, "Number of parallel jobs
SND_JOBS TYPE I VALUE 1, "Work packets sent for processing
RCV_JOBS TYPE I VALUE 1, "Work packet replies received
EXCP_FLAG(1) TYPE C, "Number of RESOURCE_FAILUREs
TASKNAME(4) TYPE N VALUE '0001', "Task name (name of
"parallel processing work unit)
BEGIN OF TASKLIST OCCURS 10, "Task administration
TASKNAME(4) TYPE C,
RFCDEST LIKE RFCSI-RFCDEST,
RFCHOST LIKE RFCSI-RFCHOST,
END OF TASKLIST.
Optional call to SBPT_INITIALIZE to check the
group in which parallel processing is to take place.
Could be used to optimize sizing of work packets
work / WP_AVAILABLE).
CALL FUNCTION 'SPBT_INITIALIZE'
EXPORTING
GROUP_NAME = GROUP
"Name of group to check
IMPORTING
MAX_PBT_WPS = WP_TOTAL
"Total number of dialog work
"processes available in group
"for parallel processing
FREE_PBT_WPS = WP_AVAILABLE
"Number of work processes
"available in group for
"parallel processing at this
"moment
EXCEPTIONS
INVALID_GROUP_NAME = 1
"Incorrect group name; RFC
"group not defined. See
"transaction RZ12
INTERNAL_ERROR = 2
"R/3 System error; see the
"system log (transaction
"SM21) for diagnostic info
PBT_ENV_ALREADY_INITIALIZED = 3
"Function module may be
"called only once; is called
"automatically by R/3 if you
"do not call before starting
"parallel processing
CURRENTLY_NO_RESOURCES_AVAIL = 4
"No dialog work processes
"in the group are available;
"they are busy or server load
"is too high
NO_PBT_RESOURCES_FOUND = 5
"No servers in the group
"met the criteria of >
"two work processes
"defined.
CANT_INIT_DIFFERENT_PBT_GROUPS = 6
"You have already initialized
"one group and have now tried
"initialize a different group.
OTHERS = 7..
CASE SY-SUBRC.
WHEN 0.
"Everythings ok. Optionally set up for optimizing size of
"work packets.
WHEN 1.
"Non-existent group name. Stop report.
MESSAGE E836. "Group not defined.
WHEN 2.
"System error. Stop and check system log for error
"analysis.
WHEN 3.
"Programming error. Stop and correct program.
MESSAGE E833. "PBT environment was already initialized.
WHEN 4.
"No resources: this may be a temporary problem. You
"may wish to pause briefly and repeat the call. Otherwise
"check your RFC group administration: Group defined
"in accordance with your requirements?
MESSAGE E837. "All servers currently busy.
WHEN 5.
"Check your servers, network, operation modes.
WHEN 6.
Do parallel processing. Use CALL FUNCTION STARTING NEW TASK
DESTINATION IN GROUP to call the function module that does the
work. Make a call for each record that is to be processed, or
divide the records into work packets. In each case, provide the
set of records as an internal table in the CALL FUNCTION
keyword (EXPORT, TABLES arguments).
DO.
CALL FUNCTION 'RFC_SYSTEM_INFO' "Function module to perform
"in parallel
STARTING NEW TASK TASKNAME "Name for identifying this
"RFC call
DESTINATION IN GROUP group "Name of group of servers to
"use for parallel processing.
"Enter group name exactly
"as it appears in transaction
"RZ12 (all caps). You may
"use only one group name in a
"particular ABAP program.
PERFORMING RETURN_INFO ON END OF TASK
"This form is called when the
"RFC call completes. It can
"collect IMPORT and TABLES
"parameters from the called
"function with RECEIVE.
EXCEPTIONS
COMMUNICATION_FAILURE = 1 MESSAGE msg
"Destination server not
"reached or communication
"interrupted. MESSAGE msg
"captures any message
"returned with this
"exception (E or A messages
"from the called FM, for
"example. After exception
"1 or 2, instead of aborting
"your program, you could use
"SPBT_GET_PP_DESTINATION and
"SPBT_DO_NOT_USE_SERVER to
"exclude this server from
"further parallel processing.
"You could then re-try this
"call using a different
"server.
SYSTEM_FAILURE = 2 MESSAGE msg
"Program or other internal
"R/3 error. MESSAGE msg
"captures any message
"returned with this
"exception.
RESOURCE_FAILURE = 3. "No work processes are
"currently available. Your
"program MUST handle this
"exception.
YOUR_EXCEPTIONS = X. "Add exceptions generated by
"the called function module
"here. Exceptions are
"returned to you and you can
"respond to them here.
CASE SY-SUBRC.
WHEN 0.
"Administration of asynchronous RFC tasks
"Save name of task...
TASKLIST-TASKNAME = TASKNAME.
"... and get server that is performing RFC call.
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
EXPORTING
RFCDEST = TASKLIST-RFCDEST
EXCEPTIONS
OTHERS = 1.
APPEND TASKLIST.
WRITE: / 'Started task: ', TASKLIST-TASKNAME COLOR 2.
TASKNAME = TASKNAME + 1.
SND_JOBS = SND_JOBS + 1.
"Mechanism for determining when to leave the loop. Here, a
"simple counter of the number of parallel processing tasks.
"In production use, you would end the loop when you have
"finished dispatching the data that is to be processed.
JOBS = JOBS - 1. "Number of existing jobs
IF JOBS = 0.
EXIT. "Job processing finished
ENDIF.
WHEN 1 OR 2.
"Handle communication and system failure. Your program must
"catch these exceptions and arrange for a recoverable
"termination of the background processing job.
"Recommendation: Log the data that has been processed when
"an RFC task is started and when it returns, so that the
"job can be restarted with unprocessed data.
WRITE msg.
"Remove server from further consideration for
"parallel processing tasks in this program.
"Get name of server just called...
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
EXPORTING
RFCDEST = TASKLIST-RFCDEST
EXCEPTIONS
OTHERS = 1.
"Then remove from list of available servers.
CALL FUNCTION 'SPBT_DO_NOT_USE_SERVER'
IMPORTING
SERVERNAME = TASKLIST-RFCDEST
EXCEPTIONS
INVALID_SERVER_NAME = 1
NO_MORE_RESOURCES_LEFT = 2
"No servers left in group.
PBT_ENV_NOT_INITIALIZED_YET = 3
OTHERS = 4.
WHEN 3.
"No resources (dialog work processes) available at
"present. You need to handle this exception, waiting
"and repeating the CALL FUNCTION until processing
"can continue or it is apparent that there is a
"problem that prevents continuation.
MESSAGE I837. "All servers currently busy.
"Wait for replies to asynchronous RFC calls. Each
"reply should make a dialog work process available again.
IF EXCP_FLAG = SPACE.
EXCP_FLAG = 'X'.
"First attempt at RESOURCE_FAILURE handling. Wait
"until all RFC calls have returned or up to 1 second.
"Then repeat CALL FUNCTION.
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '1' SECONDS.
ELSE.
"Second attempt at RESOURCE_FAILURE handling
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '5' SECONDS.
"SY-SUBRC 0 from WAIT shows that replies have returned.
"The resource problem was therefore probably temporary
"and due to the workload. A non-zero RC suggests that
"no RFC calls have been completed, and there may be
"problems.
IF SY-SUBRC = 0.
CLEAR EXCP_FLAG.
ELSE. "No replies
"Endless loop handling
ENDIF.
ENDIF.
ENDCASE.
ENDDO.
Wait for end of job: replies from all RFC tasks.
Receive remaining asynchronous replies
WAIT UNTIL RCV_JOBS >= SND_JOBS.
LOOP AT TASKLIST.
WRITE:/ 'Received task:', TASKLIST-TASKNAME COLOR 1,
30 'Destination: ', TASKLIST-RFCDEST COLOR 1.
ENDLOOP.
This routine is triggered when an RFC call completes and
returns. The routine uses RECEIVE to collect IMPORT and TABLE
data from the RFC function module.
Note that the WRITE keyword is not supported in asynchronous
RFC. If you need to generate a list, then your RFC function
module should return the list data in an internal table. You
can then collect this data and output the list at the conclusion
of processing.
FORM RETURN_INFO USING TASKNAME.
DATA: INFO_RFCDEST LIKE TASKLIST-RFCDEST.
RECEIVE RESULTS FROM FUNCTION 'RFC_SYSTEM_INFO'
IMPORTING RFCSI_EXPORT = INFO
EXCEPTIONS
COMMUNICATION_FAILURE = 1
SYSTEM_FAILURE = 2.
RCV_JOBS = RCV_JOBS + 1. "Receiving data
IF SY-SUBRC NE 0.
Handle communication and system failure
ELSE.
READ TABLE TASKLIST WITH KEY TASKNAME = TASKNAME.
IF SY-SUBRC = 0. "Register data
TASKLIST-RFCHOST = INFO_RFCHOST.
MODIFY TASKLIST INDEX SY-TABIX.
ENDIF.
ENDIF.
ENDFORM
Reward points if that helps.
Manish
Message was edited by:
Manish Kumar

Process runs faster when executed independently, but runs slower when a parallel big process is running

Hi,
The issue is, a process, say "A" is running fine on certain days. On some days there are additional processes running on the server. When these additional processes are running, "Process A" suffers performance issue.
Interesting point is that, these additional processes were running since long time. But, earlier "Process A" was running fine even when these additional process were running. Suddenly from past 2 or 3 weeks there is performance issue in "Process
A", when these additional process run.
Note: Nothing has been modified for the process A.
Process A is an SQL job which has SSIS Package and stored proc in different steps
When multiple parallel processes are running, SSIS Package step suffers around 40% increase in execution time, where as stored procs have only 15-20% increase in execution time
When Process A is executed while no other big processes are running, it's execution time is fine. From past few days, issue is only when, some other big parallel processes are running.
Currently below is my analysis:
Since Process A is running fine when it is executed independently, I assume there is no issue in this process.
Since issue occurs when some other big process is running, when Process A is running, I believe it is DiskIO issue. Will the issue be resolved if RAM size is increased?
Is there any way to check if RAM is being fully utilized by the server
Is there any other possibility why there is a sudden dip in performance when parallel processes are running.
Is there any possibility of having issues in additional processes. Incase if it has some issues, does it impact "Process A"
Please let me know if you need any further information. Infact I am not able to diagnose what is the actual root cause for the performance issue in "Process A" as nothing has been modified.
Also it would be very much helpful, if I get any idea on different ways in reaching the actual root cause of this performance issue.
NOTE: This is a data warehouse
Thanks,
Raksha

When a query has a parallel plan, it will in general try to grab all cores up to the maxdop setting, but then it often uses them inefficiently.
But in that case, there are better odds for the queries not battling on resources with each other!
What Josh alludes is to the fact that SQL Server needs to partition the data, so that different partitions are processed on different cores. This partitioning is based on statistics, and the statistics may be out of date or not accurate enough. This may
result in that the data is not partitioned proportionally, and some threads gets very little data to work with. Thus, these CPUs are still idle, so there may still be room for the two processes to run at full speed. (That is, full speed with the given plan,
which is not the full speed, had the partitioning been accurate.)
I mention this because you asked about parallelism, and many systems leave maxdop at its default setting of 0 (which means "go ahead and grab everything!") even though Microsoft recommends you set it to a different number depending on this and
that.
Since this is a data warehouse, Raksha should not tamper with "max degrees of parallelism", I think.
Erland Sommarskog, SQL Server MVP, [email protected]

How to define "leading" random number in Infoset fpr parallel processing

Hello,
in Bankanalyzer we use an Infoset which consists of a selection across 4 ODS tables to gather data.
No matter which PACKNO fields we check or uncheck in the infoset definition screen (TA RSISET), the parallel frameworks always selects the same PACKNO field from one ODS table.
Unfortunately, the table that is selected by the framework is not suitable, because our
"leading" ODS table which holds most of our selection criteria is another one.
How to "convince" the parallel framework to select our leading table for the specification
of the PACKNO in addition (this would be times 20 faster due to better select options).
We even tried to assign "alternate characteristics" to the packnos we do not liek to use,
but it seems that note 999101 just fixes this for non-system-fields.
But for the random number a diffrent form routine is used in /BA1/LF3_OBJ_INDEX_READF01
fill_range_random instead of fill_range.
Has anyone managed to assign the PACKNO of his choice to the infoset selection?
How?
Thanks in advance
Volker

Well, it is a bit more complicated
ODS one, that the parallel framework selects for being the one to deliver the PACKNO
is about equal in size (~120GB each) to ODS two which has two significant field which cuts down the
amount of data to be retreived.
Currently we execute the generated SQL in the best possible manner (by faking some stats )
The problem is, that I'd like to have a Statement that has the PACKNO in the very same table.
PACKNO is a generated random number esp. to be used for parallel processing.
The job starts about 100 slaves
Each slave gets a packet to be processed from the framework, which is internaly represented
by a BETWEEN clause on this PACKNO. This is joined against ODS2 and then the selective fields
can be compared resultin in 90% of the already fetched rowes can be discarded.
Basicly it goes like
select ...
from
ods1 T_00,
ods2 T_01,
ods3 T_02,
ods4 T_03
where
... some key equivalence join-conditions ...
AND T_00.PACKNO BETWEEN '000000' and '000050' -- very selective on T_00
AND T_01.TYPE = '202' -- selective Value 10% on second table
I'd trying to change this to
AND T_01.PACKNO BETWEEN '000000' and '000050'
AND T_01.TYPE = '202' -- selective Value 10%
so I can use a combined Index on T_01 (TYPE;PACKNO)
This would be times 10 more selective on the driving table and due to the fact,
that T_00 would be joined for just the rows I need, about a calculated time 20-30 faster.
It really boosts when I do this in sqlplus
Hope this clearyfies a bit.
Problem is, that I can not change the code either for doing the
build of the packets or the one that executes the application.
I need to change the Inofset, so that the framework decides to build
proper SQL with T_01.PACKNO instead of T_00.PACKNO.
Thanks a lot
Volker

Parallel Processing - In conjuction with TWS scheduler.

We have a .Bat file that uses upshell.exe to execute a custom script we've created.
This custom script looks for files in the inbox and moves them to the relevant OpenBatches/OpenBatchesML folder. Once files have been moved it then runs a Parallel process up-to-load for all data. However, what we're finding is that because we're running this in parallel the .bat script is completing despite FDM still processing in the background. This is evident, as the TBATCH table shows the batch as not 100% complete. This unfortunately causes ourselves a problem as the scheduler (TWS) thinks all processing has finished and subsequent downstream processing kicks off. As I understand it this wouldn't be an issue if we'd used Serial processing and not Parallel.
We've chosen Parallel becuase of the volumes and the limitation of our batch window. Am I correct in assuming that in principle parallel will process data faster than serial? I am currently looking at specific performance tests on my data at present to prove this.
The real issue is that we somehow need to ensure TWS doesn't kick off downstream processes and I need to somehow have FDM create a log file for batches that complete successfully. I'm assuming it should be done within an Event script but I'm not sure which as this is quite new to me.
Has anyone come across this issue themselves? If so, I'm looking for some guidance/examples how you've managed to get round the problem.
Thanks in advance.

Hi,
Take at look at the following 2 articles. Using the concepts outlined in them you should be able to achieve what you are trying to do.
http://odiexperts.com/interface-parallel-execution-a-new-solution
http://odiexperts.com/processing-multiple-interface-through-single-package

Parallel processing in OSB

Hi everyone,
I need to implement parallel processing in the OSB. I have a proxy service which implements a request reply pattern over JMS. This service needs to invoke three such services (request-reply over JMS) in parallel and aggregate the result and send it as a reply. How can I do this in OSB?
I have looked at the Split-Join element but it seems to only support web services. Is that correct?
Best regards,
Dimo

>
Thank you for the fast reply. Wrapping in web services has probably relatively high performance impact - I need to wrap my payload in SOAP, send it via HTTP, receive it over HTTP, parse the SOAP envelope, extract the payload and then send it via JMS. And the same steps in the reverse order to send the response. Seems like a lot of overhead to me.
>
You don't have to do all of that. If you wrap your JMS-based services using WSDL-based service with local transport, then you avoid all HTTP communication because processing stays inside OSB.
>
Isn't there any other way to parallelize the execution - the application is processing synchronous requests from a voice frontend and it should be very responsive. Does anyone know how much overhead exactly goes into the whole SOAP wrapping and unwrapping?
>
I don't know of any other way for this case. Maybe someone else will bring something better.
I have never measured SOAP overhead since for my business services it is only insignificant fraction of whole processing time. I would expect SOAP overhead to be not far from JMS overhead. Especially when XML is used as a payload.

Can anyone guide me to ideas on basics of parallel processing in SQL

Hi all,
Can anyone guide me to ideas on basics of parallel processing in SQL and its usage in the the performance tuning of the query. If so what's the syntax to be followed and how to arrive at the optimized query after tuning

My 2'c on the subject.
Don't break your head over parallel query (PQ) processing. It should be something that is of concern to the DBA/Oracle architect - not the developer.
Yes, it is good to be aware of how it works. But you should not write hints in your code that forces PQ processing.
Simple example. You use hints to force a PQ of degrees 5 (meaning 5 PQ processes will be used). It works great on development. In production, 10 users are running that query round about the same time. The PQ ceiling is 20 PQ processes. The 1st 4 users each gets 5 PQs processes and the remaining 6 get none. Or another developer did the same thing, only he was very greedy and coded a PQ of degrees 20 into his SQL. So his SQL is now consuming all available PQ processes. So how did forcing PQ in your code addressed performance? It did just for a couple of users with the majority of users now facing even worse performance.
The DBA is the one that should be tuning this. He/she can set the degree of PQ per table. Control the size of the PQ etc.
And it does not stop there. The primary reason for PQ in Oracle is to lower the latency of I/O. Let's say Oracle needs to reads 100MB worth of data (full table scan or fast full index scan for example). It uses multi block (sequential) reads to read bigger chunks at a time. Still, a single process can only read I/O so fast - the speed of which is entirely dependant on the I/O subsystem and hardware.
Using a second process to assist with the I/O can reduce the overall time. Instead of 1 process reading 100MB data, there are now 2 processes only having to read 50MB each.
But as I mentioned, the actual I/O thruput is a lower level function. Let's say you start a 100 processes. Great - each only have to read 1MB worth of data. Response should be fast.
Wrong. Those 100 processes seriously overload the I/O subsystem and throttle it so badly that the complete platform's performance is degraded severely. So instead of these 100 process speeding up I/O performance, they trash the performance of the entire platform.
Sure, this is an extreme example. But the basic concepts are usually illustrated much better using such an example.
So, you as developer deciding on just whether to use PQ and just how many PQ processes to use...? Wrong. It is not your decision, not your area of responsibility and usually not your area of expertise.
Know what PQ is. Know how PQ works. But think long and hard before forcing PQ via your code (using hints) on a production platform.

Parallel Processing...requirements???

Hi,
I understand that implementing Parallel Processing is also dependant on your HW, but to what level? For example, if I have a server with 1 CPU, can I still take advantage of Parallel Processing like when running a large query I use the Parallel hints? But if I have a server with 2 CPU's then can I use the Parallel Processing???
Sorry, but I just read up on this and this is the only part that was unclear to me.

One of the best sources of information on parallel query is Doug Burns' blog. There are a couple of technical papers here: http://oracledoug.com/papers.html and if you dop a search for "Parallel" at the main page of the website there are a number of other informal articles that have appeared over the last two or three years - including some links to articles by other writers.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
PS: It is certainly possible to get some benefit from running parallel queries with a single CPU - other constraints on the action come from the number of disk drives, and the number of controllers, and a single fast CPU could easily keep half a dozen discs busy by controlling a few PX slaves.
The trouble is, the more "useful" parallel queries tend to use multiple layers of PX slaves, and once the disc activity is over the messaging between layers can become CPU intensive - which means a single CPU may do well with some types of parallel query and badly with others.

Parallel processing BI data extraction

Hi,
I want use parallel processing to load the data to the PSA using multiple infopackage,
is there any option to run parallel info package which is delta enabled to load the data to the PSA using process chain to trigger the info package.
Regards,
vibhav.

Vibhav,
The question that i have is still the same. Are you trying to load data parallel into the exact same PSA table? This is not possible as it will cause an error while loading. However, you can utilize a Process Chain to kick off any number of InfoPackages irrespective if they are Delta enabled or not if you are extracting data into different PSA tables, in other words, different DataSources. We do this all the time and in terms of parallel processing, BI does this automatically if enough system resources are available.
In terms of the InfoPackages having Delta Capabilities, this depends on the DataSource. Some DataSources such as COPA does not allow Delta Packages seeing that the DataSource on the ERP side manages all Delta's.
In Terms of Performance, i suspect that the reason why you are posting this question is due to data taking too long to extract and you want to make it faster. In which area are you extracting your data from?

Parallel processing open items (FPO4P)

Hello,
I have a question about transaction FPO4p (parallel processing of open items).
When saving the parameters the following message always appears : "Report cannot be evaluated in parallel". The information details tells that when you use a specific parallel processing object, you also need to use that field to sort on.
I my case I use the object GPART for parallel processing (see tab technical settings). In the tab output control I selected a line layout which is sorted by business partner (GPART). Furthermore no selection options are used.
Does anyone know why the transaction cannot save the parameters and shows the error message specified above. I really don't know what goes wrong.
Thank you in advance.
Regards, Ramon.

Ramon
Apply note 1115456.
Maybe that note can help you
Regards
Arcturus

How to do parallel processing with dynamic internal table

Hi All,
I need to implement parallel processing that involves dynamically created internal tables. I tried doing so using RFC function modules (using starting new task and other such methods) but didn't get success this requires RFC enabled function modules and at the same time RFC enabled function modules do not allow generic data type (STANDARD TABLE) which is needed for passing dynamic internal tables. My exact requirement is as follows:
1. I've large chunk of data in two internal tables, one of them is formed dynamically and hence it's structure is not known at the time of coding.
2. This data has to be processed together to generate another internal table, whose structure is pre-defined. But this data processing is taking very long time as the number of records are close to a million.
3. I need to divide the dynamic internal table into (say) 1000 records each and pass to a function module and submit it to run in another task. Many such tasks will be executed in parallel.
4. The function module running in parallel can insert the processed data into a database table and the main program can access it from there.
Unfortunately, due to the limitation of not allowing generic data types in RFC, I'm unable to do this. Does anyone has any idea how to implement parallel processing using dynamic internal tables in these type of conditions.
Any help will be highly appreciated.
Thanks and regards,
Ashin

try the below code...
DATA: w_subrc TYPE sy-subrc.
DATA: w_infty(5) TYPE c.
data: w_string type string.
FIELD-SYMBOLS: <f1> TYPE table.
FIELD-SYMBOLS: <f1_wa> TYPE ANY.
DATA: ref_tab TYPE REF TO data.
CONCATENATE 'P' infty INTO w_infty.
CREATE DATA ref_tab TYPE STANDARD TABLE OF (w_infty).
ASSIGN ref_tab->* TO <f1>.
* Create dynamic work area
CREATE DATA ref_tab TYPE (w_infty).
ASSIGN ref_tab->* TO <f1_wa>.
IF begda IS INITIAL.
 begda = '18000101'.
ENDIF.
IF endda IS INITIAL.
 endda = '99991231'.
ENDIF.
CALL FUNCTION 'HR_READ_INFOTYPE'
 EXPORTING
 pernr = pernr
 infty = infty
 begda = '18000101'
 endda = '99991231'
 IMPORTING
 subrc = w_subrc
 TABLES
 infty_tab = <f1>
 EXCEPTIONS
 infty_not_found = 1
 OTHERS = 2.
IF sy-subrc <> 0.
 subrc = w_subrc.
ELSE.
ENDIF.

How to achieve parallel processing in a single request?

Hi all,
I have a method in a Session EJB that will perform some business logic before it returns an answer to the client. The logic it will perform is to collect data from the applications database and two external systems, before sending all data to a third external system to get a response and send it back to the client. Each external system is quite slow so I would like to do all the collecting of data concurrent, parallel processing. How should I handle this? I'm not allowed to create my own threads in EJB's. Can I use MDB in some way? To the calling client this should be a synchronous call...
Greatfull for any suggestions
Cheers
Anders =)

Usually, the request is received by a component located in the web container, such as by an HTTP request (including Web Services). This component is able to start threads to allow parallel processing. Now, if for some reason the request arrives directly at EJB level and that you cannot move its receiver to web component, I think JMS is not a viable solution because you will switch to asynchronous processing and you have no way to make your EJB wait for the responses while preserving the client request (waiting implies programmatic life cycle management, which is forbidden in EJB container). Maybe a resource adapter (JCA) can bring a solution. A resource adapter acts as a datasource (a datasource is a specialization of a resource adapter) and thus it is a logical way to implement an adapter to an external, eventually non-J2EE, resource, as the name implies :) But I don't have enough knowledge in JCA to be sure of this.
Hope it helps.
Bruno Collet
http://www.practicalsoftwarearchitect.com

BI 7.0 parallel processing of queries in a web application

Hi,
I'm currently having problems with a web application / web template with 10 data providers (different queries). When executing the web application the 10 queries are executed sequentially. Since each query takes about 30 sec., the complete execution time exceeds 300 seconds which is not satisfactory.
Is there any way to enable parallel processing?
Thanx in advance,
Patrick

Hello Patrick
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/41c97a30-0901-0010-61a5-d7abc01410ee
/thread/351419 [original link is broken]
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/media/uuid/ff5186ad-0701-0010-1aa1-e11f4f3f2f68
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/2b79ba90-0201-0010-1b9a-fa13a8f38127
Thanks
Chandran

BPM - Parallel process

Hi All,
I have multipe messages in a multiline container in a BPM.
Now i want to send each message individually to a webservice and get the response.Want to acheive this requirment in parallel process so that the total time is less.
Please let me know how to implement this requirement in BPM.
Regards,
Srinivas.

Hi Adish,
>> I want to send message to syn webservice
From which system you are sending data and what manner(synchronous or asynchronous).
>>Where I dont know exactly what will be the value for n in transformations step. It can vary depend upon input message that BPM will receive.
For this you can use a container variable of type xsd:integer and store the value in it and later you can use it for the next steps.
but give a arrow diagram and brief your exact requirement.(e.g., File->XI->HTTP)
Thanks,
Gujjeit

Parallel Processing in Oracle 10g

Dear Oracle Experts,
I would like to use the Parallel Processing feature on My production database running on Unix Box.
No: of CPU in each node is 8 and its RAC database
Before going for this option i would like to certain things regarding Parallel Processing.
1. According to my server specification how much DOP i can specify.
2. Which option for Setting Parallel is good
a. Using the 'alter table A parallel 4' or passing the parallel hints in the sql statements
3. We have a batch processing jobs which are loading data into the tables from flat files (24*7) using sql loader. is it possible to parallel this operation and any negative effect if enabled parallel.
4. Query or DML - which one will be perform best with parallel option.
5. What are the negative issue if parallel option is enabled.
6. what are the things to be taken care while enabling the parallel option.
Thanks in Advance
Edited by: user585870 on Jun 7, 2009 12:04 PM

Hi,
first of all, you should read [Using Parallel Execution|http://download.oracle.com/docs/cd/B19306_01/server.102/b14223/usingpe.htm#DWHSG024] in documentation for your version - almost all of these topics are covered there.
1. According to my server specification how much DOP i can specify.It depends not only on number of CPU. More important factors are settings of PARALLEL_MAX_SERVERS and PARALLEL_ADAPTIVE_MULTI_USER.
2. Which option for Setting Parallel is good - Using the 'alter table A parallel 4' or passing the parallel hints in the sql statementsIt depends on your application. When setting PARALLEL on a table, all SQL dealing with that table would be considered for parallel execution. So if it is normal for your app to use parallel access to that table, it's OK. If you want to use PX on a limited set of SQL, then hints or session settings are more appropriate.
3. We have a batch processing jobs which are loading data into the tables from flat files (24*7) using sql loader. is it possible to parallel this operation and any negative effect if enabled parallel.Yes, refer to documentation.
4. Query or DML - which one will be perform best with parallel option.Both may take advantages of using PX (with some restrictions to Parallel DML) and both may run slower than non-PX versions.
5. What are the negative issue if parallel option is enabled.1) Object checkpoint happens before starting parallel FTS (true for >=10gR2, before that version tablespace checkpoint was used)
2) More CPU and memory resources are used with PX - it may be both benefit and an issue, especially with concurrent PX.
6. what are the things to be taken care while enabling the parallel option.Read the documentation - it contains almost all you need to know. Since you are using RAC, you sould not forget about method of PX slaves load balancing between nodes. If you are on 10g, refer to INSTANSE_GROUPS/PARALLEL_INSTANCE_GROUPS parameters, if you are using 11g then properly configure services.

Which Parallel processing is faster?

Similar Messages

Maybe you are looking for