BPM - Independent Parallel Processing

Hi,
i need a suggestion.
Via JDBC Outbound Adapter a BPM receives a multiline element from a database periodicaly . For each row the BPM does a sync rfc call and following a sync JDBC update call back to the database.
Currently i have tried par foreach and foreach, but with a message size of more then 1000 rows the bpm needs over an hour to complete all steps.
What is the best solution for parallel processing the multiline message in the bpm ?
Thanks and Regards
Sebastian

Sebastian,
For-Each or Par-For-Each is the only way in your case. But what u can do is, reduce the number of rows. If you have the polling time as 5 mins, then reduce it to 2 mins. So the rows might get reduced.
your thoughts!
raj.

Similar Messages

Parallel processing and throttle

Hi experts,
where this parallel processing and throttle used exactly in XI.

You can develop scenarios in BPM for parallel processing. Simple use cases involve like splitting incomming file into multiple files and sending them out in parallel. A block with par for each would be needed in this case.
Search in help for more information.
VJ

BPM - Parallel process

Hi All,
I have multipe messages in a multiline container in a BPM.
Now i want to send each message individually to a webservice and get the response.Want to acheive this requirment in parallel process so that the total time is less.
Please let me know how to implement this requirement in BPM.
Regards,
Srinivas.

Hi Adish,
>> I want to send message to syn webservice
From which system you are sending data and what manner(synchronous or asynchronous).
>>Where I dont know exactly what will be the value for n in transformations step. It can vary depend upon input message that BPM will receive.
For this you can use a container variable of type xsd:integer and store the value in it and later you can use it for the next steps.
but give a arrow diagram and brief your exact requirement.(e.g., File->XI->HTTP)
Thanks,
Gujjeit

BPM Parallel Process with Exclusive Gateway

Hi,
I am facing issue with Exclusive Gateway in Parallel Process.
Issu is, process always in In-Progress state at parallel Join. I mean process stops at Parallel Join and more over there are no errors in the process. If I delete Exclusive Gateway in Parallel process, the process is going to next level human task through Parallel Join. It means working fine.
I have designed my process in such a way that, 1st task is Human Task ---> then Parallel Split with 2 Human tasks, out of one task performing throught Excusive Gateway and another one is just simple approval. Finally I am merging these two Human tasks using Parallel Join then finally triggerting Approval Human task, and closing the Process.
Appreciate your quick suggestions to fix this issue.
Thanks in advance,
Dev...

Hi Unni,
Thanks for your reply.
I have checked all the parallel tasks and all are in completed state. No errors.
If I delete Exclusive Gateway it is working fine. I have checked step by step tasks in NWA, and every thing goes well.
Please let me know If I missout anything.
Thanks in advance,
Dev

Parallel process run independently, but do not stop when vi completes

So I posted a problem yesterday about getting 'elapsed time' express vi to provide updates from a sub-vi to the calling vi. It was suggested that I create a parallel process in the sub-vi that will run the elapsed time function at the same time the other processes are running. I tried to implement this idea, but ran into a problem. The elapsed time process is a While loop paralleled with the main While loop in the sub-vi, It updates every second, based on my time delay, and when I view the running sub-vi, the elapsed time indicator is updating as it should. The calling vi, however, is not seeing these updates. I have wired an indicator to the sub-vi icon, but does not change until the sub-vi finishes.
Also, the other problem with the parallel process is that it runs forever, regardless of the other loop finishing. I have tried to wire an or'd bool to the stop inside the While loop, but when I do that, the elapsed time process does not start.
I have tried data binding a shared variable in the project, and then dragging that to my calling vi, but again I get no updates on the elapsed time.
Any ideas????

A VI must finish looping before it has an available output (the terminal on the sub-vi icon). Research how to communicate between loops. What you are doing can be accomplished with Notifiers (that's what I'd use), Queues, or Global/Shared Variables.
Your issues appear to be due to a lack of understanding of the LabVIEW data-flow paradigm. Check out the Producer / Consumer example. Post code here and see if one of us can give more guidance.
Richard

Parallel processing of mass data : sy-subrc value is not changed

Hi,
I have used the Parallel processing of mass data using the "Start New Task" . In my function module I am handling the exceptions and finally raise the application specific old exception to be handled in my main report program. Somehow the sy-subrc is not getting changed and always returns 0 even if the expection is raised.
Can anyone help me about the same.
Thanks & Regards,
Nitin

Hi Silky,
I've build a block of code to explain this.
DATA: ls_edgar TYPE zedgar,
        l_task(40).
DELETE FROM zedgar.
COMMIT WORK.
l_task = 'task1'.
ls_edgar-matnr = '123'.
ls_edgar-text = 'qwe'.
CALL FUNCTION 'Z_EDGAR_COMMIT_ROLLBACK' STARTING NEW TASK l_task PERFORMING f_go ON END OF TASK
    EXPORTING
      line = ls_edgar.
l_task = 'task2'.
ls_edgar-matnr = 'abc'.
ls_edgar-text = 'def'.
CALL FUNCTION 'Z_EDGAR_COMMIT_ROLLBACK' STARTING NEW TASK l_task PERFORMING f_go ON END OF TASK
    EXPORTING
      line = ls_edgar.
l_task = 'task3'.
ls_edgar-matnr = '456'.
ls_edgar-text = 'xyz'.
CALL FUNCTION 'Z_EDGAR_COMMIT_ROLLBACK' STARTING NEW TASK l_task PERFORMING f_go ON END OF TASK
    EXPORTING
      line = ls_edgar.
*&      Form f_go
FORM f_go USING p_c TYPE ctype.
RECEIVE RESULTS FROM FUNCTION 'Z_EDGAR_COMMIT_ROLLBACK' EXCEPTIONS err = 2.
IF sy-subrc = 2.
*this won't affect the LUW of the received function
    ROLLBACK WORK.
ELSE.
*this won't affect the LUW of the received function
    COMMIT WORK.
ENDIF.
ENDFORM.                    "f_go
and the function is:
FUNCTION z_edgar_commit_rollback.
*"*"Interface local:
*" IMPORTING
*"     VALUE(LINE) TYPE ZEDGAR
*" EXCEPTIONS
*"      ERR
MODIFY zedgar FROM line.
IF line-matnr CP 'a*'.
*comment raise or rollback/commit to test
*    RAISE err.
    ROLLBACK WORK.
ELSE.
    COMMIT WORK.
ENDIF.
ENDFUNCTION.
ok.
In your main program you have a Logical Unit of Work (LUW), witch consists of an application transaction and is associated with a database transaction. Once you start a new task, your creating an independent LUW, with it's own database transaction.
So if you do a commit or rollback in your function the effect is only on the records your processing in the function.
There is a way to capture the event when this LUW concludes in the main LUW. That is the PERFORMING whatever ON END OF TASK. In there you can get the result of the function but you cannot commit or rollback the LUW from the function since it already have implicitly happened at the conclusion of the funtion. You can test it by correctly comment the code I've supplied.
So, if you want to rollback the LUW of the function you better do it inside it.
I don't think it matches exactly your question, maybe it lead you on the right track. Give me more details if it doesn't.
Hope it helps,
Edgar

FORK is Not happening Parallel processing- It's working sequential

Hi,
   we are into PI 7.O and SP 13.
   I am trying to test Parallel processing using Fork step. (With Two branches)
My problem is sxm_moni both branches are not executed simultenously and it's executing one after the other.
Did any body done in XI parallel processing using BPM...both calls has to finish at the same time. I mean first call 10 min and second call aslo has to finish first 10 min ..not other 10 min.
I heard this problem from XI 3.0 and PI 7.O. But PI 7.1 did any body test the Parallel processing using Fork step.
   Pls help me is this issue will resolve if I go to PI 7.1.
Regards,
Venu.

Hi Henrique,
they would not necessarily start at the same time but shouldnt also be queued - Customer expecting the response within a 17 sec or 20 Sec but coming response 34 sec will not ok for the customer..tomorrow need add some more target again 17 sec will take...How PI can handle the Multi threading they are checking...I am not sure this problem fixed in PI 7.1 or not.
there're # of connection restrictions in your system? Check that - Where can I check connections restrictions...If you know pls through some light on this.
Also, how's your BPM transactional behavior (did you flag the create new transaction steps)?
- I did not checked the flag for create new transaction step..once my server is up I can check the flag and I can test.
Regards,
Venu.

Unable to clone File Adapter receiver channel for parallel processing

Hi Experts,
I am using variable substitution for File - RFC - File with out BPM scenario(using request response, oneway bean).
While i placed the file in the sender FTP folder, the file didnt get picked up, but in communication channel monitoring, i am getting error ' Unable to clone File Adapter receiver channel for parallel processing'.
Can anybody provide me suggestions to solve this error.
Note : without variable substitution , the interface is working good.
Is it due to, i am trying the source structure field in response file adapter?

Hi,
In your CC, do you use some additional paramaters ?
like these one of point 47/48 of [Oss note 821267 - FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter|https://service.sap.com/sap/support/notes/821267]
Maybe there is conflict with a parallel connexion and the bean used to do asynch-synch bridge...
Mickael

Parallel processing using ABAP objects

Hello friends,
                    I had posted in the performance tuning forum , regarding a performance issue problem , I am reposting it as it involves OO concept .
the link for the previous posting
Link: [Independent processing of elements inside internal table;
Here is the scenario,
I have a internal table with 10 records(indepentent) , and i need to process them .The processing of one record doesnt have any influence on the another . When we go for loop , the performance issue is that , the 10 th record has to wait until the 9 records get processed even though there is no dependency on the output.
Could some one tell a way out to improve the performance..
If i am not clear with the question , i would explain it still clearer...
A internal table has 5 numbers , say( 1,3,4,6,7)
we are trying to find square of each number ,,,
If it is a loop the finding of suare of 7 has to wait until 6 is getting completed and it is waste of time ...
This is related to parallel processing , I have refered to parallel processing documents,But I want to do this conceptually ..
I am not using conventional procedural paradigm but Object orientedness...I am having a method which is performing this action .What am I supposed to do in that regard.
Comradely ,
K.Sibi

Hi,
As examplified by Edward, there is no RFC/asynchronous support for Methods of ABAP Objects as such. You would indeed need to "wrap" your method or ABAP Object in a Function Module, that you can then call with the addition "STARTING NEW TASK". Optionally, you can define a Method that will process the results of the Function Module that is executed asynchronously, as demonstrated as well in Edward's program.
You do need some additional code to avoid the situation where your program takes all the available resources on the Application Server. Theoretically, you cannot bring the server or system down, as there is a system profile parameter that determines the maximum number of asynchronous tasks that the system will allow. However, in a productive environment, it would be a good idea to limit the number of asynchronous tasks started from your program so that other programs can use some as well.
Function Group SPBT contains a set of Function Modules to manage parallel processing. In particular, FM SPBT_INITIALIZE will "initialize" a Server Group and return the maximum number of Parallel Tasks, as well as the number of free ones at the time of the initialization. The other FM of interest is SPBT_GET_CURR_RESOURCE_INFO, that can be called after the Server Group has been initialized, whenever you want to "fork" a new asynchronous task. This FM will give you the number of free tasks available for Parallel Processing at the time of calling the Function Module.
Below is a code snippet showing how these Function Modules could be used, so that your program always leaves a minimum of 2 tasks for Parallel Processing, that will be available for other programs in the system.
      IF md_parallel IS NOT INITIAL.
        IF md_parallel_init IS INITIAL.
*----- Server Group not initialized yet => Initialize it, and get the number of tasks available
          CALL FUNCTION 'SPBT_INITIALIZE'
          EXPORTING
            GROUP_NAME                           = ' '
            IMPORTING
              max_pbt_wps                          = ld_max_tasks
              free_pbt_wps                         = ld_free_tasks
            EXCEPTIONS
              invalid_group_name                   = 1
              internal_error                       = 2
              pbt_env_already_initialized          = 3
              currently_no_resources_avail         = 4
              no_pbt_resources_found               = 5
              cant_init_different_pbt_groups       = 6
              OTHERS                               = 7.
          md_parallel_init = 'X'.
        ELSE.
*----- Server Group initialized => check how many free tasks are available in the Server Group
      for parallel processing
          CALL FUNCTION 'SPBT_GET_CURR_RESOURCE_INFO'
            IMPORTING
              max_pbt_wps                 = ld_max_tasks
              free_pbt_wps                = ld_free_tasks
            EXCEPTIONS
              internal_error              = 1
              pbt_env_not_initialized_yet = 2
              OTHERS                      = 3.
        ENDIF.
        IF ld_free_tasks GE 2.
*----- We have at leasr 2 remaining available tasks => reserve one
          ld_taskid = ld_taskid + 1.
        ENDIF.
    ENDIF.
You may also need to program a WAIT statement, to wait until all asynchronous tasks "forked" from your program have completed their processing. Otherwise, you might find yourself in the situation where your main program has finished its processing, but some of the asynchronous tasks that it started are still running. If you do not need to report on the results of these asynchronous tasks, then that is not an issue. But, if you need to report on the success/failure of the processing performed by the asynchronous tasks, you would most likely report incomplete results in your program.
In the example where you have 10 entries to process asynchronously in an internal table, if you do not WAIT until all asynchronous tasks have completed, your program might report success/failure for only 8 of the 10 entries, because your program has completed before the asynchronous tasks for entries 9 and 10 in your internal table.
Given the complexity of Parallel Processing, you would only consider it in a customer program for situations where you have many (ie, thousands, if not tens of thousands) records to process, that the processing for each record tends to take a long time (like creating a Sales Order or Material via BAPI calls), and that you have a limited time window to process all of these records.
Well, whatever your decision is, good luck.

Parallel Processing for a single Package

Hi,
I have PKg1 that have mixture of For Each Loop container, DFT's and Seq containers and I want to run more than one thread for this package where i can process data in parallel.
Please let me know how i can create this using SSIS 2012.
Thanks,

Hi,
DFTs connected by precedence constraints and I want to run this package more than once (multiple threads) at a given point of time. is this possible? if
yes, please let me know how I can achieve this.
Thanks..
If the DFTs are connected then there will be absolutely no parallel processing. Running the same package in parallel most likely result in a lock. It depends how it is architectured, but with a RDBMS in default installation or files it is not going to fly.
When you have a DFT with say OLEDB destination each using its own connection, and they are not connected then each gets opened independently and thus allowing you to ingress data simultaneously.
Arthur My Blog

Parallel processing--share your suggestions

Hi Experts,
I have a program which is taking long time (min 10days)to run because of 4 millian data.Thing is i am planning to use parallel processing, i have gone through help documents.They have used one Syntax
"CALL FUNCTION func ...STARTING NEW TASK task name "
ans some additions.
What this function module name?.Do i need to create a
"RFC function module" and call my report from this function module.My problem is how to use my progarm and this SYNTAX.
Can any will explain and give some suggestion on it.
Note : I fine tunned this program Max.extend
Thanks in advance.
Murali

Hello Murali,
There are many considerations for starting the job in parallel.
1. u should be able to logically break the job for parallel process to handle.
2. When the jobs starts in parallel mode there is no sequence in which they will complete and so should be independant of each other.
3. The FM which is called in CALL FUNCTION func ...STARTING NEW TASK task name should be RFC enabled.
4. In order to process the jobs in parallel ur system should have atleast 3 dialog work process and should have one dialog process free when u start parallel processing.
I am assuming u don't have the FM which is why u are asking what is the fucntion module name. I would suggest u need to modify ur code in such a way that u are able to create a FM which is going to be called in the CALL FUNCTION func ...STARTING NEW TASK task name.
Abhijit

Parallel processing end condition problem

Hi,
i use a block with parallel processing for each row (parforeach). In every branch i create a workitem where a user has to input something. This is stored in my field status.
Now i want that if the status is 'X' all branches will be closed like it would be in a fork.
I entered the end condition &status& = X
But the end condition of parallel processing doesn't work. I tested it and it works but into the workflow the branches won't be ended.
Does someone has a solution or can explain me the problem?
Thx

Hi,
The dynamic parallelism using 'parforeach' is not the same as fork with multiple branches with number of required branches to end the fork !! Its like this, when you use dynamic parallelism , for each index of your multiline container element used for parallelism, the sub-workflow ( or the associated Task having dynamic parallelism) will be called parallely and each branch ( parallely called sub workflow or task ) is independent now having no relation ( by default ) and workflow will not continue to next step until unless all these branches are completed.
However, to solve your probelm i suggest the following
1. How are you handling the process in each branch, is it through a sub workflow? if yes, in your sub workflow you can create a fork parallel to your normal process. In that fork, create a 'Wait for Event' step and wait for a new custom event ( for this you have to define a new custom event on your BO) . Put the necessary branches required as 1 and Join this branch to end of this sub-workflow. and
2. Whenever your requirement to end all the branches is fulfilled ( say in your case status=X) raise this new custom event using Create Event and this will be captured by 'Wait Event' step in the fork of your sub-workflow and it will end that sub workflow ( meaning, your branch is ended now) .Make sure that you pass the BO Object Instance to your sub-workflow through binding from your main workflow. !!
Hope this helps you !!
Regards
Krishna Mohan

Parallel Processing : Unable to capture return results using RECIEVE

Hi,
I am using parallel processing in one of my program and it is working fine but I am not able to collect return results using RECIEVE statement.
I am using
CALL FUNCTION <FUNCTION MODULE NAME>
STARTING NEW TASK TASKNAME DESTINATION IN GROUP DEFAULT_GROUP
PERFORMING RETURN_INFO ON END OF TASK
and then in subroutine RETURN_INFO I am using RECEIVE statement.
My RFC is calling another BAPI and doing explicit commit as well.
Any pointer will be of great help.
Regards,
Deepak Bhalla
Message was edited by: Deepak Bhalla
I used the wait command after rfc call and it worked additionally I have used Message switch in Receive statement because RECIEVE statement was returing sy-subrc 2.

Not sure what's going on here. Possibly a corrupt drive? Or the target drive is full?
Try running the imagex command manually from a F8 cmd window (in WinPE)
"\\OCS-MDT\CCBShare$\Tools\X64\imagex.exe" /capture /compress maximum C: "\\OCS-MDT\CCBShare$\Captures\CCB01-8_15_14.wim" "CCB01CDrive" /flags ENTERPRISE
Keith Garner - Principal Consultant [owner] -
http://DeploymentLive.com

Parallel Processing and Capacity Utilization

Dear Guru's,
We have following requirement.
Workcenter A Capacity is 1000.   (Operations are similar)
Workcenter B Capacity is 1500.   (Operations are similar)
Workcenter C Capacity is 2000.   (Operations are similar)
1) For Product A: Production Order Qty is 4500. Can we use all workcenter as a parallel processing through Routing.
2) For Product B: Production Order Qty is 2500. Can we use only W/C A and B as a parallel processing through Routing.
If yes, plz explain how?
Regards,
Rashid Masood

May be you can create a virtual WC VWCA=ABC (connected with a hierarchy with transaction CR22) and another VWCB=A+B and route your products to each VWC

Parallel processing open items (FPO4P)

Hello,
I have a question about transaction FPO4p (parallel processing of open items).
When saving the parameters the following message always appears : "Report cannot be evaluated in parallel". The information details tells that when you use a specific parallel processing object, you also need to use that field to sort on.
I my case I use the object GPART for parallel processing (see tab technical settings). In the tab output control I selected a line layout which is sorted by business partner (GPART). Furthermore no selection options are used.
Does anyone know why the transaction cannot save the parameters and shows the error message specified above. I really don't know what goes wrong.
Thank you in advance.
Regards, Ramon.

Ramon
Apply note 1115456.
Maybe that note can help you
Regards
Arcturus

BPM - Independent Parallel Processing

Similar Messages

Maybe you are looking for