Types of Parallel Processing

Question

Hi SDNites,
I have a requiremnet where I have huge amount of data in my internal table and we want that data to be processed using parallel processing. Can you please let me know what are the different ways using which I can meet my objective.
Please provide me details with some example as I
Regards,
Abhi

Answer 1

Also can you please tell me whether the parallel processing requirement can be done using FM (JOB_OPEN / JOB_CLOSE). If yes how can that be done?
Yes, it can be done this way, that's what I was referring to when I mentioned multiple background jobs. If you're not familiar with how to do it, check out the ABAP online help on the [SUBMIT .. VIA JOB|http://help.sap.com/abapdocu_70/en/ABAPSUBMIT_VIA_JOB.htm] statement, which also contains a short example. A more in-depth and good description of your options can be found in the online help [Programming with the Background Processing System|http://help.sap.com/saphelp_nw70ehp2/helpdata/en/fa/096c53543b11d1898e0000e8322d00/frameset.htm]. Here you can also find example programs for the RFC and the background job approach: [Implementing parallel processing|http://help.sap.com/saphelp_nw70ehp2/helpdata/en/fa/096e92543b11d1898e0000e8322d00/frameset.htm].
I personally think the RFC option has the major advantage that you have already a built-in capability for managing resource consumption (using RFC server groups). I'm not aware that you get this for free with background jobs (though if you have long-running tasks you cannot use RFC, because they are executed in dialog processes and thus face the same maximum runtime limit as normal dialog users).
Anyhow, in theory your task is rather simple, though of course the actual technical implementation can be quite challenging: Essentially you're trying to implement a [divide and conquer algorithm|http://en.wikipedia.org/wiki/Divide_and_conquer_algorithm], a real prominent example is Google's [MapReduce framework|http://en.wikipedia.org/wiki/MapReduce] (has nothing to do with ABAP, but the concept is of course valid and can be found nowadays in many applications).
So you just need to find a way to split-up your problem into smaller sub-problems, which you then solve individually. Should you have the need for a final step that combines all individual results (the reduce step in Google's framework), you would need with background jobs an approach for figuring out if all job steps are done (so essentially a [semaphore|http://en.wikipedia.org/wiki/Semaphore_%28programming%29] or [fork-join queue|http://en.wikipedia.org/wiki/Fork-join_queue], however you want to look at it).