Thread execution time problem

Question

Hi everyone.
I�m developing an application that creates multiple threads, does certain calculations, and runs a couple programs.
The problem is: for some reason, the time that takes to run each thread tends to increment as the number of threads increments.
That is, for 2 threads the time is x for 4 is x + n, and so on.
I�d checked that every thread is doing the same (with few differences that not influences in a reasonable fashion), and that, skipping the external programs execution, the problem doesn�t exist. Moreover, there are no inter thread communication.
My questions: there is any way in which I can reduce the amount of time?
          There is some reason for the amount of time the external process called takes to run to increment? (I checked that if I not call those programs the amount of time each thread takes to execute is near 0 millisecs).
          Something like Windows limits the use of a program multiple times, or something like that.(yes i have only tested it in Windows environment, and yes, the external programs i use are posix compliant)
The actually time is about 120 secs., and I have to cut it to, at least, aprox. 40.
A thread alone can run in less than 30 secs., in the worst case.
Thanks in advance.

Answer 1

Now imagine that most of the tasks involve fetching
something from the refrigerator. Even if you have
multiple chefs in the kitchen, they will, unless
scheduled properly, all be contending to open the
door and reach inside.I think that could be the case. All the threads are trying to execute the same archive. As far as I remember, a file isn't locked if it is accessed for reading (or executing) only, but, on the other hand, you're right, there is only one refrigerator on this kitchen, and too many chefs.
Something else attracted my attention about this. When I run my project, there are two things that grow considerably: first the CPU's usage, and second, the amount of memory each of the external process called by my program consumes I think this could be related with my problem.
Again: I ran my project commenting the calls to Runtime.exec(�command�) and the time dropped significantly. Running normally, the threads tend to increment their time of execution. Lets say the first thread takes to execute X seconds, the nth thread will take X plus M. In the test I ran, the first thread lived for 30 seconds and the last (150th) lived for about 130 seconds. Without the calls to Runtime.exec() the first threads lived for 0 milliseconds and the last(150th) lived for 6 milliseconds. Therefore, the CPU�s overhead for everything else isn�t the problem here, I think.
I tried something else to discard the problem of the file access:
I created ten copies of the ping program in a directory other than the system default, and called them �ping1�, �ping2�, and so on.
Then, I created a program that creates some threads and executes a ping through a call to the program pingN (with Runtime.exec(), again), so each thread will access a different file (the first thread access ping1, the second ping2 �), and so, the bottle neck problem on the file system shouldn�t happen. But that hasn�t changed anything. With the calls to Runtime.exec() the execution time goes to the clouds, and without it, it�s near zero. And neither is fault of the ping, the I set the timeout to 3 seconds and the retries to 4, so the max amount of time it�ll take to execute is about 12 seconds, but it takes 40+-