CPU load unbalanced and too low in multi-processor computers

Question

I have a rather large test system 5-6000 VIs and we have noted some peculiar behaviour.
On a single processor system the application can take up to 100% of the available resources.
On a two processor system the application can take up to 50% of the available resources.
On a four processor system the application can take up to 25% of the available resources.
On an eight processor system the application can take up to 12-13% of the available resources.
I think you guys can get the gist of what I stated above (I didn't get pictures for 1 and 2 CPUs but I have observed the behaviour, personally)
We haven't done any specific multithreading code (we do lauch many parrallel processing loops dynamically = should be a good thing)
Why is the application behaving this way? Does anyone have a similar problem? Any tips?
Thanks for any help you can offer.
//David

Answer 1

You really don't provide enough information.
Is there really always something to do to keep a core busy at all times? Unless there is serious computations involved for solving a math problem, a typical VI should not be using a lot of CPU. (For example if it needs a full core today, it would not have been possible to even run the test a couple of years ago. That would be hard to believe). Don't overestimate how much a modern computer can do using only 5% of the CPU at any given time. I still run an acquisition system on a 120MHz Pentium 1 and LabVIEW 4. There are no performance problems even though the hardware is orders of magnitude slower than even a modern Atom processor on a netbook.
How many things can really occur at the same time? If you make your code overly sequential by lining everything up along error wires or by overuse of sequence structures, the code cannot be efficiently parallelized, no matter how hard the compiler tries.
How many of the 6000 VIs are called concurrently? How many require full CPU? If this is a test system, I assume that there is interaction with a device under test? Are the tests sequential or parallel? How many devices are tested at the same time? How much is post-processing compared to waiting for results? Are the tests really CPU limited??? Really???
Have you done any profiling? Did you identify the part that is most demanding? Could it be that a single greedy loop is consuming most of the CPU doing nothing?
Except for lengthy simulations or complex data processing, a typical application should never be CPU limited. The timings should be fully software controlled such that the code runs the same, independent of the power of the computer.
All that said, LabVIEW can easily keep all CPU cores busy if really needed and if programmed correctly and with parallel processing in mind. I have a complicated lengthy fitting code where I was able to keep 64 cores at near 100% (4 AMD 6274 CPUs with 16 cores each). As a more typical example, and compared to sequential execution, the parallelized code is 4.5x faster on a quad core I7 (if I disable hyperthreading in the bios, I get a 3.8x speed increase), so not only can it keep all four cores at 100%, it can even get a measurable boost from hyperthreading. On a six core I7, I get a 6.9x speedup, also keeping all cores busy.
I am not going to look at a project with 6000 VIs, but please show us some profiling data. Find the VI that carries most of the load and attach it if you want.
LabVIEW Champion . Do more with less code and in less time .