64 bits work slower then 64 bits - Q1

Question

Hi,
This very much related to question http://forum.sun.com/thread.jspa?threadID=29241&tstart=0
Noticed certain functions use up longer time for execution when its compiled as 64-bit compared to the same function in 32-bit.
Here is a stripped down example:
"perf.c" 17 lines, 184 characters
     1  #include <stdio.h>
     2
     3  int a(void) {
     4   int z=0;
     5   if (z == 0) z++;
     6  }
     7
     8  int main(void) {
     9   int i, j, z=0;
    10
    11   for (i = 0; i < 50000; i++)
    12    for (j = 0; j < 10000; j++)
    13     a();
    14
    15   return 0;
    16  }
    17Compiled it as follow:
64-bit
cc -xarch=v9 perf.c -o perf.6432-bit
cc perf.c -o perf.32Machine:
SunOS MachineA Generic_108528-24 sun4u sparc SUNW,Ultra-5_10 (UltraSparc IIi 440 MHz)
Compiler:
Forte[tm] Developer 7: C Compiler
Timing it, consistently 64-bit slower than 32-bit:
$ timex perf.64
real       27.73
user       12.45
sys         0.01
$ timex perf.32
real       20.58
user        9.18
sys         0.00The same result could be observed in other few machines I have tried:
1. SunOS MachineB Generic_108528-15 sun4u sparc SUNW,Ultra-5_10 (IIi 360 MHz)
2. SunOS MachineC 5.9 Generic_118558-19 sun4u sparc SUNW,Ultra-60 (II 450 MHz)
However, 64-bits same speed/faster in some machines:
3. SunOS MachineD 5.9 Generic_117171-02 sun4u sparc SUNW,Sun-Fire-280R (III+ 1200 MHz)
$ timex perf.64
real        1.34
user        1.33
sys         0.01
$ timex perf.32
real        1.34
user        1.34
sys         0.004. SunOS MachineE 5.10 Generic_118822-11 sun4u sparc SUNW,Sun-Fire-280R (1200 MHz)
$ timex perf.64
real        6.94
user        6.72
sys         0.05
$ timex perf.32
real        7.12
user        6.73
sys         0.08However, I tweak the code as following:
1) If line 4 is changed to/from:
     4   int z=0;
     5   if (z == 0) z++;and compared to
     4   long z=0;
     5   if (z == 0) z++;long has some improvement (however still slower than 32-bit (in MachineA, MachineB and MachineC) )
2) If compiled with Optimization on still slower than non-optimised 32-bit.
$ cc -O  -xarch=v9 perf.c -o perf.64
$ timex perf.64
real       24.48
user       10.89
sys         0.02
$ timex perf.32
real       21.48
user        9.23
sys         0.013) Moving statement line 5 to line 13 (without the function) - this gives the same timing (64b ~= 32b)!
Question that I have based on these are:
1) How could we explain this above scenario? does this related to stack, variables?
2) What makes it faster in MachineD and MachineE? Looking at this, its seems to be possible to make 64-bit as same performance as 32-bit. So is this something to do with software or hardware? which component?
Thanks in advance for those who could shed some light on this.

Answer 1

Peter, Thank you for the machine instruction level - detailed answer.
Yup, I agree stripping down a function probably not a best way to benchmark it, the reason we have done so is because our application (just would like to add - which has intensive use of IO) become slower in certain modules when moved to 64 bits. I have used gprof for detailed function level timing, based on the data it shows self-execution time for certain functions has increased dramatically. However when I stripped down those functions - with dummy function calls, and loop it as many time - I was not able to see the actual reported difference anymore - the stripped down program show lower difference. As you described caching may have played a role here.
As for the machine I have used, yes - it was not in idle however since both 32-bit and 64-bit were run at the same - assumption made was 1) both were tested on server with same amount of load - therefore this factor could be eliminated. 2) The code has no/minimal system calls - therefore server load may not effect the "user time".
As for performance analyser, yup will try to get this up.
Now getting abit specific to our issue - at the moment we are looking if there is any possible fix/workaround with minimal impact to our application . Therefore, we try not to:
1) Move to newer version of compiler (we planning to do this eventually and gradually).
2) Change optimization level (currently compiled as -O). (I believe -fast could lead to addtional memory usage)
Besides the fact that 64-bit app uses twice the size of pointers that may effect amount of data in cache and thus effect the performance.
From your reply I also leaned that :
1) Compiler Forte 7 - The generated instruction could have some overheads/(redundant perhaps?) instructions =~ which could cause 2-3 extra CPU cycles per funtion/section (in any specific case, could this be true for codes that compiled with -O on as well?).
2) Processor/HW
Is there any other possible factors:
1) Is there any tunable kernel parameters for 64-bits app performance that may be used as workaround?
2) Dynamic libraries (e.g libc)? Is there any patches known to improve performance.
Libraries used in our application using is as follow:
        libm.so.1 =>     /usr/lib/64/libm.so.1
        libnsl.so.1 =>   /usr/lib/64/libnsl.so.1
        libsocket.so.1 =>        /usr/lib/64/libsocket.so.1
        librt.so.1 =>    /usr/lib/64/librt.so.1
        libc.so.1 =>     /usr/lib/64/libc.so.1
        libdl.so.1 =>    /usr/lib/64/libdl.so.1
        libmp.so.2 =>    /usr/lib/64/libmp.so.2
        libaio.so.1 =>   /usr/lib/64/libaio.so.1
        /usr/platform/SUNW,Ultra-5_10/lib/sparcv9/libc_psr.so.1I am glad to provide more data if needed.
Thanks.