AMD cc:NUMA support...

Question

Hi guy's,
If you have a board that supports ccNUMA, turn it on!!!
Boards that support this are:
K8D Master (series) with 2 CPU's
K8D Master3 (series) with 2 CPU's
You need to have min 2 sticks of memory installed, but better is 4, I talk further about 4 sticks.
2 sticks connected to each CPU.
Now you need to go into the BIOS and turn Node (Memory) Interleave OFF!
Do not confuse this with Memory Interleave, that's per CPU and should stay on.
You need to turn this off, because it should not automaticly use any memory that it can find.
Further you need WinXP SP2! (or Win2000 Datacenter or simular)
Check your memory speed, normaly Sandra or something should show a normal speed for your memory.
E.g.:
Normal Dual channel -> PC-2700x2 is about 4800MB/s (just a sample!)
Even with 2 CPU's and 4 sticks installed (2 for echt CPU)
Sandra shows NUMA not supported.
Now edit your boot.ini and add the /PAE switch!
This is very important, as that will turn your Windows to use ccNUMA.
Then it means that every single program will be using it's closest memory.
And share if needed the speed from the other CPU's memory.
Good written programs, multi-threading can use this to improve a lot.
Now check with Sandra memory bench again!
It should go upto 9000MB/s or such.
Also Sandra should show NUMA as being supported.
This should speedup your system under Windows a lot.
Linux can also use this, but I don't know how to check it.
Anyway, it will give your system a whole lot more memory bandwidth to play with.
This only works on the boards listed, as they have memory connected to each CPU.
The K8T-Master or K8N-Master series with 2 CPU's can't do it because they have only memory connected to CPU1.
Windows Vista should have this by default, but I'm not sure, you need to Google on that.

Answer 1

Well a little update, the K8N Master can do NUMA too according to Sandra.
The /PAE switch should be enough.
For other boards that have memory per CPU.
Disable Memory node interleave.
Turn SRAT on in the BIOS.
Edit the BOOT.INI and add the /PAE switch.
Now reboot and NUMA should be working, but you need at least WinXP SP2 or higher.
Beware, if you don't disable Memory Node Interleave, NUMA will not work!!
NUMA does make a big difference, so use it if you can.
The following AMD boards have NUMA full blown:
K8D Master-133 series
K8N Speedster series
And other DUAL AMD CPU systems that have memory connected to each CPU.
The K8N Master seems to support it, but I'm not quite sure what it does with it, maybe somebody can test and find out.