Coherence Bulk Load Memory Problem

Question

Hi,
I am trying to load like 35 million object(java object) into stand alone cache servers. According to reports of coherence, average item size is 360 byte. the thing is, to do this implementation I am using invocationservice's. And it means that, my invocable threads are running on on separate nodes(I have arranged them to be run on storage enabled nodes).
But the thing is; I am getting some memory probems liks:
8 nodes with 5 GB memory for each. Means 40 GB but even with this config, it makes full gc s several times in a day.
So to avoid full gc a little bit, I did not use invocalble threads, instead, I used regular threads in my application and it makes the loading work but it does not seem nice I think.
Any ideas that which one is better?(or maybe a new method)
Thanks

Answer 1

Have you taken a look at your GC logs to see what is happening?
Also, my rough calculations put you at an under sized grid:
35M objects @ 360 bytes means you need ~12GB of storage, not taking into account indexes, etc.
Optimistically, you'd need over 50GB of heap space in your grid. I'd suggest over 60GB. Why? Because you need space for backups copies of your data, (assuming you're running distributed services), working heap for serialisation, entry processes, etc, and your grid should never be more than 80% full.
My guess? You need more nodes/capacity in your grid - hence GC.