Hard Drive filling up: SubmitDiagInfo, /private/var & log issues

Question

This topic is part question, part help for people I've seen having a similar problem in other threads. My issue is different enough from the other issues, that I wanted to post it as a separate topic.
I've had about 24GB of available space for weeks now. I haven't downloaded any large files recently, nor created any myself.
Suddenly I noticed the hard drive space decreasing. I went from 24GB, to 19GB, to 10GB, then 8, 5, 1.5... next thing I knew, it said I had 260MB available. This was within maybe two days, with a big ramp-down within a couple of hours this morning, while I was on a train, and not even connected to a network.
I started getting error messages saying my hard drive was full; then, I got errors saying I didn't have enough application memory, and applications were being paused, and I would have to force quit them. Within about 5 minutes, I dropped from 1.5GB to 260MB.
Thinking back, the last week or so, I remembered that sometimes when I came into my office in the morning, my Mac would be awake (I usually sleep it at the end of the day, and an error message would be displayed saying SubmitDiagInfo had crashed. I thought this might have something to do with it.
I did some detective work in these forums. I checked Activity Monitor and the Console. Sure enough, SubmitDiagInfo was taking up a bunch of CPU time.
The Console revealed I had a lot of Diagnostic Logs. I tried deleting them with the Console's Clear button, but it didn't seem to do anything. I checked with OmniDiskSweep, and sure enough, I had over 20GB of log files.
At this point, my hard drive was so full, I had to force quit OmniDiskSweep. I used the Go To Folder command and entered /private/var/log - The Diagnostic Logs folder was nearly 20GB. I deleted it and emptied the trash immediately.
I looked at the /private/var/vm folder and noticed it was also very large (10GB), but I had read elsewhere not to mess with this folder.
From the console, I quit SubmitDiagInfo.
I check my hard drive, and I'm suddenly back to 24GB free space.
When I look again at /private/var/vm, and sure enough it is 5GB smaller.
This is weird stuff, and I hope this description helps anyone else having this problem.
What's interesting is, because the /private folder is invisible, you can't use it to calculate your disk space usage. For example, the folders on my drive had the following space usage:
62.46GB34.53GB+22.96GB+4.78GB4.02GB = 128.75GB (plus a few "small" folders with maybe 300MB)
Since the drive showed 159GB total space available, I couldn't figure out where the other 30GB was.
Once I found /private/log, I could see it had 19.29GB:
62.4634.53+22.96+4.78+4.0219.29 = 148.04 - ah, this makes more sense, but I'm still missing 10GB! So, I look at /private/vm, and boom, there is the extra 10.6GB hiding. All 159GB present and accounted for.
The question I have is, of course, can anyone explain why this is happening, so I can prevent it from happening again?
I hope someone finds this helpful.

Answer 1

Thanks... that is helpful. I have Carbon Copy Cloner, not
Super Duper, so I will keep that solution in mind.
Do not, repeat do not, clone your current suspect hard drive
to your CCC backup. That would defeat the purpose of having
a good backup. I don't know if you can boot from you CCC
backup. That would be a good test. I was able to boot from
my SuperDuper backup and notice normal hard drive space.
In the meantime, I ran the extended system test, and the
computer passed.
Mine with the growing hard drive passed every test I had plus
the test Apple recommended. They were quick place the blame
on a Virtual Windows XP Pro. I do not thing that was the
problem. My normal Macintosh HD is usually about 52 GB
and it was showing 110 GB used and I never could figure out
what was hogging the hard drive.
Good luck!