Time Machine tossed my backups. Why?

Question

Here is the sequence of events:
1) Had full macbook (160GB) & full external FW (320GB) HD (Not yet using TM)
2) Bought new external USB (640GB) HD, answered 'yes' to 'use for TM?' prompt
3) After TM finished full backup, deleted some 'not critical' files to make room on internal HD
4) Copied old iMovie projects from external FW HD to internal 'Movies' folder. TM didn't see change, didn't backup.
5) Dinked around and finally got TM to backup by creating new 'oldimovie' folder in home directory and copying old iMovie projects there.
6) Replaced contents of the 'oldimovie' folder with different iMovie projects from FW HD. TM backed them up.
7) Repeated process for all old content on FW HD over period of a few days, watching TM backup each change.
8) Put FW HD away, confident that I had transfered all content to new USB HD via TM.
9) TM continues to do its thing for several weeks.
10) I go into TM to look at older backups and can't find any evidence that 'oldimovie' folder ever existed. Can't find any evidence of any of the content that I copied in from the external FW drive.
11) Thank myself for not reformatting external FW drive yet.
Obviously, I do not understand how TM is supposed to work. Can someone please educate me? Is there a way to accomplish what I was trying to do that works (transfer contents of my internal and external HDs into one single TM backup on a new HD?)
Thanks!

Answer 1

zduffields,
Well, the first thing that comes to mind is to tell you that Time Machine will make a backup not just of your boot volume, but of any secondary/external volumes that are mounted when the backup is initiated. That's part of its "thing." So, you could have done this all in one lump, by having the older external connected and mounted during that initial backup.
All this said (and this is important!), you'll need more than just 640 GBs to make regular backups of two volumes, consisting of a total of 480 GBs. Note that I said "regular backups." While you can make a single backup of these two volumes, and you could probably make several subsequent backups (provided there's not too much data "churning" in the meantime. More on this in a bit), Time Machine will fail at some point, with the message that there's not enough room on the backup drive to make further backups.
That's because Time Machine really needs at least twice as much space as needed for the initial backup, in order to continue subsequent backups indefinitely. That "at least" phrase in the prior sentence is really the critical part, and I usually recommend closer to a 3X figure. It is true that Time Machine will begin deleting older backups at some point, to make room for new ones. However, it needs plenty of extra room as "overhead" in order for this to take place properly. More specifically, Time Machine operates in a certain sequence when performing its duties: First, it determines what has happened since the last backup, and decides what new/changed files must be copied to the destination volume. Second, it creates the new backup, copying those files identified in step 1. Third, it determines how to go about "thinning" older backups to make room for new ones, and it does so based on the latest backup (which it just made).
Time Machine uses a quite brilliant scheme to actually make new backups. Central to this scheme is the use of "multiple hard links" that point to the data for every file being backed up. The end result of this is that many, many full backups of a given installation can be made, all without using more than the space needed for the initial backup (equal to the amount of space being used on the source). That is, provided nothing ever changes on the source!! Every change, to the extent that it is changed, must be reflected within any subsequent backup in the form of additional data copied to the destination. This changed data, which must then be copied to the backup drive, is called "data churning." Included in this is any new file, any changed file, and any file that existed before but was moved to another location in the file system.
Given the way you have been using Time Machine thus far, it is no wonder that your backups have been deleted at an accelerated pace. Yes, that is what has happened; the earlier backups you made in an attempt to include your iMovie projects were deleted, in order to make room for the new data that was substituted. Time Machine will always try to leave enough room for its next backup before finishing, so it copied the newly added data to the destination drive, then deleted the old data as a last step.
So, you have two choices: Erase the backup drive, then start over with your older external connected (and include it in a single backup). In this way, you will have at least made one good backup of all your data. I will recommend against allowing TM to perform further backups, because you simply won't have enough room.
Or, you could trade in that 640 GB drive for something realistically large enough to handle your total of 480 GBs. For this, a 1 TB drive would be a bare minimum.
Scott