Command line transfer of TM backup to Time Capsule

Question

I have just purchased a new 3TB Time Capsule to replace my ailing 750GB Firewire drive as my Time Machine Backup drive. However, the 750GB drive has none TM files on it, that I do not want to transfer to the new Time Capsule (see details below)
file://localhost/Volumes/LaCie%20BackUp/Backups.backupdb/
file://localhost/Volumes/LaCie%20BackUp/IBM%20DDRS-39130D%20Media/
file://localhost/Volumes/LaCie%20BackUp/Orphans%20Folder/
file://localhost/Volumes/LaCie%20BackUp/Files%20from%20Server/
The reason I am doing it this way is also because my old drive occassionaly goes into a 'pause' that requires me to disconnect it from my MacBook Pro and reconnect it to try again. Thus I need to be able to transfer my backup files in smaller chunks than a whole disk image would entail, so that if the drive stops, I can pick up from the point of failure and carry on rather than start from scratch again.
Only the Backups.backupdb needs to be transferred to the Time Capsule, so I don't want to use Disk Utility and have been trying to copy the relevant backup directories using Terminal and the 'cp' command as SuperUser. However, I am finding that even so I am getting error messages back stating 'Operation not permitted', mainly to do with cp trying to run 'lchmod'.
I have read the existing FAQ article https://discussions.apple.com/message/11422415#5 and have all the correct journaling settings, etc. and the data seems to be transferring fine. Just not sure yet if I will have fully functional backups when I have finished.
Any command line help such as using the appropriate flags, etc. would be helpful.
Thanks, Russ

Answer 1

Pondini's notes are quite useful and I tried using SuperDuper and Disk Utility. I transferred my backups from my old 750GB Firewire400 backup drive to my new 3TB Time Capsule using both of them. Each time I tried, the transfer took to an inordinately long time - it failed any time up to a maximum of 18 hours from starting, the best only achieved about 90% transfer!
Also, I discovered by monitoring my MacBook's performance with Activity Monitor that both these programs are CPU intensive and indicated between 150% to 200% activity (both CPUs virtually maxed out), mostly with Finder activity. This was the limiting factor in the transfer and the process was achieving peak Disk I/O and network transfer rates of 10MB/s. The MacBook's fans were running full-blast during this time just trying to keep the thing cool!
I eventually found the most efficient method to transfer backups by reading through some of the Unix command manual pages and discovered Apple Software Restore (ASR). This command is used as the underlying tool by both SuperDuper and Disk Utility and does all the actual work. The final transfer tool less than 7 hours.
Network Transfer Rate 36MB/s (sustained)
Disk I/O Rate 52MB/s (peak), >32MB/s (sustained)
CPU activity 20% (peak), <15% (typical)
Main process is 'diskimages-helper'
The reason it works much better is that only the essential work is being done and that the command allows you to alter the buffer size to make it more efficient for the larger file transfer method. The whole transfer was therefore limited purely by the maximum transfer speed of the Firewire 400 drive itself. So here is how I did it using terminal:
$ su
Password:
[change to superuser so you don't have problems with passwords and permissions, etc.]
sh-3.2# nice -n -20 asr restore --source <source drive> --target <target drive> --erase --noprompt --allowfragmentedcatalog --buffers 8 --buffersize 64m --noverify
Validating target...done
Validating source...done
Repartitioning target device...done
Warning: You may not be able to start up a computer with the target volume.
Validating sizes...done
Restoring  ....10....20....30....40....50....60....70....80....90....100
Remounting target volume...done
After mounting the resulting Time Capsule BackUp, it had the name of my old drive, which confused me somewhat, so I simply renamed it in the Finder.
Explanation:
The 'nice -n -20' command tells the OS X kernal to prioritise the commands that follow by giving them preferential access to CPU time (kind of like putting go-faster-stripes on).
The <source drive> is selected by dragging the drive icon onto the terminal window.
The <target drive>  has to have been created first by running an initial Time Machine Backup on the Time Capsule, which creates the .sparsebundle directory. Once this is created, stop Time Machine, then use the finder to log onto the Time Capsule and double click the .sparsebundle file. This will mount a disk called 'Time Capsule Backup'. Drag this image onto the terminal window as the target drive.
--erase removes any other stuff on the target drive and matches the journaling to the source drive
--noprompt stops you getting "do you want to continue? messages
--allowfragmentedcatalog ensures the directory catalogue on the source drive is read as is
--buffers 8 the default is 8, but I tried it with different numbers of buffers, but it didn't affect transfer speeds much
--buffersize 64m this made the most difference as the default is 1MB. Changing the buffer size makes the biggest impact as large file transfers work better when you give them large buffers.
--noverify - I used this as the transfer itself takes so long that I wasn't really wanting to wait for it to verify as well. On the occasions that I left it to verify, that function operated at a VERY slow speed and would have taken more than 100 hours to complete. I did not have the patience to investigate as to why that was, but essentially the data transfer rate during that stage was <2MB/s. Whatever was causing it to run so slow was not limited by any resources on my MacBook Pro or the network connection, so I will let someone else work out what the issue was there.