How to migrate from a standard store setup in a splitted store (msg - idx) setup

Question

How can I migrate from a standard store setup in a splitted setup described in
https://wikis.oracle.com/display/CommSuite/Best+Practices+for+Messaging+Server+and+ZFS
can a 'reconstruct' run do the migration or have I do a
imsbackup - imsrestore ?

Answer 1

If your new setup would use the same filesystem layout as the old one (i.e. directory paths to the files would be the same when your migration is complete) you can just copy the existing store into the new structure, rename the old store directory into some other name, and mount the new hierarchy instead of it (zfs set mountpoint=...). The CommSuite Wiki also includes pages on more complex migrations, such as splitting the user populace into several stores (on different storage) and/or separate mailhosts. That generally requires that you lock the user in LDAP (perhaps deferring his incoming mail for later processing into the new location), migrate his mailbox, rewrite the pointers from LDAP, reenable account. The devil is in the details, for both methods. For the latter, see Wiki; for the former I'll elaborate a bit here
1) To avoid any surprises, you should stop the messaging services before making the filesystem switch, finalize the data migration (probably with prepared data already mostly correct in the new hierarchy before you shut down the server, just resync'ing the recent changes into new structure), make the switch and reenable the server. If this is a lightly-used server which can tolerate some downtime - good for you If it is a production server, you should schedule some time when it is not very used so you can shut it down, and try to be fast - so perhaps practice on a test system or a clone first.
I'd strongly recommend taking this adventure in small reversible steps, using snapshots and backups, and renaming old files and directories instead of removing them - until you're sure it all works, at least.
2) If your current setup already includes a message store on ZFS, and it is large enough for size to be a problem, you can save some time and space by tricks that lead to direct re-use of existing files as if they are the dataset with a prepopulated message store.
* If this is a single dataset with lots of irrelevant data (i.e. one dataset for the messaging local zone root with everything in it, from OS to mailboxes) you can try zfs-cloning a snapshot of the existing filesystem and moving the message files to that clone's root (eradicating all irrelevant directories and files on the clone). Likewise, you'd remove the mailbox files on the original system (when the time is right, and after sync-ing).
* If this is already a dedicated store dataset which contains the directories like dbdata/    mboxlist/  partition/ session/   and which you want to split further to store just some files (indices, databases) separately, you might find it easier to just make new filesystem datasets with proper recordsizes and relocate these files there, and move the partition/primary to the remaining dataset's root, as above. In our setups, the other directories only take up a few megabytes and are not worth the hassle of cloning - which you can also do for larger setups (i.e. make 4 clones and make different data at each one's root). Either way, when you're done, you can and should make sure that these datasets can mount properly into the hierarchy, yielding the pathnames you need.
3) You might also look into separating the various log-file directories into datasets, perhaps with gzip-9 compression. In fact, to reduce needed IOPS and disk space at expense of available CPU-time, you might use lightweight compression (lzjb) on all messaging data, and gzip on WORM data sets - local zone, but not global OS, roots; logs; etc. Structured databases might better be left without compression, especially if you use reduced record sizes - they might just not compress enough to make a difference, just burning CPU cycles. Though you could look into "zle" compression which would eliminate strings of null bytes only - there's lots of these in fresh database files.
4) If you need to recompress the data as suggested in point (3), or if you migrate from some other storage to ZFS, rsync may be your friend (at least, if your systems don't rely on ZFS/NFSv4 ACLs - in that case you're limited to Solaris tar or cpio, or perhaps to very recent rsync versions which claim ACL support). Namely, I'd suggest "rsync -acvPHK --delete-after $SRC/ $DST/" with maybe some more flags added for your needs. This would retain the hardlink structure which Messaging server uses a lot, and with "-c" it verifies file contents to make sure you've copied everything over (i.e. if a file changes without touching the timestamp).
Also, if you were busy preparing the new data hierarchy with a running server, you'd need to rsync old data to new while the services are down. Note that reading and comparing the two structures can take considerable time - translating to downtime for the services.
Note that if you migrate from ZFS to ZFS (splitting as described in (2)), you might benefit from "zfs diff" if your ZFS version supports it - this *should* report all ofjects that changes since the named snapshot, and you can try to parse and feed this to rsync or some other migration tool.
Hope this helps and you don't nuke your system,
//Jim Klimov