Parallel backup vs section backup

Hi, i would like to know whats the difference between a configured backup with parallel channels and the "section size" parameter in these cases.
CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO BACKUPSET;
run{
ALLOCATE CHANNEL disk1 DEVICE TYPE DISK FORMAT '/disk1/backups/%U';
ALLOCATE CHANNEL disk2 DEVICE TYPE DISK FORMAT '/disk2/backups/%U';
ALLOCATE CHANNEL disk3 DEVICE TYPE DISK FORMAT '/disk3/backups/%U';
ALLOCATE CHANNEL disk4 DEVICE TYPE DISK FORMAT '/disk4/backups/%U';
BACKUP TABLESPACE USERS;
comparated with this:
CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO BACKUPSET;
run{
ALLOCATE CHANNEL disk1 DEVICE TYPE DISK FORMAT '/disk1/backups/%U';
ALLOCATE CHANNEL disk2 DEVICE TYPE DISK FORMAT '/disk2/backups/%U';
ALLOCATE CHANNEL disk3 DEVICE TYPE DISK FORMAT '/disk3/backups/%U';
ALLOCATE CHANNEL disk4 DEVICE TYPE DISK FORMAT '/disk4/backups/%U';
BACKUP TABLESPACE USERS SECTION SIZE 25G;
In this case could SECTION SIZE parameter improve performance?
Thanks.
Paulo.

Parallel backup would be helpful if your USERS tablespace have multiple datafiles. If it has only one datafile the RMAN will not use parallelism and the file would be backup by single slave process.
To overcome this limitation RMAN has SECTION parameter which helps a single large datafile to be backed up in parallel. So if your datafile is of 100GB then SECTION SIZE 25G will cause RMAN to distribute the backup of each section (part or range of blocks of datafile) to different channels/slaves.

Similar Messages

Time Machine doesn't do External HD to External HD Live parallel backups...

I didn't know where else to post this...
I have 12GB of external storage devices attached to my MacBook Pro 17. They are a variety of sizes, and I'm trying to figure out how to use some for storage, and the rest for backing up those storage drives. I don't think Time Machine supports this...
so, I'm looking for a Time Machine styler backup software recommendation that also allows you to have live parallel backups from one external drive to another. I've checked out all the obvious things - SuperDuper, Data Backup, Backup, and others. They only support backups of the INTERNAL hard drive.
If that doesn't work, I'll have to resort to scheduled incremental back ups, but again, I can't find software that will do external to external backups.
Does anyone know of a Backup utility like Time Machine but for EXTERNAL Drives.

SFjoe1966 wrote:
I didn't know where else to post this...
I have 12GB of external storage devices attached to my MacBook Pro 17. They are a variety of sizes, and I'm trying to figure out how to use some for storage, and the rest for backing up those storage drives. I don't think Time Machine supports this...
Time Machine can back up external drives, if they're formatted HFS+, along with your internal, but only to a single volume. So unless you have one of those big enclosures that you can configure as a monster hardware +Concatenated RAID set,+ Time Machine may not be your whole solution.
so, I'm looking for a Time Machine styler backup software recommendation that also allows you to have live parallel backups from one external drive to another. I've checked out all the obvious things - SuperDuper, Data Backup, Backup, and others. They only support backups of the INTERNAL hard drive.
I don't use SuperDuper, but I'd be very surprised if it can't back up external HDs. The similar CarbonCopyCloner, which I do use in addition to Time Machine, can. But it only backs-up one volume at a time, so you'd need one "task" for each. Depending on the sizes, it could back up, say, two 500 GB disks to a single 1 TB drive (I don't know offhand if it would have to be partitioned, but that would probably be wise).
With that much data, your best bet may be a combination: Time Machine for your OSX volume and one or more externals (probably the ones with the most changes), and CCC, SD, or others for the other volumes.
You might want to review the Time Machine Tutorial and perhaps browse Time Machine - Frequently Asked Questions (or use the link in *User Tips* at the top of this forum).
Also see Kappy's post on Basic Backup, complete with links to the web sites of each product.

Parallel Backup script written in python

I'm writing a backup script in python, based on one I wrote in BASH earlier on. Both of these leverage rsync, but I decided to move to a python implementation because python is much more flexible than BASH. My goals with the new implementation are to back up the system, home folder, and my documents folder to a set of multiple primary, secondary, and tertiary disks, respectively. The method is as follows:
1. check for the existence of disks and create folders which will contain mountpoints for each category of disk.
2. decrypt and mount disks found under subfolders within those category folders, creating the mountpoints if they don't exist.
3. syncronize the aforementioned data to the mounted disks using rsync, doing all three classes of disk in parallel.
4. unmount and close disks
This is really my first serious python program, and I realize that it's a bit complicated. My code is rather sloppy, as well, perhaps understandably so given my novice status. My only other programming experience is with BASH scripts, but I digress.
Here is the code for the script (about 250 lines). It is written as a series of functions, and I'm uncertain as to whether functions or objects would work better. Additionally, I'm sure there's a python function provided by the os module analogous to the sync system call, but I've yet to find it in my python desk reference. The backup functions need work, and I'm still trying to figure out how to get them to loop through the mounted disks in each folder and copy to them. I require assistance in determining how to write the backup functions to do as outlined above, and how to run them in parallel. This is still a work in progress, mind.
#!/usr/bin/python
# Backup Script
#### preferences ####
# set your primary/secondary backup disks, encryption, and keyfile (if applicable) here.
# backup disks
# primary and secondary backups. As used here,
# primary backups refer to backups of the entire system made to an external drive
# secondary backups refer to backups made of individual folders, such as documents
# primary backup disks by UUID:
global PDISKS
PDISKS = ("/dev/disk/by-uuid/16d64026-28bd-4e1f-a452-74e76bb4d47b","")
# secondary backups by UUID.
global SDISKS
SDISKS = ()
# tertiary disks by UUID:
global TDISKS
TDISKS = ("/dev/disk/by-uuid/39543e6e-cf50-4416-9669-e97a6abd2a37","")
# backup paths
# these are the paths of the folders you wish to back up to secondary
# and tertiary disks, respectively. Primary disks are set to back up the
# contents of the root filesystem (/*). NO TRAILING SLASHES.
global SBACKUP
SBACKUP = "/home/bryant"
global TBACKUP
TBACKUP = "/home/bryant/docs"
# use encryption:
use_encryption = True
# keyfile
# set the full path to your keyfile here
# this assumes a single keyfile for all backup disks
# set this to None if you don't have a single keyfile for all of your backups
keyfile = "/usr/local/bin/backup.keyfile"
# import modules
import os, subprocess, sys
### preliminary functions ###
# these do the setup and post-copy work
def check_dirs():
"""checks that the folders which contain the mountpoints exist, creates them if they don't"""
print("checking for mountpoints...")
p = os.path.isdir("/mnt/pbackup")
if p == True:
print("primary mountpoint exists.")
elif p == False:
print("mountpoint /mnt/pbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/pbackup")
s = os.path.isdir("/mnt/sbackup")
if s == True:
print("secondary mountpoint exists.")
elif s == False:
print("mountpoint /mnt/pbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/sbackup")
t = os.path.isdir("/mnt/tbackup")
if t == True:
print("tertiary mountpoint exists.")
elif t == False:
print("mountpoint /mnt/tbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/tbackup")
def mount_disks():
"""mounts available backup disks in their respective subdirectories"""
pfolder = 1
sfolder = 1
tfolder = 1
pmapper = "pbackup" + str(pfolder)
smapper = "sbackup" + str(sfolder)
tmapper = "tbackup" + str(tfolder)
for pdisk in PDISKS:
e = os.path.islink(pdisk)
if e == True:
subprocess.call("sync",shell=True)
kf=os.path.isfile(keyfile)
if kf == True:
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + pdisk + " " + pmapper + " --key-file " + keyfile,shell=True)
if kf == False:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + pdisk + " " + pmapper,shell=True)
f = os.path.isdir("/mnt/pbackup/pbak" + str(pfolder))
if f == True:
subprocess.call("mount " + "/dev/mapper/" + pmapper + " /mnt/pbak" + str(pfolder),shell=True)
pfolder += 1
elif f == False:
os.mkdir("/mnt/pbackup/pbak" + str(pfolder))
subprocess.call("mount " + "/dev/mapper/" + pmapper + " /mnt/pbak" + str(pfolder),shell=True)
pfolder += 1
for sdisk in SDISKS:
e = os.path.islink(sdisk)
if e == True:
subprocess.call("sync",shell=True)
kf=os.path.isfile(keyfile)
if kf == True:
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + sdisk + " " + smapper + " --key-file " + keyfile,shell=True)
if kf == False:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + sdisk + " " + smapper,shell=True)
f = os.path.isdir("/mnt/sbackup/sbak" + str(sfolder))
if f == True:
subprocess.call("mount " + "/dev/mapper/" + smapper + " /mnt/sbackup/sbak" + str(sfolder),shell=True)
sfolder += 1
elif f == False:
os.mkdir("/mnt/sbackup/sbak" + str(folder))
subprocess.call("mount " + "/dev/mapper/" + smapper + " /mnt/sbackup/sbak" + str(sfolder),shell=True)
sfolder += 1
for tdisk in TDISKS:
e = os.path.islink(tdisk)
if e == True:
subprocess.call("sync",shell=True)
kf=os.path.isfile(keyfile)
if kf == True:
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + tdisk + " " + tmapper + " --key-file " + keyfile,shell=True)
if kf == False:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + tdisk + " " + tmapper,shell=True)
f = os.path.isdir("/mnt/tbackup/tbak" + str(tfolder))
if f == True:
subprocess.call("mount " + "/dev/mapper/" + tmapper + " /mnt/pbak" + str(tfolder),shell=True)
tfolder += 1
elif f == False:
os.mkdir("/mnt/tbackup/tbak" + str(tfolder))
subprocess.call("mount " + "/dev/mapper/" + tmapper + " /mnt/tbak" + str(tfolder),shell=True)
tfolder += 1
def umount_disks():
"""unmounts and relocks disks"""
subprocess.call("umount /mnt/pbackup*",shell=True)
subprocess.call("umount /mnt/sbackup*",shell=True)
subprocess.call("umount /mnt/tbackup*",shell=True)
subprocess.call("cryptsetup luksClose /dev/mapper/pbackup*",shell=True)
subprocess.call("cryptsetup luksClose /dev/mapper/sbackup*",shell=True)
subprocess.call("cryptsetup luksClose /dev/mapper/tbackup*",shell=True)
def check_disks():
"""checks to see how many disks exist, exits program if none are attached"""
pdisknum = 0
sdisknum = 0
tdisknum = 0
for pdisk in PDISKS:
p = os.path.islink(pdisk)
if p == True:
pdisknum += 1
elif p == False:
print("disk " + pdisk + " not detected.")
for sdisk in SDISKS:
s = os.path.islink(sdisk)
if s == True:
sdisknum += 1
elif s == False:
print("disk " + sdisk + " not detected.")
for tdisk in TDISKS:
t = os.path.islink(tdisk)
if t == True:
tdisknum += 1
elif t == False:
print("disk " + tdisk + " not detected.")
total = pdisknum + sdisknum + tdisknum
if total == 0:
print("ERROR: no disks detected.")
sys.exit()
elif total > 0:
print("found " + str(total) + " attached backup disks")
print(str(pdisknum) + " Primary")
print(str(sdisknum) + " secondary")
print(str(tdisknum) + " tertiary")
return total, pdisknum, sdisknum, tdisknum
### backup functions ###
# these need serious work. Need to get them to loop through available mounted
# disks in their categories and then execute rsync
def pbackup():
"""calls rsync to backup the entire system to all pdisks"""
dirs = os.listdir("/mnt/pbackup")
for dir in dirs:
m = os.path.ismount(dir)
if m == True:
subprocess.call("sync",shell=True)
print("syncing disks with rsync...")
# subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} /* /mnt/pbackup/" + dir + "/ --exclude={/sys/*,/mnt/*,/proc/*,/dev/*,/lost+found,/media/*,/tmp/*,/home/*/.gvfs/*,/home/*/downloads/*,/opt/*,/run/*",shell=True)
print("test1")
subprocess.call("sync",shell=True)
print("disk synced with disk " + pdisk + ".")
print("sync with disk " + pdisk + " complete.")
elif m == False:
continue
def sbackup():
"""calls rsync to backup everything under SBACKUP folder to all sdisks"""
dirs = os.listdir("/mnt/sbackup")
for dir in dirs:
m = os.path.ismount(dir)
if m == True:
subprocess.call("sync",shell=True)
# subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} SBACKUP/* /mnt/sbackup/" + dir + "/",shell=True)
print("test2")
subprocess.call("sync",shell=True)
print("disk synced with disk " + pdisk + ".")
print("sync with disk " + sdisk + " complete.")
elif m == False:
continue
def tbackup():
"""calls rsync to backup everything under TBACKUP folder to all tdisks"""
dirs = os.listdir("/mnt/tbackup")
for dir in dirs:
m = os.path.ismount(dir)
if m == True:
subprocess.call("sync",shell=True)
# subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} TBACKUP/* /mnt/sbackup/" + dir + "/",shell=True)
print("test3")
subprocess.call("sync",shell=True)
print("disk synced with disk " + pdisk + ".")
print("sync with disk " + sdisk + " complete.")
elif m == False:
continue
#### main ####
# check for root access:
r=os.getuid()
if r != 0:
print("ERROR: script not run as root.\n\tThis script MUST be run as root user.")
sys.exit()
elif r == 0:
# program body
check_dirs()
check_disks()
mount_disks()
# pbackup()
# sbackup()
tbackup()
umount_disks()
print("backup process complete.")
Last edited by ParanoidAndroid (2013-08-07 20:01:07)

I've run into a problem on line 149. I'm asking the program to list the directories under the top-level backup directories under /mnt, check to see if each one is a mountpoint, and if it is unmount it. It does this, but it appears to recurse into the directories under the directories I'm asking it to check. The output is:
checking for mountpoints...
primary mountpoint exists.
secondary mountpoint exists.
tertiary mountpoint exists.
disk /dev/disk/by-uuid/16d64026-28bd-4e1f-a452-74e76bb4d47b not detected.
found 1 attached backup disks
0 Primary
0 secondary
1 tertiary
keyfile found. Using keyfile to decrypt...
mounting tbackup1 at /mnt/tbak1
test3
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
not a mountpoint
Device /dev/mapper/pbackup* is not active.
Device /dev/mapper/sbackup* is not active.
backup process complete.
here is the code for the entire script. It's been much modified from the previously posted version, so I included all of the code versus the section in question for reference. As I said, the section that seems to be causing the issue is on line 149.
#!/usr/bin/python
# Backup Script
#### preferences ####
# set your primary/secondary backup disks, encryption, and keyfile (if applicable) here.
# backup disks
# primary and secondary backups. As used here,
# primary backups refer to backups of the entire system made to an external drive
# secondary backups refer to backups made of individual folders, such as documents
# primary backup disks by UUID:
global PDISKS
PDISKS = ["/dev/disk/by-uuid/16d64026-28bd-4e1f-a452-74e76bb4d47b"]
# secondary backups by UUID.
global SDISKS
SDISKS = []
# tertiary disks by UUID:
global TDISKS
TDISKS = ["/dev/disk/by-uuid/39543e6e-cf50-4416-9669-e97a6abd2a37"]
# backup paths
# these are the paths of the folders you wish to back up to secondary
# and tertiary disks, respectively. Primary disks are set to back up the
# contents of the root filesystem (/*). NO TRAILING SLASHES.
global SBACKUP
SBACKUP = "/home/bryant"
global TBACKUP
TBACKUP = "/home/bryant/docs"
# use encryption:
use_encryption = True
# keyfile
# set the full path to your keyfile here
# this assumes a single keyfile for all backup disks
# set this to None if you don't have a single keyfile for all of your backups
keyfile = "/usr/local/bin/backup.keyfile"
# import modules
import os, subprocess, sys
### preliminary functions ###
# these do the setup and post-copy work
def check_dirs():
"""checks that the folders which contain the mountpoints exist, creates them if they don't"""
print("checking for mountpoints...")
if os.path.isdir("/mnt/pbackup"):
print("primary mountpoint exists.")
else:
print("mountpoint /mnt/pbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/pbackup")
if os.path.isdir("/mnt/sbackup"):
print("secondary mountpoint exists.")
else:
print("mountpoint /mnt/pbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/sbackup")
if os.path.isdir("/mnt/tbackup"):
print("tertiary mountpoint exists.")
else:
print("mountpoint /mnt/tbackup does not exist.\n\tcreating...")
os.mkdir("/mnt/tbackup")
def mount_disks(wdisk):
"""mounts available backup disks in their respective subdirectories"""
pfolder = 1
sfolder = 1
tfolder = 1
pmapper = "pbackup"
smapper = "sbackup"
tmapper = "tbackup"
if wdisk == "p":
for pdisk in PDISKS:
if os.path.islink(pdisk):
subprocess.call("sync",shell=True)
if os.path.isfile(keyfile):
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + pdisk + " " + pmapper + str(pfolder) + " --key-file " + keyfile,shell=True)
else:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + pdisk + " " + pmapper + str(pfolder),shell=True)
if os.path.isdir("/mnt/pbackup/pbak" + str(pfolder)):
print("mounting " + pmapper + str(pfolder) + " at /mnt/pbak" + str(pfolder))
subprocess.call("mount " + "/dev/mapper/" + pmapper + str(pfolder) + " /mnt/pbackup/pbak" + str(pfolder),shell=True)
pfolder += 1
else:
os.mkdir("/mnt/pbackup/pbak" + str(pfolder))
subprocess.call("mount " + "/dev/mapper/" + pmapper + str(pfolder) + " /mnt/pbackup/pbak" + str(pfolder),shell=True)
pfolder += 1
elif wdisk == "s":
for sdisk in SDISKS:
if os.path.islink(sdisk):
subprocess.call("sync",shell=True)
if os.path.isfile(keyfile):
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + sdisk + " " + smapper + str(sfolder) + " --key-file " + keyfile,shell=True)
else:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + sdisk + " " + smapper + str(sfolder),shell=True)
if os.path.isdir("/mnt/sbackup/sbak" + str(sfolder)):
print("mounting " + smapper + str(sfolder) + " at /mnt/sbak" + str(sfolder))
subprocess.call("mount " + "/dev/mapper/" + smapper + str(sfolder) + " /mnt/sbackup/sbak" + str(sfolder),shell=True)
sfolder += 1
else:
os.mkdir("/mnt/sbackup/sbak" + str(folder))
subprocess.call("mount " + "/dev/mapper/" + smapper + str(sfolder) + " /mnt/sbackup/sbak" + str(sfolder),shell=True)
sfolder += 1
elif wdisk == "t":
for tdisk in TDISKS:
if os.path.islink(tdisk):
subprocess.call("sync",shell=True)
if os.path.isfile(keyfile):
print("keyfile found. Using keyfile to decrypt...")
subprocess.call("sudo cryptsetup luksOpen " + tdisk + " " + tmapper + str(tfolder) + " --key-file " + keyfile,shell=True)
else:
print("keyfile not found or keyfile not set. \t\nAsking for passphrase...")
subprocess.call("sudo cryptsetup luksOpen " + tdisk + " " + tmapper + str(tfolder),shell=True)
if os.path.isdir("/mnt/tbackup/tbak" + str(tfolder)):
print("mounting " + tmapper + str(tfolder) + " at /mnt/tbak" + str(tfolder))
subprocess.call("mount " + "/dev/mapper/" + tmapper + str(tfolder) + " /mnt/tbackup/tbak" + str(tfolder),shell=True)
if os.path.islink(tdisk):
tfolder += 1
else:
os.mkdir("/mnt/tbackup/tbak" + str(tfolder))
subprocess.call("mount " + "/dev/mapper/" + tmapper + " /mnt/tbackup/tbak" + str(tfolder),shell=True)
tfolder += 1
def umount_disks():
"""unmounts and relocks disks"""
pdirs = os.listdir("/mnt/pbackup")
sdirs = os.listdir("/mnt/sbackup")
tdirs = os.listdir("/mnt/tbackup")
for pdir in pdirs:
if os.path.ismount("/mnt/pbackup/" + pdir):
subprocess.call("umount /mnt/pbackup/" + pdir,shell=True)
else:
print("not a mountpoint")
for sdir in sdirs:
if os.path.ismount("/mnt/sbackup/" + sdir):
subprocess.call("umount /mnt/sbackup/" + sdir,shell=True)
else:
print("not a mountpoint")
for tdir in tdirs:
if os.path.ismount("/mnt/tbackup/" + tdir):
subprocess.call("umount /mnt/tbackup/" + tdir,shell=True)
else:
print("not a mountpoint")
subprocess.call("cryptsetup luksClose /dev/mapper/pbackup*",shell=True)
subprocess.call("cryptsetup luksClose /dev/mapper/sbackup*",shell=True)
subprocess.call("cryptsetup luksClose /dev/mapper/tbackup*",shell=True)
def check_disks():
"""checks to see how many disks exist, exits program if none are attached"""
pdisknum = 0
sdisknum = 0
tdisknum = 0
for pdisk in PDISKS:
if os.path.islink(pdisk):
pdisknum += 1
else:
print("\ndisk " + pdisk + " not detected.")
for sdisk in SDISKS:
if os.path.islink(sdisk):
sdisknum += 1
else:
print("\ndisk " + sdisk + " not detected.")
for tdisk in TDISKS:
if os.path.islink(tdisk):
tdisknum += 1
else:
print("\ndisk " + tdisk + " not detected.")
total = pdisknum + sdisknum + tdisknum
if total == 0:
print("\nERROR: no disks detected.")
sys.exit()
elif total > 0:
print("found " + str(total) + " attached backup disks")
print(str(pdisknum) + " Primary")
print(str(sdisknum) + " secondary")
print(str(tdisknum) + " tertiary")
return total, pdisknum, sdisknum, tdisknum
### backup functions ###
# these need serious work. Need to get them to loop through available mounted
# disks in their categories and then execute rsync
def pbackup():
"""calls rsync to backup the entire system to all pdisks"""
dirs = os.listdir("/mnt/pbackup")
for dir in dirs:
if os.path.ismount("/mnt/pbackup/" + dir) == True:
subprocess.call("sync",shell=True)
print("syncing disks with rsync...")
subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} /* /mnt/pbackup/" + dir + "/ --exclude={/sys/*,/mnt/*,/proc/*,/dev/*,/lost+found,/media/*,/tmp/*,/home/*/.gvfs/*,/home/*/downloads/*,/opt/*,/run/*",shell=True)
subprocess.call("sync",shell=True)
else:
continue
def sbackup():
"""calls rsync to backup everything under SBACKUP folder to all sdisks"""
dirs = os.listdir("/mnt/sbackup")
for dir in dirs:
if os.path.ismount("/mnt/sbackup/" + dir):
subprocess.call("sync",shell=True)
subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} " + SBACKUP + "/* /mnt/sbackup/" + dir + "/",shell=True)
subprocess.call("sync",shell=True)
else:
continue
def tbackup():
"""calls rsync to backup everything under TBACKUP folder to all tdisks"""
dirs = os.listdir("/mnt/tbackup")
for dir in dirs:
if os.path.ismount("/mnt/tbackup/" + dir):
subprocess.call("sync",shell=True)
subprocess.call("rsync --progress --human-readable --numeric-ids --inplace --verbose --archive --delete-after --hard-links --xattrs --delete --compress --skip-compress={*.jpg,*.bz2,*.gz,*.tar,*.tar.gz,*.ogg,*.mp3,*.tar.xz,*.avi} " + TBACKUP + "/* /mnt/sbackup/" + dir + "/",shell=True)
subprocess.call("sync",shell=True)
else:
continue
#### main ####
# check for root access:
r=os.getuid()
if r != 0:
print("ERROR: script not run as root.\n\tThis script MUST be run as root user.")
sys.exit()
elif r == 0:
# program body
check_dirs()
d=check_disks()
if d[1] > 0:
mount_disks("p")
pbackup()
elif d[2] > 0:
mount_disks("s")
sbackup()
elif d[3] > 0:
mount_disks("t")
tbackup()
umount_disks()
print("backup process complete.")
Last edited by ParanoidAndroid (2013-08-11 00:32:02)

RMAN Parallel Backup Validate

Hi,
OS= Solaris 10
Database = 10gR2
I am using following script to validate the physical and logical corruption in the database through RMAN:
rman target /
RMAN> RUN {
CONFIGURE DEVICE TYPE DISK PARALLELISM 8;
ALLOCATE CHANNEL d1 DEVICE TYPE DISK;
ALLOCATE CHANNEL d2 DEVICE TYPE DISK;
ALLOCATE CHANNEL d3 DEVICE TYPE DISK;
ALLOCATE CHANNEL d4 DEVICE TYPE DISK;
BACKUP VALIDATE DATABASE;
RELEASE CHANNEL;
My machine has 4 CPUs.
I just want to verify that will the above RMAN script open up 4 channels in parallel and validate the database?
regards

Hi Fahd,
Can you please study the below link also from the forum.
Backup Time
Best regards,
Rafi.
http://rafioracledba.blogspot.com/

How does Parallels backup exclusion work with Time Machine?

Running Parallels version 9 there is a means by which you may (supposedly) request that a VM is not backed up via Time Machine (Mavericks). In the Parallel Desktop app, one chooses from the menu Virtual Machine -> Configure, selects the Security tab, and clicks in the box for "Time Machine Do not back up virtual machine.". Having done that, however, if you launch Time Machine preferences, click on "Options...", this brings up the window of "Exclude these items from backups." and there is nothing in the box indicating that the Parallels VM is being excluded. The Time Machine exclude list in that window on my machine is completely empty. I think this is pretty crummy. How do I know that the Parallels VM is really being excluded, and why doesn't it appear in the list ???

I'll follow up to my own post. I found that one can execute some Time Machine commands via the command line and so:
$ sudo tmutil isexcluded ~/Documents/Parallels/Windows\ 7.pvm/
[Excluded] /Users/myuserid/Documents/Parallels/Windows 7.pvm
It's unfortunate that there did not appear to be a "listexcluded" verb (sigh), and it's really terrible that one can see that something is excluded via the command-line, but it doesn't appear in the Time Machine whiz-bang graphical interface. I'd call that a major bug.

Parallelism backup in RMAN Orcale 10g RAC

Hi All,
I want to know how parallelism works for RMAN RAC
Can you forward me to some useful links where I can get this information.
One way of doing it is:
CONFIGURE DEVICE TYPE DISK PARALLELISM 2;
CONFIGURE CHANNEL 1 DEVICE TYPE DISK connect 'SYS/rac@node1';
CONFIGURE CHANNEL 2 DEVICE TYPE DISK connect 'SYS/rac@node2';
But in 10.2.0.2 onwards it says that we dont have to specify nodes. Oracle will automatically spread it accross node. is that true.. but doesnt seesm to be working?
node1> rman target /
run {
CONFIGURE RETENTION POLICY TO REDUNDANCY 1;
CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE CHANNEL DEVICE TYPE DISK MAXPIECESIZE 8G;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO'$RMANOUT/%F';
CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET PARALLELISM 2;
CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET;
CONFIGURE DEVICE TYPE DISK PARALLELISM 2;
allocate channel c1 device type disk ;
allocate channel c2 device type disk ;
backup incremental level 0
database format '$RMANOUT/%d_%s_%T_%c_DB0' tag="DB_LEV0";
backup current controlfile for standby format
'$RMANOUT/%d_%s_%T_CONTROL_STDBY' tag "CNTRL_STDBY";
sql "ALTER SYSTEM ARCHIVE LOG CURRENT";
backup archivelog from time='sysdate-5/24'
format '$RMANOUT/%d_%s_%T_%c_ARCH0' tag "ARCH_LEV0" ;
backup current controlfile format '$RMANOUT/%d_%s_%T_CONTROL' tag "CNTRLFL" ;
backup spfile format '$RMANOUT/%d_%s_%T_SPFILE' tag "SPFILE" ;
will the above script run parallel sessions in RMAN.
I queried gv$session I cant see 2 sessions, I can only see 1.

if you want to enable parallelism you have (so to say) three ways related to channel allocation:
1. in rman settings directly
configure default device type to disk;
configure device type disk parallelism 2;
configure channel c1 device type disk connect 'aaa/bbb@node1'
configure channel c2 device type disk connect 'aaa/bbb@node2'
2. or directly in rman script, something like:
RUN
allocate channel c1 device type disk connect 'XXX/YYY@node1';
allocate channel c1 device type disk connect 'XXX/YYY@node2';
backup database plus archived log delete input;
3. or you can use automatic workload balancing with setting:
configure device type <disk or sbt> parallelism <number of channels>;
and after that issue backup command.
parallelism you may monitor by checking gv$session_longops
also rman have option for 'node affinity awareness' which means that rman during backup determines which nodes in cluster have faster access to certain datafiles than to other datafiles.
pay attention that the granule of parallelization for backupsets is the backupset...inappropriate setting of the parameter "filesperset" can lead backup to be serialized...specially if you use tapes...

RMAN behavior under heavy load (parallel backups)

Hello all
Oracle 10g on Linux, Veritas NetBackup for Oracle - Release 6.5,
Many instances on same Host
Did you experience any problem when backup two or more instances on same host at the same time? Can RMAN/the catalog handle this heavy load?
Will potential errors be reported by RMAN? I log RMAN's output to a textfile, grep for <RMAN-> and mail the log if any <RMAN-> are found in the log?
Thanks for your answers Casi
Edited by: Casi on May 8, 2009 10:50 AM

Hello,
Casi wrote:
Did you experience any problem when backup two or more instances on same host at the same time? Can RMAN/the catalog handle this heavy load? The catalog will be fine: those are Oracle tables that have no problem with concurrency.
Although you could always run the backups without connecting to the catalog ( nocatalog ), and then only connecting when the backup is complete to resync the catalog -- that would prevent the holding of connections across the network (presuming your catalog is remote) for the durations of the backups, in addition to removing that connection to the catalog as a potential point of failure for the backup itself.
Will potential errors be reported by RMAN? I log RMAN's output to a textfile, grep for <RMAN-> and mail the log if any <RMAN-> are found in the log?Yes, any errors will be logged, and if you 'grep -i error <your_error_log>' you'll find those errors, if there are any.
I should imagine the only issues you'll have will be the resources to run concurrent backups, but that all depends on your hardware set-up.

HT201250 time machine parallels backup

Will time machine to an external drive include Parallels and all the Windows files?

What I do is set the flag inside each Parallels VM to exclude that VM from Time Machine. Then, I archive my VM folder onto an external drive that is not used for Time Machine.
If you are using Windows more often than me and have files that you want backed up more regularly, you can configure Parallels to use your Mac home directory for equivalent places in Windows. Then, your user files would be backed up in Time Machine, in a slightly crude fashion. You have to make sure that your Windows applications are saving data to a place that actually lives on the Mac. This might take some investigation.

How to configure backup parallel in SAP with oracle database?

Dear SAP,
We need your suggestion on oracle database backup parallel using brbackup, our SAP system run on:
OS: UNIX - SUN Solaris SPARC 10
SAP: ECC 6.0 SR3
Database: Oracle 10.2.0.4
Our production database are growing rapidly, estimate growing is 2GB/days. Right now we backup our Production using one tape that estimated time to online backup is 4-5hours.
In there future, if we stayed this configuration. It will take long time to backup and restore it's not applicable. That's why we are planning to add more tapes during backup (4 tape parallel).
I read from help.sap, that we can configure this on $ORACLE_HOME/dbs/init<SID>.sap. in parameter:
compress = harware
backup_dev_type = tape
tape_address = (tape1, tape2, tape3, tape4)
tape_address_rew = (tape1, tape2, tape3, tape4)
http://help.sap.com/saphelp_nw04/helpdata/en/88/a2a041a5d4af4db168992639d65d3b/frameset.htm
My question,
If we want to backup parallel using 4 tapes, is this configuration that we need to do (init<SID>.sap) from sap side (DB13)?
Thanks and Best Regards,
Widi

Dear Mark,
Thanks for your help. actually I have one more question:
How do I restore my backup form this 4 tapes, if I'm using parallel backup?
Will the command same like I restore form 1 tape:
example:
brrestore u2013b beczqlnk.ant u2013m full
brrestore u2013a 5371-5372=/oracle/<SID>/oraarch/
Below are the paramter that I will configure:
- tape_address = /dev/rmt/0cbn
- compress = hardware
- backup_dev_type = tape
- exec_parallel = 4
- tape_size = 1600000
- tape_address = (dev/rmt/0mn, /dev/rmt/1mn, ..)
- tape_address_rew = (dev/rmt/0m, /dev/rmt/1m, ..)
Appreciate you help and suggestion on this.
Dear Venkat,
thanks for your suggestion

Parallels-how to delete prior TC/TM backups

MacBook Pro 10.5.4
TC 500 Gig
I have been running TC/TM Backup for a few months.
I have had TC/TM back up everything on my MBP internal drive - knowing full well that it backed up everything that changed - even slightly - as did my Parallels folder
Now I want to save/conserve some space.
I know how to exclude my Parallels folder from future TM/TC backups.
My question is --
How do I delete all of my previous Parallels files from my multiple previous TM/TC backups.
I know about hard links, I know about bundles ---
I just can't delete al of my previous Parallels backups.
Thank you.

Open Finder and locate the VM folder that you want to delete. Click the Time Machine icon on the Dock. Click on the monolithic file you need to delete. Click on the Gears button. Select delete and all instances of this file past and present will be permanently deleted.

Rman backup 11g rel2

Hello Guys
I was testing rman backup on oracle 11g rel2 , logging time for backup using rman as in 2 ways 1) full database backup
2)full database backup /parallel feature say "section size 200M" in script
i was expecting due to new 11g feature option 2 should complete faster ,but its reverse without using this feature
its running more fast (completing by 4-5 minutes less time)
I also tried taking backup of 1 datafile or 1 tablespace using both the option and also placing backupsets
on different folders/mountpoints ,but result is same
if you had worked on this feature ,please give details what is benefit of using this feature,i can see disadvantage here
Thanks

804719 wrote:
so without section size this operation is not actually parallelYes, it's not parallel for datafile, but it parallel for the entire job.
You can analyze the backup log. It should contains the records about opening several channels simultaneously, for instance:
Starting backup at 26/04/12 23:14:19
channel tape1: starting full datafile backup set
channel tape1: specifying datafile(s) in backup set
input datafile file number=00049 name=/home/oracle/oradata/...
channel tape1: starting piece 1 at 26/04/12 23:14:21
channel tape2: starting full datafile backup set
channel tape2: specifying datafile(s) in backup set
input datafile file number=00050 name=/home/oracle/oradata/...
channel tape2: starting piece 1 at 26/04/12 23:14:22
channel tape3: starting full datafile backup set
channel tape3: specifying datafile(s) in backup set
All within 3 seconds, and working in parallel...

Moving Complete Keynote with linked media to a backup machine

Hi all, I am using keynote for a major live event presentation. It's mostly HD videos with PSD photos as graphics. I've designed the show so that everything is linking to my media. Everything is working great except that I need to have this running on a parallel backup machine during the show. When I copy the media to a new drive and then open the project I can't figure out how to tell keynote where the footage now lives. I get a progress bar that stay's on forever. How can I do this properly? Do I need to embed the media on the secondary machine? The file size will be in the 8 gig or more range. If I have to will I have problems running this live?
Thanks

You should have thought about that before installing the SSD. Unfortunately when you have a configuration with more than one primary job, your backing up task becomes utterly more complicated. Now you need a mirror drive for both drives, and/or backups more than twice the capacity. Time to pair down the files you don't need frequently to some DVDs, or another drive*:
http://www.macmaps.com/backup.html
And I assume you are aware SSDs can't fully be erased?

Backup fail with VSSwriter error

Hello,
we currently are using Symantec Backup on tape and Windows Server Backup to make two differents backups on a SBS 2008.
We are getting failed backup randomly during the week for both. Sometimes one is succeed and not the others.
We have found several post about the VSS issues and how to fix it but nothing has changed.
What we have done:
- check the VSS writer and all is in "no error" state
- check the settings of the backups
- restart the VSS services
- Configure the Shadow Copy on the drives
- reboot the server
- Tried several solutions from forums
What we can see in the event viewer
ID 12289
"Volume Shadow Copy Service error: Unexpected error VSS_E_WRITER_STATUS_NOT_AVAILABLE. An older active writer session state is being overwritten by a newer session. The most common cause is that the number of parallel backups has exceeded the maximum
supported limit. hr = 0x80042409.
Operation:
Aborting Writer
ID 7001
VssAdmin: Unable to create a shadow copy: The shadow copy provider had an error. Please see the system and application event logs for more information.
Command-line: 'C:\Windows\system32\vssadmin.exe Create Shadow /AutoRetry=15 /For=C:\'. "
ID 12341
Volume Shadow Copy Warning: VSS spent 75 seconds trying to flush and hold the volume \\?\Volume{c22d5d46-0540-11e0-9496-806e6f6e6963}\. This might cause problems when other volumes in the shadow-copy set timeout waiting for the release-writes phase,
and it can cause the shadow-copy creation to fail. Trying again when disk activity is lower may solve this problem.
Operation:
Executing Asynchronous Operation
Context:
Current State: flush-and-hold writes
Volume Name: \\?\Volume{c22d5d46-0540-11e0-9496-806e6f6e6963}\"
Some will post the link of "how to troubleshoot VSSwriter issue": we have already used it.
Does someone have a fresh idea about this issue ?
Thanks in advance

Hi Nerea,
Before going further, would you please let me confirm if temporarily disable either of the two backup, will
another backup operation run smoothly? In other words, if temporarily disable Symantec Backup, will Windows Server Backup always run successfully? Or if disable Windows Server Backup, will Symantec Backup always run successfully? Please check if Windows Server
Backup conflicts with Symantec Backup.
Based on the error message, please run
chkdsk command and check if there is any issue in the disk. Then use
Vssadmin resize shadowstorage to resizes the maximum amount of storage space that can be used for shadow copy storage. And check if this issue still persists.
In addition, would you please let me know whether re-register Volume VSS components? If this issue still persists,
please re-register and monitor the result.
From command prompt:
cd windows\system32
Net stop vss
Net stop swprv
regsvr32 ole32.dll
regsvr32 vss_ps.dll
Vssvc /Register
regsvr32 /i swprv.dll
regsvr32 /i eventcls.dll
regsvr32 es.dll
regsvr32 stdprov.dll
regsvr32 vssui.dll
regsvr32 msxml.dll
regsvr32 msxml3.dll
regsvr32 msxml4.dll
net start swprv
net start vss
By the way, did the Symantec Backup and Windows Server Backup run at the same time? Or before one backup finished,
had another backup started?
Hope this helps.
Best regards,
Justin Gu

Parrallelization of RMAN Full Database Backup Confusion

Hi,
I am running Oracle 10gR2 on Solaris 5.10 on sparc machine having 16 CPUs.
I am using following RMAN script to take full database backup:
#!/bin/bash
ORACLE_HOME=/u01/app/oracle/product/10g
PATH=$ORACLE_HOME/bin:$PATH
ORACLE_SID=mtcedwp
export ORACLE_HOME ORACLE_SID PATH
rman target / <<EOF
run
CONFIGURE RETENTION POLICY TO REDUNDANCY 1;
CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET PARALLELISM 8;
CONFIGURE CONTROLFILE AUTOBACKUP ON;
ALLOCATE CHANNEL d1 DEVICE TYPE DISK;
ALLOCATE CHANNEL d2 DEVICE TYPE DISK;
ALLOCATE CHANNEL d3 DEVICE TYPE DISK;
ALLOCATE CHANNEL d4 DEVICE TYPE DISK;
ALLOCATE CHANNEL d5 DEVICE TYPE DISK;
ALLOCATE CHANNEL d6 DEVICE TYPE DISK;
ALLOCATE CHANNEL d7 DEVICE TYPE DISK;
ALLOCATE CHANNEL d8 DEVICE TYPE DISK;
BACKUP TAG ATHEER_FULL_DB DATABASE FORMAT '/u01/BACKUPS/WEEKLY_DB_BACKUP/backup_%U';
CROSSCHECK BACKUPSET;
CROSSCHECK BACKUP;
DELETE NOPROMPT OBSOLETE;
RELEASE CHANNEL d1;
RELEASE CHANNEL d2;
RELEASE CHANNEL d3;
RELEASE CHANNEL d4;
RELEASE CHANNEL d5;
RELEASE CHANNEL d6;
RELEASE CHANNEL d7;
RELEASE CHANNEL d8;
EOFWill the above script take the 8-way parrallel backup? or do I need to configure location for every channel separately?
regards

This channel can be allocated manually or can be pre-configured by using
automatic channel allocation.
The number of allocated channels determines the maximum degree of
parallelism that is used during backup restore or recovery. For example if
you allocate 8 channels for a backup operation 8 background processes for the
operation can run concurrently.
Parallelism of backup sets allocates multiple channels and assigns
files to specific channels. You can configure parallel backups by setting a
PARALLELISM option of the CONFIGURE command to a value greater than 1 or by
manually allocating multiple channels.
http://download.oracle.com/docs/cd/B19306_01/backup.102/b14191/rcmconc1.htm#i1012864

Online Database Backup Error

Hi,
While trying to take the Online database backup , i get the following error
BR0278E Command output of 'LANG=C cd /oracle/CPD/sapdata1/sr3_1 && echo sr3.data1 | LANG=C cpio -ovB > /dev/rmt0.1':
<sr3.data1> ?
BR0280I BRBACKUP time stamp: 2007-09-14 15.42.29
BR0279E Return code from 'LANG=C cd /oracle/CPD/sapdata1/sr3_1 && echo sr3.data1 | LANG=C cpio -ovB > /dev/rmt0.1': 255
BR0222E Copying /oracle/CPD/sapdata1/sr3_1/sr3.data1 to/from /dev/rmt0.1 failed due to previous errors
Is there anything related with the expiration period as i had made changes withthat too.
Kindly suggest what needs to be done?
Regards,
Radeesh V

Hi Radeesh,
Pls check the following
1)The file to be backed up cannot be read. this can be due to hardware problems on the disk on which the file is stored.
2)The file to be backed up cannot be written to the volume. this can be due to hardware problems with the volume or the backup device.
3)The backup device is not responding, for example if no volume (tape) was inserted in the backup device.
4) The input or output file cannot be opened. The operating system user is experiencing authorization problems.
5) The physical end of the tape was reached. However, since a parallel backup is performed or the option -c force was used, cpio cannot request a cpio continuation volume. The tape size (parameter tape_size) must be reduced.
Hope this is useful
Regards
Umesh

Parallel backup vs section backup

Similar Messages

Maybe you are looking for