Backup database and external files

We have a web application that stores files in a separate folder outside the database, we want to do real time backup (while the users are connected to the application). If we run a scheduled task to backup the database and then the files, then the database and files will be out of sync, because if the database backup takes 5 mins for instance, then we will be backing up the files that are related to some records that were not added to backup.
What is the best way to do real time backup for database and external files so that we have them both in synch?

user11738458 wrote:
We have a web application that stores files in a separate folder outside the database, we want to do real time backup (while the users are connected to the application). If we run a scheduled task to backup the database and then the files, then the database and files will be out of sync, because if the database backup takes 5 mins for instance, then we will be backing up the files that are related to some records that were not added to backup.
What is the best way to do real time backup for database and external files so that we have them both in synch?Then why not backup only those files that match up to a certain point in time.
Exclude all newer files.

Similar Messages

2005 database and log file locations

Is there a SQL query to list where exactly the database and log files reside for all databases on an instance (sql server 2005)?

You can query the Information form DMV
sys.master_files (Transact-SQL)
SELECT *
FROM [sys].[master_files]
Olaf Helper
[ Blog] [ Xing] [ MVP]

Create server role for backups (BACKUP DATABASE and RESTORE HEADERONLY)

Hello-
I wish to create a server role that will allow any users assigned to it to be able to issue a BACKUP DATABASE and RESTORE HEADERONLY for any database on the server. I've been searching Google and MSDN but haven't had much luck. I can create the
user (CREATE SERVER ROLE backupuser;) but don't really know how to assign the permissions to it that I want. I wish to keep the permissions as narrow as possible for good practice; I could just assign the user to the sysadmin group but that shouldn't
be necessary.
If anyone can help, I would much appreciate it.

Whenever you need to find out the required permissions for a command, the best way is to look up these commands in Books Online, and to the Permissions/Securities section. This is what I always do when people asks these questions, because I rarely knows
the answer by heart.
So, let's see what the topic for BACKUP has to say:
   BACKUP DATABASE and BACKUP LOG permissions default to members of the
   sysadmin fixed server role and the db_owner and db_backupoperator fixed
   database roles
The permission section for RESTORE HEADERONLY says:
   Beginning in SQL Server 2008, obtaining information about a backup set or
   backup device requires CREATE DATABASE permission. For more information,
   see GRANT Database Permissions (Transact-SQL).
So it seems that if you want a user that has server-level rights to back up databases, this must be sysadmin, which is a little disappointing. One possibility is to package the BACKUP command in a stored procedure, which you sign with a certificate. Then
you create a login from the certiticate, and add that login to sysadmin. BACKUP accepts parameters for many of its arguments, so you don't have to build dynamic SQL. Still it is not really a satisfactory solution, since this user will have to work with
BACKUP "commands" that will be procedure calls.
Erland Sommarskog, SQL Server MVP, [email protected]

T-SQL Code to backup database with multiple files - Syntax error

Hello,
I'm trying to backup a database into multiple files but I'm getting a syntax error.
Here is the code:
declare @DBName varchar(100)
declare @DBFileName varchar(256)
declare @FolderName varchar(256)
declare @Path varchar(100)
set @Path = '\\Backup-Server\Test\'
set @DBName = 'DayNite'
set @DBFileName = 'DayNite-Full' + '-' + (SELECT CONVERT(char(10), GetDate(),110)) + '-' + 'P'
set @FolderName =(SELECT CONVERT(char(10), GetDate(),110))
set @Path = @Path + @FolderName + '\'
EXEC master.dbo.xp_create_subdir @Path
--Calculate broken files for BACKUP DATBASE Function
/*declare @dbsize int
set @dbsize = (SELECT ((size*8)/1024)/1000 as SizeGB FROM sys.database_files WHERE file_id = '1')
set @dbsize = @dbsize / 4
print @dbsize*/
EXEC
BACKUP DATABASE [test] TO
DISK = @Path + @DBFileName + '1.bak',
DISK = @Path + @DBFileName + '2.bak',
DISK = @Path + @DBFileName + '3.bak',
DISK = @Path + @DBFileName + '4.bak',
DISK = @Path + @DBFileName + '5.bak',
DISK = @Path + @DBFileName + '6.bak',
DISK = @Path + @DBFileName + '7.bak',
DISK = @Path + @DBFileName + '8.bak',
DISK = @Path + @DBFileName + '9.bak',
DISK = @Path + @DBFileName + '10.bak',
DISK = @Path + @DBFileName + '11.bak',
DISK = @Path + @DBFileName + '12.bak',
DISK = @Path + @DBFileName + '13.bak'
WITH INIT , NOUNLOAD , NAME = 'DayNite Full Backup', NOSKIP , NOFORMAT

Made some slight modifications to your script and it should work(worked when tested).. if you wnat more complete solution -- google -OLA Hallengren backups -- you should get some good scripts..
declare @DBName varchar(100)
declare @DBFileName varchar(256)
declare @FolderName varchar(256)
declare @Path varchar(100)
set @Path = '\\Backup-Server\Test\'
set @DBName = 'DayNite'
set @DBFileName = 'DayNite-Full' +''+ '-' +''+ (SELECT CONVERT(char(10), GetDate(),110)) +''+ '-' +''+ 'P'
set @FolderName =(SELECT CONVERT(char(10), GetDate(),110))
set @Path = @Path +''+ @FolderName +''+ '\'
EXEC master.dbo.xp_create_subdir @Path
--Calculate broken files for BACKUP DATBASE Function
/*declare @dbsize int
set @dbsize = (SELECT ((size*8)/1024)/1000 as SizeGB FROM sys.database_files WHERE file_id = '1')
set @dbsize = @dbsize / 4
print @dbsize*/
select @Path,@DbFileName
declare @SQL nvarchar(2000)
Set @SQL ='Backup DATABASE [DAYNITE] TO DISK = '''+@Path +''+ @DBFileName +''+ '1.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '2.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '3.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '4.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '5.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '6.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '7.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '8.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '9.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '10.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '11.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '12.bak'',
DISK = '''+@Path +''+ @DBFileName +''+ '13.bak''
WITH INIT , NOUNLOAD , NAME = ''DayNite Full Backup'', NOSKIP , NOFORMAT '
print @SQL
exec (@SQL)
Hope it Helps!!

Backup internal and external hard drives-TC and offsite

I now have Mavericks 10.9.3 on my iMac with a 2TB (1.25 TB used) internal hard drive.
I also have some external drives attached to my iMac with older iPhoto libraries and other files (total file sizes 940 GB and 505 GB). I primarily use Aperture now on my internal hard drive, but still access those iPhoto libraries on my external drives on occasion. I have Time Machine backup my iMac internal hard drive on my 2TB Time Capsule regularly.
My primary question is in regards to getting another copy onto a larger external drive that covers my internal and external drives so I can have a backup off-site. Online backup services seem to always exclude external drives. So a physical drive I can have offsite seems to be the best option.
A year ago I changed the destination of my Time Machine backup to be on a 3TB external drive (backed up iMac internal hard drive and the two externals). However, when I changed the destination of the backup back to the Time Capsule, the TM initiated a brand new backup (it did not recall that I had backed up prior to that on my Time Capsule).
I want to backup monthly, if not quarterly for my off-site storage. But, if every time I change the destination drive for TM, a new backup profile is created, it will overwork my drives unnecessarily.
Is there a backup program or a process on "disk utility" I could run parallel to TM that I just use quarterly capturing only the changes/additions in those few months for both the internal and external hard drives? Also, is there a way to add an external drive to my Time Capsule that is solely used to wirelessly backup the two externals on a regular basis (i.e. keep the internal 2TB drive backing up to the Time Capsule; and the external hard drive attached to Time Capsule via USB used as the backup drive for the external hard drives)?
Summary: I need to backup regularly to the local Time Capsule/additional external hard drive. The data will come from my internal hard drive and my two external hard drives. I also want to do quarterly backups of the additions/changes to all three drives to have on an offsite external drive that I manually backup to quarterly. Any help is greatly appreciated.

Carbon Copy Cloner is not on the App store.
Correct.. it is not approved because Apple do not like the fact that CCC (and most likely superduper) which are the most popular backup software for Mac because it makes a bootable clone. Apple will never approve of that. But let me assure you that is the genius of it. If the internal disk fails, you simply boot from the external. It is $40 but you can use on all the computers in your home.
CCC is a clone.. ie when it does the backup, any changes on the drive are changed on the clone. It does not work like Time Machine which simply piles up incrementals until the drive fills up. The idea of CCC is a backup of the drive as it exists at any point in time. TM btw is also not a reliable archive, ie it thins backups constantly.. so you should never rely on it to archive old versions.. but in the middle of a project it does a good job to keep various versions of your files. That is why I specifically said in my last post do not stop using it.
and you can keep using the TC just for the internal drive.
Keep TM running to the TC.. that will then keep a current hourly incremental of your drive. You can set CCC in a way which is a lot more flexible. ie backup just at the end of the day. There is no need for constant hourly backups. So to answer the second question.. you are still using your TC and TM.. but I suggest you only backup the internal drive.
Please read a bit from forum expert Pondini on the value of clones and TM.
http://pondini.org/TM/Clones.html
That's why many folks use both Time Machine and a bootable clone, to have two separate, independent backups, with the advantages of both. If one fails, the other remains.
Now the ports issue.
You can of course continue to use USB2. Just that moving large volumes around will be slow.. as doubtless you already know.
On your particular Mac since you missed out on USB3 which is a pain.. you can buy a Thunderbolt to USB3 adapter like the belkin.
http://www.belkin.com/au/p/P-F4U055/
I suggested the Thunderbolt to Esata (it is an older interface and the adapter is rather cheaper but I hear more reliable.. check reviews for both).
http://store.apple.com/au/product/H8875ZM/A/lacie-esata-hub-thunderbolt-series
Sata is the interface of the hard disk.. Esata just means external sata.. so it is native and without conversion.
I was merely suggesting ways to speed things up. But as long as you don't run CCC on more than daily basis then I think you will be fine just with USB 2. It will take a while on the day you do the swap over for the archive volume, but if you turn off the power saving in the Mac and leave it run overnight it should be able to do most of it.. you need to realise it will have to deep scan both the disks.. to compare files.. but CCC is based on Rsync and it is extremely fast and efficient. I am just not sure I know how long it will take to do. Anyway.. there is a plan.. tweak and adapt as you see fits your needs.

Time machine backups cache and temp files?

Just bought a new external usb disk and used it as the time machine backup volume. All time machine configurations are remained as default, so it would backup all my current HD. While watching it estimating the time that the first backup would take, a question jump out in my mind:
Will time machine backup all files "honestly"? Will it make a copy of every file including the Internet cache and temp files when I'm browsing websites? I'm used to watch youtube a lot, and the temp flv file would be large (though I don't know where it is stored on Leopard). If these files would be backup-ed as well, I think it would be a great waste of disk space.
Can anyone confirm that those caches and temp files would be copied or not? If it would, how to prevent this? Thank you.

If you wish to exclude anything from the backup simply drag the item into the Exclude list. For example, you can save considerable space by excluding the /System/ folder. Cache files would be found in the /Library/ and /Home/Library/ folders. Each folder contains a folder named Caches. There is no specific folder in which temporary items as stored that is visible to the user because the operating system will usually delete temporary files on its own. This may not be the case for a specific program, however. Browsers may maintain their own reserved cache and temporary files, and manage them on their own.
I do not think, however, that TM backs up caches or temporary files. I find no such items in my TM backup - only the folders. I do not back up my /System/ folder.

Move Exchange Server 2010 SP1 databases and log files to another partition

My company has been running Exchange Server 2010 SP1 on virtual machine for almost three years. My colleague created this virtual machine (Windows Server 2008 R2 SP1) before Exchange Server installation with following partitions:
C partition - for Windows Server 2008 R2 SP1 OS (size 40 GB)
E partition - for Exchange Server 2010 SP1 databases and logs (size 410 GB)
Those two partitions are on the same virtual hard disk "next to one another" so I can not extend C partition which gets filled up with IIS logs (when C partition free space is under 10 GB I am backing up these logs with backup software and afterwards
deleting them thus freeing up space on C partition). I have 2 databases (along with Public Folders) and size of these databases and their logs is around 235 GB.
Is it safe to move these databases and their logs to another partition which I would create on separate virtual hard disk for this purpose only and how much time this process might take since during this process databases
would be dismounted and inaccessible to all users? He made big blunder in my opinion but I have to solve this.

Yes, its safe to move the DB/Logs. You can follow the procedure mentioned in this article to do it via EMC or Shell... http://exchangeserverpro.com/move-exchange-2010-database-folder/
But it depends on various factors on how long it is going take, like size of the database/logs, server performance, storage performance etc...
Besides this, upgrade Exchange 2010 SP1 to SP3, SP1/SP2 aren't supported anymore! http://blogs.technet.com/b/rmilne/archive/2014/04/09/end-of-exchange-2010-sp2-support.aspx
Blog |
Get Your Exchange Powershell Tip of the Day from here

How can I get Time Machine to only backup new and changed files?

Time machine is backing up more than just new and changed files and is filling up my back up disk too fast. How can I change it so that it will only back up the new or changed files instead of the whole drive? I used to have my backups go back months, but now it is backing up only a couple days back, since there is no room left on my back up hard drive. Each back up is like 400 gigs, instead of just the amount of the changed or new files.
Thanks for any ideas or help!
kt

Thank you Eric! I did go look at this info and tried to do a full reset of TM and I am still getting full backups of my HD instead of just new/changed files. The weird thing is that I have 9 BU's from over the last 2 days and they all seem to be around 415 gigs and my backup HD is only 750 gigs! I don't get it. It shows that I have around 75 gigs of free space on that 750 gig drive. It doesn't make sense.
So I am having the same problems even after doing the full reset of TM. I am using Mavericks 10.9.5 which they don't talk about in this info.
Does anyone have any further ideas on this? I don't understand how I could possibly have that many full BU's on my HD that is only 750 gigs.
Thank you,
kt

When group-working from SAN, where is the best place to store Cache Files, the Cache Database, and render files?

I have a small post production house, and we are spinning up into editing our first series. We're all working from a SAN, and sharing projects and sequences via the Media Browser. It's working great so far, but we've been having issues (could be user error) with trying to figure out where best to store the Media Cache database, the cache files, and the preview files so when a project is moved from one machine to another, it doesn't constantly have to re-conform all of the media in the project.
We have a very large master project, as this is a documentary style show, so there are hours and hours of clips.
When I open a project on a new or different machine it has to conform EVERYTHING in the project again.
Where should I store all these conform and cache files so that new people can get started working without having a machine literally be unusable until the conform process finishes?
Thanks!

Hi Jon,
I have a small post production house, and we are spinning up into editing our first series. We're all working from a SAN, and sharing projects and sequences via the Media Browser.
Glad it is working for you. Just to let you know, this workflow is not officially supported so your mileage may vary based on the kind of system you're working with. There are a lot of variations to the quality of these kinds of systems. Some are turn key, some home built - so I think you understand.
It's working great so far, but we've been having issues (could be user error) with trying to figure out where best to store the Media Cache database, the cache files, and the preview files so when a project is moved from one machine to another, it doesn't constantly have to re-conform all of the media in the project.
While it's possible to have the cache files (and the database) in a central location (I only know of one place that pulls that off, and it's a very, very large place), the systems that have this data stored locally are far more reliable. You can work around all the reconfirming by avoiding moving the project data around. You really don't need to move the project around from place to place. Have one computer as the "hub" for inputting data from other workstations. Wouldn't that work?
When I open a project on a new or different machine it has to conform EVERYTHING in the project again.
Can't you just have the master project on one machine and work on shorter sequences of your doc on the other machines? When done, just import to the master machine.
Where should I store all these conform and cache files so that new people can get started working without having a machine literally be unusable until the conform process finishes?
Premiere Pro simply must have those cache files in order to work. Sorry, there's no way around this problem unless you change your workflow.
Thanks,
Kevin

Backup methods and external hard drive questions

I usually copy files and photos to DVDs but wonder if that's enough and the most efficient method to back up. So I purchased a Seagate 1TB external hard drive. Here are my questions:
- What is the best way to copy all my data to this hard drive?
- Is it necessary to copy everything?
- Is it possible to partition the external hard drive?
- Is it possible to use this hard drive to copy data from more than one computer, for instance, my powerbook and a PC laptop? If so, how?
Any information on this would be so appreciated!

Here are some suggestions for backup:
Backup Software Recommendations
Carbon Copy Cloner
Data Backup
Deja Vu
Silver Keeper
SuperDuper!
Synchronize Pro! X
Synk Pro
Synk Standard
Tri-Backup
Others may be found at VersionTracker or MacUpdate.
Visit The XLab FAQs and read the FAQ on backup and restore. Also read How to Back Up and Restore Your Files.
You can partition the drive if you need to, but that will depend upon why you would need to partition the drive.
Personally, I would clone your startup volume to the external drive. You can do this with any of the referenced software as well as by using the Restore option of Disk Utility. You can then incrementally update the clone using a backup utility.
It would be possible to backup more than one computer in which case you would need to partition the drive. Backing up a Windows machine, however, would require partitioning the drive using MBR and formatting it FAT32 (MSDOS).
For clarification do you really have a PowerBook (PPC machine) or a MacBook (Intel?)

Rebuilding iPhoto database and missing files

Heya,
I got a corrupt iPhoto6 database after a power failure (but had a backup) while travelling. I didnt have the backup drive handy(was far away), so I kept adding new files to this corrupt database. I also began making modifications.
Now I want the two together and keep all the modifications too. I foolishly thought I could: add to the backup the new "rolls" with the Originals and Modified folders and then rebuild the database.
Why 'rebuilding the database' doesnt involve scanning through these two directories I don't know, because this doesnt work. I thought maybe the permissions were wrong, but that doesnt seem to be it.
So what the best way - while retaining the modifications Ive made to my new rolls - to add everything back to the Master/non-corrupted database. A Merge with iPhoto Library Manager? Could a rebuilt with it work(Looks like no, since it only scans the xml file).
cheers,
matt
Intel Mac mini Mac OS X (10.4.8)

Matt:
Here's what I think is your only option to retain some of the modifications that are in the library and get all of your photos back.
1 - create a new library.
2 - with the paid version of iPhoto Library Manager copy all of the albums and rolls from the current library into the new one. This will retain your rolls and albums, keywords, and comments but not slideshows, books, etc.
3 - copy the Originals folder from the current library to the desktop.
4 - drag the Originals folder into the open iPhoto window. You should get a window asking if you want to import duplicates or not - select not.
That will give you the closest you can get to what you had before the crash with the after crash modifications.
Do you Twango?
TIP: For insurance against the iPhoto database corruption that many users have experienced I recommend making a backup copy of the Library6.iPhoto database file and keep it current. If problems crop up where iPhoto suddenly can't see any photos or thinks there are no photos in the library, replacing the working Library6.iPhoto file with the backup will often get the library back. By keeping it current I mean backup after each import and/or any serious editing or work on books, slideshows, calendars, cards, etc. That insures that if a problem pops up and you do need to replace the database file, you'll retain all those efforts. It doesn't take long to make the backup and it's good insurance.
I've written an Automator workflow application (requires Tiger), iPhoto dB file backup, that will copy the selected Library6.iPhoto file from your iPhoto Library folder to the Pictures folder, replacing any previous version of it. You can download it at Toad's Cellar. Be sure to read the Read Me pdf file.

Backup drive and transferred files not found

I recently purchased a time capsule. I transferred some files from an older hard drive to the TC (they appeared on the TC) and then backed up my PowerMac G5 (wired directly) and my MacBook Pro (wirelessly) to the TC. The initial backups for these two computers worked fine, but I am now encountering two problems.
1. The files I originally transferred from the old hard drive now don't appear on the TC (although it looks like they might be there based on the amount of TC drive space occupied). Is there anyway to access these older files (assuming they are still there)?
2. Frequently when I try to enter TC from the PowerBook Pro, it says that hard drive not found. Also, it doesn't back up automatically. The only back ups are those I have done manually. The TC seems to be working fine with the wired PowerMac G5.
I would appreciate suggestions on either topic.

1. Have you changed the file sharing access on your Time Capsule since you copied your files? When you use the Time Capsule file sharing without accounts setup you have access to all of the disk. When file sharing with accounts is enabled you have access to a Shared file (where everyone has access and where your Time Machine backups are kept) and then to your own personal file (where only you alone have access). You no longer have access to anything that is not in the Shared file or your account file. If you have enabled account access then I would suggest turning it off which would then give you access to the whole disk and then moving the files you are referring to either into your Shared folder or your Account folder.
2. Go to System Preferences, Time Machine, and then Change Disk. Select the name of your Time Capsule (this won't overwrite or delete any of your previous Time Machine backups). This should hopefully fix the problem.

Username And External Files plz help!

Hey there,
Hope someone can help me out with this even pointing into the right direction will be very helpful.
I kind of understand how to create a basic username and password 'menu' in flash.
password = inputName
if (password == "kirupa") {
answer = "Access granted!" ;
} else {
answer = "Access denied!" ;
But I would like to create a menu where a user can create a new username with then will be added to an exteral file ( data base I guess).
From there on anything the user does is remembered on that name. And if that user exits the program and loads it again the user's stats are loaded.
If you have any info on this please let me know, where lost here.
thx
pavel

u can not write anything from flash in text file . What u can write through flash is shared object. u can store ur user name and password in shared oject and then get and set those value same u can achive through javascirpt cookies also
But both option would be system specific
otherwise
database is the option as suggested before

Can I use my Time Capsule as a backup disk and storage?

Can I use my Time Capsule as a backup disk and put files from my other USB external hard drive on it, using the TC as an external as well?

Actually, there is a way to limit the amount of space Time Machine uses for backup so that it won't eat up all your disk space...for example, I have mine set to only use 65GB, and then it starts to overwrite older backups...Google it for more info. To begin with, I deleted my Sparse Disk Image Bundle from Time Capsule, and used Disk Utility to create a Sparse Disk Image Bundle (turn off backups first!!!) with a capacity of 65GB in HFS Journaled, Case Sensitive...The Image must be named as follows "Full Computer Name_Ethernet MAC Address" (The ethernet MAC address is used even if you plan to solely use wifi for backups). for example, my Sparse Disk Image Bundle name is "Ahmad Atiya's MacBook Pro_00254b9d8f72.sparsebundle" Move this image to your time capsule's main directory (if you didn't create it there to begin with). Go to your Time Machine preferences, turn it on, and set it to use your Time Capsule. It should start using the image you created. If you click "get info" on the disk image, it'll show a capacity of whatever you created.

Re-connecting older backup drive to restore files from

Can you take a backup disk 'offline' by disconnecting it, start using a new backup disk, then at some point in the future disconnect the newer backup disk, re-connect the first (older) backup disk and recover files off of it?

bokehboy wrote:
I am thinking of creating longer-term backup 'records'. I understand the difference between archiving and backing up, and this would be somewhat of a hybrid -- sort of like an informal form of archive. External hard drives are getting so cheap these days (ballpark of a dime per GB) that I was thinking of saving each year's worth of backups on a separate hard drive (the time factor may vary, depending on the rate at which I am creating new content).
That would certainly work, at least for a while.
For really long-term storage, there are at least two unpredictables: will future versions of OSX (say, 10.9.x) be able to read and restore TM backups that are a couple of major releases old; and will an external disk that gathers dust for a few years still work reliably?
Another method would be to burn a set of DVDs, but Time Machine is just so dang convenient.
Similar unpredictability. DVDs may be even less reliable, long term, than external HDs.
Big data centers periodically read-in old backup tapes and write out new ones, as tapes can deteriorate, too.

Backup database and external files

Similar Messages

Maybe you are looking for