Managing large libraries

I have upwards of 40,000+ images in my referenced Aperture library. I create a folder for each year and a project for each shoot. I'm fairly diligent about adding keywords and keeping my files clean but as the cameras keep getting better the megapixels grow larger and the hard drive space (internal) gets gobbled up. I had just about filled a 1TB hard drive with photos dating back to 2004. I decided to get a 3TB, it formatted beautifully and I used SuperClone to move files from the 1TB to the the 3TB. I then removed the 1TB, put the 3TB in its bay and changed the name of the 3TB to "Pictures" ... just like the old 1TB. I thought this strategy would avoid Aperture "losing" the referenced files. It didn't. I know how to reconnect the referenced files to Aperture and my method of organization makes it doable but it will be somewhat tedious. I can do large batches so that should be fine.
But all this makes me wonder if I should be doing something different in terms of organization. I'd be curious to hear from others who have similar numbers of photos and how they organize them as well as any views about moving into an increasingly large megapixel world and what that means in terms of computers. I read somewhere that I can probably put a 3TB drive in each bay of my aging Mac Pro (2.66, Dual Core Intel Xeon) but should I really move towards that or do I need to think about some kind of external storage arrangement that can be connected to the stunning new iMacs that try to seduce me everytime I go to the Apple Store?
Oh, and if there is a more clever way to reacquaint the referenced files and Aperture please let me know.
Thanks very much.

Your set-up is both good and standard.
Once you commit a file to management by Aperture, you must use Aperture and only Aperture to manage that file. When you need to move your Referenced Masters, hook up both drives (if moving from one drive to another), and use "File→Relocate Masters".
For 1 TB of data, this will take some time.
Since you used another program to move your Masters (you used SuperClone), you must then re-connect your Images to their masters. There shouldn't be anything tedious to _you_ about this, and you shouldn't have to do any more but one very big batch. Just select all your Images and run "File→Locate Referenced Masters". It, too, will take some time to run. (Added: after it's done, confirm that no files got left out. Filter "Photos" with the Rule "File Status" set to "missing".)
I recommend buying four identical drives for each large Library:
- one for the Library
- one for the Referenced Masters
- one to back up the Library (I use SuperDuper!)
- one to back up the Referenced Masters (also SuperDuper!)
(All drives FW800 or faster, 7200 rpm, large caches.)
Having each of these on its own external drive makes administering back-ups easy (and in making them easy, they are more likely to be done). I tell people that a pair of 1 TB drives holds about one photographer-year of work, at a cost of $200 US. Imho, this is a bargain.
Message was edited by: Kirby Krieger

Similar Messages

Cannot resize the albums in album view in Itunes 11. For those of us with large libraries (160gb), this is extremely frustrating. I want to make the artwork smaller. This seems to be a serious oversight.

Cannot resize the albums in album view in Itunes 11. For those of us with large libraries (160gb), this is extremely frustrating. I want to make the artwork smaller. This seems to be a serious oversight.

I agree.
It was very useful to reduce everything to small and quickly scan down looking for missing artwork.
Particularly useful when I only connect to the machine with iTunes via screen sharing, so everything can be a bit slower to scroll through at times.
In fact, itunes 11 when used purely as a server for Apple TV etc is a downgrade in usability IMO, as they've got rid of some of the useful views that helped me manage my library remotely.

Repair itunes library (damaged) file. Multiple iTunes libraries on an external harddrive. All libraries have been working fine. Do some "housecleaning" on a one of the large libraries 296GB. iTunes quit and now has damaged file. 296 GB still on HD.

How to repair itunes library (damaged) file. Multiple iTunes libraries on an external harddrive. All libraries have been working fine. Do some "housecleaning" on a one of the large libraries 296GB. iTunes quits and now has damaged file. 296 GB still on HD.

The duplication is unnecessary. Exporting creates a duplicate of the file. So now you'll have a duplicate of a duplicate. Exporting is not "working on" a file.
No it's not merging, it's exporting from one to the Finder and into another. No matter what lose something. If you export the Original you leave behind all the work you've done in iPhoto. If you export anything else, you lose the non-destructive editing and the ability to revert to the original. With merging you preserve that work. Yes, you can trash the old Library when you have completed the manoevre but no it's not the same thing as merging.
This User Tip
https://discussions.apple.com/docs/DOC-4921
has details of the options in the Export dialogue. But in brief:
Current gets you the iPhoto Preview, used for sharing via media browsers. It's a jpg, medium quality missing metadata. Original gets the file you imported, unchanged and then you can export different version of the current version at different qualities. If you choose to export anything except the Original you do not get a Raw. There's no such thing as an "edited Raw", and you lose the connection between the original and the exported version. That means you've taken a non-destructive workflow and turned it into a destructive one.
The Tiff will certainly be higher qulaity and less likely to suffer generational loss in future editing but the file sizes are vast, often more than 10 times the size of the jpeg.
Put it this way, it would be cheaper to buy Library Manager than the disk you'll need to contain all the Tiffs. Unless you plan on a lot of editing, I'd go for a high quality jpeg as a reasonable compromise.
To be clear:
So, I guess my new question is, how can I edit the Raw image and keep it in a RAW format that IPhoto recognizes so that I can reprocess without any loss at a future time, or is it that once you edit a RAW image, then it is no longer a RAW image?
Once you edit a Raw it can no longer be a Raw. End of.

Best storage set-ups for large Libraries?

Let's say "large Library" means 500 GB and/or 500,000 Images, and above.
What set-ups for storage, on a Mac maximized with RAM, will provide the best Aperture performance? The best bang-for-the-buck? Assume Thunderbolt is available. I don't know much about hardware: RAID, hybrid drives, etc.
What sensible considerations should one make for backup and data-redundancy?
How can I determine if the storage media is a bottleneck in the performance of Aperture on my system?
I run most libraries off of external USB-3 drives mounted (sometimes directly, often via powered USB-3 hubs) to my
MacBook Pro
Retina, 15-inch, Early 2013
Processor 2.7 GHz Intel Core i7
Memory 16 GB 1600 MHz DDR3
Graphics NVIDIA GeForce GT 650M 1024 MB
System Drive APPLE SSD SD512E
This works well for small and medium-size Libraries, but for large Libraries, I'm spending costly time waiting for the system to catch up.
Some of that, demonstrably, comes when I run a three-monitor set-up. (But this provides some welcome advantages to me.)
Additionally, some back-ups and database repairs now take 12 hr. or longer.
Thanks.

Thanks William,
I kept my c. 2011 MPB and use it for making back-ups, which is done automatically after I leave for the day. My early-2013 MPB is so much faster (and has such a higher-resolution screen) that I don't use the older computer at all.
William Lloyd wrote:
Probably the fastest storage you can get is the La Cie "little big disk" which is a pair of 512 GB SSDs with a Thunderbolt 2 connection. The issue is, only the newest Mac Pro, and the newest retina MacBook Pros, have Thunderbolt 2 right now.
OWC tech explained to me that TBolt2 allows 2x the throughput, but that the drives have to be able to provide it. TBolt1 should provide 1,000 MB/s (do I have the units correct?), which is faster than most drives can provide. So the bottleneck, for me, isn't likely to be the port, but rather the drives. USB-3 can move 500 MB/s, which is still faster than -- afaict -- what my WD Passport drives can provide.
As I currently see it, I need faster throughput, and I need to either trim my large Libraries or find a way to manage them so that the regularly used files are more speedily available.
Regarding faster throughput, an external SSD comes to mind.
The problem, for me, is that the large Libraries are close to 1TB (with Previews). While I don't expect them to grow (the data acquisition phase is, for now, done), it would be short-sighted to assume they won't. That brings up the second consideration, which is how to best use spanned drives that contain an Aperture Library.
As I see it (with my limited understanding of hardware), what I really want is _a lot more RAM_, or, barring that, a huge SSD scratch disk. I have 200 GB free on my system drive, which is an Apple-supplied SSD, but it doesn't seen to be used as effectively as I'd like.
WD is now selling a new portable TBolt SSD, the "My Passport Pro", available with 4 GB of storage, and with a throughput of 230 MB/s. My current largest Library is on WB Passport drives, whose throughput is not faster than 480 Mb/s (Max) (I assume BITS, not BYTES, so 40 MB/s). That's a huge difference, while still only 1/4 of the speed possible with TBolt1, and 1/8 the throughput possible with TBolt2 (which my early-2013 MBP does not have, afaict).
These are the questions I am trying to answer:
- How can I measure my current throughput?
- Can I determine the bottleneck in my use of large Libraries in Aperture?
- Will new storage give me faster performance?
- Which storage will give me the best "bang-for-my-bucks"? (The La Cie "little big disk" seems to have it's bang limited by my computer, which greatly lowers its quotient value.)
In short: how can I get 900-1000 MB/s throughput on my machine, and is there any way to set up my Library so that, even though it is close to 1 TB, I can use a smaller very fast drive for the most read/written files?
--Kirby.

Managing Two Libraries With Time Capsule

Hi,
I just got a Time Capsule and an iPhone. I already have an iPod. The space on my Hard Disk is running low - mainly due to a large iTunes music selection and large number of photos.
My question is this. Is there any way for me to manage two libraries - a small one (that will merge with my 8GB iPhone) that remains on my mac, and a large one (that merges with my 60GB iPod) on the Time Capsule? I know that I can create these two libraries, but every time I rip a new album, or download something from iTunes, I would like them to download to both libraries.
Is there a simple solution to sync certain sections of two libraries?
The same goes for iPhoto.
Thanks,
Matt

I have a similar situation. I have a 8gb ipod touch and a 30gb ipod video. I only have one library and what I did was to have the video sync with my entire library (about 25 gb). For the touch, I have it sync only playlists that I select. (You can edit how a ipod syncs when you connect to the computer) this setting is under the music tab.
If you want to move the library off the computer, make sure you follow apple's instructions.
http://support.apple.com/kb/HT1449
For the photos, each ipod can be setup to either sync the entire library or selected events. the settings for this are under the photo tab.

PLL Library Size - Few Larger Libraries Or More Smaller Libraries

Hi
I am interested in people's thoughts on whether it is better to have fewer larger libraries or more smaller libraries. If you do break into smaller pll libraries, should the grouping be that some forms will not use modules in library, ie do not have to attach to all forms. For common modules that all forms require access to, do you achieve anything in having many little libraries, rather than on larger library?
Is it correct that pll libraries are loaded into memory at run time when form is loaded and library is attached?
What are issues to consider here
Thanks

Hi Till,
My honest opinion...do not merge the libraries. Switch between them using iPhoto Library manager and leave it at that.
A 22 gig library is way too big to run efficiently.
Lori

Managing multiple libraries for multiple family members

I have four family members with multiple iTune libraries, including two that need to be age appropriate for our children. To add to this burden, we now have an iPad as well. How do I best manage these libraries, minimizing duplicate purchases, enabling each to have store accounts for gift card purchases, and sync each iPod simply for those not familiar with the apple software? Can I merge our libraries, create different users with their songs and accounts for purchases and syncing logins? We have two operating systems in the home, mac and pc? Argh, this is cumbersome, my library has over 4000 items, other family members have much smaller libraries, but would like their own individual libraries. Should I merge and make separate playlists for each user? If so, how can I incorporate my own vast library of playlists already created?
Help me please, any suggestions?
Rob Mc

There are 3 options. See this page:
http://support.apple.com/kb/ht1495
My advice is to use option #1 and have each user with a separate windows userID. They can each keep their own itunes library database file with ratings, play counts, playlists.
Have all the library content in a public folder to avoid filling the PC's hard drive with duplicates.
You can use a 3rd-party utility program like iTunes Folder Watch to update all the libraries when new content gets added.
Method #2 is easiest, but then ratings and play counts will not be separated. Also, they have to remember to log out of the itunes store and log in with their own ID when making purchases. Something to watch for when kids are involved with their own gift cards....this is not a problem with method #1.
Method #3 seems to give a lot of people grief because all users MUST use a shift-start of itunes to choose their own library. Woe has fallen on many who have tried this method.
As for using both Mac & PC, itunes & ipods are designed to only sync with one PC. There is a way around that, but you either have to do a lot of manual backing up and copying, or buy something like TuneRanger.

Managing multiple libraries with different file types (AAC v. Lossless)

I have set up two libraries, one for my AAC music and the other for my lossless music. The files themselves are located in two separate places on my hard drive. When I switch libraries (option-clicking iTunes), and go to Preferences/Advanced tab, the directory for the files and the import method doesn't change -- I have to change it manually. This is annoying.
Does anyone know of a way to automate this change, so that if I open a new library, the default location of the files and the import format changes automatically? I tried ituneslibrarymanager from Dougscripts, but for some reason can't get this program to install on my machine.

No, it's the same Windows XP user account, but I have two separate music libraries - the music files in library are in a lossless format, while those in the other are in AAC. For convenience, I wish to keep the two libraries separate.
In iTunes 7 it is possible to have separate libraries - start iTunes and hold down the SHIFT key (Windows) while the program starts. A dialogue box then appears where the required library can be selected. However, as far as I can tell, it's not possible to have a different set of preferences for each library.
Say the preferences are set up for my lossless library, which is located on a separate network drive. If I then make the AAC library active and open iTunes, iTunes reindexes the library according to the settings for the lossless library and I end up with the AAC library full of lossless files. I've tried unchecking various things in the preferences box, but I've had no luck fixing the problem. Thanks for your suggestion - I have already reported the problem.
The whole thing is a rather inelegant way of managing multiple libraries in iTunes. Surely the Apple developers can come up with a way of doing it from within iTunes??

Managing multiple libraries

Hello everyone,
is there a way (even through a third-party add-on) to manage multiple libraries (I actually need to have only 2 libraries) in iTunes?
I'm asking so because I need to move part of my podcast collection to an external drive (I want to keep only the most "important" podcasts in my Mac and the rest of them in a external drive). But I'd like to make this change transparent to iTunes (I don't want to choose the library I want to mount every time I use iTunes).
Thanks and have a nice weekend.
Sig

I'm asking so because I need to move part of my podcast collection to an external drive (I want to keep only the most "important" podcasts in my Mac and the rest of them in a external drive).
You don't need a separate library to do this.
iTunes prefs -> Advanced.
Set the *iTunes media folder location* to the external.
In iTunes, select the items you want on the external then right click -> Consolidate selected files.
This will copy these to the external drive.
You can then delete them from the internal.
Then set iTunes prefs -> Advanced, *iTunes media folder location* back to the internal.
I'm looking for something integrated within iTunes
Then no, you cannot manage separate libraries except holding Option when starting and selecting the library you want.

Managing two libraries

hi there,
I am having problems managing two libraries. I am trying to create two separate libraries, AAC and Lossless. I was given the advice below, which I followed, problem is when you switch back and forth between the libraries, if switches songs from my Lossless library which are stored correctly in the Lossless folder to files from the AAC library and vice versa.
Also, each time you switch libraries you have to go to advance preferences and indicate where you want to store the files. Example, I have the Lossless library open, the preference is set as G:/music/lossless; when I exit that library and open the AAC library the preference is set to the Lossless library. I don't mind changing that if I have to but the problem of the libraries switching content from AAC to Lossless is the big problem. Any advice?
PREVIOUS POST INFO:
Do this.
Create 2 folders on the external.
G:\Music\Lossless
G:\Music\AAC
Hold Shift (for Windows), Option (for Mac) and launch iTunes
Select Create new library and select the AAC folder you created above.
This will create a new iTunes folder in G:\Music\AAC. You should now have G:\Music\AAC\iTunes\.
Go to Windows Exploder and move all your AAC files into G:\Music\AAC\iTunes\iTunes music\ folder.
Go to iTunes menu File -> Add folder to library and select G:\Music\AAC\iTunes\iTunes music\.
Quit iTunes and restart holding Shift (for Windows), Option (for Mac).
Select Create new library and select the Lossless folder you created above.
This will create a new iTunes folder in G:\Music\Lossless. You should now have G:\Music\Lossless\iTunes\.
Go to Windows Exploder and move all your Lossless files into G:\Music\Lossless\iTunes\iTunes music\ folder.
Go to iTunes menu File -> Add folder to library and select G:\Music\Lossless\iTunes\iTunes music\.
Next time you launch iTunes hold Shift* (for Windows), Option (for Mac) select Choose library and select teh iTunes library.itl file in teh folder you want.
Make sure you first set your Import prefs to AAC or Lossless, depending on which folder you are in.

I have a similar situation. I have a 8gb ipod touch and a 30gb ipod video. I only have one library and what I did was to have the video sync with my entire library (about 25 gb). For the touch, I have it sync only playlists that I select. (You can edit how a ipod syncs when you connect to the computer) this setting is under the music tab.
If you want to move the library off the computer, make sure you follow apple's instructions.
http://support.apple.com/kb/HT1449
For the photos, each ipod can be setup to either sync the entire library or selected events. the settings for this are under the photo tab.

Newbie question abount managing java libraries

Hello all,
I'm rahter new to Java and have a question about how to manage java libraries in a central location. I've been told to use Maven, but I don't need/want to use it.
I'm only asking a way to have centralized the different Java libraries that a Developer Team may use in a form that they could download to their PCs and use in their projects, thinking in a free IDE environment where could be JDeveloper, NetBeans and Eclipse in use.
I'm thinking in use Subversion to upload the different Java libraries files to a Subversion Repository and then each Developer Team doing a checkout of what they need, but I think it's not the primary use to do with the versioning system.
Please, could somebody point me out if there is any product that could fit my needs about managing Java libraries ?
Thanks in advance.
Angel.

- limit the set of packages to those which have been approved by the processYou can constrain Maven to only use an internal repository from which to pull libraries. If you don't put the libraries in the repository you can't use them in your project.
- limit the packages by licenseI don't think there's anything built in to do this. There's a space for that information in the pom (metadata) file though, so it would be possible to do so. You'd probably have to do some work on the repository to bring ensure the fields was populated though - I get the impression that it tends not to be by default.
- produce audit logs of what the dependencies and licenses used by each project isDependencies yes, certainly. Licenses again I don't think there's anything existing but you could probably adapt the dependency plugin to report this.
Example dependency tree generated by Maven for JA-SIG CAS:
[INFO] [dependency:tree]
[INFO] org.jasig.cas:cas-server-webapp:war:3.2.1
[INFO] +- log4j:log4j:jar:1.2.15:runtime
[INFO] +- net.sf.ehcache:ehcache:jar:1.4.0-beta2:runtime
[INFO] | +- net.sf.jsr107cache:jsr107cache:jar:1.0:runtime
[INFO] | \- backport-util-concurrent:backport-util-concurrent:jar:3.0:runtime
[INFO] +- commons-lang:commons-lang:jar:2.2:runtime (scope not updated to compile)
[INFO] +- quartz:quartz:jar:1.5.2:compile
[INFO] +- org.springframework.ldap:spring-ldap:jar:1.2.1:compile
[INFO] +- org.springframework.ldap:spring-ldap-tiger:jar:1.2.1:compile
[INFO] +- org.springframework:spring-webmvc:jar:2.5.1:compile
[INFO] | +- org.springframework:spring-beans:jar:2.5.1:compile
[INFO] | +- org.springframework:spring-context:jar:2.5.1:compile
[INFO] | +- org.springframework:spring-context-support:jar:2.5.1:compile
[INFO] | +- org.springframework:spring-core:jar:2.5.1:compile
[INFO] | \- org.springframework:spring-web:jar:2.5.1:compile
[INFO] +- javax.servlet:jstl:jar:1.1.2:compile
[INFO] +- javax.servlet:servlet-api:jar:2.4:provided (scope not updated to compile)
[INFO] +- ognl:ognl:jar:2.6.9:runtime (scope not updated to compile)
[INFO] +- junit:junit:jar:3.8.1:test
[INFO] +- org.acegisecurity:acegi-security:jar:1.0.6:runtime
[INFO] | +- commons-codec:commons-codec:jar:1.3:runtime
[INFO] | \- oro:oro:jar:2.0.8:runtime
[INFO] +- cas:casclient:jar:2.1.1:runtime
[INFO] +- org.springframework:spring-aop:jar:2.5.1:compile
[INFO] | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] +- org.springframework:spring-test:jar:2.5.1:test
[INFO] +- commons-logging:commons-logging:jar:1.1:compile
[INFO] +- org.springframework:spring-webflow:jar:1.0.5:compile
[INFO] | \- org.springframework:spring-binding:jar:1.0.5:compile
[INFO] +- org.inspektr:inspektr-core:jar:0.6.1:compile
[INFO] | +- aspectj:aspectjweaver:jar:1.5.3:runtime
[INFO] | +- aspectj:aspectjrt:jar:1.5.3:compile
[INFO] | \- org.springframework:spring-jdbc:jar:1.2.9:compile
[INFO] | \- org.springframework:spring-tx:jar:2.5.1:compile
[INFO] +- org.jasig.cas:cas-server-core:jar:3.2.1:compile
[INFO] | +- org.jasig.service:person-directory-impl:jar:1.1.1:compile
[INFO] | | +- org.jasig.service:person-directory-api:jar:1.1.1:compile
[INFO] | | \- commons-collections:commons-collections:jar:3.2:compile
[INFO] | +- jdom:jdom:jar:1.0:compile
[INFO] | +- org.springframework:spring-orm:jar:2.5.1:compile
[INFO] | +- org.apache.santuario:xmlsec:jar:1.4.0:runtime
[INFO] | +- org.opensaml:opensaml:jar:1.1b:compile
[INFO] | +- javax.persistence:persistence-api:jar:1.0:compile
[INFO] | \- javax.xml:xmldsig:jar:1.0:compile
[INFO] \- taglibs:standard:jar:1.1.2:compile

How to manage iPhoto libraries

How to easily switch from one iphoto library to another, move photos from one library to another etc.

David Neale1 wrote:
iPhoto '11 9.4.2 has the menu item File>Switch to Library…
Can you not manage libraries that way?
No - you can not manage libraries that way - you can switch libraries that way - exactly the same as the more general
You can switch libraries by depressing the option key, launching iPhoto and using the select library command
but you can not manage them that way or move photos between them that way
You can move photos (but not versions) by exporting from one and importing to the other one
But all that being said twice
IPLM. http://www.fatcatsoftware.com/iplm/ is the best tool for managing multiple libraries
It lets you move photos between libraries keeping all versions and all information
LN

Working With Large Libraries

I use a portable hard drive for music storage (no music is stored in the Macbook). My library is 14,000+ and until 8.1 I have never had issues of any kind. Today I downloaded 8.1.1 and I still have the same problems. Beach ball spinning forever, HUGE delays in starting of songs, and delays in dragging songs to playlists. Does anyone have any suggestions to fix the problem? Kind of frustrating when 8.1 was supposed to be better for large libraries.

Hold down the option key while you launch iTunes. This presents you with an option to choose a Library (or create a new one). Navigate to your current Library, & with a bit of luck iTunes will stop looking in the wrong place.
Usually these problems are caused by the User moving something or other… not that I'm accusing you of anything!
If that doesn't work, go to +Preferences > Advanced+ & make sure that the folder iTunes uses for storing your music is the right one.

Final Cut 10.2 keeps crashing when trying to update my larger libraries

My Final Cut just updated to 10.2 and each time I attempt to load one of my larger libraries, it crashes. Does anyone have a work around or fix?

My Final Cut just updated to 10.2 and each time I attempt to load one of my larger libraries, it crashes. Does anyone have a work around or fix?

Managing large numbers of SPAx102s

What are service providers using to manage large numbers of 2102s or 3102s? How are firmware versions controlled and rolled out? What about configuration & password management?
Thanks!
Solved!
Go to Solution.

Hello,
as called "provisioning" is used for this. The SPA-xxxx devices has ability to remote download (http and tftp) the settings file on regular basis, check for changes, deploy the settings file and reboot. The file is loaded based on device's MAC address, so every device will realy load only it's own file.
Provisioning can set the adapter password, and can also fil-in rule for the FW update when needed ... and SP-xxxx adapter will check FW's file header and if newer (or better different) than the one running in adapter it will download the FW, flash the FW on idle time and then reboot.

Managing large libraries

Similar Messages

Maybe you are looking for