Is there an upper threshold of catalog size that shouldn't be crossed?

so here's a quick version of my workflow:
Shoot a job,
Download cards via Image Capture to a pocket drive off my laptop and if the shoot is more than a one day ordeal,
Import into Lightroom and have it render 1:1 previews of the day's bounty at the hotel at night, otherwise...
Transfer the Canon 21mp CR2 files to the 4 drive RAID-0 on my primary machine and either import & build previews into a new job catalog there or if already built, transfer the catalog and previews off the pocket drive to that same RAID-0
edit
rename and organize
process out to Tiffs for client and build web indexes
convert any selected files to DNG, discard unstarred raw files
move the catalog, preview, web and DNG files to my mirrored archive (online/offline depending on what drive is mounted)
import the job catalog into a master catalog that resides permanently on both my primary and secondary MacPros.
Images viewed in the master catalog are most likely offline and are viewed via thumbnails and 1:1 previews, the processed Tiffs get killed from my 6TB RAID after 30 days time from delivery of the job to client.
I'm in the process, as I have time, of applying this process to archive drives of past shoots that had originally been cataloged with Aperture, or not cataloged at all.
As of this writing, I have 17 individual, 1TB & 1.5TB hard drives containing RAW & DNG files from 16.7, 21, 22, 39 and 60 megapixel cameras, that each have mirrored backups stored offline and off premises.
Easily 300,000+ unique images.
Is one .lrcat library referencing one lrdata set of previews going to be left standing when I'm finished?

While I have no experience with such massive amount of data in Lightroom, I can tell you that there is no upper limit built-in to the software that would prevent a catalog from operating on 300,000 images. You give us great detail about your hard drive set-up, and the speed of these drives certainly does affect catalog performance in LR, but the computer specs also affect catalog performance, and the only way to determine if it will be fast enough for you, is to try a large catalog. I also point out that processing 21mb originals (and larger) will be a source of slowness that is distinct from the size of the catalog.
I personally do not like multiple catalogs for a variety of reasons, but I am not a professional photographer. I do acknowledge that if your photographs can be split into non-overlapping sets (such as personal and work; or wedding photos vs landscape photos), then multiple catalogs certainly makes sense. Some pro photographers use one catalog per job, and other pro photographers use one large catalog.

Similar Messages

Is there a practical limit to catalog size?

I get frequent enough requests to use old photos so I keep everything in one catalog, and all raw files on disk in directories by year\day. Currently there are several hundred thousand files (about half unedited), and the catalog size at last check is over 4gb. I just keep adding adding primary and backup disk drives when needed because they are relatively cheap (IBM has announced the first exabyte array so that trend should continue). The only problem I've seen is the catalog is slow to backup but nothing that cant be worked around. The access to photosm search, or editing has not seemed to have slowed at all.
My question is: Has anyone run into a problem with this approach?

What worked for me was to turn off the auto normalise preference in Garageband.
See this thread -
http://discussions.apple.com/thread.jspa?threadID=1403099&tstart=0
Hope it works for you

Catalog size and Speed

I have 8GB of Ram. Approximately how big would my catalog have to be in Lightroom 4 before I started seeing slow down?
Thank You for any information,
Sandy

Catalog is not kept in ram, so not much of a factor here...
Most features of Lightroom are not affected by catalog size, since Lightroom is usually reading/writing pre-computed locations on disk... so for most features, there will be no slowdown whatsoever regardless of catalog size
That said, when entire catalog will need an update, or you are viewing consolidated metadata for entire catalog (or searching entire catalog...), then speed will be very much influenced by catalog size (not so much by ram amount). - but it's not a threshold thing: in other words, these things will gradually consume more time the more photos you have...

Changing to a paper size that is not listed

How do I change to a paper size that is not listed in the drop down menu? Thanks /

What paper size? There are minimum and maximum paper sizes the printer can handle. There are additional limits on the sizes that can be printed borderless. To set custom paper sizes in Windows 8.1 do the following: go to the Devices and Printers folder. Single click on the HP Photosmart 7520 series. Near the top of the Devices and Printers folder will be a new item "Print Server Properties". Click on this , then click Forms, Create a new form, type a Form name, set the proper size and margin size, Save Form, Apply, OK. Next close any open printer windows, the right click on the photosmart 7520 and select Printing Printing preferences. The new paper size should now be available.

Are there practical limits on elements organizer catalog size?

I have 90K items in my PSE11 organizer catalog. I'm running Win8 and I'm getting nervious that catalog size will become an issue. Backing up the catalog is cumbersome since it is 337G. If I need to break it up into smaller pieces, what is the best way to do this? Is there a bridge or link between catalogs? Thanks

See:
http://www.johnrellis.com/psedbtool/photoshop-elements-faq.htm#_Limits_on_the
and:
http://www.johnrellis.com/psedbtool/photoshop-elements-faq.htm#_Splitting_and_rearranging
and:
http://www.johnrellis.com/psedbtool/photoshop-elements-faq.htm#_Merging_Catalogs
The only real problem with big catalogs is what you have found : it's long and tedious to do backups. From the above links you can see that it's a bad idea to split catalogs, unless there is nothing common within them. Contrary to Lightroom, the organizer offers no satisfactory solution to merge catalogs. And the size of the catalog has no real impact on the speed of your organizing.
My suggestion would be to think about alternative backup procedures : anyway, if you want to upgrade to Lightroom, you'll have to choose your own backup system for picture files. LR only offers backups for the catalogs (database).
- Once you know where your big catalog is stored (menu help / system info) it's easy to create a desktop shortcut so that you can save the catalog itself. That's very quick compared to the backup of all your files (several hours).
- Consider using PSE incremental backups. I am not fond of them, but they are acceptable if you master the process : they save the database fully each time and let you restore to a given backup sesssion. Many users rely on incremental backups without understanding the process and are unable to do a complete restore in case of a big problem.
- Consider using external backup tools (as if you were with Lightroom). They can be easier and they offer automatic and periodic backup workflows. I am using Synctoy with Win7 and do periodic clones of my drives with Acronis
- Continue to use the integrated backup procedure when you have to change OS, computer, drives... and migrate all your pictures and catalog.

Iphoto and downloading from Flickr - is there an upper limmit?

Hi
I am a flickr user and have a album of over 3.000 pictures that I want to download to my mac using Iphoto. But I can only get 500 of those photos?
Is there an upper limmit in Iphoto of 500 and can this be fixed so that I can get all of my 3.000 photos down to my mac?

While I have no experience with such massive amount of data in Lightroom, I can tell you that there is no upper limit built-in to the software that would prevent a catalog from operating on 300,000 images. You give us great detail about your hard drive set-up, and the speed of these drives certainly does affect catalog performance in LR, but the computer specs also affect catalog performance, and the only way to determine if it will be fast enough for you, is to try a large catalog. I also point out that processing 21mb originals (and larger) will be a source of slowness that is distinct from the size of the catalog.
I personally do not like multiple catalogs for a variety of reasons, but I am not a professional photographer. I do acknowledge that if your photographs can be split into non-overlapping sets (such as personal and work; or wedding photos vs landscape photos), then multiple catalogs certainly makes sense. Some pro photographers use one catalog per job, and other pro photographers use one large catalog.

PSE 8 Catalog Size?

Can anyone tell me if there is an optimal catalog size in PSE 8? I just started using the software and I've imported my 26,000 images (20% raw / 80 % jpeg) and I'm wondering if I should break that up to get better performance on my PC.

Thanks,
I did not import (upload) so many images to PSE8 in organizer yet. I did de-activate the auto-analynizer, optimized and repair the cataloque as you mentioned. However the organizer just shut down or react so slow or even jam, don´t work. So I have to close the organizer. Somehow I found it is not a easy organizer to use. Too many functions that I do not need and some I need, but not provided. I like my thumbnail showed as my ordered filename. And it ordered them by date (oldest or newest). I would prefer a simple organizer instead of such useless monster.

Catalog Size (PE6)

Is there a limit to the size of the catalog in PE6. My Catalog is approaching 178 MB and thumbs cache is nearing 1.5GB.
The Organizer Keyword mapping function is responding very sluggish even after performed a Catalog repair and optimazition.
Any suggestions on a fix?

No, there isn't any limit on catalog size. Depending upon the number of assets in catalog, size of catalog keeps on growing.
Just curious to know following-
- As thumbs cache size is 1.5 GB, how may assets do you have in catalog?
- How come, catalog size is 178 mb and thumbs cache size is 1.5 GB (I guess catalog size you mentioned is for catalog.pse6db file only)?
~Andromeda

Elements 4.0 Organizer - Catalog Size?

Hey all,
I am about to organize a lot of digital scrapbooking files and I am considering doing it in my current pictures catalog. I will put an old date on them and tag them so they will not interfere with my photos. I am approaching 10,000 items in my current catalog. Is there a practical limit on the number of items in a catalog. I can always start another catalog, but I had rather not if I can get away with it.
Thanks in advance,
Rob Hix

There is no firm limit on catalog size. Most likely, it depends on your computer's memory and speed, and your tolerance for the appearance of speed or the appearance of slowness. Catalogs of 65K have been reported running well.

Cataloge size & location

When I go to System info under help it says that my catalog is 49.6MB & cache Size is 532.9 MB & gives me the location.
I know for a fact that My Pictures file is over 15GB.
Can I assume therefore that the catalog is not kept with the actual pictures & that the catalog size is correct?
Thanks in advance

The Old Fart a écrit:
When I go to System info under help it says that my catalog is 49.6MB & cache Size is 532.9 MB & gives me the location.
I know for a fact that My Pictures file is over 15GB.
Can I assume therefore that the catalog is not kept with the actual pictures & that the catalog size is correct?
You are right. 'My Pictures' is your 'library', it holds all the picture files. The catalog location you found holds the database which is the heart of the catalog, together with other files for the thumbnail cache, face recognition etc. Generally, the thumbnail cache (thumb.5.cache) is the biggest element in the catalog.
By default, the catalog is where you found it. You can move that catalog folder elsewhere, for instance if your picture files are on an external drive in a master folder such as 'F:\My external Pictures', you can move the catalog folder as 'F:\My ext catalog'.
I suppose you have all your picture files in 'My Pictures', so the size of more than 15GB seems right.
For users having their pictures scattered over several drives or subfolders, it may be harder to know the total size of image files.
The way to calculate it is to do the first steps of a 'full backup'. There is a step before the real backup where that size is calculated. You get the size of the 'library' plus the size of the 'catalog', that will be the size of the backup folder which will hold both.

Is there a way to only catalog backup set not in the catalog? "CATALOG START WITH" generates double entry.

The DB version is 11.1.0.7.
The "CATALOG START WITH" not only catalog backup set that are not in the RMAN catalog, but also catalog a second copy for files that are already in the catalog.
Files are not a second copy; they are the same file in the same directory.
Having two copies in the catalog, through an error when deleting explicitly the backups.
We have to add the force clause to avoid the error.
RMAN> delete FORCE backup tag 'offsite-1';
The force clause ignores any I/O error.
The delete is part of our scheduled offsite backup script, which returns a false positive error when the force clause is ommited.
I don't want the force clause, in order to be notify if the backup set that are expected to be on disk are no more there, e.g., if someone had deleted files manually at the OS level bypassing rman.
Is there a way to only catalog backup set not in the catalog?
or May be remove the copy #2 entry from the catalog?
Find bellow a test case example:
RMAN> list backup summary TAG 'OFFSITE-1';
List of Backups
===============
Key     TY LV S Device Type Completion Time    #Pieces #Copies Compressed Tag
12      B F A DISK        11-JUN-13 09:41:45 1       1       NO         OFFSITE-1
13      B F A DISK        11-JUN-13 09:43:31 1       1       NO         OFFSITE-1
14      B A A DISK        11-JUN-13 09:43:33 1       1       NO         OFFSITE-1
15      B F A DISK        11-JUN-13 09:43:34 1       1       NO         OFFSITE-1
RMAN> CATALOG START WITH 'C:\backups\offsite_disk\IBISdatabase\scheduled_offsite';
searching for all files that match the pattern C:\backups\offsite_disk\IBISdatabase\scheduled_offsite
cataloging done
List of Cataloged Files
=======================
File Name: C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\scheduled_offsite\06OBTJEP_1_1
File Name: C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\scheduled_offsite\07OBTJI3_1_1
File Name: C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\scheduled_offsite\08OBTJI5_1_1
File Name: C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\scheduled_offsite\09OBTJI6_1_1
RMAN> list backup summary TAG 'OFFSITE-1';
List of Backups
===============
Key     TY LV S Device Type Completion Time    #Pieces #Copies Compressed Tag
12      B F A DISK        11-JUN-13 09:41:45 1       2       NO         OFFSITE-1
13      B F A DISK        11-JUN-13 09:43:31 1       2       NO         OFFSITE-1
14      B A A DISK        11-JUN-13 09:43:33 1       2       NO         OFFSITE-1
15      B F A DISK        11-JUN-13 09:43:34 1       2       NO         OFFSITE-1
RMAN> DELETE BACKUP TAG 'OFFSITE-1';
List of Backup Pieces
BP Key BS Key Pc# Cp# Status      Device Type Piece Name
24      18      1   1   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\06OBTJEP_1_1   <<--- same file
30      18      1   2   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\06OBTJEP_1_1   <<--- same file
25      19      1   1   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\07OBTJI3_1_1
31      19      1   2   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\07OBTJI3_1_1
26      20      1   1   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\08OBTJI5_1_1
32      20      1   2   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\08OBTJI5_1_1
27      21      1   1   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\09OBTJI6_1_1
33      21      1   2   AVAILABLE   DISK        C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\09OBTJI6_1_1
deleted backup piece
backup piece handle=C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\06OBTJEP_1_1 RECID=24 STAMP=821470356
deleted backup piece
backup piece handle=C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\07OBTJI3_1_1 RECID=25 STAMP=821470356
deleted backup piece
backup piece handle=C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\08OBTJI5_1_1 RECID=26 STAMP=821470356
deleted backup piece
backup piece handle=C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\09OBTJI6_1_1 RECID=27 STAMP=821470357
Deleted 6 objects
RMAN-06207: WARNING: 4 objects could not be deleted for DISK channel(s) due
RMAN-06208:          to mismatched status. Use CROSSCHECK command to fix status
RMAN-06210: List of Mismatched objects
RMAN-06211: ==========================
RMAN-06212:   Object Type   Filename/Handle
RMAN-06213: --------------- ---------------------------------------------------
RMAN-06214: Backup Piece    C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\06OBTJEP_1_1
RMAN-06214: Backup Piece    C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\07OBTJI3_1_1
RMAN-06214: Backup Piece    C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\08OBTJI5_1_1
RMAN-06214: Backup Piece    C:\BACKUPS\OFFSITE_DISK\IBISDATABASE\SCHEDULED_OFFSITE\09OBTJI6_1_1

Hi JM,
Your demo with Oracle 10.2.0.5
At line 36 you answered "No" to "Do you really want to catalog the above files (enter YES or NO)?".
Just repeate your demo and answer Yes.
Even if he might have replied with "YES" then also, the RMAN search for the files which it can recognize as a valid rman backup file. Else it would show the list of files which are not recognized by RMAN as a valid file.
RMAN> catalog start with '/home/oracle/scripts';
using target database control file instead of recovery catalog
searching for all files that match the pattern /home/oracle/scripts
List of Files Unknown to the Database
=====================================
File Name: /home/oracle/scripts/RMAN_BACKUP.sh
File Name: /home/oracle/scripts/rman_clone.log
File Name: /home/oracle/scripts/Db_Refresh.sh
File Name: /home/oracle/scripts/rman_erp.scr
File Name: /home/oracle/scripts/RMAN_14FEB.log
File Name: /home/oracle/scripts/nid.sh
File Name: /home/oracle/scripts/Full_Clone.sh
File Name: /home/oracle/scripts/rman_staging.scr
Do you really want to catalog the above files (enter YES or NO)? yes
cataloging files...
no files cataloged
List of Files Which Where Not Cataloged
=======================================
File Name: /home/oracle/scripts/RMAN_BACKUP.sh
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/rman_clone.log
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/Db_Refresh.sh
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/rman_erp.scr
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/RMAN_14FEB.log
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/nid.sh
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/Full_Clone.sh
RMAN-07517: Reason: The file header is corrupted
File Name: /home/oracle/scripts/rman_staging.scr
RMAN-07517: Reason: The file header is corrupted
About the RMAN catalog, it will not catalog the same file twice. It will only catalog the files which are not known to the database.
See this demo. (/backup/RMAN/STAGING/ is the location where all the backup pieces are.)
RMAN> catalog start with '/backup/RMAN/STAGING/';
searching for all files that match the pattern /backup/RMAN/STAGING/
no files found to be unknown to the database
This was the location based test, Another demo showing piece by piece catalogging .
RMAN> list backuppiece '/backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00';
List of Backup Pieces
BP Key BS Key Pc# Cp# Status      Device Type Piece Name
1323    1321    1   1   AVAILABLE   DISK        /backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00
RMAN> catalog backuppiece '/backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00';
cataloged backup piece
backup piece handle=/backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00 RECID=1324 STAMP=821717283
RMAN> list backuppiece '/backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00';
List of Backup Pieces
BP Key BS Key Pc# Cp# Status      Device Type Piece Name
1324    1321    1   1   AVAILABLE   DISK        /backup/RMAN/STAGING/ERP_CNTRL_c-13949842-20130724-00
Hope its understandable, if any query please revert.
Happy to Help.
Thanks and Regards,
Prashant

When using the "Fill & Sign PDF" feature, is there a way to stretch to size the text box, for a specific area on the page...instead of it going on in one straight long line...?

When using the "Fill & Sign PDF" feature, is there a way to stretch to size the text box on a specific area of the page...instead of the text box going on in one straight long line....? I'm not seeing there's an option or ability to do so, just wanting to confirm.

Improving the handling of multiple line text fields is in our plans, but for now, you will have to add manual carriage returns (Enter).
Thanks,
Josh

Is there a way to change the size of a sticky note attached to a page? I would like to make the √ l

Is there a way to change the size of a sticky note attached to a page? I would like to make the √ larger.

Yes - click on the themes button in the toolbar and select the new size and type - depending on your selection some text may not fit the new theme - usually everything is ok
LN

When converting a document on Pages to PDF, is there a way to shrink the size of the PDF document?

When converting a document on Pages to PDF, is there a way to shrink the size of the PDF document?

When exporting you are given 3 options:
Good, Better, Best
But I am guessing you are asking something else, just haven't said so.
Peter

Is there a way to reduce the size of my PDF file, created (or I should say, Exported) in InDesign.

I created a 10-page sale flyer in InDesign CS5 which will be viewed on our website. When I export it as a PDF file, even with the compression at "minimum", the size is still 3.8 MB.
Is there a way to reduce the size of the PDF file? with InDesign or another tool out there?
Thanks for your help!
Blee

Rik Ramsay wrote:
To expand a little on what Jongware said, normally choosing vector over bitmap will decrease the filesize. If the vector art is very complex though, this will not be true.
Yup, a logo as bitmap takes up more space than the same as a vector -- unless the vector image is bitmap heavy (think Drop Shadow, Outline Glow; these are actually bitmap effects).
Also, check your PDF version. "Older" versions do not support live transparency, the latest do. Then again, you cannot use live transparency if you suspect your viewers may be using Mac OS X, because Apple's own PDF viewer sucks big time.

Is there an upper threshold of catalog size that shouldn't be crossed?

Similar Messages

Maybe you are looking for