AME watch folder - hangs when too many files

I have about 350 vob files in a watch folder, ever time I start up AME it seems to start processing them but it just starts "not responding". I noticed that it is writing files to the Media Cache folder but then just eventually stops after about a min or two. If I only put like 20 files in the folder it works ok.
Is there a way to disable the media cache files from being written? I don't know that they are necessary just for what I'm doing (i'm not previewing or anything). I don't know that it is the issue but it seems AME is hanging on the # of files and the fact that they are VOB/mpeg2. Has anybody ever experienced this?
The # of files is not a one time deal, this will be a reoccurring issue and it would be much nicer to be able to just dump all the files and be done.
I'm using CS5.5, Win 7 64bit, recent i7 Sandy Bridge proc, 16GB ram. The watch folder is on a Synology NAS connected via gigibit, maybe that is part of the problem (that it is not local). The media cache folder is on a local 2TB drive (WD Black) with plenty of room.
Any insight would be appreciated.
thanks
Ryan

The fact that the watch folder is not local to the AME machine may indeed factor in.The process is definitely more robust in a local scenario; network environments complicate matters in a number of ways. Can you run a test where you copy the same batch of files to a watch folder on the AME box?
A few other questions:
Is the "Auto-Encode Watch Folders" option enabled (at the top right of the Queue panel)? And are all 350 files getting added to the AME queue?
If Auto-Encode *is* enabled and some jobs are *not* getting added to the queue, then please try with Auto-Encode enabled.
We're not suggesting this as a workaround but just to give us more insight into where things are failing.
What are CPU and memory usage though the process, from the time you drop files into the watch folder until AME hangs? Are either getting maxed out?
How big are the VOB files? How long does it take to copy all of them into the watch folder?
I know of no way to disable the creation of media cache files.
Thanks for bringing this issue to our attention and giving us an opportunity to troubleshoot it.
Mark Mapes
Premiere Pro/AME Sr. Quality Engineer

Similar Messages

CS5 Photomerge hanging when using many files

Hello,
I have managed to work around the issue, but it's annoying nonetheless.
The basic problem is that I've started creating very wide and tall Bokeh Panoramas or 'The Brenzier Method' http://www.ryanbrenizer.com/category/brenizer-method/
From Lightroom I select the images I want and 'merge to panorama in PS'. When I use say 2 - 30 images it's fine. However when I use say 160 (ridiculous I know) it will hang when merging the files, processor usage stays on 13%, but the progress dialogue bar doesn't animate. I tried it through photomerge and 'align all layers' from the edit menu. I reduced this ridiculous amount to 60 and if I chose the 'vignette removal' option on the photomerge dialogue it will hang on this process after starting to create the panorama. If I don't do the vignette, it will create the panorama.
I know that it will depend upon the complexity of the merge, it's just annoying that it would fail whereas Microsoft ICE will complete the panorama without fail, it just may be of a lesser accuracy due to parallax.
Other than using less images and attempting to overlap correctly, is there much else that can be done?
I'm using an up to date version of CS5 64bit windows. running on a 16gb Core i7 2600k. The merged final image occupies 13gig of ram. I have the scratch disc on a spare 2tb drive with about 1.5 tb free.
Paul

Mr_Fujisawa, I just wanted to come back on here and thank you for opening my mind to the "Brenizer Method", which I had not heard of before. I've been experimenting with it today and enjoying the results! Up to now I put effort into getting everything in focus all across my panos, which are generally landscapes. It didn't dawn on me to use stitching on subjects up close.
-Noel

Too many files open when parsing xml

HI,
I have an application which needs to parse about 13 XML files multiple times.After parsing the XML and creating a DOM I put in values to create an XML message which is then put onto a message queue.I am getting a 'Too many files open' problem.I have run the loop a maximum of 105 times after which i get the problem.However I will need to do this atleast more than 10000 times.Can someone suggest a solution to read the XML, create a DOM and populate the DOM without encountering the above mentioned problem.If I just parse the XML and store the 13 DOMs in a HashMap and use the same, the text values from the previous call get appended resulting in an invalid XML.To overcome this we parse the XML each time a message is to be sent.We also tried increasing the system memory as suggested at some sites but the solution was of no help.
Thanks & Regards

Don't know what this has to do with Web Start but you should make sure that, after parsing a file, the corresponding InputStream is properly closed.

FCP XMLs into AME watch folder

Hi, recently Adobe Media Encoder added support for FCP XML files. I am exporting FCP XMLs from Premiere into an AME watch folder but no job is being created in the queue, despite having 'Auto-Encode Watch Folders' turned on and 'Start encoding automatically when idle for:' set to 0 minutes. If I manually add the same FCP XML file to the encode queue, the job is created and processes correctly. Has anyone been able to make AME process FCP XMLs automatically using watch folders?
The point of doing this is to potentially free up workstation resources by doing all rendering on dedicated AME machines, allowing quicker turnaround of the kind of short, simple edits that our users do.
This is on OSX Mavericks, Adobe CC Premiere 7.2.1(4) and Adobe Media Encoder 7.2.0.43.
thanks

[Moved to AME forum.]

HT2506 Too many files in Preview

I opened too many files in preview. 5 or so Powerpoints and 3 PDFs. They are all stuck in a pinwheel load page. Is there a way i can exit them? they don't respond at all but the application is not recognized as "not responding". Force quit only closes preview, when i open it back up it just starts again trying to open the windows.

From the menu bar, select
 ▹ Force Quit...
then select the frozen application from the list and press return. Relaunch it by holding down the shift key and clicking its icon in the Dock or in the Applications folder.

Word cannot complete the operation because too many files are open

Hello,
I'm working on a 60 pg. document, and when I hit 'save', I get this annoying msg.:
"word cannot complete the operation because too many files are open." I have no other files open, and I am unable to continue to work, because I cannot save - can anyone help? Thanks!

Hello Kristina,
Have you been able to save the file at all?
More than anything, it seems to be a bug when you're trying to save to a server. There's more on the problem here. So what you can also try is simply saving the file to your desktop, or somewhere on your Mac's hard drive. That way you're saving the file locally instead of to a server. If that works, you've at least saved the file and can continue on. Otherwise, read on.
If you can't do it from Word, then at least save your changes from another application. Open TextEdit. You should get a blank file on the desktop. Make sure it's set as a rich text file. You'll know just be looking at it. If you see a ruler and other buttons at the top of the TextEdit document window, it's rich text. If not, make it rich text by pressing CommandShiftT.
Copy and paste your entire Word text into the TextEdit window. Save the TextEdit document to the desktop. Now that you've at least gotten your work saved, you can try some other things.
If you're not trying to work on the file from, or save it to a remote server, then don't worry about the first workaround. It doesn't apply. Number two was poorly worded by John. He seems to assume everyone knows what RH is. I had to look it up. It means Remote Home folder. So open the Word preferences and click on FIle Locations. Highlight the first choice, Documents, and click on the Modify button. Choose any location you want on your Mac's hard drive. Click "Choose" and close the preferences. Can you now save the document to the folder you defined as the default save location?
Spotlight is that little magnifying glass in the upper right corner of your screen by the time/date. There are a few methods to disable Spotlight. What John is saying that if the Mac is spending a lot of time indexing your drives at the same time you're trying to save your Word document, the intense processing time to index may be keeping Word from saving.
If you've managed to at least save your file as a rich text TextEdit document, then shut all of your applications down and restart your Mac. Launch Word and open the TextEdit file. You may lose some formating if you've done tables or other things TextEdit can't save in Word's original form.

Photoshop CS6 error "could not complete ... too many files open. Try closing........." with 3D psd?

Im working in a .psd file with multiple 3D layers. The file size totals at 356mb.
When opening the file, I get this error: "could not complete your request because there are too many files open. Try closing some windows and try again." This doesn't any sense to me, as I was not experiencing any chugging when the file was open, and I am on a relatively fast system.
Win 7 64bit
Intel i7-2760QM @ 2.4GHz (8 CPU)
16 gigs ram
Nvidia GTX 560M (2752mb)

Having the same issue on the Mac!
Worked on a file yesterday, fairly big, but nothing that I haven't handled before... Only difference was the 3DS file I imported has over 270 textures.
I think there is a limit on how many textures can be imported, essentially corrupting the file on save. It would be a lot more helpful to have Photoshop warn us that the 3D model is too complicated before we import it and save our work looking forward to finishing off the next day.
I tried the same model again today, did nothing other than position it and render it, then saved, same error on opeing: "Could not complete your request because there are too many files open. Try closing some windows and try again."
If anyone at Adobe would like to have a look ta the file the smaller version that I did today I can upload somewhere or send a YouSendIt.
Cheers.

Photoshop CS5.1 Extended: Could not complete your request because there are too many Files Open

I saved a file last night and now this morning I get an error message stating: "Photoshop CS5.1 Extended: Could not complete your request because there are too many Files Open. Try closing some windows and try again.". I've rebooted the system and made sure Photoshop was the only program running and that there were no other files open prior to trying to open my file. I increased Photoshops ram usage, checked the OpenGL settings, 3D and VRAM settings, cleared my temp files and am at a loss for opening this file.
System Specs:
Win 7 64
6 core I7 processor
32gb Ram
3GB VRAM
600gb free space on 7200 rpm drive
File specs:
1 3D object replicated maybe 5 times (yes, I should have rasterized them, and will do that to 4 of them if I get the file open)
Maybe 8 textures
Not sure how many total layers.
Is there anyway to get Photoshop to reopen this file?

No, more VRAM is useless. I had the same thought as you so I tried with a friend's computer with 3GB of VRAM for 8GB RAM (mine has 1GB VRAM for 16GB RAM). Still got the same error message. Sorry guys but our files are lost until Adobe fix the error in Photoshop code. I hope in the future they will find a way to either fix this error or give a clear limitation, it is really a shame such a dangerous problem exists and persists for more than 3 photoshop gen.
Still you may be able to have at least a flat image of your latest file by trying to open the file as smart object. I started my work over by doing that. Not a lot but better than nothing.
However for my new file I am trying something different : Each time I think a 3D file is final, I do not rasterize it but make it into a smart object and save the new smart object into a single separated .psd file. If any change is needed I can still edit all of my 3D objects by opening them separately. So far it works, the global file is lighter with few to no 3D objects inside but I will not say it is a 100% no-risk method yet.
Last advice I can give everyone : Before closing the file you are working on, save it as a copy (File / Save as... and check 'As copy' in the options below) then with your main file still open, try to open the copy. You may need a lot of memory if your file is heavy but still if the dreaded error message appears when you try to open the copy then remove or rasterize some 3D objects in your main one. Save again as copy and try to open the copy until you do not get the message anymore.

JspSmartUpload uploading too many files

Has anyone else had a problem with too many files being uploaded?
My code is as follows:
<FORM NAME="frmAddDoc" ACTION="process_add.jsp" METHOD="post" ENCTYPE="multipart/form-data">
<%@ include file="attachments.html"%>
and attachments.html code specifies two files to be uploaded as follows:
<TR>
<TD CLASS="formFieldLabel">Source Document</TD>
<TD><INPUT TYPE="File" NAME="sourceDocFileName" SIZE="12" CLASS="fileField"></TD>
<TD><IMG SRC="../shared/images/shim_trans.gif"></TD>
</TR>
<TR>
<TD CLASS="formFieldLabel">Published Document</TD>
<TD><INPUT TYPE="File" NAME="docFileName" SIZE="12" CLASS="fileField"></TD>
<TD><IMG SRC="../shared/images/shim_trans.gif"></TD>
</TR>
In process_add.jsp there is a call to the UploadBean.upload method which is as follows:
public UploadResult upload(PageContext pageContext, String CSVFileTypesAllowed)
StringBuffer statusMsg = new StringBuffer();
int uploadedFilesCount = 0;
// Upload initialization
uploadHelper.initialize(pageContext);
uploadHelper.setAllowedFilesList(CSVFileTypesAllowed);
uploadHelper.upload();
uploadedFilesCount = uploadHelper.save(uploadTempDir, uploadHelper.SAVE_PHYSICAL);
return uploadResult;
This works most of the time but occasionally four files are uploaded instead of two (i.e. uploadedFilesCount and uploadedFiles.getCount() are both 4 when they should be 2).
(UploadHelper = new SmartUpload)
Has anybody any ideas what could be causing this?
I having been chasing this "bug" for a week now and it has me beat
Can anybody help?
Thanks

Hi
Try looking for the jspSmartUpload documentation, nobody here or atleast 98 % of the people do not use these third party tools. But rather develop them on their own. So it might be difficult for you to get an answer for your post.
Swaraj

Too many files on 10.4 Desktop - Finder won't load

2.66 DualCore Intel Xeon + OSX 10.4.10 + 2Gb DDR2 DIMM + 230Gb ST3250820AS + OPTIARC DVD RW AD-7170A
Despite recurrent warnings, a relative kept leaving too many files on the Desktop, up to 300 files (JPGs, downloaded archives). Last week, after attempting to move/copy them to a folder, OSX crashed, and now Finder won't load; after password screen, menu bar and Dock appear, the 'MyDay' app (from Entourage) starts bouncing, and system freezes. Sometimes it goes back to password screen, and from there again to a frozen, empty finder with dead mouse pointer. Other times the spinning clock or wheel remain spinning forever.
Tried DiskUtility, repaired disk and permissions, reset PRAM, no dice.
Tried to boot off the 10.4.10 DVD, it doesn't allow to run OSX from the disc (I have a vague recollection of doing this once, but...)
Started in SafeBoot mode, finder froze at same stage.
Re-Installed OSX 10.4.10 in Archive and Install mode, same freeze at same stage.
The only explanation for the crash, as no updates/new app installing happened, is that there are, again, too many loose files on the desktop and as it tries to load the Finder it crashes. I'm hauling the machine ASAP to try with my own Intel to try access his disk via FireWire target mode but worry the same thing will happen on my system, that it will balk at too many files.
Is there a way to try to fix this without resorting to clean install (needless to say, last backup was months ago)? If I install Leopard on that disk (it comes out fine in Disk Utility) will the same thing happen as soon as it tries to load the Finder? Is there a way to boot 10.4 off a CD/DVD?
Thanks for any info or tips!

Welcome to the forums!
Make your relative write this out 1000 times:
Performance tip: Keep the Desktop clutter-free (empty, if possible)
Mac OS X's Desktop is the de facto location for downloaded files, and for many users, in-progress works that will either be organized later or deleted altogether. The desktop can also be gluttonous, however, becoming a catch-all for files that linger indefinitely.
Unfortunately - aside from the effect of disarray it creates - keeping dozens or hundreds of files on the Desktop can significantly degrade performance. Not necessarily because the system is sluggish with regard to rendering the icons on the desktop and storing them in memory persistently (which may be true in some cases), but more likely because keeping an excessive number of items on the Desktop can cause the windowserver process to generate reams of logfiles, which obviously draws resources away from other system tasks. Each of your icons on your desktop is stored as a window in the window server, not as an alias. The more you have stored, the more strain it puts on the window server. Check your desktop for unnecessary icons and clear them out.
Keeping as few items as possible on the Desktop can prove a surprisingly effective performance boon. Even creating a single folder on your Desktop and placing all current and future clutter inside, then logging out and back in can provide an immediately noticeable speed boost, particularly for the Finder.
And it is why Apple invented 'Stacks' for Leopard.

Too Many Files Opened

All,
I am running on Fedora Core 10 (2.6.27.21-170.2.56.fc10.x86_64) with Java 1.6.0_12-b04 (32-bit and 64-bit get the same issue) and am getting a "java.io.FileNotFoundException: ... (Too many open files)" error when running my unit tests. I have been very careful about placing things in a try { } finally { stream.close(); } block and can't figure out what is actually going wrong.
The unit tests actually work fine on Windows XP, which points me to either Java or Linux itself.
After doing some research online I found some articles and postings:
- http://lj4newbies.blogspot.com/2007/04/too-many-open-files.html
- http://www.mail-archive.com/[email protected]/msg15750.html
- http://www.linuxforums.org/forum/redhat-fedora-linux-help/73419-we-facing-too-many-open-files-regularly.html
However, is there anything else I can do within Java itself (using a VM argument, code "best practices" that I may be missing, etc. to combat this?
My Linux core configuration as based on the articles are as follows:
$ cat /proc/sys/fs/file-max
564721
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 55296
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

tjacobs01 wrote:
The unit tests actually work fine on Windows XP, which points me to either Java or Linux itself.OR another app on your linux machine that's got a lot of files open
OR you installed java as yourself and not system and you don't have priveledges to open the files
OR your code has a bug somewhere that doesn't effect running it on windows
99.999% of the time, the problem is with you, not with Java or LinuxThis actually seems to be it. I rebooted the system into console mode to avoid X from loading. I then reset the JDK to be setup by root (I use the self-extracting, non-RPM version). When I ran the test suite via Ant here it did not show the problem. Therefore, I'm fairly convinced it is actually some other application eating up too many files.

Too Many Files Open Currently

When I try to sync my iPod Touch 2G and there are a large number of files to sync/update (100+), I receive an error message that says "Attempting to copy to the disk "iPod Touch 2G" failed. There are too many files currently open."
When I receive that message, I have to click OK and then restart my Airport Extreme (AE) via the Airport Utility. Before I restart the AE, I can still access the internet, but I cannot access the USB External Drive to which the AE is connected and which contains my iTunes library.
It seems as though iTunes and the Airport Express are not communicating "nicely" and that something iTunes is doing is causing the Airport Express to "choke" on the data being synced. The network is hardwired/ethernet rather than wireless.
This problem has occurred with both iPod touches in our house. I have tried uninstalling and reinstalling iTunes and Quicktime along with resetting both iPod Touches to their factory settings.
Thanks in advance for any ideas you might have!

turn off itunes sharing

IPod won't sync "too many files currently open"

Every time I try to connect my iPod there is an error that says "cannoy sync iPod too many files currently open." I'm not sure what this means because I only have iTunes open and it still will not sync. Please help! Thanks!

I had the same problem. I just turned off my shared playlists and restarted my itunes. it worked after that

Final Cut Express keeps stopping and saying "Too many files open"

Final Cut Express keeps stopping and saying "Too many files open". I don't see how this is the case as I only have the one project open...

Try here:
http://www.fcpbook.com/Misc1.html
Al

Sync fails in iTunes 9.0 & 9.01: "too many files open" and error -6999

I've been trying to synch my iPod (Touch, g1) for the last couple weeks but have been getting errors ever since going to iTunes 9.0.
First it blew my library and now says "Previous iTunes Libraries" at the top. Can live with that (after going through and reloading about 2/3s of the album artwork).
But now am getting two errors every time I try to sync. One is, "Attempting to copy to the disk '[my iPod's name]' failed. There are too many files open currently."
Elsewhere I saw it suggested for this error that I turn off sharing, but, alas, sharing isn't turned on.
The second error message is, "The iPod '[my iPod's name]' cannot be synced. An unknown error occurred (-6999)." and I can't find any mention by Apple about what to do about this.
HELP. Please.

There has been some advice to turn off "sharing" to stop getting the "too many files open" error message, but, as noted in my question, sharing was already turned off in the iTunes Preferences.
But, it turns out, of course, that there is another kind of sharing in the 9.x iTunes -- the one for sharing libraries on your local network. Don't know why they are different because the function seems the same, but I noticed that there was still an icon for "home sharing" or "house sharing" (little picture of a house) on the library/playlist list in iTunes.
I right clicked on the house icon and found another checkbox switch there. I turned that one off too (Why Apple?) and, voila, the "too many files" error has (apparently) gone away.
Hope this helps some others out there.
Apparently you have to turn off sharing in two different places. (Again; why is that, Apple?)

AME watch folder - hangs when too many files

Similar Messages

Maybe you are looking for