Qmaster cluster batch results in random -2048
I used to notice this back in the day with 24bit audio exports... it's been since fixed, although now it's returned in a far more brutal way
I have 3 machines on a qmaster cluster... an xserve, and two g5 towers. The xserve works as the controller. I recently submitted a whole batch, including files that are either animation or 8bit uncompressed, with either 16 or 24 bit audio. I have each clip compressing down to a mix of h264, sorenson and mpeg4.
After 2 days of exporting, i opened up the output folder and checked the files and notice of 288 files, 65 of them return a "-2048" quicktime error and do not open. The funny thing, its completely random which files get it (not related to certain export codecs or input codecs or anyting of the sort)
So either way, I got no idea what's going on. I'm using Qmaster 2.0.1 across the board, and the only thing I could possibly think of causing this is that I'm clustering off a xsan volume (1.3).
Any ideas would be greatly appriciated.
Hey gusx,
Not the exact same, but I've done a lot of research and talked to a lot of people since my last post on here. Here's another parallel post more related to yours:
http://discussions.apple.com/thread.jspa?threadID=523121&tstart=0
Turns out, Final cut pro, compressor, qmaster and almost anything related to FCP studio that runs off xsan will have this sort of corruption. I personally don't use FCP, but use compressor a ton. I've found compressing a quicktime file directly using "this computer" results in a perfect file.
On the other hand, batching a file over a qmaster cluster results in a damaged 2048 file every one out of four, and ive found doing a 60i to 24p transfer on the highest quality settings very typically produce the green noise over my export (even when not batching over a qmaster). Ive heard tons of similar instances to the green noise/2048 from FCP editors as well.
I spoke with xsan engineering and told them i had my volume set to default 4kb and no dice, and they simply said "yes... we have been able to reproduce this issue... its been an issue since september.. I'm actually shocked the literature hasn't been updated since december... there is no current solution to the problem, no matter what the settings"
Since then, I have had someone from xsan development email me directly and i have reported it to them as a "huge bug" and haven't heard back, so I think they're at least looking into it. I know this is a huge hit to our productivity so I definitly encourage anyone else who is having similar problems to post it up here and hopefully they will solve it in xsan 1.4!
Similar Messages
-
Error when submitting job to Qmaster cluster
Hi all,
I'm new to working with the Qmaster cluster but I created a cluster (at least I think I did it right) using the distributed processing apple document from the help menu. Everything looks right...I have an active cluster with two machines. What is a little weird is that the Cluster that I can choose in Compressor when submitting a job has a format like "ThisComputer.RScomputer.local:50411" instead of the name of the cluster I made (I called it Zeus Cluster).
So, I choose this long cluster name and submit the job but I get this error:
Error: An internal error occurred: NSException raised by 'NSPortTimeoutException', reason = '[NSPortCoder sendBeforeTime:sendReplyPort:] timed out'.
Has anyone seen this error. What could I be doing wrong? Apple Qadministrator has the Cluster active and both machines are sharing fine.
Any help would be appreciated. Thank you!Have you looked in the /Library/Logs/Qmaster for any specific detail? (use /applications/utilities/console.app )
there's usually some detail in there that will give you an insight. If you see smething in there of significance, by all means post it here so w can examine and suggest to you.
I have had his before, and in my case is related to the cluster setup I had. -
Hello, I'm totally new to programming. I started with LabView and test stand to make some test as VI's. When test stand secuence ends, the batch result box (batchmodel.seq) pops out for the several uut's tested in every socket. I'd like to add to the status message the information fo the test wich failed instead of searching this info in the xml (or html) report.
Joel Urbaez
Test Engineer ,
Napco Alarm Lock Intl. Group.Hi Joel:
Every report in TestStand comes with message on top of the report (below the header) that tells you which step in the sequence caused the failure. Hence, you do not need to read through the entire report to get this information. If you still want to use the banner to report this information you can do the following:
1. After the sequence has executed you will have to add a step in the Post-UUT sequence.
2. Here you can either choose to show a message popup where you can display the step that failed
3. You would have to use the following variables:
If ( RunState.Sequence.Locals.ResultList[0].Status = "Failed") ? RunState.Sequence.Locals.ResultList[0].TS.StepName
4. You can also try to modify the modelsupport2.dll file for the functions that bring up the popup banner to add more information.
Thank you
Nandini
NI -
UUT serial number is not displayed in batch results popup window?
UUt serial number is not displayed in batch results popup window. I am using TestStand 2012 SP1 f1 (5.0.1.66). I am also using Prebatch callback and setting up
Parameters.ModelData.TestSockets[index].UUT.SerialNumber with the UUT serial number. I have used the exact sequence file with TestStand 4.2 (4.2.0.134) with no problems.Sorry for my late reply I was away for work. I actually revisited the code that I implemented with TestStand 4.2 (4.2.0.134) and it is showing the exact problem. Sorry for misleading you. however the UUT report is showing the correct serial number for each unit. This would be something that is nice to have but not critical for my project. I am basically auto discovering all my UUT so instead of scanning the serial number I read it from the UUT memory so there are a lot of steps in my PreBatch call-back but they are all related to reading the UUT serial number. Once I read all the UUTs serial numbers I set the following in a for loop:
Parameters.ModelData.TestSockets[Locals.For_Loop_Index].UUT.SerialNumber =StationGlobals.UUT_Serial_Numbers[Locals.For_Loop_Index]
I tried to implement my a simple PreBatch with only a for loop and an array of serial number and I ended up with the same issue. So I am suspecting that the batch report is getting the serial numbers from a different variable and that variable is set when you run the batch option without the PreBatch call-back.
Kind Regards,
Oshana -
Dear All,
I have an issue of quality related inventory movement from one plant to other plant.
From plant X i will move unrestricted stock to Plant Y.
Then stock goes into quality inspection in plant Y.Then again UD is given and the stock will post into unrestricted stock.
But can we copy the batch results from Plant X to plant Y without recording the results again.
Thanks and regards
G.V.RAGHU RAMGVR,
Try to use a UD - Followup action with a custom function module, performing what is required for you.
Regards,
Prasobh -
Qmaster Cluster Storage Location Creating Duplicate Clusters
I'm in the process of setting up a Qmaster cluster for transcoding jobs and also for Final Cut Server. We have an Xserve serving as the cluster controller with a RAID attached via fiber that is serving out an NFS Share over 10GB Ethernet to 8 other Xserves that make up the cluster. The 8 other Xserves all are automounting the NFS Share. The problem we are running into is that we need to change the default "Cluster Storage" location (Qmaster preference pane) to the attached RAID rather than the default location on the system drive. Primarily because the size of transcodes we are doing will fill the system drive and the transcodes will fail if it is left in the default location.
Everytime we try and set the "Cluster Storage" location to a directory on the RAID and then create a cluster using QAdministrator it generates a duplicate cluster spontaneously and prevents you from being able to modify the cluster you originally made. It says that it's currently in use or being edited by someone else.
Duplicated Cluster.
Currently be used by someone else.
If you close Qadmin and then try and modify the cluster it says it is locked and prompts for a password despite the fact that no password was setup for the cluster. Alternatively if you do setup a password on the cluster it actually does not work in this situation.
If the "Cluster Storage" location is set back to it's default location none of this duplicated cluster business happens at all. I went and checked and verified that permissions were the same between the directory on the RAID and the default location on the system drive (/var/spool/qmaster). I also cleared out previous entries in the /etc/exports and that didn't resolve anything. Also, everytime any change has been made servives have been reset and started again. The only thing I can see that is different between using the /var/spool/qmaster and another directory on our RAID is that once a controller has been assigned in QAdmin the storage path that shows up is different. The default is nfs://engel.local/private/var/spool/qmaster/4D3BF996-903A30FF/shared and the cusstom is file2nfs://localhost/Volumes/FCServer/clustertemp/869AE873-7B26C2D9/shared. Screenshots are below.
Default Location
Custom Location
Kind of at loss at this point any help would be much appreciated. Thanks.Starting from the beginning, did you have a working cluster to begin with or is this a new implementation?
a Few major housekeepings (assuming this is a new implementation) - Qmaster nodes have the same versions of : Qmaster, Quicktime, and Compressor (if they have compressor loaded).
The only box that really matters as far as cluster storage location is the controller. It tells the rest of the boxes where to look. On your shared storage, create a folder structure that is "CLUSTER_STORAGE" or something to that affect... then on the controller's preferences pane set the controller's storage to that location. It will create a new folder that is cryptic with numbers and letters and dashes and use that for the storage location for any computer in the cluster.
Now... What I'm seeing in your first screen shot however worries me a little bit.. I have had the same issue and the only way I've found to remedy that is to pull all things Final Cut Studio off that box and do a completely fresh reinstall... then update everything again to the same versions. I'm assuming you're using FCStudio 7.x?
We should be able to get you on your feet with this. Configuring is always the hardest part, but when it gets done.. man it's worth it
Cheers -
Batch Scheduler Failing Randomly
Hi all -
I have a batch scheduled to run on demand. It runs on ~130 custom members. A few months ago it started failing on random custom members. It will run successfully on most, but failed on some with the error:
Problems encountered in "Financial Reports/01 - Monthly Financials/GA Reports/GA Burst Reports"
ADM Error processing results from query: The remote procedure call failed.
Every time the members it fails on are different and the number of failures fluctuates as well.
Oracle Support suggested trying fewer members, but halving the number of customs involved still produces errors.
Has anyone else had experience with this issue?
Thanks,
SarahHi,
I am not aware of a way to schedule calc scripts through workspace.
Though as Rosi said there is the business rules command line launcher, CmdLineLauncher in the bin folder of planning.
You can also run calc scripts using command line with maxl.
Once you have the scripts then you can schedule them.
Cheers
John
http://john-goodwin.blogspot.com/ -
Make sure Compressor 3.5.3 uses Qmaster cluster
How do you make sure Compressor 3.5.3 uses the Q master cluster? I know you have to set up Qmaster, but I don't know what the settings are supposed to be. My MacPro 2,1 has 8 cores.
Also, how do you tell if your version of Compressor is running as 32 bit or 64 bit and how do you change it from one to the other?
I have two identical MacPros but one takes three times longer to process the same files in Compressor. I'd like to get them on even ground. They have the same separate raids as well and they both test equally with AJA and Black Magic speed tests.Steve Garman wrote:
I have two identical MacPros but one takes three times longer to process the same files in Compressor. I'd like to get them on even ground. They have the same separate raids as well and they both test equally with AJA and Black Magic speed tests.
Also, just to point out that you have three threads on this question.
Russ -
SQL Query Result with Random Sorting
Hi Experts,
My Oracle Version : Oracle9i
I have three tables which are given below,
Table Name: check_team
team_id team_code
100 A
101 B
102 C
103 D
Table Name: check_product
product_id product_code
1 XXX
2 XYZ
Table Name: check_team_products
tprod_id tprod_team_id tprod_product_id
1 100 1
2 100 2
3 101 1
4 101 2
5 102 1
6 102 2
7 103 1
8 103 2
Required Output First Time:
team_id team_code product_id product_code
100 A 1 XXX
101 B 2 XYZ
102 A 1 XXX
103 B 2 XYZ
Required Output Second Time:
team_id team_code product_id product_code
100 B 2 XYZ
101 A 1 XXX
102 B 2 XYZ
103 A 1 XXXI need the result as Required Output specified above and also the result has to be random too.. Can someone help me in writing a SQL Query to get results as that?
Added Oracle VersionSo, is it something like this you want?
SQL> ed
Wrote file afiedt.buf
1 with check_team as (select 100 as team_id, 'A' as team_code from dual union all
2 select 101, 'B' from dual union all
3 select 102, 'C' from dual union all
4 select 103, 'D' from dual)
5 ,check_product as (select 1 as product_id, 'XXX' as product_code from dual union all
6 select 2, 'XYZ' from dual)
7 ,check_team_products as (select 1 as tprod_id, 100 as tprod_team_id, 1 as tprod_product_id from dual union all
8 select 2, 100, 2 from dual union all
9 select 3, 101, 1 from dual union all
10 select 4, 101, 2 from dual union all
11 select 5, 102, 1 from dual union all
12 select 6, 102, 2 from dual union all
13 select 7, 103, 1 from dual union all
14 select 8, 103, 2 from dual)
15 --
16 -- end of test data
17 --
18 select team_id, team_code, product_id, product_code
19 from (
20 select t.team_id, t.team_code, p.product_id, p.product_code
21 ,row_number() over (partition by team_id order by dbms_random.random()) as rn
22 from check_team t join check_team_products tp on (tp.tprod_team_id = t.team_id)
23 join check_product p on (p.product_id = tp.tprod_product_id)
24 )
25* where rn = 1
SQL> /
TEAM_ID T PRODUCT_ID PRO
100 A 2 XYZ
101 B 1 XXX
102 C 2 XYZ
103 D 1 XXX
SQL> /
TEAM_ID T PRODUCT_ID PRO
100 A 2 XYZ
101 B 1 XXX
102 C 2 XYZ
103 D 1 XXX
SQL> /
TEAM_ID T PRODUCT_ID PRO
100 A 1 XXX
101 B 2 XYZ
102 C 1 XXX
103 D 1 XXX -
When I do certain OData queries, the results in the Excel worksheet are in random order. In the Power Query preview they are ordered correctly and in PowerPivot they are also ordered correctly. How does Power Query decide how to order rows? And why is
it reordering them?By default, we attempt to download OData feeds in parallel by issuing multiple simultaneous requests for different ranges of the data. When this happens, we don't try to reorder the data when it comes back. Because the return order of the requests is nondeterministic,
it's possible and even likely that the original sort order is lost.
Setting the third parameter to false disables parallel requests, which restores determinism to the data being fetched. -
Batch Rename---in Random Order
Is it possible to batch rename a set of files in a random order?
Making a slideshow of a number of events, but I don't want them in the order shot. I can manually rename and move them, or I could sort by some other order than by name or date, such as file size, I guess, but those options haven't randomized the collection as well as I would like.
Just askin'...
Thanks in advance.I’m not great with Scripts, but this might work:
// adds random five digits numbers at the beginning of jpgs’ filenames in a folder;
// use it at your own risk;
var theFolder = Folder.selectDialog ("select folder that contains the files to be renamed");
// for renaming other files than jpgs change the term in the brackets accordingly;
var theFiles = theFolder.getFiles ("*.jpg");
for (var m = 0; m < theFiles.length; m++) {
var theRandomNumber = String(Math.random()).slice(3,8);
File(theFiles[m]).rename(theRandomNumber + "_" + (theFiles[m]).name)
One can copy the text into a new file in ExtendScript Toolkit (which should be part of Photoshop’s installation) and start running it by hitting the play-button or save it as a jsx-file into Photoshop’s Presets – Scripts; but it really doesn’t do anything within Photoshop, so it may be better not stored there.
Anyway it’s probably be best to run it only on a copy of a folder, because getting the numbers back off later on might be bothersome. -
Random results with random access query
Hi,
when I execute a random access query
(fetchConfig.setResultSetType(ResultSet.TYPE_SCROLL_INSENSITIVE);
fetchConfig.setFetchDirection(ResultSet.FETCH_UNKNOWN)) and iterate over the
result set twice, the second time the objects are retrieved in a different
order, also when I set the ordering of the objects. This only happens if the
first loop touches the last item. If not, everything works fine. I am using
MS Sql Server, microsoft driver, kodo 3.4
regards,
ChristiaanHi,
I am using datastore identity. I do implement equals and hashcode in the
super class, not sure whether that is of any impact. The order of the items
look random to me, however it stays constant after the second loop. The same
order is also produced when I re-start the test application. The output
shows the index of the object I am retrieving, the description of the object
(which is also the order in which the items are created) and their order
attribute. The query sorts on the order attribute. As you can see the first
loop is correct. I now and then have the situation where the last item in
the first loop (retrieve 7) results in the first item (item nr 0) being
retrieved. Same behaviour applies for both jtds and microsoft driver.
first loop:
retrieve 0: item nr: 0 order: 0
retrieve 1: item nr: 1 order: 1
retrieve 2: item nr: 2 order: 2
retrieve 3: item nr: 3 order: 3
retrieve 4: item nr: 4 order: 4
retrieve 5: item nr: 5 order: 5
retrieve 6: item nr: 6 order: 6
retrieve 7: item nr: 7 order: 7
second loop
retrieve 0: item nr: 3 order: 3
retrieve 1: item nr: 6 order: 6
retrieve 2: item nr: 7 order: 7
retrieve 3: item nr: 0 order: 0
retrieve 4: item nr: 5 order: 5
retrieve 5: item nr: 4 order: 4
retrieve 6: item nr: 2 order: 2
retrieve 7: item nr: 1 order: 1
third loop
retrieve 0: item nr: 3 order: 3
retrieve 1: item nr: 6 order: 6
retrieve 2: item nr: 7 order: 7
retrieve 3: item nr: 0 order: 0
retrieve 4: item nr: 5 order: 5
retrieve 5: item nr: 4 order: 4
retrieve 6: item nr: 2 order: 2
retrieve 7: item nr: 1 order: 1
"Marc Prud'hommeaux" <[email protected]> wrote in message
news:[email protected]...
Christiaan-
My first guess is are you using application identity, and if so, are you
correctly implementing the equals() and hashCode() method? Failure to do
so may conceivably cause this problem.
Otherwise, can you provide some details about how the results are our of
order? It it exactly reverse or something, or are elements in random
order?
Christiaan wrote:
Hi,
when I execute a random access query
(fetchConfig.setResultSetType(ResultSet.TYPE_SCROLL_INSENSITIVE);
fetchConfig.setFetchDirection(ResultSet.FETCH_UNKNOWN)) and iterate over
the result set twice, the second time the objects are retrieved in a
different order, also when I set the ordering of the objects. This only
happens if the first loop touches the last item. If not, everything works
fine. I am using MS Sql Server, microsoft driver, kodo 3.4
regards,
Christiaan
Marc Prud'hommeaux
SolarMetric Inc. -
Hi all
So if we have recorded insp results for the Bulk material, then we need the results of this batch to be copied on the 20kg, 100kg pack size materials as well, since itu2019s the same batch.
please check and revert with your comments. Also let me know if you have any other solution on this.It is possible two ways.
1) through QM and inspection results copy. Are you using QM?
2) Batch derivation: See these links for some more info:
http://www.sapsword.com/home/sap-batch-management/sap-batch-derivation-1
http://help.sap.com/saphelp_470/helpdata/en/eb/e289e3b36f6e4890da5b09a5daf8dc/content.htm
FF -
Buggy Qmaster cluster when compressing
I have set up and used the cluster/qmaster method of maximizing the amount of cores used for faster processing of video. I find it only works after I use Onyxx to clean up my computer. It just hangs with 00:00 when I send it to the cluster, it never engages and starts the job.
If I run Onyxx first it works fine.
I wonder what is causing the problem: Permissions, caches.Running Compressor Repair should help determine what is happening.
-
Does a QMaster Cluster really make sense for Final Cut Server?
Correct me if I'm wrong, and I often am, but my understanding of the difference between setting up a "Cluster" and just using "This Machine" is the cluster allows multiple jobs to be processed simultaneously, while an unmanaged system will only do one job at a time. However, when it does that one job, it uses all the system's processors.
Yesterday I created a cluster consisting of two Quad Core Intel systems, and started Final Cut Server cataloging about a half a TB of data. I noticed that Server submits the Proxy render jobs one at a time, and that the jobs were being submitted to only one processor core, in a rotating fashion. Is there a way to make FCServer submit jobs in "parallel," or am I better off with using just the local system for transcoding?The default proxy transcode setting seems to be specifically tuned to work at a small scale, which makes sense if you are asking a box to host the database, serve the app, and do all the proxy transcodes. I observed the same phenomenon, so I made my own transcode setting for proxies and I chose to lower the bitrate a bit to help with file size and transfer time. The upside is the floodgates are wide open when my 32 core Xeon cluster (four 8-cores) goes to town on a proxy encoding job.
Your other observation is also right, that only one item is encoded at a time across the cluster rather than working on all of them at once across the cluster. So what you get with a fast cluster is shorter encode time, not parallel encodings that all finish at the same time.
If you have the spare machines to dedicate to cluster work, I strongly endorse setting up that way. I would not recommend Quick Clusters for Final Cut Server encoding duty. Either set up a dedicated Managed Cluster that FCSvr can use or none at all.
Maybe you are looking for
-
How do I add another Iphone to my laptop?? I have creadted a new user for the other Iphone however it wont let me sync to the new user??? PLEASE HELP
-
Hi, I created a document in Indesign with layers. I have that in acrobat and created buttons to show the different layers. I click a button, and that layer shows up. Great. But when I submit or save the pdf, the layers do not show up visible upon
-
Dump issue in standard transaction
I am getting a dump error in a standard transaction while clicking save option, could you please confirm me whether the issue is with SAP standard transaction path fill missing or some thing else since when we will do the same steps in same transacti
-
HI all i have the following piece of code, please explain me how the logic works.. LOOP AT T_BAL INTO V_H_BALANCES. MOVE V_H_BALANCES TO V_TEMP_BALANCES. AT END OF MATNR. SUM. MOVE V_H_BALANCES-KSPEM TO V_TEMP_BALANCES-KSPEM.
-
I have to select the 'no proxy' setting every time I load Firefox. It will not keep the setting. I cannot find a way to get it to retain that information.