DPX sequence performance

Hi,
I'm running Premiere Pro 5.0.3 under Windows 7 64bit on a Core i7 950 CPU with nvidia GTX470 graphics and a RAID that benchmarks at over 300MB/s.
Running R210 AVI files is not a problem, other than all the usual problems that exist with R210 AVI files.
Running DPX sequences representing exactly the same image data goes at about 5fps.
I'm not very surprised; still sequences in Premiere have been slow since time immemorial, but I'm a bit disappointed by this.
Since DPX is a format with a lot of options and I'm in the process of writing some DPX handling code, I'll list the characteristics of the files in question:
DPX 2.0 per SMPTE-268M-2003
XPDS (least significant byte first ordering)
Total image size is 8192K per frame, padded from 7.91MB with 92160 bytes EOI padding on image element 0
Left-to-right, top-to-bottom ordering.
1 image element at offset 2048
Packing method value 1, "Filled to 32-bit words, method A"
Reference low code value is 0
Reference high code value is 1023 (this is not a "core" value per the DPX spec, but is required by apps such as Apple Color)
No encoding (value 0)
1920x1080
10-bit RGB (Image Element Descriptor value 50d)
In other words an absolutely basic standard 10-bit RGB DPX as produced and used by absolutely everyone.
What gives?
P

Jim,
V210 is the uncompressed 10-bit YUV version. Aja I think have R10K which I believe avoids the luminance clipping issue endemic to Blackmagic, and a few other technical conveniences. Uncompressed 8-bit formats are well served by preexisting 8-bit fourCCs (HDYC and UYVY for YUV, usually RGBA for RGB).
Steve,
It is not disk speed as other software (including my own code) will happily read and write 1080p24 10-bit RGB DPX files on this system all day long, and at rates always in excess of 40fps. The problem is is not the disks, the problem the way Premiere is trying to read them.
I don't think it's my DPX files either, as Premiere won't even read it's own output (which is not padded to any particular length) at anything other than a crawl.
Could you please find out what if any assumptions are made in the DPX reader with regard to file size, alignment on disk and in memory, disk cluster and block size end-of-image padding and its placement on the various image elements? Also whether this was tested with anything other than 2K, because if the reader relies on alignment and buffering characteristics of a very precise file size, that would also cause breakage.
Something is broke, and I'd like to either fix it my end, help you fix it, or at least have something worthwhile to say to other people when I get asked about this.
Thanks,
P

Similar Messages

Trouble importing an XML that uses DPX sequences

We are doing an offline edit using a DPX sequence. We're testing now to ensure that an XML of the sequence when we're done will work. (Our DPX is in HD, and the online house will be taking a 4K DPX version of the same piece - identical except for the resolution - for the online.) However, when we start up a new project and import the XML to test it out, it isn't making the edits to the DPX sequence that we made. Everywhere there is an edit, it just starts the DPX sequence at the beginning again. This is the same exact HD DPX sequence that we used when we made the edits.
Here is the original project:
http://img607.imageshack.us/img607/1802/i2ht.jpg
Here is what it looks like when we create an XML from that project and import it into a new project:
http://img14.imageshack.us/img14/6389/swfw.jpg
The text block you see in the thumbnail for the videos in the sequence is the slate at the top of the DPX sequence, so you can see that it's just starting at the beginning of the DPX sequence every time there's a cut. What should we be doing differently?

My apologies Jim. My understanding - which was probably incorrect - was that the XML schema that was in use by FCP7, which has since been implemented for use in Smoke, Resolve, Premiere, etc., was developed by Apple. Thanks for educating me!
XML as currently coded cannot convey sequences created from image-sequence-based media.
What has been an issue for me and a few others, which may be in play here, is relinking to media coming from a third party grading session. Smoke, Hiero, Resolve, SpeedGrade can all conform DPXs to an EDL or XML using timecode (independent of reel name, clip name, etc) while Premiere as yet does not. Premiere's relinking struggles if there's any disparity between the source file names and the file names it's relinking to. It would be great to see some more flexibility there, such as timecode-based relinking coupled with optional fields for partial clip name and partial reel name relinking.
Since the original post refers to a third party vendor doing the finish on the project, simply relinking in Premiere may not be an option. If the vendor in question is going to be conforming in either Smoke, Flame or Hiero, just sending an EDL over will just about guarantee a successful conform. I'd contact them and find out what they prefer.
Cheers!
Jon

Export RED DPX Sequence

I have some RED 4K material. When I try to export out a DPX sequence and import that into AE it is not maintainng the modified RED RAW image charateristics of how it was shot or how I altered it. It looks as if it's showing me the RED RAW Flat gamma setting.

i've had to manually override timecodes on export, not at all ideal and tedious as ****, please tell me there is a simpler way of doing this, this plug is supposed to make your life easier, any feedback/advice greatly appreciated.

How to import DPX sequence?

I am trying to import a short sequence of DPX files directly into the timeline. How do I multiple select a sequence? I can single click and drag but can't tell Color that what I am importing is a whole sequence. Is this even possible?

You can browse and import DPX sequences directly into Color. But if the sequences are numbered and not named, Color doesn't see them as sequences but as individual files
So files numbered 90000.dpx to 91334.dpx won't work.
Opening.90000.dpx to Opening.91334.dpx will show up as a 'clip' that you can import and drag into the timeline.
Neil Sadwelkar

Exporting DPX Sequence from Premiere Pro CS6

I edited a .mov premiere sequence from my 7D and 5D MK II in SpeedGrade by using the "send to adobe speedgrade" option. I exported the grade as a .dpx sequence and brought it back into premiere. Now when I try to export it in h.264 "for youtube hd 1920x1080 23.976", I get a huge loss in color range and blacks become so pixelated and I even lose a lot of darkness in the blacks. The grade no longer looks good. The preview monitor looks great in premiere but it does not match the final export. I need help with this one!
Software: Production Premium CS6
Footage: 1920x1080 / 23.976fps
Camera: Canon 7D + 5D MKII
System Specs:
W7 Professional 64-bit
i7 930 (stock)
Gigabyte X58A-UD3R
12gb G.skill Pi Series 1333mhz
EVGA GTX 470 SC
180gb Corsair Force GT SSD (boot)
1TB WD Caviar Black (production premium program files)
2TB Seagate Barracuda (speedgrade .dpx files)
750gb Seagate Barracuda (source footage/scratch disks + cache)

Use the MPEG2-DVD preset
CS5-thru-CC PPro/Encore tutorial list http://forums.adobe.com/thread/1448923 may help

Exporting DPX sequence: frame rate incorrect in exported file

Hello all, I'm new here. I'd really appreciate some help with this issue. My sequence and source files I am working with are 2048x1240, (1.0), 23.976 fps . I am exporting a DPX file sequence with matching settings.
The problem is, when I import that new DPX file sequence back into my Premiere project, or an AE project with the same settings, the new DPX file has a frame rate of 29.97 fps.
I've tried numerous times and checked all my settings. I can't figure out why the new DPX sequence is not 23.976 fps. Any ideas what I am doing wrong here?
here are my export settings, note output and source have matching frame rates:
and here is a screen grab showing the file properties of my new exported DPX sequence, after importing back into Premiere Pro CS5, with a frame rate of 29.97 fps.
Any help will be greatly appreciated!
thanks,
Jeremy

Hi there.
I am having the same exact problem, in CS6.
BUT, I have exported this project countless times, and only after my last edit has it started to export at 29. My entire movie is in 23.97. I have tried interpreting the one clip that is 29, but this does not help. It will not allow me to interpret the whole project at once.
Are there any other solutions?

Playback of DPX sequences with quadro nvidia

I am considering buying a nvidia quadro 4000 for mac to playback my 1080P DPX sequences out threw a BlackMagic HD Extreme 3D card.
At the moment i can do it when i select 1/2 Resolution in playback from Premiere CS 5.5.
My disk speed is about 300MB pr sek so that is not the problem.
Does annybody know if the Quadro card will give me any aceleration with that and make me able to play it back at full res ??
Thank you

> Hardware acceleration doesn't make it easier to "play" any media files (assuming they match the sequence). What get's accelerated are certain effects, scaling, deinterlacing, blending modes, etc. But not simple playback.
That's correct.
The graphics card doesn't help at all with decoding, nor does it help with reading the data off the disk faster. The reason that people talk about the GPU helping with formats like RED is that you are nearly always scaling RED footage down to use it (since you're almost certainly not using a 4K sequence, and you're almost certainly not previewing on a 4K monitor). The GPU helps enormously with scaling operations if you have a card that Premiere Pro can use for CUDA processing.
DPX image sequences are slow to preview for two reasons: data rate off of the hard disk and CPU speed for the relatively slow decoder. Fast disks and fast CPUs are what you need for fast DPX playback.

Conformed DPX sequence?

Hi there folks,
I'm editing a music video next week and the client has asked for it delivered to them as a conformed DPX sequence with EDL....
Eeeeek.
An EDL should be easy enough, but a conformed DPX sequence, that's me lost...
Can anyone help? Can I do this through FCP?
Thanks,
Matty

that's usually for a film recorder. It is going to film output?
You can do a DPX sequence out of Compressor, if that helps.
in the compressor inspector panel change Quicktime movie to Image sequence
http://dl.dropbox.com/u/129533/changeQTmovie.png
then select DPX.
http://dl.dropbox.com/u/129533/select_dpx.png
See if that's what they want. I'd do a short test first.

Color Importing a 24fps EDL as 23.98 when conforming from DPX Sequences

I'm trying to do a DPX conform in Color and I am having some issues with importing my EDL from FCP. I have 2k 24fps DPX files, from which I created 720p 24fps ProRes quicktimes to do my offline edit. My offline edit is now complete and I'm looking to conform from the DPX files.
I exported my EDL from Final Cut. When importing my EDL in color I make sure that "Project Frame Rate", "EDL Frame Rate", and "Source Frame Rate" are sll set to 24. It links to the correct DPX files, however no matter what I do the project setting is set to 23.98 not 24fps. The sequence looks fine inside Color, however when I send it back to Final Cut there are frames missing. All the clips start at the correct timecode, however some are missing 4-5 frames on the tail end.
I saw on this forum that someone had the same issue about a year ago, and doesnt seem like it has been solved.
Any Ideas?!?!
Thanks.

Read this: http://www.cinematography.com/index.php?showtopic=16732

Select statement Table Join sequence (Performance)

Hi,
I have 3 tables Header (Hdr), Item (Itm) and Schedule Line (Sdl) table. The number of entries is max in Sdl table (Eg: Hdr = 1000; Item = 20000 and Sdl = 200000 records). I need to join these 3 tables in my SELECT statement to fetch the required records.
Case A:
Header AS HDR
JOIN Item AS ITM
ON HDRORDID = ITMORDID
AND HDRVRSIOID = ITMVRSIOID
JOIN ScheduleLine AS SDL
ON ITMITMID = SDLITMID
Case B:
ScheduleLine AS SDL
JOIN Item AS ITM
ON ITMITMID = SDLITMID
JOIN Header AS HDR
ON HDRORDID = ITMORDID
AND HDRVRSIOID = ITMVRSIOID
Questions:
1) The performance of the Select query is really bad with Case A and very good (really fast) with Case B. Will the number of records in the tables used determine the sequence of join in the Select statement ?
2) If yes, is this independent of database used ? That is, will the behaviour be the same in all databases (Eg: Oracle, DB2, SQL Server, etc) ?
Appreciate your help.
Thanks,
Karthik

I believe that the performance discrepancy between the two versions is caused by you not specifying the joins correctly (i.e. not qualifying the header and item at schedule level as I mentioned previously).
CAse A.
Header AS HDR
JOIN Item AS ITM
ON HDRORDID = ITMORDID
AND HDRVRSIOID = ITMVRSIOID
JOIN ScheduleLine AS SDL
ON ITMITMID = SDLITMID
reads all 1000 hDRs, joins all 20000 items correctly to these BUT then joins schedule lines only by item id so there will be a very large number of matches. i.e. any item with itemid 10 will match to EVERY SDL with itemid 10. as there are 20000 items and 200000 schedule lines this will be massive!
Case B:
ScheduleLine AS SDL
JOIN Item AS ITM
ON ITMITMID = SDLITMID
JOIN Header AS HDR
ON HDRORDID = ITMORDID
AND HDRVRSIOID = ITMVRSIOID
This will not have the same amount of DB traffic as the SDL is now the driving force. It will still match more than you intend but will only be reading the items redundantly trather than the more voluminous schedules.
Basically, your sql is wrong and should qualify the matches right down to schedule line level as per my earlier post. When you have changed the sql, try the two alternatives again and see if there is any performance differences. I suspect that there will not be, but it will be interesting to find out for sure.

Sequences Performance

Hello.
I have a problem in the sequences.
Retrieving the next value of the sequence consumes 5 milliseconds of the procedure execution time.
When the sequence value goes high for millions... retrieving the sequence next value consumes about 300 milliseconds !!!!!
The sequence is cached, with values 20, .... and also tested with cache value 1000 and the performance is the same. the sequence currently cached with value 20
Why it takes 300 milliseconds, while it was consumes only 5 milliseconds?
Any Idea

Perhaps this questions belongs in the PL/SQL.
Post relevant parts or examples of your code, to explain your question better. How did you measure or arrive to the timings? Is nothing else running that takes processor time? And why do you worry about the performance of sequences?
I did a small test (on 9.2.0.4, not state of the art unix box but quite idle) where "next" values took less than a milisecond each to retrieve. Likely what accounts for the time in the following output is the output itself i.e. dbms_output contributes too. The cache value is set to 20, and you can see from the jump to get to 21 that it takes about a milisecond.
-- 17-JUL-07 08.44.25.273886000 AM +02:00 seq number: 16
-- 17-JUL-07 08.44.25.274136000 AM +02:00 seq number: 17
-- 17-JUL-07 08.44.25.274385000 AM +02:00 seq number: 18
-- 17-JUL-07 08.44.25.274636000 AM +02:00 seq number: 19
-- 17-JUL-07 08.44.25.274885000 AM +02:00 seq number: 20
-- 17-JUL-07 08.44.25.275844000 AM +02:00 seq number: 21
-- 17-JUL-07 08.44.25.276174000 AM +02:00 seq number: 22
-- 17-JUL-07 08.44.25.276427000 AM +02:00 seq number: 23
-- 17-JUL-07 08.44.25.276678000 AM +02:00 seq number: 24
-- 17-JUL-07 08.44.25.276929000 AM +02:00 seq number: 25
-- 17-JUL-07 08.44.25.277179000 AM +02:00 seq number: 26

Sequences Performance Issue

Hello.
I have a problem in the sequences.
Retrieving the next value of the sequence consumes 5 milliseconds of the procedure execution time.
When the sequence value goes high for millions... retrieving the sequence next value consumes about 300 milliseconds !!!!!
The sequence is cached, with values 20, .... and also tested with cache value 1000 and the performance is the same. the sequence currently cached with value 20
Why it takes 300 milliseconds, while it was consumes only 5 milliseconds?
Any Idea

So selecting the sequence can be almost a hundred times slower, depending on how large the sequence value is. Somehow I don;t think so unless it is a bug in your version. What happens if you do this?
SQL> create sequence s start with 100000 increment by 1 cache 1000;
Sequence created.
Elapsed: 00:00:00.00
SQL> declare
2    n number;
3 begin
4    for i in 1 .. 100000 loop
5      select s.nextval into n from dual;
6    end loop;
7 end;
8 /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.07
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.06
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.02
SQL> drop sequence s;
Sequence dropped.
Elapsed: 00:00:00.05
SQL> create sequence s start with 10000000000 increment by 1 cache 1000;
Sequence created.
Elapsed: 00:00:00.00
SQL> declare
2    n number;
3 begin
4    for i in 1 .. 100000 loop
5      select s.nextval into n from dual;
6    end loop;
7 end;
8 /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.02
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.05
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:05.01
SQL>

Conforming project to dpx sequences

Hi there,
I have an advert I've cut in premier cc and had graded professionally - Ive had dpx image sequences given back to me . Is there a straightforward way to get premier to relink to these sequences?

OK, I chatted with Adobe and after they got back to me via email, here's what they wrote:
"Hi Sami,
This is Adobe Technical Support.
We are responding in reference to your case# 0185837302 regarding the Adobe Premiere Pro, I researched the issue and found out that the issue is reproducible on our end.
I have reported the issue to the higher level of support team. They will research on this issue and will try roll out a patch update to fix it."

Hard drive performance issue with DPX & MXF files

Hi everyone,
I spent hours on this forum reading articles about hardware to use for editing large files. I also have to admit, that I made some cut backs due to budge limitations.
I spent most of the money on CPU, motherboard, & ram (6 core 32 gigs , hell of nice MB) ( Video card= gtx 760 2 gig )So ok, I cheaped out on the hard drives. (I think)
I put (2) 7200 rpm drives (raid 0) as my Operating system
(1) External 8TB RAID G drive (USB 3.0 ) as my Media drive. (footage)
(2) 7200 drives (Raid 0) as my Cache Drive
I then opened up an OLD project using canon 5D footage (h264 mov) and some DPX sequence files.
The 5D fooatge played in real time BUT as soon as it ran into the DPX sequence, the frames dropped.
I wanted to cry. I just spent 1,600.00 on parts (&3000.00 pc retail if I bought it instead of building it)
And I got the same performance as my old quad core 6600/ 6 ram system. WHAT GIVES????
So, can someone recommend the cheapest workaround that can get me editing in realtime?
Do you think it's the 3.0 usb connection?
The footage I will be editing will be from the canon c300 (MXF files)
My fear is, if I am dropping frames with DPX file, what's going to happen when I get the MXF files?
Fun Facts:
I tested my OS drives, cache & media,and I was getting 290-301mb/sec READ SPEEDS
Thanks in advance !

Loucann,
I do not have the data on a three drive RAID 0 but here is a two drive RAID 0 with good Seagate ST2000DM001 drives. I have to say "good" because there have been two varieties available, some with two platters and some with three platters. The preferable are with two platters and my experience shows these drive to have an "E" in the third character of the Serial Number like Z1E1ME4V They are slightly faster than the three platter versions. Here is the HD Tune Read of the two better ST2000DM001 drives in RAID 0 on my system.
The best connection is to have all three on a RAIDable Intel SATA III connection but I am doubtfull if this can be accomplished on many motherboards, maybe Eric can comment and enlighten us.
Message was edited by: Bill Gehrke
What motherboard do you have? I guess from the info above it is a socket 2011.

Encoding MPEG2-DVD from dpx image sequences produced random encoding artifacts

I'm using PPro CS 6. I've exported my 90 minute film as a DPX sequence. This image sequence is effectively my master. I then export this DPX sequence as a MPEG2-DVD and I get the following artifacts on random frames. It happens whether I'm exporting PAL (non interlaced) or NTSC (interlaced). I've now run the encoder three separate times and I get random artifacts on completely different frames. I'm checked the DPX images and they are fine so I know this problem is happening in the encoding.
Thanks.

Here is another example. This time it I was encoding PAL from the DPX sequence.

DPX sequence performance

Similar Messages

Maybe you are looking for