DNG SDK thinks 0xFFFFFFFF is invalid white level for 32 bit integers

As discussed in another message the DNG SDK does not seem to be able to write floating point DNGs. Since I need higher dynamic range than a 16 bit sample I instead went for 32 bit integers.
This works, and I manage to create a file but when I run dng_validate on it it says the white level is invalid. The white level I have of course set to 0xFFFFFFFF to make use of the full 32 bit range. Looking into the code in dng_ifd.cpp where this test is made it seems like the default max white level is set to 65535.0 and the file's white level is compared against that, regardless if the sample type is 16 or 32 bit. This means that I can only make use of 16 bits of the 32 bit integer which seems kind of strange. Looking into the DNG spec I don't see anything there that forbids using the full 32 bit range of 32 bit samples. So this looks like a bug to me.
This is with version 1.4
the created file can be opened in Lightroom 4, so the only problem seems to be that dng_validate does not think its valid.
Message was edited by: torger76, removed clipping issue, that was a fault in my code.

Hello Charles,
I would be concerned too if my MacBook Prowas running slowly. I found a couple of articles I recommend to help isolate and troubleshoot this issue.
I recommend reviewing this article first for possible causes of the slowness:
OS X Mavericks: If your Mac runs slowly
http://support.apple.com/kb/PH13895
You can further isolate the issue by determining if it is only happening in your user account or if it is happening system-wide:
Isolating an issue by using another user account
http://support.apple.com/kb/TS4053
Thank you for using Apple Support Communities.
Best,
Sheila M.

Similar Messages

White level for HDV sequence

I am working on a series of shows shot with HDV (Sony Z1) and destined for broadcast in both HD and PAL networks.
I edit in HD and apply colour correction filters etc to the timeline and monitor the levels using the FCP toolbench levels and "excess luma" option on the canvas window.
So, the luma in HD is declared to be within tolerance - just some green lines indicating whites are OK. But when I convert the finished show to PAL for our UK broadcaster the same footage now regularly goes into illegal white levels - lots of red indicated "excess luma".
Anyone run into this before or seen anything that explains legal levels of HD versus PAL (or NTSC). I guess now that I see the whites declared illegal in PAL I am starting to worry they are to hot for HD as well. I just sent out our first PAL version done this way and just applied a "broadcast safe" filter to the whole timeline but this seems a bit crude.
Any ideas?
Paul Shard
Producer - Distant Shores
Dual 1GHz Mac OS X (10.4.3) 1.75GB ATI9800, FCStudio
Dual 1GHz Mac OS X (10.4.3) 1.75GB ATI9800, FCStudio

how are you doing the conversion? i'm not an
engineer - although i play one at work - so i can't
really speak to what is causing the luma level
shift.
I am exporting the HD sequence as a AIC 1080i50 sequence (it was shot natively in 1080i50) and creating a PAL Uncompressed 8-bit sequence. I put the AIC HD sequence in there and do the 14by9 letterbox there.
is it a global, uniform shift in luma or are you
seeing a luma shift in things like specular
highlights? can you post a screen shot anywhere?
It does seem to be a global increase in Luma. I must do more tests and see if a screen shot is possible.
Right now I'm off to teach a sailing seminar though back tomorrow. Thanks for your suggestions
Paul
Dual 1GHz Mac OS X (10.4.3) 1.75GB ATI9800, FCStudio

DNG SDK vs Lightroom - difference(s) in TIFF output?

Hi, I'm thinking about writing a batch program that uses the DNG SDK to create full-size JPEGs of images i've edited (by exporting to TIFF from the DNG SDK and then using imagemagick to convert to JPEG). I often forget to export the image after I've developed it in Lightroom, and it'd be handy to just batch process any DNG I've made changes to since import.
If I save the develop metadata into a DNG from lightroom, how much (if any) of those development changes will the DNG SDK use when rendering a TIFF? Examples:
- I typically make basic adjustments - exposure, recovery, highlights, contrast - will those be read automatically from the DNG?
- If I embed one of the new color profiles into the DNG, will that be used when rendering?
I'm guessing that the answer is a no, but I thought it was worth asking...
Thanks,
Joe

The DNG SDK is mostly sample code, and is not intended to be used "as is" for actual work. It does not contain the all the actual rendering algorithms used by Adobe (for example, the de-mosaic algorithm in the SDK is very basic). It does implement the full DNG 1.2.0.0 profile color model, however, as a guide to others trying to add DNG 1.2.0.0 support.
P.S. If you need to use the actual Adobe rendering algorithms, they are currently available in binary form (not source code like the DNG SDK), via the DNG WIC codec. This is Windows only, however.

Adobe Releases DNG SDK

Adobe just released an SDK for the DNG format. What does this mean fro better DNG support in Aperture if anything? Are we to assume that Apple hasn't had access to this before now? I'm hoping what this means is that in the future we'll have better support for DNG which means it will support virtually any RAW format. Does anyone know, if Apple didn't have access to this before how long this might take to get working? Is it a matter of a couple months, or would we have to wait until 2.0?
http://www.macworld.com/news/2006/04/18/sdk/index.php?lsrc=mwrss

Well, first off, dcraw produces pretty remarkable
conversions. Perhaps that's why so many products out
there (including ACR) use the code as part of their
converter.
No argument that the output is decent, but I tend to think the adoption of DCRaw is because it is (a) Free (as in beer or speech), and (b) way easier than doing it yourself from scratch. Did I mention it was free?
I mean, lets say you are writing "Bob's RAW Readin' App". What else are you really going to use? If you're running on OSX you could rely on the OS conversion but many of these apps want to support windows as well. DCRaw is just sitting there, quite free for the taking, and the "Readin'" part of that app is done. Now that Apple's conversions are better that idea may take root a little more than it has in the past.
I honestly don't know that there is even another library around, commercial or not, that does what DCRaw can do.
Secondly, it's rather spurious reasoning to assume
that Apple is only capable of producing broken
renderers (though, I'll admit that between
image-quality problems in 1.0 and performance
problems in 1.1 I don't have a lot of confidence) -
and it's absurd to keep adding support for
oddball/exotic cameras while ignoring DNG.
Here's the problem though. In order to support something like the Fuji, they are going to have to be able to parse that offset image sensor data (rotated 45 degrees) wether or not it comes from a Fuji RAW or a DNG converted from a Fuji. Thanks to DCRaw we all know right where that RAW data is held, so it's not a problem to reach it even without assitance from Fuji. The trick is producing a decent conversion. Even now there's a thread in the Lightroom forums complaining about the Fuji S2 conversion being not that great:
http://www.macromedia.com/cfusion/webforums/forum/messageview.cfm?catid=589&thre adid=1125459&enterthread=y
and I'll bet ACR has put more thought into Fuji RAW than even DCRaw has so far! Is it not fair to say that if Aperture had the same problems with conversions we'd have a hue and cry across the land? Far better to say you can't do something and have people know not to buy the product, than to lead people into thinking you can do something when some people are going to be very dissapointed in your "support". Even now I feel Aperture should simly remove "DNG" from the list of supported formats, because it does not meet what I consider to be the absolute minimum for that format - DNG Linear.
I actually partly agree with you in that I think they should add DNG Linear support first, which would placate a lot of people and be super easy to add since there is only one form of DNG Linear. Then they can finish support for the more offbeat formats.
As for why to use DNG - it's simple - there are a LOT
of cameras out there that are not supported
otherwise, and there always will be. It's wonderful
to have a simple, compatible format.
Except that every true variant of a Bayer array (Canon and Nikon are not variants that differ significantly in this regard) requires different processing techniques. The Fuji 45 degree rotation is replicated exactly (well, as exactly as it can be) in that DNG file and requires different techiniques to work with and get a good conversion. I shudder to think how the SuperCCD SR data from the S3 is represented in a DNG file since it has two sensors at each pixel - I suspect it is probably stored as a linear DNG file (in short it demosiacs the RAW data before storage and does not really hold the raw data [unprooved speculation]).
It's bizarre that you'd advocate for not supporting
this, it could only be a good thing.
It would be an awesome thing if it really worked. When I first read about DNG I was very, very excited! It was only after I read the specs and realized you still would need a different conversions algorithm for any of the numberless variants of RAW data that DNG is capible of holding that my joy turned to anguish.
The reality on the ground is that pretty much all cameras that do not have bog-standard bayer CFA grids will be shunt off to the Island of Misfit Cameras if DNG were really to take hold, because most programs would really only bother to include conversion algorithms for the most popular kinds of formats held in DNG. I dearly love the Fujis and Sigmas and Sonys (cyan color in grid) of the world and want to see alternate choices flourish.
The problem is not exactly with DNG so much as that what DNG is trying to do (pretend to normalize all sensor data as if it were the same) is simply not possible, and in my mind not even desireable because, as I said, I want to keep sensor diversity alive and alternate sensor designs coming down the pipe.

DNG SDK not working on Mac (Leopard)

Hi, I am writing a program to create dng files via the latest adobe dng sdk. I follow the same basic recipe as outlined in the example dng_validate code. On a Windows PC, the created thumbnails and previews appear just fine. However, the same code on the Mac creates dng files that do not show the thumbnails or the previews. In addition, apple programs like Aperture do not recognize these dng files either, even though they validate correctly by the provided dng_validate program on the Mac. Is this a bug in the Mac version of the SDK? OR, are there additional steps that are needed for these dng files to work???

Hi Eric,
An example image is given below. The first one (test.dng) is the image we created with the Adobe DNG SDK on a Mac. The second image (test_dngv.dng) is the result of running it through the dng_validate program (which uses a compressed image by default with the -dng option), and therefore much smaller in size.
Neither of these images produce visible thumbs or previews on the Mac (not in any of the Apple programs such as Finder, Preview, Aperture, iPhoto etc.). However, when placed onto a Windows machine, they appear just fine everywhere.
As the previous poster has indicated, this may not be the fault of the Adobe DNG SDK, but for all practical purposes it is a hindrance to us. Whoever is at fault, the bottom line is that the DNG format is proving to be less portable than other image formats such as TIFF or JPG. We have no problem moving these other formats from any computer to any other computer, whether it be Mac or Windows. We are trying to support Adobe and its efforts with new standards, but this is costing us valuable engineering time in dealing with these matters, that just do not exist with other formats.
And, I apologize if this is presented in the wrong forum. However, we would appreciate if you can help us in this matter. Obviously, it can be done, since the Adobe DNG Converter program is capable of generating dng files with proper thumbs and previews on a Mac. It is just not documented anywhere for developers trying to work with the DNG SDK.
http://www.davidklepacki.com/test.dng
http://www.davidklepacki.com/test_dngv.dng
Thank you,
David

Quicktime wont load because the installer thinks I have a newer level quicktime installed already.

quicktime wont load because the installer thinks I have a newer level quicktime installed already. Thus the newest level of iTunes and quicktime will not install, help... Thanks.

Unfortunately, this sort of trouble has gotten more complicated to deal with ever since Microsoft pulled the Windows Installer CleanUp utility from their Download Center on June 25. First we have to find a copy of the utility.
Let's try Googling. (Best not to use Bing, I think.) Look for a working download site for at least version 3.0 of the Windows Installer CleanUp utility. (The results from mydigitallife and Major Geeks are worth checking.)
After downloading the utility installer file (msicuu2.exe), scan the file for malware, just in case. (I use the free version of Malwarebytes AntiMalware to do single-file scans for that.)
If the file is clean, to install the utility, doubleclick the msicuu2.exe file you've downloaded.
Now run the utility ("Start > All Programs > Windows Install Clean Up"). In the list of programs that appears in CleanUp, select any QuickTime entries and click "Remove".
Restart the PC, and try another install. Does it go through properly this time?

How to compile DNG SDK with GCC?

I tried to compile DNG SDK with GCC, but I got stuck to the following error message:
... dng_flags.h:33:28: error: RawEnvironment.h: No such file or directory ...
In other words, it seems that file 'RawEnvironment.h' is missing. I cannot find this file from DNG SDK or XMP SDK. Do you know where I could find this file or do you know how to compile DNG SDK with GCC?
Regards Sami

Please define either qMacOS or qWinOS (i.e., in your list of preprocessor defines). As you can see in line 32 of dng_flags.h this is required.

DNG SDK build project cannot find the xml sdk

On win7/x64. Installed dng sdk and all dependencies per the readme files. xml sdk built OK with vc10 express, as 64 bit static and dll.
But the vc solution for dng_validate has incorrect names for the xml sdk build projects, so it cannot find those components. Specifically it looks for "..\..\..\xmp_sdk\build\vc\XMPCore.vcxproj". the actual paths in the xmp sdk are different and longer, like "..\..\..\xmp_sdk\build\vc10\dynamic\windows_x64". Moreover there is no XMPCore.vcxproj there.
I suppose the real problem may be that the dng sdk project is trying to build it as .net managed code, while the xml sdk was built as open code. So I guess what I am looking for is a way to build the dng sdk open.

I would confirm your post, but I have not found a solution. I wish someone at Adobe would look into this.

Build DNG SDK on Mac with XCode 6.2

Hello everyone,
I am trying to build the DNG_SDK Version 1.4 with XCode 6.2. I have downloaded all the third party libraries for xmp and dng sdk. After linking errors with the current xmp sdk 201412, I tried to compile DNG_SDK with XMP SDK Version CS6. The XMP Subproject is found in my DNG_SDK and is build successfully.
For the DNG SDK I still get linker errors and I have no clue why (regardless which architecture I select i386 or x64):
Ld /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release/dng_validate normal i386
    cd /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/projects/mac
    export MACOSX_DEPLOYMENT_TARGET=10.9
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cl ang++ -arch i386 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOS X10.9.sdk -L/Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release -F/Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release -filelist /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Intermediates/dng_validate.build/Default/dng_validate\ release.build/Objects-normal/i386/dng_validate.LinkFileList -mmacosx-version-min=10.9 /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Products/libXMPFilesStatic.a /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Products/libXMPCoreStatic.a -framework Carbon -Xlinker -dependency_info -Xlinker /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Intermediates/dng_validate.build/Default/dng_validate\ release.build/Objects-normal/i386/dng_validate_dependency_info.dat -o /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release/dng_validate
Undefined symbols for architecture i386:
"TXMPMeta<std::string>::Initialize()", referenced from:
      XMPFiles::Initialize(unsigned int, char const*, char const*) in libXMPFilesStatic.a(XMPFiles.o)
"TXMPMeta<std::string>::SetProperty(char const*, char const*, std::string const&, unsigned int)", referenced from:
      P2_MetaHandler::SetAudioInfoFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetVideoFrameInfoFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)
      MP3_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MP3_Handler.o)
      ReadIDXFile(std::string const&, std::string const&, TXMPMeta<std::string>*, bool&, MD5_CTX*, bool) in libXMPFilesStatic.a(SonyHDV_Handler.o)
      XDCAMEX_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(XDCAMEX_Handler.o)
      XDCAM_Support::GetLegacyMetadata(TXMPMeta<std::string>*, XML_Node*, char const*, bool, std::string&) in libXMPFilesStatic.a(XDCAM_Support.o)
      RIFF::importProperties(RIFF_MetaHandler*) in libXMPFilesStatic.a(RIFF_Support.o)
"TXMPMeta<std::string>::SetProperty(char const*, char const*, char const*, unsigned int)", referenced from:
      ID3_Support::ID3v1Tag::read(XMP_IO*, TXMPMeta<std::string>*) in libXMPFilesStatic.a(ID3_Support.o)
      P2_MetaHandler::SetXMPPropertyFromLegacyXML(bool, XML_Node*, char const*, char const*, char const*, bool) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetGPSPropertyFromLegacyXML(XML_Node*, bool, char const*, char const*) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetAltitudeFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)
      MP3_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MP3_Handler.o)
      MPEG4_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MPEG4_Handler.o)
      ReadIDXFile(std::string const&, std::string const&, TXMPMeta<std::string>*, bool&, MD5_CTX*, bool) in libXMPFilesStatic.a(SonyHDV_Handler.o)
"TXMPMeta<std::string>::DeleteProperty(char const*, char const*)", referenced from:
      P2_MetaHandler::SetRelationsFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetDurationFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetGPSPropertyFromLegacyXML(XML_Node*, bool, char const*, char const*) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::SetAltitudeFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)
      P2_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(P2_Handler.o)
      JPEG_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(JPEG_Handler.o)
      MPEG4_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MPEG4_Handler.o)
Any hint is appreciated. Thanks
I found a hint here (Xcode Compile Error Undefined symbols for architecture x86_64:   "TXMPMeta<std::string>::Initialize() that a include statement would be missing, but that is defined in the dng sdk (dng_xmp_sdk.cpp):
#if qMacOS
#ifndef MAC_ENV
#define MAC_ENV 1
#endif
#endif
#if qWinOS
#ifndef WIN_ENV
#define WIN_ENV 1
#endif
#endif
#include <new>
#include <string>
#define TXMP_STRING_TYPE std::string
#define XMP_INCLUDE_XMPFILES qDNGXMPFiles
#define XMP_StaticBuild 1
#include "XMP.incl_cpp"
This cannot be the reason..

Thank you so much! That has worked!
I have downloaded XCode 5.1.1 and the build ran without errors. But still I is unclear to me what has changed. The Xcode-project file included in the dng sdk was configured compiling on Mac OS SDK 1.6. Anyway, I can go on with that. Thanks again

Dng sdk question

hi!
I was wondering, did anyone write a simple conversion program from dng to tiff format using this sdk?
I was trying to do that, but for some reason i cannot get results that look like the pictures from the photoshop itself
Thanks,
Ray

Ray, the DNG SDK contains a command-line tool called dng_validate which does exactly that, i.e., produces a TIFF from a DNG.
It will not produce identical results to Camera Raw, because Camera Raw contains other editing adjustments and image processing algorithms which are not contained in the DNG SDK.

Using DNG SDK for 10bit DNG files

Is there any simple way one can use DNG SDK to write 10bit DNG files?
From what I have seen, DNG SDK can only validate/read/write 8 or 16bit files. I have quite a lot of 10bit raw data to be saved and performing a 16 bit conversion would be both time and space consuming.
If there is no simple workaround to achieve this, is there any chance for future SDK versions to support nonpower of 2 pixel bit depth?
Thank you for your time.

> See these specifications for more information on TIFF and TIFF-EP
I do know that specification. However, DNG deviates from the normal TIFF in enough points to warrant explicite specifications, particularly because several cameras' raw data is compressed in like but not identical ways.
> I'm curious if CR will open 10bit DNG....
The Pentax K10D creates 12bit, uncompressed DNG in this packed format and that will be processed by ACR correctly, so I guess you won't have any problem.
On the other hand, it is interesting, that the DNG converter stores 12 and 14bit data in 16bit form, thereby avoiding the issue; a huge waste of space.

Get DNG Date Created from dng sdk

I have successfully compiled the dng sdk and the validate app and I wondered what is a simple way to just get the date created out?

Maybe your scanner software is doing something strange saving as a DNG that ACR does not like? Try scanning as a TIF and open in ACR if that works for you.
Scanning large format negatives as a 16 bit RGB tiff is and then opening in Camera RAW is my preferred workflow and I get to use all of the ACR tools and adjustments, much easier to do some of these than in Photoshop.
Maybe the TIFF route might work for you - and once in ACR you can always save the output as DNG
Mike

Compile DNG-SDK with VC6

Hi all,
I downloaded the DNG-SDK, and now am trying to compile it with VC6.
There are two main problems:
1) The data type 'unsigned long long' doesnt exist. What could I use instead?
2) Is '__int64' ok to use instad of 'signed long long'?
3) The constructor of class range_tag_set in Dng_Image_Writer.cpp does not compile. Does someone know the problem? What should I change?

Please define either qMacOS or qWinOS (i.e., in your list of preprocessor defines). As you can see in line 32 of dng_flags.h this is required.

DNG SDK - What a Complete MESS !!!!!

Hi - and sorry for this to be HIGHLY infriendly message
I have downloaded and installed ALL latest greatest of anything pertaining to the DNG SDK -
Followed ANY read me, configure, make, make install files available there.
Installed CMAKE and everything else....
I am on OS X 10.9 and USE the xCODE version of Mavericks
I am developing MANY different things and NEVER have problems.
I have in my life NEVER exprienced something as cumbersome and clumsy as this. NEVER!!!!
I am trying to compile/build the inlcuded and generic dng_validate project. And this proves to be IMPOSSILBE.
Simply put. I cannot figure it out. And I cannot do more that to follow the instructions provided in the SDK.
This is incredible and I am missing the words to describe anger I am feeling.
I realize this sdk is something that adobe cares nothing about - hence all the problems adobe are having with their own apps, which arent even capable of reading DNGs correctly...
To whomever, maintaining this SDK ---- You are doing a really bad job at it !!!!
Regards
Morten

The SDK is from 2 years ago. XCode has changed quite a bit since then, and the build settings are therefore a bit out of date.
It's not any different with other projects in XCode that haven't kept up with the changes to the tool chain.
It's not easy/trivial, but it is far from impossible to get this to build if you have experience writing your own code and not just building other people's code.
You also have to make sure you are using the correct versions of the XMP SDK and the third party code.
David Blake

DNG SDK

With the intention of being able to read DNG SDK codes, I've started to study C++,
But now I can not find where the main function is ))
As you guess, I have no experience with SDKs ...
I want to learn the process of raw conversion from the very beginning ...
So, where should I start in the SDK for finding where the ADU numbers from the camera raw file are entered into the conversion pipeline?

There isn't really a main function in the SDK itself. The closest thing is the dng_validate program, which shows how to call the various SDK functions.
Sandy

DNG SDK thinks 0xFFFFFFFF is invalid white level for 32 bit integers

Similar Messages

Maybe you are looking for