Pixel Bender metadata hinting proposal

Question

I put this on my blog earlier, but I wanted to repost here for discussion.This was what I wrote:
We tried to be semantically agnostic in the original design of the Pixel Bender language. We'd seen other languages go down rabbit holes of over-specification around what parameters really meant, locking them into archaic and insufficient implementations or clumsy hierarchies of meanings. We didn't want to limit Pixel Bender developers into what we could conceive at the time of the invention of the language.
We were being a bit too idealistic :). It is completely true that the community of Pixel Bender developers continues to blow our minds at what they accomplish with the language and we never would have anticipated half the stuff that you guys are using Pixel Bender for. However, having some generally useful semantic meanings for Pixel Bender parameters would definitely help those who design general user interfaces for Pixel Bender filters.
One smart thing we did (if I do say myself) was to allow parameter and kernel metadata to be extensible. It provided developers like After Effects and Picnik a way to add custom metadata to Pixel Bender that specified semantics or actual UI controls for Pixel Bender kernels and graphs. The picnik metadata and the After Effects metadata are different though. I started to get concerned that by not having something around this that we could end up with many different Pixel Bender semantic metadata mechanisms floating around. To that end, I created the proposal below and started floating it around Adobe and some of the sites that are heavy users of Pixel Bender.
In this design, I tried to follow some general rules:
Pixel Bender is not only a way to write image and video filters for Adobe applications, but also a way for you to host 3rd party filters in your Flash-based applications. Any guidelines we create need to be implemented by us, but also by any independent Flash developers creating apps that use Pixel Bender kernels. To that end, I tried to keep the design relatively simple so that it wouldn't be too difficult to implement in Flash.
This proposal adds semantic metadata, but avoids specifying specific user interface controls. Rather than specifying a slider as an editor for a parameter, I think it makes more sense to say that the parameter is a percentage and allow someone designing a UI to make a good percentage editor. We are definitely thinking about how to specify custom editor UIs for Pixel Bender filters, but this proposal does not approach that.
The proposal is not a complete answer for all Pixel Bender filters. I'm trying to get the most universal semantics represented. The most generally useful, as opposed to trying to give the complete solution. If you have an application that is uses Pixel Bender kernels for something that makes sense to augment the metadata you can still choose to do so.
The proposal represents guidelines, not requirements. As a developer consuming Pixel Bender kernels, you can choose to enumerate the metadata as described in the proposal or not. The suggested metadata is metadata. It is not required. The goal is that if you wish to take advantage of it when it is present, that you can provide a more compelling user interface to your users because you understand the intent of a parameter, not just its type.
There are a couple open questions in the proposal that I'd like feedback on. They are called out pretty clearly. I'd really like your feedback on this proposal. I hope to issue the final version soon. Reply to this post or in the Pixel Bender forums on Adobe Labs. If you decide to post a reply on your own blog, please post a link to your post here or in the forum. Right now trackbacks are off so I won't know about it otherwise. You can also tweet a reply to the Pixel Bender twitter account (if you can fit it in 140 characters ).

Answer 1

Overall I like the proposal. To address the issues called out:
For both integers and floats concerning a "time", I think it is best
to have that be relative to the time that the filter has been running.
The time a filter is running would be determined by context the filter
is being used in. For video I can see it being tied to video time,
while for a filter being applied to an image it probably corresponds
to how much real time the filter has been running. The general idea
being that if I run my application multiple times I should expect to
see the same results, which is harder to do if time is tied to actual
time versus relative time. If that makes any sense.
I am not in favor of the InputSizeX types. While I understand the
desire to have the image size is a common request, given the ability
to chain filters it promotes a false sense of accuracy. Better to let
people have workarounds with known caveats instead of a semi supported
standard.
As a side note, the mousePos example on page 8 is specified as float2,
but the values only contain 1 number.
-- Daniel R. <[email protected]> http://danielr.neophi.com/