Native Canvas Painting and Refresh

Question

Hi,
I have a project where I get live video from a firewire camera, and paint it to an awt Canvas natively using JNI. I need additional items painted on the canvas (on top of the video), like a pair of crosshairs and rulers to measure certain widths, etc. The software tracks an object moving across the screen, so the crosshairs need to be moved around and refreshed often.
I am getting a strange problem - the natively painted video is working just fine, but the painted objects are not refreshing properly. For example, instead of getting a crosshair that moves around the screen, I get a trail of crosshairs. And underneath the crosshairs, the video is refreshing just fine. What's going on? I'm using libXv, an X-windows extension to paint YUV4:2:2 camera buffers directly to the screen.
Here are relevant class snippets:
public class NativeCanvas extends Canvas{
    public void update(Graphics g)                 <-- overridden to eliminate flicker
        paint(g);
    public native void paint(Graphics g);         <-- here's the native video drawing function
public class CanvasWithOverlays extends NativeCanvas
    public void paint(Graphics g)
           super.paint();
           Graphics2D g2 = (Graphics2D)g;
           //(paint crosshairs, etc)
} Any help will be greatly appreciated. Thanks very much!
--Ismail Degani
High Energy Synchrotron Source
Cornell University

Answer 1

Hi,
I'm not sure how the crosshairs can be out of sync with the video stream - the canvas paint routines paint the video frame natively, and then paint the crosshairs. It's all sequential -
super.paint();    // goes into a native function XvPutImage that quickly blits the frame on screen
Graphics2D g2 = (Graphics2D)g;          
//(paint crosshairs, etc) This should work properly with the Event Queue, to the best of my knowledge. I schedule a TimerTask that continually calls the repaint() method to refresh the video:
public class LiveVideoPainter extends TimerTask
    static Logger logger = Logger.getLogger(LiveVideoPainter.class.getName());
    NativeCanvas nc;
    VideoDataSource vs;
    public LiveVideoPainter(NativeCanvas nc, VideoDataSource vs)
        if(nc == null)  {
            logger.error("The Native Canvas is null!");
            return;
        if(vs == null)  {
            logger.error("The Video Data Source is null!");
            return;
        this.nc = nc;
        this.vs = vs;
    public void run()
        vs.getFrame(nc.buffer);
        nc.repaint();
} I actually had this same problem when designing this application with C++ using the Qt windowing toolkit a year ago. Basically, if I called XvPutimage, and then called regular X drawing routines like XDrawLine etc in a loop, it would draw successfully, but never refresh properly:
while(true)
get_decompressed_frame(buf);
xv_image=XvCreateImage(display,info[0].base_id, XV_UYVY, (char*)buf, 1024, 768);
XvPutImage(display, info[0].base_id, window, gc, xv_image,
                   0,0,1024,768,
                   0,0, 1024,768);
if(crossHairDisplayed)
// Draw Horizontal CrossHair                                                                                                                                                           
            XDrawLine(display, window, gc,
                      0,   (int)(DataSource::pixPerMicron*DataSource :: crossY + DataSource::zcenY),
                      1024,(int)(DataSource::pixPerMicron*DataSource :: crossY + DataSource::zcenY));
            // Draw Vertical CrossHair                                                                                                                                                             
            XDrawLine(display, window, gc,
                      (int)(DataSource::pixPerMicron*DataSource :: crossX + DataSource::zcenX), 0,
                      (int)(DataSource::pixPerMicron*DataSource :: crossX + DataSource::zcenX) , 768);
}In this code bit, the crosshairs should move when the DataSource object changes member variables, line CrossX. But, the old crosshairs would not go away until the window was moved or resized. I'd get two crosshairs on the screen. I had to use a hack from the xwindows utility xrefresh everytime the crosshairs changed. It essentially simulated an x-window that opened over the Qt window, and then immediately closed. This worked well, but I thought I'd be free of that hack when I moved to java and JNI. Isn't this bizarre? Why would the window hold on to the old crosshair even when the varialbes change and there isn't any code that repaints it there after the video frame gets blitted?
hack adapted from xrefresh.c:
      if(newCrossHair)
                          Visual visual;
                  XSetWindowAttributes xswa;
                  Display *dpy;
                  unsigned long mask = 0;
                  int screen;
                  Window win;
                  if ((dpy = XOpenDisplay(NULL)) == NULL) {
                    fprintf (stderr, "unable to open display\n");
                    return;
                  screen = DefaultScreen (dpy);
                  xswa.background_pixmap = ParentRelative;
                  mask |= CWBackPixmap;
                  xswa.override_redirect = True;
                  xswa.backing_store = NotUseful;
                  xswa.save_under = False;
                  mask |= (CWOverrideRedirect | CWBackingStore | CWSaveUnder);
                  visual.visualid = CopyFromParent;
                  win = XCreateWindow(dpy, DefaultRootWindow(dpy), 400, 600, 1, 1,
                                      0, DefaultDepth(dpy, screen), InputOutput, &visual, mask, &xswa);
                  XMapWindow (dpy, win);
                  /* the following will free the color that we might have allocateded */
                  XCloseDisplay (dpy);
                  newCrossHair = false;
      } Any ideas? There's probably just some type of refresh call I need to use.
--Ismail