Recent

Author Topic: [SOLVED] - "Tween" two images (Onionskin) & Chroma Key  (Read 30938 times)

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
[SOLVED] - "Tween" two images (Onionskin) & Chroma Key
« on: May 12, 2012, 10:03:48 pm »
Hi

I'm looking for the "correct" way to tween two images - when I say "tween" I mean create an image with a ghost of another image (or images) sort of overlayed into it (with varying weights) - "onion skin" is another term for this.

I've sort of got it working with this code;

Code: [Select]

if chkbxOverlay.Checked then begin
    Overlay := TBGRALayeredBitmap.Create(Video.Width, Video.Height);
    OnionSkin := TBGRABitmap.Create('test20120511154948.png');

    Overlay.AddLayer(VideoImg, boLinearBlend, 10);
    Overlay.AddLayer(OnionSkin, boLinearBlend, 10);

    Composite := OverLay.ComputeFlatImage;
    Onionskin.Free;
    Composite.Draw( BGRAPanel1.Canvas , 0, 0 );
    Overlay.Free;
  end else begin
    VideoImg.Draw( BGRAPanel1.Canvas, 0, 0, True );
  end;

but it doesn't "feel" right at the minute and not sure how to vary the faded amount between the two images.  Not sure I need layered images (unless this is the best way to load several images and ghost them all).

there's an example of tweening on the efg2 website (http://www.efg2.com/Lab/ImageProcessing/Tween.htm) where a trackbar is used to control this fading from one image to another with a middle position where each image equally "ghosts" onto the resultant image.

In the above code the VideoImg is a frame from a webcam (from the 5dpo library).  The onionskin loaded image is a previous snapshot image (i.e. the last image grabbed from the webcam) - an onion skin operation is useful to show movement between the current position and the last position typically for animation ("scrubbing" allows other previous images to be loaded and it's easier then to see whether a set of animation frames are smooth or not).  The "Composite" above, is just a reference to a TBGRABitmap which isn't instantiated or free'd (do I need to do that here?).

When the overlay checkbox is unchecked the videoimg goes direct to the canvas - but when it's checked I want to load the previous image and ghost it over the live webcam video feed.

TheBlackSheep
« Last Edit: May 24, 2012, 11:03:37 pm by TheBlackSheep »

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #1 on: May 13, 2012, 12:09:40 pm »
If I understand what "tween" means, the best way to do it is this way :

Code: [Select]
function MakeTween(img1,img2: TBGRABitmap; position: byte): TBGRABitmap;
begin
  if position = 0 then
    result := img1.Duplicate
  else if position = 255 then
    result := img2.Dupliacte
  else
  begin
    result := img1.Duplicate;
    result.PutImage(0,0,img2,dmSet,position);
  end;
end;

Then you can draw and then free the resulting image. By using this method, you can have a transition when there are transparent parts.

If there is no transparency, you can just do :
Code: [Select]
img1.PutImage(...)
img2.PutImage(...,dmSet,position);

Note : Composite needs to be freed, it is not a property.
Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #2 on: May 13, 2012, 01:21:15 pm »
hi circular,

I must be missing something still - as a quick test (in Windows) I've put 3 TImages on a form (the two images from the Tween example on the efg2 page, a blank TImage for the result and a trackbar to control the tween amount.  I've loaded the images into Image1 and Image2 at design-time and put this in the trackbar change function;

Code: [Select]
procedure TForm1.TrackBar1Change(Sender: TObject);
var
   bmp1,bmp2, resultbmp: TBGRABitmap;

   function MakeTween(img1,img2: TBGRABitmap; position: byte): TBGRABitmap;
   begin
     if position = 0
        then result := TBGRABitmap(img1.Duplicate)
        else if position = 255
               then result := TBGRABitmap(img2.Duplicate)
     else
     begin
       result := TBGRABitmap(img1.Duplicate);
       result.PutImage(0,0,img2,dmSet,position);
     end;
   end;

begin
  bmp1 := TBGRABitmap.Create(Image1.Picture.Bitmap);
  bmp2 := TBGRABitmap.Create(Image2.Picture.Bitmap);
  resultbmp := MakeTween(bmp1,bmp2,trackbar1.position);
//  Image3.Picture.Bitmap.Assign(resultbmp.Bitmap);
  resultbmp.Draw(Image3.Canvas,0,0,true);
  resultbmp.Free;
end; 

When I move the trackbar and use the commented out line above it produces no image in Image3, the version as it stands above just shows a garbled image.  The sample images I'm using are here;

http://www.efg2.com/Lab/Bitmap/TulipSmall.ZIP
http://www.efg2.com/Lab/Bitmap/SunflowerSmall.ZIP

although any images in theory should do.

TheBlackSheep

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #3 on: May 13, 2012, 02:46:26 pm »
hi circular

after a bit of playing I seem to have it working...

Code: [Select]
procedure TForm1.TrackBar1Change(Sender: TObject);
var
   bmp1,bmp2, resultbmp: TBGRABitmap;

   function MakeTween(img1,img2: TBGRABitmap; position: byte): TBGRABitmap;
   begin
     if position = 0
        then result := TBGRABitmap(img1.Duplicate)
        else if position = 255
               then result := TBGRABitmap(img2.Duplicate)
     else
     begin
       result := TBGRABitmap(img1.Duplicate);
       result.PutImage(0,0,img2,dmDrawWithTransparency,position);
     end;
   end;

begin
  bmp1 := TBGRABitmap.Create('SunflowerSmall.bmp');
  bmp2 := TBGRABitmap.Create('TulipSmall.bmp');
  resultbmp := MakeTween(bmp1,bmp2,trackbar1.position);
  Image3.Picture.Bitmap.Assign(resultbmp.Bitmap);
  resultbmp.Draw(Image3.Canvas,0,0,true);
  resultbmp.Free;
end;
   

will try it in the 5dpo application - I guess next I need to work out how to do a Chromakey (any ideas?)...

thanks

TheBlackSheep

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #4 on: May 13, 2012, 11:54:53 pm »
Well, no you should not replace dmSet by dmDrawWithTransparency, because when the images are transparent, you don't get what you want. If the images are not transparent, there should be no difference, so you can leave dmSet anyway.

In order to draw the image, if using the Bitmap property does not work, you can use MakeBitmapCopy. This way you get a non transparent image with a background color already applied, assign it and then free it.
Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #5 on: May 14, 2012, 11:14:59 am »
when I change it to dmSet it's just fades in the second image from no image to full opaque in the resultant window.  I did notice that in my previous code I was setting the resultant image twice (although it needed this to make it work for some reason). 

I've now changed it to this and this does seem to work;

Code: [Select]
procedure TForm1.TrackBar1Change(Sender: TObject);
var
   bmp1,bmp2, resultbmp: TBGRABitmap;

   function MakeTween(img1,img2: TBGRABitmap; position: byte): TBGRABitmap;
   begin
     if position = 0
        then result := TBGRABitmap(img1.Duplicate)
        else if position = 255
               then result := TBGRABitmap(img2.Duplicate)
     else
     begin
       result := TBGRABitmap(img1.Duplicate);
       result.PutImage(0,0,img2,dmSetWithTransparency,position);
     end;
   end;

begin
  bmp1 := TBGRABitmap.Create('000034.JPG');
  bmp2 := TBGRABitmap.Create('000014.JPG');
  resultbmp := MakeTween(bmp1,bmp2,trackbar1.position);
  Image3.Picture.Bitmap := resultbmp.MakeBitmapCopy(clNone);
  resultbmp.Free;
end; 

although you'll probably tell me that's still wrong  :(

I think the effect is better described as "translucency" as opposed to transparency - i.e. the second image is superimposed on top of the first image and with a 50:50 weighting each image is there in the output  - by adjusitng the trackbar you can increase or decrease the translucency of the image on top. 

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #6 on: May 15, 2012, 12:21:20 pm »
Oh I think there is a bug in the dmSet mode of PutImage. In fact, several bugs. I've just fixed them on subversion. Thanks for testing.

You wrote "dmSetWithTransparency" ? You mean "dmDrawWithTransparency" ?

You are right about translucency, but I did not find the right word at the time I defined it.

About memory usage, I suppose you should rather write something like :
Code: [Select]
var copy: TBitmap;
begin
  bmp1 := TBGRABitmap.Create('000034.JPG');
  bmp2 := TBGRABitmap.Create('000014.JPG');
  resultbmp := MakeTween(bmp1,bmp2,trackbar1.position);
  bmp1.Free;
  bmp2.Free;

  copy := resultbmp.MakeBitmapCopy(clBtnFace);
  resultbmp.Free;

  Image3.Picture.Bitmap := copy;
  copy.Free;
end; 

Try this with the last svn version, and see the difference between dmSet and dmDrawWithTransparency, so you'll understand why I was saying that you would rather want dmSet.
Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #7 on: May 16, 2012, 12:44:29 am »
hi circular

yep, that seems to have worked now for the "translucent" image - after a bit of searching I think this is also known as "uniform transparency".

One side effect of the new bgrabitmap source appears that when I save a snapshot of the original 5dpo webcam video image as a png I appear to get a transparent mask applied across the whole image which I wasn't seeing before (this is not the same "tweened" image but the raw image prior to being blended).

If I save the same image as a bitmap it's fine - save it as a png and it's completely transparent - so it's definitely something to do with transparency within png's that's changed between this and the previous version (it could of course be that the current version is correct and the previous version was saving the png incorrectly but looked ok from the manner in which I was using it).

Code: [Select]

procedure TFMain.btnSnapshotClick(Sender: TObject);
begin
 if CBVideoActive.Checked and (VideoImg <> nil)
   then begin
     VideoImg.SaveToFile('test'+FormatDateTime('yyyymmddhhnnss',now)+'.bmp');  //this works
     VideoImg.SaveToFile('test'+FormatDateTime('yyyymmddhhnnss',now)+'.png');  //fully transparent
   end;
end;

procedure BGR24_to_TrueColor(src: PRGB24Pixel; dest: PLongWord; size: Integer);
var i: integer;
begin
 // move(src^, dest^, size*4);
  for i := 0 to size -1 do begin
    // this is the reason why often BGR instead of RGB
    // is used, we don't need shifting like above, we can
    // simply dump entire longwords into their new locations.
    dest[i] := PLongWord(@src[i])^;
  end;
end;

procedure TFMain.VideoFrame(Sender: TObject; FramePtr: PByte);
var Composite:TBGRABitmap;

  function MakeTween(img1,img2: TBGRABitmap; position: byte): TBGRABitmap;
   begin
     if position = 0
        then result := TBGRABitmap(img1.Duplicate)
        else if position = 255
               then result := TBGRABitmap(img2.Duplicate)
     else
     begin
       result := TBGRABitmap(img1.Duplicate);
       result.PutImage(0,0,img2,dmSet,position);
     end;
   end;

begin
  FrameRate:=round(1/((GetTickCount-FrameTime)/1000));
  FrameTime:=GetTickCount;
  StatusBar.SimpleText := format('(%d, %d) %d fps', [video.Width, Video.Height, FrameRate]);
  case Video.PixelFormat of
    uvcpf_YUYV: YUYV_to_Gray(PLongWord(FramePtr), PLongWord(VideoImg.Data), Video.Width * Video.Height);
    uvcpf_YUV420: YUV420_to_Gray(FramePtr, PLongWord(VideoImg.Data), Video.Width * Video.Height);
    uvcpf_RGB24: RGB24_to_TrueColor(PRGB24Pixel(FramePtr), PLongWord(VideoImg.Data), Video.Width * Video.Height);
    uvcpf_BGR24: BGR24_to_TrueColor(PRGB24Pixel(FramePtr), PLongWord(VideoImg.Data), Video.Width * Video.Height);
   end;

  if chkbxOverlay.Checked then begin
    OnionSkin := TBGRABitmap.Create('test20120511145457.png');
    Composite := MakeTween(videoimg,onionskin,(255 div trackbar1.max) * trackbar1.position);
    Composite.Draw( BGRAVirtualScreen1.Canvas, 0, 0, True );
    Composite.Free;
    OnionSkin.Free;
  end else begin
    VideoImg.Draw( BGRAVirtualScreen1.Canvas, 0, 0, True );
  end;
end;

this is basically the example from the SdpoVideo4L2 component - I've set the webcam to uvcpf_BGR24 pixelformat (the default I think) - the videoframe function is constantly running and I have a "snapshot" button which allows the user to take the current frame and save it - as you can see, VideoImg is a source not a target image for the "tween" operation and it looks ok on the BGRAVirtualScreen - we're copying the images to make the tween anyway so not sure how the image is now going fully transparent (unless I'm doing something really obviously stupid here  :o)  Any ideas?

TheBlackSheep

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #8 on: May 16, 2012, 08:34:38 pm »
Well the PNG writer is buggy sometimes, so it may just be that. Anyway, you can check this by applying AlphaFill(255) before saving to PNG. If there is no difference, then it's only PNG saving that is buggy.

It may also be compiling issue. To be sure it is not, go to bgrabitmap folder and delete the "lib" subfolder.

Oh and I'm sorry to tell you but the way you draw on the virtual screen is not the right way. Well it works under Windows and Linux to a certain extent, but if the form is redrawn for some reason, the image will disappear. And no MacOS, it won't appear at all.

The main idea is that you should not access with BGRAVirtualScreen1.Canvas, because it is only the visible result of the virtual screen. To draw on the virtual screen, you need to override the OnRedraw event. So for example, in the VideoFrame, you can keep the current bitmap (VideoImg) and call BGRAVirtualScreen1.Redraw. In the OnRedraw handler, put your "composite" code and then PutImage into the virtual screen (which is a parameter of the OnRedraw handler).

Is it clear or do you need more explanations about this ?
Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #9 on: May 16, 2012, 09:38:04 pm »
thanks circular

it works!!!! sorted the "correct" way to draw on the bitmap (calling BGRAVirtualScreen1.redrawBitmap now) and in the OnRedraw event do putImage's instead of trying to draw directly to the canvas. 

the snapshot works with .bmp, .jpg and even .png using the alphafill (on a duplicated image) to produce the png (and the quality of the image being produced is much improved - it was a bit noisy before and I was thinking I might need to add in some kind of averaging across a few frames although I might still need to do this to avoid ambient light flicker which is fairly typical in animation software).

The onionskin also works much better so the canvas drawing was definitely wrong even though it looked like it was sort of working.

I'll tidy this up now to produce a simple animation grabber and publish the source on github.

Many thanks for your support on this, it's much appreciated.

TheBlackSheep

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #10 on: May 17, 2012, 01:39:05 am »
Cool.  :)

One final comment, you should avoid loading the OnionSkin image at each frame.
Code: [Select]
OnionSkin := TBGRABitmap.Create('test20120511145457.png');
Rather load it once at startup and free at the end, or something like that.

Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images
« Reply #11 on: May 17, 2012, 08:51:37 am »
yep - it was just in there for testing at the minute, in practice it would normally be the previous snapshot image and loaded dynamically - I'll avoid loading it on every frame as it is currently.

Have you ever done anything with chromakey? (i.e. green screen transparency) - I'm thinking it's just a case of making a mask based on a pixel color bias towards green (or could be any color really) - there'd need to be a tolerance on the color as the greenscreens are never uniform to one color because of lighting/shading differences.

TheBlackSheep

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #12 on: May 17, 2012, 12:02:49 pm »
Never done chromakey but you can use BGRADiff or BGRAWordDiff functions to compare each pixel to some color.

So you can go through each scan line, compare each pixel and in the destination bitmap put either from one source or another.
Conscience is the debugger of the mind

TheBlackSheep

  • Jr. Member
  • **
  • Posts: 93
Re: "Tween" two images / ChromaKey
« Reply #13 on: May 21, 2012, 06:39:40 pm »
hi circular

I thought I had the chromakey (greenscreen) working (it looks like it does on Win32) but on Linux it fails badly - any ideas?

I have 3 images on the form, the foreground has a green screen background - just a standard image found on a google image lookup for greenscreen images - the background can be any image and I've just used a random landscape picture. - the Composite one is the final "blended" image.  There are 4 trackbars - 1 for each color and a final one which is the BGRADiff threshold (named "tolerance");

All the trackbars are tied to this event to do the chromakey; 

Code: [Select]
procedure TfrmChromaKeyTest.TrackBarChange(Sender: TObject);
var
   BG,FG,Composite: TBGRABitmap;
   p1,p2,p3: PBGRAPixel;
   n : integer;
   diff : integer;
begin
  BG := TBGRABitmap.Create(imgBackground.Picture.Bitmap);//background source
  FG := TBGRABitmap.Create(imgForeground.Picture.Bitmap);//foreground source with greenscreen
  if (BG.width <> FG.width) or (BG.Height <> FG.Height)
    then BGRAReplace(BG,BG.Resample(FG.Width,FG.Height));
  Composite := TBGRABitmap.Create(FG.Width, FG.Height); //target

  p1 := BG.Data;
  p2 := FG.Data;
  p3 := Composite.Data;

  lblRedChromaKeyValue.Caption:=IntToStr(trkbrRedChromakey.Position);
  lblGreenChromaKeyValue.Caption:=IntToStr(trkbrGreenChromakey.Position);
  lblBlueChromaKeyValue.Caption:=IntToStr(trkbrBlueChromakey.Position);
  lblToleranceValue.Caption:=IntToStr(trkbrTolerance.Position);

  for n := FG.NbPixels-1 downto 0 do begin
    diff := BGRADiff(p2^, BGRA(trkbrRedChromakey.Position,trkbrGreenChromakey.Position,trkbrBlueChromakey.Position));
    if diff <= trkbrTolerance.Position
      then p3^ := p1^
      else p3^ := p2^;
    inc(p1);
    inc(p2);
    inc(p3);
  end;
  Composite.InvalidateBitmap;
  imgComposite.Picture.Bitmap := Composite.MakeBitmapCopy(clBlack);
  Composite.Free;
  FG.Free;
  BG.Free;
  DrawZoom;
end;   

I think this is what you'd suggested and the wiki example to get access to the pixels wthin the image and then compare using BGRADiff - it seems to work ok in Win32 but Linux the composite image is all messed up with the colors bleeding all over the image (moving the trackbars does have an effect but the "color bleed" is always there)

TheBlacksheep

circular

  • Hero Member
  • *****
  • Posts: 4195
    • Personal webpage
Re: "Tween" two images
« Reply #14 on: May 21, 2012, 10:15:46 pm »
Maybe there is a bug in BGRADiff.

Your code seems good but :
- you could store BGRA(trkbrRedChromakey.Position,trkbrGreenChromakey.Position,trkbrBlueChromakey.Position) in a variable to avoid making it for each pixel
- Composite.MakeBitmapCopy(clBlack) returns an object that need to be freed, so you need to assign it a variable, assign it and then free it. When you write imgComposite.Picture.Bitmap := Composite.MakeBitmapCopy(clBlack), it is not a real assignment, it is a property that duplicates the bitmap.

Can you put a screenshot of the awful result on Linux, to see if it can be a bug of BGRADiff ? What if you try with BGRAWordDiff ?
Conscience is the debugger of the mind

 

TinyPortal © 2005-2018