Lazarus

Programming => Graphics and Multimedia => Graphics => Topic started by: aradeonas on October 15, 2015, 06:32:23 pm

Title: Fast hardware image blur
Post by: aradeonas on October 15, 2015, 06:32:23 pm
Hi,

I want to do fast blur so I need hardware acceleration, for this I found these sources but I dont know how to do it in FPC with OpenGL or OpenCL with or without BGRABitmap.
http://www.eriksmistad.no/gaussian-blur-using-opencl-and-the-built-in-images-textures/
https://software.intel.com/en-us/blogs/2014/07/15/an-investigation-of-fast-real-time-gpu-based-image-blur-algorithms 
Title: Re: Fast hardware image blur
Post by: ChrisR on May 30, 2016, 10:41:11 pm
You can do a blur easily on the CPU. The trick is that the 2D Gaussian blur is separable: a 2D blur can be computed much faster as two 1D blurs instead of an integrated 2D blur (and a 3D blur is much faster when computed as three 1D blurs). When you know this trick, you can get good performance using just the CPU for all but the largest images
  http://blog.ivank.net/fastest-gaussian-blur.html
  http://www.cabiatl.com/mricro/bounty/
  http://mantis.freepascal.org/view.php?id=10275

For very large images, the GPU will be faster. However, remember that transfers from main memory to the GPU is slow, so if you are using OpenGL on the GPU to display your images you will certainly benefit from using the GPU to blur, but if you want to move the image back to the CPU to use a library like BBRABitmap you may find that it is hard to see a huge speedup with the GPU (since you will have to both copy the raw data to the GPU and then retrieve the blurred data). If you want to go this route, I would recommend GLSL rather than OpenCL. This reason is that many older graphics drivers do not support OpenCL and it is harder to get this to work on all platforms. The code I wrote uses GLSL and works on Intel/NVidia/AMD cards with Windows/Linux/OSX. The shaders kSmoothShaderFrag  and kSobelShaderFrag demonstrate how to call GLSL from Lazarus/FPC/Delphi to blur and blur/edgedetect a 3D image. These would be a good starting point for a 2D blur.
  https://github.com/neurolabusc/MRIcroGL/blob/master/raycastglsl.pas
Title: Re: Fast hardware image blur
Post by: aradeonas on May 31, 2016, 09:20:30 am
Thank you very much for a useful answer ;)
Do you have a performance test that how much time it gets for a simple blur in you machine with or without OpenGL?
About MRIcroGL, What it does? and in that unit I see functions that seem to do blur, is that right? is there any demo to give me a easier start point? I'm new in this field :D
About OpenCL, how old machines dont support it? If they are much old I dont care about that if the OpenCL results is better than GLSL (they are?)
Title: Re: Fast hardware image blur
Post by: circular on May 31, 2016, 01:49:41 pm
Hello!

Regarding software blur, in the latest versions of BGRABitmap, the fast blur it is done using the something similar to algorithm 4 described on http://blog.ivank.net/fastest-gaussian-blur.html

To do a "hardware" blur, basically the idea is the following (using TBGLVirtualScreen):

Here attached is a sample project for a shader (here it flips the image vertically).
Title: Re: Fast hardware image blur
Post by: aradeonas on May 31, 2016, 02:34:53 pm
Hi Circular and thank you for the demo,
I didnt know BGRABitmap can do these stuff this easily :D . I will play with it more but I can understand  whet you said about blur "that does the blur (you may need to derive the class to add "variables")".

Also please add a simple demo when you add an ability. complicated demo is complicated and they are good for showing power not how to do that. Doing that is easier than making documentation and also Im sure you make self test project and probably you can put them in tests folder with some cleaning.

About blur speed, I tested again BGRABitmap blur and used image from the linked page and in Normal mode with r=10 it takes 700ms in my system (that is not old) and in Fast mode it takes 90 ms. Are you sure it is optimized like that or my machine is heavily slow? (I checked your code and it seems like that but I cant understand why is there such a huge difference)
Title: Re: Fast hardware image blur
Post by: ChrisR on May 31, 2016, 03:10:05 pm
My sense is the example from Circular is a great starting point. If you create a blur shader using TBGLVirtualScreen I think it would be useful for a lot of users. My own MRIcroGL is tuned as a 3D volume renderer, rather than a general purpose rendering tool. Since you ask, MRIcroGL uses GLSL for the volume rendering but it can also use GLSL for hardware accelerated computation of image gradients.
  http://www.mccauslandcenter.sc.edu/mricrogl/gradients
With MRIcroGL the user can set the preferences to either calculate the gradients on the CPU or the GPU. The relative benefit really depends on the quality of the GPU and the CPU, but modern Intel graphics seem about x8 faster on the GPU than on the Intel CPU, with a bigger benefit if you have a dedicated Nvidia or AMD GPU.

For writing your shader, I would suggest reading this article:
  https://developer.apple.com/library/mac/documentation/Performance/Conceptual/OpenCL_MacProgGuide/TuningPerformanceOntheGPU/TuningPerformanceOntheGPU.html
Title: Re: Fast hardware image blur
Post by: circular on May 31, 2016, 03:35:41 pm
Hi Circular and thank you for the demo,
I didnt know BGRABitmap can do these stuff this easily :D . I will play with it more but I can understand  whet you said about blur "that does the blur (you may need to derive the class to add "variables")".
Cool.

Quote
Also please add a simple demo when you add an ability
Yes.

Quote
About blur speed, I tested again BGRABitmap blur and used image from the linked page and in Normal mode with r=10 it takes 700ms in my system (that is not old) and in Fast mode it takes 90 ms. Are you sure it is optimized like that or my machine is heavily slow? (I checked your code and it seems like that but I cant understand why is there such a huge difference)
In Normal mode, it doesn't use the optimisation with box blurs. Only fast blur with radius >= 10 does.
Title: Re: Fast hardware image blur
Post by: aradeonas on May 31, 2016, 03:56:23 pm
Thank you ChrisR.

circular I had a typo :
I can't understand what you said about blur "that does the blur (you may need to derive the class to add "variables")".
Quote
In Normal mode, it doesn't use the optimisation with box blurs. Only fast blur with radius >= 10 does.
I just tested and it takes 62ms for r=9 and 90 for r=11. it seems not not to me it has a much difference and I expect faster result.
Title: Re: Fast hardware image blur
Post by: circular on May 31, 2016, 05:40:50 pm
Quote
I can't understand what you said about blur "that does the blur (you may need to derive the class to add "variables")".
Oh ok. I suggest to look at the example I posted, and look at the GLSL files. There is the code for the shader. And that's where you would need to put some other code.

The GLSL code is related to a TBGLShader3D class. In the example it is defined in umyshader.pas. The uniform variables are like parameters that are transmitted to the GLSL code. They are defined both in the class and in the GLSL (in mix.fragment.glsl).

For example the blur radius would be a uniform variable, like the fade_factor in the example.

Quote
I just tested and it takes 62ms for r=9 and 90 for r=11. it seems not not to me it has a much difference and I expect faster result.
Surprising. Well the fast blur without box blur is already optimized so the difference is not that big with a small radius. The difference is significant with a radius like 80 for example. To see the difference, you would need to comment out the "if" block in the fast blur where it calls the box blur.

After that, you can try to call box blur directly. Maybe the box blur could be optimized?



Title: Re: Fast hardware image blur
Post by: aradeonas on October 03, 2016, 10:25:26 am
Hi circular,

I totally forgot this topic and today I tried to test my version but I coulnt port such thing to your demo, it is more complicated and I dont know about them:
http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/
Can you help me with that? and also please include previous example with BGRA, its good to have these samples as reference.
Title: Re: Fast hardware image blur
Post by: circular on October 03, 2016, 11:55:57 am
To do this example, you only need to change the code in mix.fragment.glsl file.

Some variables need to be replaced:

"image" becomes "textures[0]" (what the application refers to)
"FragmentColor" becomes "gl_FragColor" because the GLSL variable is used instead of using an output variable.
"gl_FragCoord" becomes "texcoord" (what the vertex shader yields).

So you would get something like:
Code: C  [Select]
  1. uniform float fade_factor;
  2. uniform sampler2D textures[1];
  3.  
  4. varying vec2 texcoord;
  5.  
  6. uniform float offset[5] = float[]( 0.0, 1.0, 2.0, 3.0, 4.0 );
  7. uniform float weight[5] = float[]( 0.2270270270, 0.1945945946, 0.1216216216,
  8.                                    0.0540540541, 0.0162162162 );
  9. uniform float textureHeight = 200;                                                               
  10.                                                                            
  11. void main()
  12. {
  13.         gl_FragColor = texture2D(textures[0], texcoord) * weight[0];
  14.         for (int i=1; i<5; i++) {
  15.                 gl_FragColor +=
  16.                         texture2D(textures[0], vec2(texcoord)+vec2(0.0, offset[i]/textureHeight) )
  17.                                 * weight[i];
  18.                 gl_FragColor +=
  19.                         texture2D(textures[0], vec2(texcoord)-vec2(0.0, offset[i]/textureHeight) )
  20.                                 * weight[i];
  21.         }
  22.  
  23.         // use the fade factor to choose the level of blur
  24.         gl_FragColor = gl_FragColor*fade_factor + texture2D(textures[0], texcoord)*(1 - fade_factor);
  25. }
Title: Re: Fast hardware image blur
Post by: aradeonas on October 03, 2016, 12:25:25 pm
Thank you very much!
It made it more clear for me.
I tested it and result is fast but I coulnt make the blur more strong. This post seems a good one and I wanted to do such thing like sample in the post or like this implemention:
https://github.com/Jam3/glsl-fast-gaussian-blur
What is your opinion abut this way? Is it good?
Title: Re: Fast hardware image blur
Post by: circular on October 03, 2016, 12:46:08 pm
The level of blur here via fade_factor is just from non-blurred to blurred. It does not go further. For a wider blur, there would be more coefficients of course.

Note that this is not the optimised version using the linear interpolation. That's would be the second example of
http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/ and I leave that to you.

Also the blur here is one-dimensional. You can make it two dimensional, but that requires two "for" loops, that's slower obviously. In principle, Guassian blur can be done in two passes, one horizontal and one vertical but that would require to store a temporary image that is blurred in one direction.
Code: C  [Select]
  1. uniform float fade_factor;
  2. uniform sampler2D textures[1];
  3.  
  4. varying vec2 texcoord;
  5.  
  6. uniform float offset[5] = float[]( 0.0, 1.0, 2.0, 3.0, 4.0 );
  7. uniform float weight[5] = float[]( 0.2270270270, 0.1945945946, 0.1216216216,
  8.                                    0.0540540541, 0.0162162162 );
  9. uniform float textureWidth = 320;
  10. uniform float textureHeight = 200;                                  
  11.                                        
  12. void main()
  13. {
  14.     float w;
  15.     gl_FragColor = vec4(0);
  16.     for (int y = 0; y < 5; y++) {
  17.     for (int x = 0; x < 5; x++) {
  18.         w = weight[x]*weight[y];
  19.         gl_FragColor += texture2D(textures[0], vec2(texcoord)+vec2(offset[x]/textureWidth, offset[y]/textureHeight) ) * w;
  20.         if (x > 0)
  21.             gl_FragColor += texture2D(textures[0], vec2(texcoord)+vec2(-offset[x]/textureWidth, offset[y]/textureHeight) ) * w;
  22.         if (y > 0)
  23.         {
  24.             gl_FragColor += texture2D(textures[0], vec2(texcoord)+vec2(offset[x]/textureWidth, -offset[y]/textureHeight) ) * w;
  25.             if (x > 0)
  26.                 gl_FragColor += texture2D(textures[0], vec2(texcoord)+vec2(-offset[x]/textureWidth, -offset[y]/textureHeight) ) * w;
  27.         }
  28.     }
  29.     }
  30.  
  31.     // use the fade factor to choose the level of blur
  32.     gl_FragColor = gl_FragColor*fade_factor + texture2D(textures[0], texcoord)*(1 - fade_factor);
  33. }

On the page https://github.com/Jam3/glsl-fast-gaussian-blur the algorithm is also one-dimensional. It needs to be called twice to get a regular blur (two-dimensional).

I wonder if that would be doable by rendering the texture first in a framebuffer with the blur in one direction and then displaying from that framebuffer while applying the blur in the other direction.
Title: Re: Fast hardware image blur
Post by: aradeonas on October 03, 2016, 01:15:24 pm
Thanks. So I should learn this way and make a better version of it.
Title: Re: Fast hardware image blur
Post by: aradeonas on October 03, 2016, 02:41:16 pm
I am learning GLSL as I didnt knew it before.
Im reading the code too and here :
Code: C  [Select]
  1.                         for (int i=1; i<iterations; i++) {
  2.                                 glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fbo[1]);
  3.                                 glBindTexture(GL_TEXTURE_2D, fbtex[0]);
  4.  
  5.                                 glUseProgram(prog[mode]);
  6.                                 glDrawArrays(GL_TRIANGLES, 0, 6);
  7.  
  8.                                 glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fbo[0]);
  9.                                 glBindTexture(GL_TEXTURE_2D, fbtex[1]);
  10.  
  11.                                 glUseProgram(prog[mode+2]);
  12.                                 glDrawArrays(GL_TRIANGLES, 0, 6);
  13.                         }
  14.  
iterations will be set from num keys so you can set it for example 9 at it seems it repeat the shader 9 times as he says :
Quote
f you have to apply the filter several times in order to get a more strong blur effect, the only thing you have to do is ping-pong between two framebuffers and apply the shaders to the result of the previous step.
How can I do such thing in BGL?
Title: Re: Fast hardware image blur
Post by: circular on October 03, 2016, 09:15:32 pm
Interesting.

I haven't implemented framebuffers yet.  :-[

I am learning as well. So using glBindFrameBuffer can change the current canvas. Hmmm...
Title: Re: Fast hardware image blur
Post by: aradeonas on October 04, 2016, 11:09:15 am
OK, So if you got time to implement it please let me know. Excuse me for not implementing this myself, you know m, I just know how to use these to make  higher level work. these are way more complicated for me :D
Title: Re: Fast hardware image blur
Post by: circular on October 04, 2016, 02:10:26 pm
I will do when I have some time. That will be a nice feature and will allow to do antialiasing via resampling. At least now I have a rough idea of how to do it. There are some examples there:
https://www.opengl.org/wiki/Framebuffer_Object_Examples
Title: Re: Fast hardware image blur
Post by: aradeonas on October 04, 2016, 02:21:03 pm
Happty to hear that ;)
Title: Re: Fast hardware image blur
Post by: simin_sh on January 19, 2017, 01:25:15 pm
Hi

I was testing Circular testshader1 project in this post:
http://forum.lazarus.freepascal.org/index.php/topic,30012.msg211701.html#msg211701

with the new fragment glsl code in this post for fast blur:
http://forum.lazarus.freepascal.org/index.php/topic,30012.msg224043.html#msg224043

and I got this error you can see in the picture attached, then I set the version in the code "440" because my OpenGL version is 4.4, but it seams that the version in TBGLShader3D did not assign correctly, or I am not setting the version correctly in the create shader function in UMyShader class, and I do not know where is the problem with glsl version. Here is the code :

Code: Pascal  [Select]
  1. constructor TMyShader.Create(ACanvas: TBGLCustomCanvas);
  2. begin
  3.   inherited Create(ACanvas, ReadFileToString('mix.vertex.glsl'), ReadFileToString('mix.fragment.glsl'), '', '440');
  4.   FFadeFactorUniform := UniformSingle['fade_factor'];
  5.   FTexturesUniform[0] := UniformInteger['textures[0]'];
  6. end;
  7.  

I am using windows 10, Lazarus 1.6 and BGRABitmap from Git.

Any help appreciated prior
Title: Re: Fast hardware image blur
Post by: simin_sh on January 22, 2017, 12:12:43 pm
Hi

I was testing Circular testshader1 project in this post:
http://forum.lazarus.freepascal.org/index.php/topic,30012.msg211701.html#msg211701

with the new fragment glsl code in this post for fast blur:
http://forum.lazarus.freepascal.org/index.php/topic,30012.msg224043.html#msg224043

and I got this error you can see in the picture attached, then I set the version in the code "440" because my OpenGL version is 4.4, but it seams that the version in TBGLShader3D did not assign correctly, or I am not setting the version correctly in the create shader function in UMyShader class, and I do not know where is the problem with glsl version. Here is the code :

Code: Pascal  [Select]
  1. constructor TMyShader.Create(ACanvas: TBGLCustomCanvas);
  2. begin
  3.   inherited Create(ACanvas, ReadFileToString('mix.vertex.glsl'), ReadFileToString('mix.fragment.glsl'), '', '440');
  4.   FFadeFactorUniform := UniformSingle['fade_factor'];
  5.   FTexturesUniform[0] := UniformInteger['textures[0]'];
  6. end;
  7.  

I am using windows 10, Lazarus 1.6 and BGRABitmap from Git.

Any help appreciated prior



Here is what I did to solve the problem: As said in https://en.wikipedia.org/wiki/OpenGL_Shading_Language#Versions, setting version should be like #version440, then when I remove word "define" in "TBGLShader3D.Create" constructor, it sets version correctly and runs the program.
Title: Re: Fast hardware image blur
Post by: circular on January 23, 2017, 06:16:48 pm
Ok thanks. I've applied the change on Git.
Title: Re: Fast hardware image blur
Post by: simin_sh on January 24, 2017, 07:12:29 am
You are welcome.

How can I have multiple fragments glsl code files and a vertex glsl code?? for having a blur in vertical and horizontal orientations for example. I know I can use glBindFramebuffer in glsl and they are not yet impelemented in TBGLShader3D class, but do you have any advice or guidance for doing this?? or is it possible to have framBuffers in BGRA soon??

Any help appreciated prior
Title: Re: Fast hardware image blur
Post by: circular on January 27, 2017, 01:46:45 pm
Indeed framebuffers are not implemented yet and that would be the way to go.
Title: Re: Fast hardware image blur
Post by: User137 on January 27, 2017, 06:14:37 pm
If you want to use different glsl codes per polygons, you might want to change "program". glUseProgram(n) will do that.

If you want to study such code from nxpascal you are free to https://github.com/Zaflis/nxpascal/blob/master/src/nxGL.pas#L209  It has example of that in walker_shaders demo. Also a demo about framebuffer where it draws a cube in texture that it uses on the cube faces so... it makes the "infinite mirror" effect.
Title: Re: Fast hardware image blur
Post by: simin_sh on January 28, 2017, 11:24:15 am
Thank you for your reply User137, I will check it.
Title: Re: Fast hardware image blur
Post by: simin_sh on January 28, 2017, 11:43:34 am
Indeed framebuffers are not implemented yet and that would be the way to go.

Thank you for your replay Circular.
Is it possible for you to give me an estimated time for implementing FrameBuffers??

Regards
Title: Re: Fast hardware image blur
Post by: circular on February 13, 2017, 05:43:46 pm
I've been quite busy but I guess now I will have a bit of free time. I am not sure how long it would take, but probably 6 hours would do. Now I just have to find 6 hours in my weekends.
Title: Re: Fast hardware image blur
Post by: simin_sh on February 14, 2017, 07:17:37 am
I've been quite busy but I guess now I will have a bit of free time. I am not sure how long it would take, but probably 6 hours would do. Now I just have to find 6 hours in my weekends.


Great, thank you so much. So I will waiting.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 04, 2017, 09:25:38 am
Hi Circular
Did you get a chance to look at the FrameBuffer part?
Regards
Title: Re: Fast hardware image blur
Post by: circular on March 04, 2017, 04:20:11 pm
I think I am getting somewhere. I have added a TBGLFrameBuffer class in the dev-bgrabitmap branch.

How it works: create a TBGLFrameBuffer class, then switch to it by using ActiveFrameBuffer property of BGLCanvas.

For example :
Code: Pascal  [Select]
  1. procedure TForm1.OpenGLControlPaint(Sender: TObject);
  2. var
  3.   mousePos: TPoint;
  4.   buf: TBGLFrameBuffer;
  5. begin
  6.   { Draw Background }
  7.   BGLViewPort(OpenGLControl.Width, OpenGLControl.Height, BGRAWhite);
  8.   { Create framebuffer }
  9.   buf := TBGLFrameBuffer.Create(256,256);
  10.   { Use it }
  11.   BGLCanvas.ActiveFrameBuffer := buf;
  12.  
  13.   BGLCanvas.Fill(CSSYellow);
  14.   BGLCanvas.Line(0,0,128,128, BGRABlack);
  15.   mousePos := ScreenToClient(Mouse.CursorPos);
  16.   BGLCanvas.FillRect(mousePos.x - 50, mousePos.y - 50, mousePos.x + 50, mousePos.y + 50, CSSRed, False);
  17.  
  18.   { Render framebuffer on the control }
  19.   BGLCanvas.ActiveFrameBuffer := nil;
  20.   BGLCanvas.PutImage(0,0, buf.Texture);
  21.   { Free framebuffer }
  22.   buf.Free;
  23.  
  24.   { Update }
  25.   OpenGLControl.SwapBuffers;
  26. end;
  27.  

In this example, a red square is drawn around the mouse cursor, but within the framebuffer (yellow background). So it gets clipped.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 05, 2017, 07:51:04 am
Thank you so much Circular.
Now I can try the way of using two frames for implementing a fast blur using vertical and horizontal glsl blur codes.
Title: Re: Fast hardware image blur
Post by: circular on March 05, 2017, 09:17:51 am
You're welcome.

Good luck, and if there is something to fix, just tell me.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 06, 2017, 01:20:50 pm
I tested giving 2 TBGLFrameBuffer to a BGLCanvas, by changing ActiveFrameBuffer property, and I understand how it works now. But I am not sure how to use these frame buffers functionality in TBGLShader3D, for giving 2 shader and 2 fragments together to it. any advice or idea??

Any help appreciated prior
Regards
Title: Re: Fast hardware image blur
Post by: circular on March 06, 2017, 06:53:18 pm
I would suggest to use one shader with a vec2 parameter to indicate the direction.

Then you can have two framebuffers. In the first one, you draw the texture with the shader in one direction. In the second one, you draw the first frame buffer with the shader in a perpendical direction.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 07, 2017, 10:47:53 am
I created 2 shaders for vertical and horizontal blur directions, each one exists in a separate class, and wrote this code for testing what you suggested. any comment??
Below is my code, and I attached the whole test project too.

Any help appreciated prior
Regards

Code: Pascal  [Select]
  1.  
  2. procedure TForm1.FormShow(Sender: TObject);
  3. begin
  4.   tex := BGLTexture('Pics\Src.jpg');
  5.   DataHoriz.Shader := TMyShaderHoriz.Create(BGLCanvas);
  6.   DataVert.Shader := TMyShaderVert.Create(BGLCanvas);
  7.   TexLoaded := True;
  8. end;
  9.  
  10. procedure TForm1.OpenGLControlPaint(Sender: TObject);
  11. begin
  12.   if not TexLoaded then
  13.     Exit;
  14.   BGLViewPort(OpenGLControl.Width, OpenGLControl.Height, BGRAWhite);
  15.   Buf0 := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  16.   Buf1 := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  17.  
  18.   BGLCanvas.ActiveFrameBuffer := Buf0;
  19.   BGLCanvas.Lighting.ActiveShader := DataHoriz.Shader;
  20.   tex.Draw(0, 0);
  21.  
  22.   BGLCanvas.ActiveFrameBuffer := Buf1;
  23.   BGLCanvas.Lighting.ActiveShader := DataVert.Shader;
  24.   Buf0.Texture.Draw(0, 0);
  25.  
  26.   BGLCanvas.ActiveFrameBuffer := Buf0;
  27.   BGLCanvas.Lighting.ActiveShader := DataHoriz.Shader;
  28.   Buf1.Texture.Draw(0, 0);
  29.  
  30.  
  31.   BGLCanvas.ActiveFrameBuffer := nil;
  32.   BGLCanvas.Lighting.ActiveShader := DataVert.Shader;
  33.   Buf0.Texture.Draw(0, 0);
  34.   BGLCanvas.Lighting.ActiveShader := nil;
  35.  
  36.   Buf0.Free;
  37.   Buf1.Free;
  38.   OpenGLControl.SwapBuffers;
  39. end;      
  40.  
Title: Re: Fast hardware image blur
Post by: circular on March 07, 2017, 11:46:34 pm
Ok. This code uses OpenGL 3.3 shader language. In order to have the projection right, it would then be needed to pass the projection matrix as a parameter. So i've added support for that on dev-bgrabitmap branch. At the same time I realized I had made some mistake for multidimensional shader variables. So I fixed it as well.

Here I adapted your code accordingly and fixed a few things.

Still not perfect, because the shader does not really know the texture coordinates and assumes that they are the same as the render coordinates. That's not exactly the case as for some reason the frame buffer is vertically flipped, so one last FlipY is necessary when drawing the result.

Note : I replaced the icon to make the zip lighter.
Title: Re: Fast hardware image blur
Post by: circular on March 08, 2017, 12:06:12 am
I think it is possible from there to do a function that blurs a texture (returning a new texture that is blurred, the one from the second framebuffer). The only thing is that for now, if the framebuffer is freed, that frees the texture as well. So maybe one function to free the framebuffer but keep the texture would be handy.

Otherwise, to have the information about the vertex coordinate, the way to go with OpenGL 3 would be to use array buffers. I made some classes to use them, even if it is not much official yet.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 08, 2017, 12:05:37 pm
Thank you Circular
I checked the code and now its much better and understandable, I think my next step would be testing deeper blur functionality, and implement it in glsl code, and have Radius param to manage blur deep dependency in algorithm.

I was wondering is there a way for saving the blured image, I know we have glReadPixels in openGL, but I was wondering if there is such functionality in BGRABtimap?

Regards
Title: Re: Fast hardware image blur
Post by: circular on March 08, 2017, 01:25:28 pm
You're welcome.

Reading pixels is not implemented yet but that's of course something we can add.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 08, 2017, 02:42:56 pm
I tested a ping pong between buffers for a deeper blur, I think its the right way, but I have a problem in the size of texture coordinates and the control in draw, you can see it when draw the blured picture on control. any idea how to fix it??
Maybe it is because of the problem you said about flip, or the glsl code, I am not sure.

Here is my code
and I attached the test project with the sample pic I tested.

Any help appreciated prior
Regards


Code: Pascal  [Select]
  1.  
  2. procedure TForm1.scenePaint(Sender: TObject);
  3. var
  4.   n, i: integer;
  5. begin
  6.  
  7.   if not TexLoaded then
  8.     Exit;
  9.   BGLViewPort(scene.Width, scene.Height, BGRAWhite);
  10.   //allocate target for horizontal blur
  11.   //hb=horiz buffer, vb=vert buffer, hs=horiz shader, vs=vert shader
  12.   hb := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  13.   BGLCanvas.ActiveFrameBuffer := hb;
  14.  
  15.   //configure shader
  16.   hs.ProjectionMatrix := BGLCanvas.ProjectionMatrix;
  17.   hs.TextureSize := Point(tex.Width, tex.Height);
  18.   BGLCanvas.Lighting.ActiveShader := hs;
  19.  
  20.   tex.Draw(0, 0); //perform horiz blur
  21.  
  22.   //allocate target for vertical blur
  23.   vb := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  24.   BGLCanvas.ActiveFrameBuffer := vb;
  25.  
  26.   //configure shader
  27.   vs.ProjectionMatrix := BGLCanvas.ProjectionMatrix;
  28.   vs.TextureSize := Point(tex.Width, tex.Height);
  29.   BGLCanvas.Lighting.ActiveShader := vs;
  30.  
  31.   hb.Texture.Draw(0, 0); //perform vert blur on horizontally blurred image
  32.  
  33.   //Apply blur n times
  34.   n := 100;
  35.   for i := 0 to n - 1 do
  36.   begin
  37.     //allocate target for horizontal blur
  38.     BGLCanvas.ActiveFrameBuffer := hb;
  39.     //configure shader
  40.     BGLCanvas.Lighting.ActiveShader := hs;
  41.     vb.Texture.Draw(0, 0); //perform horiz blur
  42.     //allocate target for vertical blur
  43.     BGLCanvas.ActiveFrameBuffer := vb;
  44.     //configure shader
  45.     BGLCanvas.Lighting.ActiveShader := vs;
  46.     hb.Texture.Draw(0, 0); //perform vert blur on horizontally blurred image
  47.   end;
  48.  
  49.   //render result
  50.   BGLCanvas.ActiveFrameBuffer := nil;
  51.   BGLCanvas.Lighting.ActiveShader := nil;
  52.   vb.Texture.FlipY.Draw(0, 0); //draw on control
  53.  
  54.   vb.Free;
  55.   hb.Free;
  56.   scene.SwapBuffers;
  57. end;  
  58.  
Title: Re: Fast hardware image blur
Post by: simin_sh on March 08, 2017, 02:46:11 pm
You're welcome.

Reading pixels is not implemented yet but that's of course something we can add.


Ok, so you think we can have it on BGRA in near future?? Or I need to to find another way?? because the speed and quality of blur is so important for me in this case, and I thought that implementing the save result functionality would help me analyze these items better.

Regards
Title: Re: Fast hardware image blur
Post by: simin_sh on March 08, 2017, 03:13:27 pm
I tested a ping pong between buffers for a deeper blur, I think its the right way, but I have a problem in the size of texture coordinates and the control in draw, you can see it when draw the blured picture on control. any idea how to fix it??
Maybe it is because of the problem you said about flip, or the glsl code, I am not sure.

Here is my code
and I attached the test project with the sample pic I tested.

Any help appreciated prior
Regards


Code: Pascal  [Select]
  1.  
  2. procedure TForm1.scenePaint(Sender: TObject);
  3. var
  4.   n, i: integer;
  5. begin
  6.  
  7.   if not TexLoaded then
  8.     Exit;
  9.   BGLViewPort(scene.Width, scene.Height, BGRAWhite);
  10.   //allocate target for horizontal blur
  11.   //hb=horiz buffer, vb=vert buffer, hs=horiz shader, vs=vert shader
  12.   hb := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  13.   BGLCanvas.ActiveFrameBuffer := hb;
  14.  
  15.   //configure shader
  16.   hs.ProjectionMatrix := BGLCanvas.ProjectionMatrix;
  17.   hs.TextureSize := Point(tex.Width, tex.Height);
  18.   BGLCanvas.Lighting.ActiveShader := hs;
  19.  
  20.   tex.Draw(0, 0); //perform horiz blur
  21.  
  22.   //allocate target for vertical blur
  23.   vb := TBGLFrameBuffer.Create(tex.Width, tex.Height);
  24.   BGLCanvas.ActiveFrameBuffer := vb;
  25.  
  26.   //configure shader
  27.   vs.ProjectionMatrix := BGLCanvas.ProjectionMatrix;
  28.   vs.TextureSize := Point(tex.Width, tex.Height);
  29.   BGLCanvas.Lighting.ActiveShader := vs;
  30.  
  31.   hb.Texture.Draw(0, 0); //perform vert blur on horizontally blurred image
  32.  
  33.   //Apply blur n times
  34.   n := 100;
  35.   for i := 0 to n - 1 do
  36.   begin
  37.     //allocate target for horizontal blur
  38.     BGLCanvas.ActiveFrameBuffer := hb;
  39.     //configure shader
  40.     BGLCanvas.Lighting.ActiveShader := hs;
  41.     vb.Texture.Draw(0, 0); //perform horiz blur
  42.     //allocate target for vertical blur
  43.     BGLCanvas.ActiveFrameBuffer := vb;
  44.     //configure shader
  45.     BGLCanvas.Lighting.ActiveShader := vs;
  46.     hb.Texture.Draw(0, 0); //perform vert blur on horizontally blurred image
  47.   end;
  48.  
  49.   //render result
  50.   BGLCanvas.ActiveFrameBuffer := nil;
  51.   BGLCanvas.Lighting.ActiveShader := nil;
  52.   vb.Texture.FlipY.Draw(0, 0); //draw on control
  53.  
  54.   vb.Free;
  55.   hb.Free;
  56.   scene.SwapBuffers;
  57. end;  
  58.  



Here is the ScreenShot that shows what I mean of the size problem in control or texture, I set the form size exactly the size of picture but you can see in the picture that its not fit, and I do not know where is the problem.

Any help appreciated prior
Regards
Title: Re: Fast hardware image blur
Post by: circular on March 09, 2017, 07:43:32 am
Hmm did you set the size using ClientWidth/ClientHeight?

I will have a look at a save feature when I can.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 09, 2017, 07:55:30 am
Thank you so I will be waiting for the news of save functionality.

Yes, it does not make any difference, all the sizes including TBGLFrameBuffer, BGLViewPort, BGLTexture and the Form in the program are 900*506 which is exactly the size of my source jpg picture.
And the blur image is still like the screenshot.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 09, 2017, 10:58:39 am
I have a question about the algorithm of blur.

When I ping pong between buffers, it seems that something is wrong about the blur functionality, because with the various number of ping pong between buffers and having a deeper blur, the progress of blur is not what it should be, I mean the blur picture in 25 times blur is not that different from 35 times blur!

And you can see from the picture attached what I mean exactly, upper left picture is blur 5 time, upper right is 30 times, and downer left is 60 times and down right is 300 times!

Any idea or suggestions??

Any help appreciated prior
Regards
Title: Re: Fast hardware image blur
Post by: circular on March 09, 2017, 08:17:19 pm
That's normal. Repeating a blur does not make it much wider. The loop of the blur needs to take a wider range of pixels. And the weights need to be computed, for example in the shader.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 09, 2017, 08:34:41 pm
Thank you for your replay
Because of that I am not completely sure about the algorithm I can not say how to implement such thing, I thought I just can do it with the ping pong between buffers, any Idea of how to increase the range of pixels?? Can I do this with increasing the weights for example??

Regards
Title: Re: Fast hardware image blur
Post by: circular on March 10, 2017, 07:24:32 pm
You can use Gauss formula:
https://en.wikipedia.org/wiki/Gaussian_blur

There are sqrt and exp functions in GLSL.

The actual radius is roughly twice the sigma value. So for a sigma value of 5, you would loop from -10 to 10, or as in the algorithm you've used, sum the value at x+0 and then loop from 1 to 10 (including x-1 and x+1, then x-2 and x+2, etc.).

To be sure to have a correct sum, you also need to sum the weights, so that in the end you can divide by it. Otherwise you won't be sure that the sum of the weights is equal to 1. Instead it will be something like 0.99, so colors would get slightly darker.

Does it make sense?



Title: Re: Fast hardware image blur
Post by: simin_sh on March 11, 2017, 08:32:09 am
Thank you for your replay.

But no I can not understand the changes you described for having a deeper blur, or a blur with custom Radius, can you give me a sample code of what you mean please?

And I am not that much familiar with GLSL, But I am learning about it.

Any help appreciated prior
Regards
Title: Re: Fast hardware image blur
Post by: circular on March 11, 2017, 10:47:34 am
I am also learning GLSL my friend. However I will give you a hand.

The horizontal shader would something like:
Code: glSlang  [Select]
  1. uniform sampler2D image;
  2. uniform ivec2 textureSize;
  3. varying vec2 texCoord;
  4.  
  5. out vec4 FragmentColor;
  6.  
  7. #define PI 3.14159265358979323846
  8.  
  9. //compute gauss function for sigma and x
  10.  
  11. float gauss(float sigma, float x)
  12. {
  13.         float normalized = x/sigma;
  14.         return 1/(sqrt(2*PI)*sigma)*exp(-0.5*normalized*normalized);
  15. }
  16.  
  17. void main(void)
  18. {
  19.         float sigma = 10;
  20.         //the range of pixels for which the weight will be at least 1% is roughly 2*sigma
  21.         int range = int(2*sigma+0.5);
  22.        
  23.         float weight = gauss(sigma, 0);
  24.         float totalWeight = weight;
  25.         FragmentColor = texture2D( image, texCoord/textureSize ) * weight;
  26.        
  27.         for (int i=1; i<=range; i++) {
  28.                 weight = gauss(sigma, i);
  29.                 FragmentColor += texture2D( image, (texCoord + vec2(i, 0.0))/textureSize ) * weight;
  30.                 FragmentColor += texture2D( image, (texCoord - vec2(i, 0.0))/textureSize ) * weight;
  31.                 totalWeight += 2*weight;
  32.         }
  33.        
  34.         //normalize result because totalWeight is not exactly 1
  35.         FragmentColor /= totalWeight;
  36. }

I let you add "sigma" as a uniform parameter for the shader so that you can change it from the Pascal code.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 11, 2017, 12:58:49 pm
Yes, thank you so much Circular, it looks like exactly what it should be, comparing with the blur deep in LazPaint.

Now I should know where is the problem with the size of picture and texture and displaying the result in the openglcontrol, and also check the time, because it takes 30ms for a picture of size 900*600 (regardless of sigma value) and its too much different from what it should be!

Regards
Title: Re: Fast hardware image blur
Post by: User137 on March 11, 2017, 03:02:32 pm
it takes 30ms for a picture of size 900*600 (regardless of sigma value) and its too much different from what it should be!
Try to precalculate sqrt(2*PI) to a single float value. Calculator says it's 2.506628274631000502415765284811 ... Leave a few digits out for it to accept.
Title: Re: Fast hardware image blur
Post by: circular on March 11, 2017, 09:35:43 pm
You can also try with a constant weight to see if it is faster.
Title: Re: Fast hardware image blur
Post by: lainz on March 11, 2017, 09:55:44 pm
simin_sh You will release it? It will be nice to have opengl blur in bgrabitmap, I just need it to use in a project for the graphics contest, just for fun.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 12, 2017, 07:23:49 am
it takes 30ms for a picture of size 900*600 (regardless of sigma value) and its too much different from what it should be!
Try to precalculate sqrt(2*PI) to a single float value. Calculator says it's 2.506628274631000502415765284811 ... Leave a few digits out for it to accept.


Thank you for your replay

Yes it made it faster, for the same sample it takes 50% less time. But I am not sure about the time I am calculating, because it takes different values in different run times! Once it takes 15ms, once it takes 16ms and once it takes 0! I will test it better and with different samples and see the difference.

Regards
Title: Re: Fast hardware image blur
Post by: simin_sh on March 12, 2017, 07:27:42 am
You can also try with a constant weight to see if it is faster.


Thank you for your replay
You mean I can have weights calculated in an array for example and just use them??

Regards
Title: Re: Fast hardware image blur
Post by: simin_sh on March 12, 2017, 07:31:27 am
simin_sh You will release it? It will be nice to have opengl blur in bgrabitmap, I just need it to use in a project for the graphics contest, just for fun.


I am still testing some things, but here is my final test project if it would be useful for you. You just need to have a Src.jpg file in Pic folder, because code will use it for blur.

Regards
Title: Re: Fast hardware image blur
Post by: circular on March 12, 2017, 11:57:17 am
@simin_sh: What I am suggesting is to use weight = 1/(2*range+1)

That would give a box blur as a result.

Otherwise you can try to create the framebuffers only once, or recreate them only if the size has changed. And free them when the form is closed.

About the time begin 0 or 15 ms, that's normal because the grain of time is equal to 15 ms. To have more precision, you could use EpikTimer, or do the same thing 10 times and then divide the time by 10.
Title: Re: Fast hardware image blur
Post by: lainz on March 12, 2017, 02:02:32 pm
Thanks you both for all this.

@Circular, you will include a blur demo in bgrabitmap or maybe somewhere in another repository?
Title: Re: Fast hardware image blur
Post by: circular on March 12, 2017, 07:12:19 pm
Yep
Title: Re: Fast hardware image blur
Post by: simin_sh on March 13, 2017, 07:43:39 am
Thanks you both for all this.

@Circular, you will include a blur demo in bgrabitmap or maybe somewhere in another repository?

You are very welcome
Title: Re: Fast hardware image blur
Post by: simin_sh on March 13, 2017, 07:49:50 am
@simin_sh: What I am suggesting is to use weight = 1/(2*range+1)

That would give a box blur as a result.

Otherwise you can try to create the framebuffers only once, or recreate them only if the size has changed. And free them when the form is closed.

About the time begin 0 or 15 ms, that's normal because the grain of time is equal to 15 ms. To have more precision, you could use EpikTimer, or do the same thing 10 times and then divide the time by 10.

Thank you for your replay

I tested the formula for Box Blur, it seems that the time does not have that much different, but as you said when I calculate the average time for 30 Blur with different sigma, time is 3-5 ms for each one, I think it is okay for now, but I still need to save it in the end, and check the time I need.

Regards
Title: Re: Fast hardware image blur
Post by: circular on March 13, 2017, 07:36:54 pm
I've added BGLCanvas.GetImage. It returns a TBGRABitmap object that you can save.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 14, 2017, 08:53:39 am
I've added BGLCanvas.GetImage. It returns a TBGRABitmap object that you can save.


Thank so much Circular.
Regards
Title: Re: Fast hardware image blur
Post by: simin_sh on March 15, 2017, 03:53:53 pm
I have tested GetImage of BGLCanvas and its giving me same result, there is a gap in the canvas and I do not recognize what is it exactly, but I know its about using shaders, because when I use BGLFrame buffers separately in a test project its completely fit. You can see what I mean in the ScreenShot attached.

For simplicity of test I have removed the codes about vertical blur shader.

Any Idea why its happening??

Any help appreciated prior
Regards   

Code: Pascal  [Select]
  1.  
  2. procedure TForm1.scenePaint(Sender: TObject);
  3. var
  4.   bmp: TBGRACustomBitmap;
  5. begin
  6.   if not TexLoaded then
  7.     Exit;
  8.   BGLViewPort(w, h, BGRAWhite);
  9.  
  10.   //allocate target for horizontal blur
  11.   hb := TBGLFrameBuffer.Create(w, h);
  12.   BGLCanvas.ActiveFrameBuffer := hb;
  13.  
  14.   //configure shader
  15.   hs.ProjectionMatrix := BGLCanvas.ProjectionMatrix;
  16.   hs.TextureSize := Point(w, h);
  17.   BGLCanvas.Lighting.ActiveShader := hs;
  18.   hs.Sigma := sig;
  19.   //
  20.   BGLCanvas.PutImage(0, 0, tex);
  21.  
  22.   //render result
  23.   BGLCanvas.ActiveFrameBuffer := nil;
  24.   BGLCanvas.Lighting.ActiveShader := nil;
  25.   BGLCanvas.PutImage(0, 0, hb.Texture);
  26.   hb.Free;
  27.  
  28.   scene.SwapBuffers;
  29.  
  30.   bmp := BGLCanvas.GetImage(0, 0, BGLCanvas.Width, BGLCanvas.Height);
  31.   bmp.SaveToFile('Pics\GaussBlur1.png');
  32.   bmp.Free;
  33. end;    
  34.  
  35.  
  36. procedure TForm1.FormCreate(Sender: TObject);
  37. begin
  38.   w := 900;
  39.   h := 506;
  40.   Form1.Width := w;
  41.   Form1.Height := h;
  42.   WriteLn(w, ' , ', h);
  43.   scene := TOpenGLControl.Create(Self);
  44.   with scene do
  45.   begin
  46.     Parent := Self;
  47.     Width := w;
  48.     Height := h;
  49.     OnPaint := @scenePaint;
  50.     AutoResizeViewport := True;
  51.   end;
  52.   sig := 1;
  53. end;
  54.  
  55. procedure TForm1.FormShow(Sender: TObject);
  56. begin
  57.   tex := BGLTexture('Pics\1.jpg');
  58.   hs := TMyShaderHoriz.Create(BGLCanvas);
  59.   vs := TMyShaderVert.Create(BGLCanvas);
  60.   TexLoaded := True;
  61. end;          
  62.  
  63.  
  64.  
Title: Re: Fast hardware image blur
Post by: simin_sh on March 15, 2017, 05:08:35 pm
Regardless of the problem of size of the picture, I have a question about quality and appearance of Gauss blur.

When I compare Gauss blur we have made in OpenGL with the fastBlur in Lazpaint, it seems that the radius has at least 50% smaller size of the sigma in the GLSL Formula that I pass as the blur depth, and the quality is different too.
You can see the compare I tested in the video file that I uploaded here:
https://www.dropbox.com/s/ebnt80wgug9k02a/blur.zip?dl=0

Any idea or suggestions??

Regards
Title: Re: Fast hardware image blur
Post by: circular on March 15, 2017, 06:40:32 pm
Maybe the parameter with the projection matrix does not work. If the width and height are equal, do you still have the problem?

About the radius, it is expected to be different than the sigma value. The radius is something like sqrt(2) times bigger than the sigma. We could try to emulate the fast blur with GLSL to see if that's similar.
Title: Re: Fast hardware image blur
Post by: simin_sh on March 16, 2017, 10:06:29 am
Maybe the parameter with the projection matrix does not work. If the width and height are equal, do you still have the problem?

Yes I have tested it before and there is still a gap.
You can see the result of blur when I use a picture of size 506*506 attached.
Title: Re: Fast hardware image blur
Post by: circular on March 18, 2017, 05:38:54 pm
Does it help using old style shading language (version 120 instead of 330)? This way you can use predefined variables for vertex and texture position.

See attached project
Title: Re: Fast hardware image blur
Post by: simin_sh on March 24, 2017, 02:35:42 pm
Thank you so much Circular, yes its now okay when I use version 130 for GLSL.
I did not use GLSL 120 because it gave me the following error in the SS attached.

Now I need to try implement horizontal and vertical shaders in one shader, and then need to test other blurs like fast blur in GLSL, to have the blurs in lazpaint, which are a bit more soft and beautiful, I need to check them out.

Regards
Title: Re: Fast hardware image blur
Post by: circular on April 30, 2017, 09:19:57 pm
Hello there!

I have implemented radial blur and motion blur in BGRAOpenGL in latest version.

Simply write tex.FilterBlurMotion or tex.FilterBlurRadial with adequate parameters where tex is an IBGLTexture.

http://forum.lazarus.freepascal.org/index.php/topic,24239.msg244982.html#msg244982

Regards
Title: Re: Fast hardware image blur
Post by: simin_sh on May 01, 2017, 08:05:35 am
Hello there!

I have implemented radial blur and motion blur in BGRAOpenGL in latest version.

Simply write tex.FilterBlurMotion or tex.FilterBlurRadial with adequate parameters where tex is an IBGLTexture.

http://forum.lazarus.freepascal.org/index.php/topic,24239.msg244982.html#msg244982

Regards

GREAT, I will see them as soon as possible.
Thank you