Recent

Author Topic: openGL app/comiss/invalid floating point operation  (Read 2672 times)

quantumfoam

  • New Member
  • *
  • Posts: 23
openGL app/comiss/invalid floating point operation
« on: October 12, 2018, 08:58:28 pm »
I have an OpenGL application, programmed in Lazarus 1.8.4 64 bit (compiled as Win32GUI -WG),
where I skip creation of Lazarus main form. Instead, I create my own OpenGL window via Win API calls.
But, in the program I also use Lazarus built-in dialogs, like Color picker dialog, Font dialog etc.
When I use these dialogs (but still rendering main OpenGL window in some slow rate, like 1-10 fps via timer),
it RANDOMLY fails in rendering preparation phase (mostly after closing dialog).
I use rendering via FBO to texture + few render buffers for mouse picking. It fails on
clearing of buffer, where I store ID of rendered triangle.

Code: Pascal  [Select][+][-]
  1.   glBindFramebuffer(GL_FRAMEBUFFER, FFrameBuffer);
  2.   finalQuadTextureDest.bind(true); // bind some destination texture to get rid of: "Texture state usage warning: Texture 0 is base level inconsistent. Check texture size"
  3.   glClear( GL_COLOR_BUFFER_BIT or GL_DEPTH_BUFFER_BIT);
  4.   col := 0;
  5.   glClearBufferuiv( GL_COLOR, FRenderbufferId.RenderbufferId, @col); // IT FAILS HERE -> Project raised exception class 'External: SIGFPE'. / Invalid floating point operation
  6.   finalQuadTextureDest.bind(false);
  7.  

Here is assembler debug dump:

Quote
000000005A540CB0 f30f10642428             movss  0x28(%rsp),%xmm4
000000005A540CB6 0f57c0                   xorps  %xmm0,%xmm0
000000005A540CB9 0f2fc8                   comiss %xmm0,%xmm1
000000005A540CBC f30f102d1c6ece00         movss  0xce6e1c(%rip),%xmm5        # 0x5b227ae0 <vkCreateInstance+7878384>
000000005A540CC4 f30f118928010000         movss  %xmm1,0x128(%rcx)
000000005A540CCC f30f11912c010000         movss  %xmm2,0x12c(%rcx)
000000005A540CD4 f30f119930010000         movss  %xmm3,0x130(%rcx)
000000005A540CDC f30f11a134010000         movss  %xmm4,0x134(%rcx)
000000005A540CE4 7206                     jb     0x5a540cec
000000005A540CE6 f30f5dcd                 minss  %xmm5,%xmm1
000000005A540CEA eb03                     jmp    0x5a540cef
000000005A540CEC 0f28c8                   movaps %xmm0,%xmm1
000000005A540CEF 0f2fd0                   comiss %xmm0,%xmm2                // FAILS ON THIS LINE -> Project raised exception class 'External: SIGFPE'. / Invalid floating point operation
000000005A540CF2 f30f1189b8010000         movss  %xmm1,0x1b8(%rcx)
000000005A540CFA 7206                     jb     0x5a540d02
000000005A540CFC f30f5dd5                 minss  %xmm5,%xmm2
000000005A540D00 eb03                     jmp    0x5a540d05

xmm0 is :
{v4_float = {0, 0, 0, 0}, v2_double = {0, 0}, v16_int8 = {0 <repeats 16 times>}, v8_int16 = {0, 0, 0, 0, 0, 0, 0, 0}, v4_int32 = {0, 0, 0, 0}, v2_int64 = {0, 0}, uint128 = 0}

xmm2 is:
{v4_float = {-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}, v2_double = {-nan(0xfffffffffffff), -nan(0xfffffffffffff)}, v16_int8 = {-1 <repeats 16 times>}, v8_int16 = {-1, -1, -1, -1, -1, -1, -1, -1}, v4_int32 = {-1, -1, -1, -1}, v2_int64 = {-1, -1}, uint128 = 0xffffffffffffffffffffffffffffffff}

card: NVidia GeForce 710
driver version: 21.21.13.7866
driver creation date: 9.2.2017
app is written for OpenGL 4.3

My guess is, that something in those dialog windows leaves fpu/mmx in some inconsistent state and probably OpenGL driver can not handle it. I know this description is rather poor, but the project is quite big. I recently fixed all OpenGL debug output warnings, including change of ID render buffer from integer to float (using glClearBufferfv and uintBitsToFloat in shader) with no avail. The bug happens on Lazarus 64 bit, I could not replicate this on 32 bit.

I'm posting this in hope, that someone can give me a clue, which way to run investigation. Thanks.

Thaddy

  • Hero Member
  • *****
  • Posts: 14157
  • Probably until I exterminate Putin.
Re: openGL app/comiss/invalid floating point operation
« Reply #1 on: October 12, 2018, 09:11:47 pm »
I have seen similar things but only where the FPU is shared ( i.e. mixing x87 assembler with mmx/sse/etc)
That would be programmer error but I need  more input to test it. I can't reproduce it with the standard OpenGL unit.
Try to compile your code with -CfSSEv2 or higher depending on processor. (that's already pretty conservative!)
Note that fpc and most other compilers use the FPU x87 on 32 bit so the vector registers, which are shared with the FPU x87, may be dirty.
« Last Edit: October 12, 2018, 09:16:29 pm by Thaddy »
Specialize a type, not a var.

quantumfoam

  • New Member
  • *
  • Posts: 23
Re: openGL app/comiss/invalid floating point operation
« Reply #2 on: October 12, 2018, 10:58:08 pm »
I don't use any assembler code myself.
I tried SSE3 and SSE64 compiler switch with no avail. SSEv2 is not supported.

I will try to find some function for cleaning vector registers or maybe even use asm to do it,
as a quick and dirty test. And hopefully, in few days I will extract some minimal reproducible example.

Thanks for info.

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 844
Re: openGL app/comiss/invalid floating point operation
« Reply #3 on: October 14, 2018, 12:39:31 am »
Try to disable floating point exceptions. All drivers are written on C++, where floating point exceptions are disabled by default, so they don't bother about exceptions, their code causes. I had such problem on old GT210. My code worked well on my computer, but was causing floating point exception right on device creation on another computer. I tried to contact NVidia to report this bug, but they said, that they didn't care, as this video card was obsoleted. Then I just disabled floating point exceptions and it started to work properly. It's actually recommended by OpenGL/Direct3D docs.
« Last Edit: October 14, 2018, 12:41:11 am by Mr.Madguy »
Is it healthy for project not to have regular stable releases?
Just for fun: Code::Blocks, GCC 13 and DOS - is it possible?

quantumfoam

  • New Member
  • *
  • Posts: 23
Re: openGL app/comiss/invalid floating point operation
« Reply #4 on: October 14, 2018, 11:14:02 am »
The problem was with my misunderstanding of glClearBuffer* functions. They require 4-component vector.
I only supplied 1 uint/single. So the driver loaded other components from undefined memory.
Thanks anyway for tips.

 

TinyPortal © 2005-2018