[SOLVED] Need help with "Marked memory at $XXXXXXXX invalid Exception".

[garlar27]

Hi,
   I'm doing a Client/Server test program which works with many threads and I have this strange exception:

Marked memory at $B3DD0060 invalid
Wrong signature $AAAAAAAA instead of 02202B3D
  $080785BD
  $080785F0
  $080706A2
  $080947B2  TSYNCTHREADTEST__LOG_,  line 122 of SyncThreadTestApplication.pas
  $084344FB  TBASELOGGER__LOGFMT,  line 559 of MyBaseClass.pas
  $08436B56  TBASECLASS__LOGFMT,  line 1572 of MyBaseClass.pas
  $08096E66  TBASECONNECTIONCLASS__RECVMESSAGE,  line 930 of SyncThreadsTypes.pas
  $0809F312  TCONECTIONTHREAD__ESPERARRESPUESTA,  line 836 of ServerConnThreadClasses.pas
  $0809FDDA  TCONECTIONTHREAD__EXECUTE,  line 943 of ServerConnThreadClasses.pas

the code where the exception is rised:

Code: [Select]

procedure TSyncThreadTest.Log_(AMessage: string);
var
   fh, Msg: string;
   UnPChar: PChar;
begin
   DateTimeToString(fh, FORMATO_LOG_FECHA_HORA_CORTO, Now); { AñoMesDíaHoraMinutoSegundoMilisegundo}
{line 122}   Msg := fh + ',' + RightStr('000000000000' + IntToStr(PtrInt(GetThreadID)), 10) + ',' + CaracteresEspecialesComoHexa(AMessage, TRUE);

   UnPChar := StrAlloc(Length(Msg)+1);
   StrCopy(UnPChar, PChar(Msg));
   PostMessage(FOutputForm.Handle, WM_VERBOSE, Integer(UnPChar), 0);

   try // A CRITICAL SECTION IS USED SINCE ALL THREADS SHARE THE SAME LOG FILE.
      System.EnterCriticalsection(FLogWriteCritSect);
      SimpleWrite(Msg);
   finally
      System.LeaveCriticalsection(FLogWriteCritSect);
   end;

   //try
   //   System.EnterCriticalsection(ConsoleOutputCritSect);
   //try {$IFDEF LINUX}DebugConsole(Msg);{$ENDIF} except end;
   //finally
   //   System.LeaveCriticalsection(ConsoleOutputCritSect);
   //end;
end; {<--- LogWrite }

The exception is raised after "A LOT" of calls to this procedure.
Does anybody knows what this exception means?
Did I something wrong?

[garlar27]

Oops!!
I forgot to mention I'm using FPC 2.4.4; Lazarus 0.9.30; Ubuntu 10.04 LTS

[ludob]

It seems you are using heaptrc. That is the error you get when doing a freemem with a pointer that was not obtained with getmem.

Perhaps one of the functions used in line 122 is not thread safe. For debugging purposes, put the whole block inside the critical section.

EDIT: Also, in case you use a different memory manager like cmem, make sure that heaptrc is loaded after the memory manager by putting it in the uses clause after the memory manager unit. Don't use the -gh compiler option since it will put heaptrc in front of every other unit.

[garlar27]

Mmmmhhhh....

Yes I'm using HeapTrace from the compiler option. I removed that option and that exception disappeared.

Perhaps one of the functions used in line 122 is not thread safe. For debugging purposes, put the whole block inside the critical section.

I don't think that line 122 is causing problems, since is not shared among threads and now that the exception has disappeared confirms that.

Thanks ludob!!! =D

[Martin_fr]

Yes the exception has disappeared. But the error is still there.

Heaptrace simply increases the chance of catching the error. Without it you will have random other crashes.

Wrong signature $AAAAAAAA instead of 02202B3D

Most likely means that you accessed (written to ) memory that you already freed before.

Such memory will eventually be used for other objects, and then writing random data into it, will mean disaster.

This can be caused by missing thread synchronization. but it can also be happen without threads.

And unfortunately, it is one of the hardest to debug issues that there are....

---------
Also you are right (AFAIK, only 95% sure..)

The error could be (and probably is) on an other line.

I am *guessing* this code may want to allocate memory (for the strings).
That triggers the check of the signature.

And it fails, because at some time before, something wrote to that memory.

IIRC it is possible to modify heaptrc so you can call the check by hand. (But I do not recall how, and it needs to recompile the RTL too)

If so, checks can be placed at strategic points, and narrow down the real location ...

You can also try to analyse the memories content (but a watch with memory dump). But B3DD0060  is only the start of the memory block. It could be very big, and out of all of it, it may be a single byte, somewhere....


Does the error contain added info, where the memory was previously used (allocated) ?

[ludob]

I don't think that line 122 is causing problems, since is not shared among threads and now that the exception has disappeared confirms that.

Strange. A few lines further in the same begin...end block you are using a critical section because it is shared by several threads and the beginning of that block is not shared? RightStr or CaracteresEspecialesComoHexa are also shared. Or are they local thread procedures? AFAIK RightStr is thread safe.

Martin_fr is right in that the exception is gone because the heaptrc check is removed. Your problem is probably still there but passes unnoticed most of the times. What I tried to explain is that heaptrc could be the cause of a problem if it is loaded before a non-default memory manager. If that is not the case then the problem is with your code.

DateTimeToString is a suspect for not being thread safe. In Delphi they explicitly mark that form of DateTimeToString as not thread-safe. See http://docwiki.embarcadero.com/Libraries/en/System.SysUtils.DateTimeToString. I think fpc suffers from the same problem. Since line 122 uses fh for the last time, it is very well possible that fh is freed there. 

You better stick to heaptrc until you are sure the problem is gone. Try putting suspected areas inside a critical section and narrow the problem down to the function causing the problem.

[KpjComp]

Code: [Select]

Msg := fh + ',' + RightStr('000000000000' + IntToStr(PtrInt(GetThreadID)), 10) + ',' + CaracteresEspecialesComoHexa(AMessage, TRUE);

One idea is to temporally split the above into multiple lines to narrow down your search.

eg.

Code: [Select]

Msg := fh + ',';
Msg += RightStr('000000000000' + IntToStr(PtrInt(GetThreadID)), 10);
Msg += ',' + CaracteresEspecialesComoHexa(AMessage, TRUE);

Or if running inside the Debugger try evaluating each part and see if it looks logical.  A trick I do here if errors are intermittent is wrap everything inside a try except and place a breakpoint on the a raise, I know with Delphi sometimes the Evaluator gets all confused if left to the debugger to break on an unhandled exception, by breaking on a "raise" local var scope etc is left intact.  Also make sure optimizations are turned off while debugging.

eg.

Code: [Select]

try
  DoSomething();
except
  raise; //stick a breakpoint here,
end;


[garlar27]

Martin_fr is right. There's something wrong in my code. I'm still having random exceptions some are on a procedure "end;" line, one in an unit "end." and most of them in a while loop like this:

Code: [Select]

while Assigned(SomeObject) do begin
   Index := SomeObject.Index; // Access Violation O_o
   // ...
end;

I have a lot of logs and for sure I will have to add many more. I have 2 suspects:
   1: The objects I'm moving from thread to thread (I couldn't add a critical sections to them because that throw a nasty exception which I don't remember now).
   2: The lists and Queues I use (though every list and queue use it's own critical section).

  What you say give me ideas and places to watch. The only problem is that it seems that I'm not able to see where things start to go wrong at least not with what I have.
  Maybe I should put the logs inside an SQLite instead of using plain text....

KpjComp:
I think I have a better try except:

Code: [Select]

   try
      DoSomething();
   except
      on E: Exception do begin
         {$IFDEF DEBUG}
         LogFmt('   [EXCEPTION] TTransactionList.%S.GetTransactionAt: "%S"', [CompleteName, E.Message]);
         LogFmt('   [EXCEPTION] TTransactionList.%S.GetTransactionAt: GetDumpCallStack'#13'%S', [CompleteName, GetDumpExceptionCallStack(E)]);//GetDumpCallStack(20)]);//
         {$ENDIF}
         Terminate;
      end;
   end;
GetDumpCallStack was taken from here   
I could have done everything in the same line but sometimes the dump call stack is not recorded.

[KpjComp]

I think I have a better try except:

Yes, but my try except raise had nothing to do with logging errors.  But in making it easer for you to trace into using the Debugger.  If your not using the debugger, them my bit of code would be a waste of space. 

[garlar27]

I cant use breakpoints on Linux to debug threads  it will freeze the XServer so I have to rely on logs

[KpjComp]

I cant use breakpoints on Linux to debug threads

That's makes things a little tricky!! 

Are you able to post more of your code, maybe many eyes might help..
Or if you can add a Zip of a test project that does the same would be even better.

[garlar27]

Does anybody knows what can cause an "Invalid pointer operation" exception?

[KpjComp]

Does anybody knows what can cause an "Invalid pointer operation" exception?

Yes, it could be a billions to the power billions of reasons.

[garlar27]

I was fearing that answer....
  Maybe I should change the TFPList for open arrays, and/or change how objects go from thread to thread.

Also, I'm pondering to change the whole design, expecting not to export my errors to the new design.

By the way does anybody knows where I can find examples of how to use TSimpleIPC?
These links didn't helped much:
   http://lazarus-ccr.sourceforge.net/docs/fcl/simpleipc/tsimpleipcserver.html
   http://lazarus-ccr.sourceforge.net/docs/fcl/simpleipc/tsimpleipcclient.html

[Martin_fr]

Is it always the same pointer? At the same location?

When the exception happens, you should inspect the code. Look at the variables, (pointers, strings, objects). Check there values.
Pointers must point to memory belonging to your app...

If you can nail it down to 1 or 2 variables, then you can use watchpoints (only available in 0.9.31) on the variable, to be informed whenever it changes...

Of course you need to solve the x11/thread issue... There is a way to debug over ssh from a different PC. BUT it is very limited and has a few issues of its own...
Have you tried QT? I have no idea, but maybe it works better?