TThread descendants consuming 100% CPU

[bauglir]

Hello,

I'm on
3.16.0-4-amd64 (Debian
everytime I try to create TThread descendant, program stops (on inherited Create within descendant's constructor) and start consuming all available CPU.
This happens in several programs I was able to run on other Linux versions/distros and on Windows as well.

I've heard something about thread affinity

do_Syscall(syscall_nr_sched_setaffinity,fpgetpid,setsize, @cpu_set);

but what does those parameters mean? Which types are those, which values? Where do I put this line?

Thanks for any help

Brona

[rvk]

Could you show some code (or a test-project perhaps) ??

How do you call the TThread.Create()? With what parameter for Suspended?
If you call it with true (Suspended) are you having the same trouble with 100% in the inherited Create() ??

(if not then there might be something wrong in your Execute-procedure)

[bauglir]

this is a constructor for my thread, the problem is not in Execute procedure, program hangs on inherited Create, suspended parameter makes no difference


constructor TE2Thread.Create(aSmsX: TP3SmsE; aConfig: TP3Config; aLog: TP3Logger);
begin
  inherited Create(true); //HERE IT HANGS
  fSms := aSmsX;
  fConfig := aConfig;
  fLog := aLog;
  fQuery := TP3MySql.Create;
  fQuery.OnQuery := OnSqlQuery;
  fQuery.Connect(Config.Database.Host, Config.Database.Login, Config.Database.Password);
  fQuery.ExecuteSQL('USE ' + Config.Database.Database);
  fQuery.ExecuteSQL('SET NAMES UTF8');
  FreeOnTerminate := true;
  Log.Log('thread');
end;

[rvk]

Is this a normal Lazarus form program?

Could you try a simple TThread and see if that also hangs?

Code: [Select]

type
   tc = class(tthread)
     procedure execute;override;
   end;
 
   procedure tc.execute;
   begin
   end;
 
procedure TestThread;
begin
   with tc.create(false) do
   begin
     waitfor;
     free;
   end;
end;

If this still hangs you might want to create a small test-project to see if it also poses a problem. If it does you can post it here to see if others have it too.

[Cyrax]

this is a constructor for my thread, the problem is not in Execute procedure, program hangs on inherited Create, suspended parameter makes no difference


constructor TE2Thread.Create(aSmsX: TP3SmsE; aConfig: TP3Config; aLog: TP3Logger);
begin
  inherited Create(true); //HERE IT HANGS
  fSms := aSmsX;
  fConfig := aConfig;
  fLog := aLog;
  fQuery := TP3MySql.Create;
  fQuery.OnQuery := OnSqlQuery;
  fQuery.Connect(Config.Database.Host, Config.Database.Login, Config.Database.Password);
  fQuery.ExecuteSQL('USE ' + Config.Database.Database);
  fQuery.ExecuteSQL('SET NAMES UTF8');
  FreeOnTerminate := true;
  Log.Log('thread');
end;

Move inherited Create(true); from beginning to the end of that subroutine. Also make sure that you have enabled unix thread support by defining -dUseCThreads and/or checking that cthreads unit is in your main program uses-clause.

[bauglir]

Hi,

1/ I'm actually not new to Pascal (been doing pascal for 25 years, 15 Delphi, 10 years FPC) so of course I tried to change suspend parameter, move inherited clause all over the place and of course I have cthreads enabled, and bunch of other things...

2/ programs are written on Windows with Delphi and then compiled on Linux using FPC (no Lazarus anywhere) and as I mentioned, all have been working for years on other linux machines
2.6.36-gentoo-r5 + FPC 2.6.0 = working (all 5 programs works)
2.6.31-gentoo-r6 + FPC 2.2.2 = working (all 5 programs works)
3.16.0-4-amd64 Debian + 2.6.4+dfsg-4 = not working  (4 out of 5 not working)

it does not have problem with trivial thread programs (e.g. example by rvk). It hangs in more complex programs.
I've actually noticed 2 ways how it hangs.
1/ either on inherited Create;
2/ on some random function somewhere in the code, where the function is executed, but never returns (whole function is executed, line after the function call is not)
and it always starts to consume all CPU...

B.

[rvk]

Just some question if somebody want to troubleshoot:

How does it do with the following:
gentoo + FPC 2.6.4 = .... ?
Windows + FPC 2.6.4 = .... ?
Are you using 32bit FPC or 64bit?

Even then... if a simple TThread works we still might need a test-project. If you can strip down one of the programs to a bare minimum and it still gives the same trouble it can be investigated. Without it it's just guesswork unless somebody else already encountered something similar.

[User137]

That seems like platform dependant bug. Should be reported to FPC issues? TThread class comes strictly from FPC (not lazarus).

(But i might add the fpc version of yours is heavily outdated by now.)

[u2o]

I've some similar problems on Windows with SQlite3, but CPU consumption only reaches 50%.

• The application hangs when the problem occurs, and after crash.
• The application processes all code properly and when I close it the problem occurs (without crash).

In my experience, many problems were due to poorly optimized code, but in others it is caused by Sqlite3 DLL.

The only way to find where the problem occurs in these cases is very complex, breakdown process by process, function by function. And if we add multiple queries to a database through a database engine (external), I wish you a very nice super-exciting month of tests. 

But if you want to prove my point, you can try different version of the engine of your database, and see if that is what happens.

[User137]

As far as i understood he has the problem on a empty app without databases?

[bauglir]

It's almost impossible for me to minimize dozens of thousands of lines of codes, but I have new issue with threads (again only on one server noted above, no problems elsewhere)

the following code is exiting with approx. 100 times

An unhandled exception occurred at $000000000041740C :
EAccessViolation :
  $000000000041740C


exception is not somewhere in the middle but when exiting application, all queries are fine, working.

The thing is, when I comment out

Code: [Select]

mt := TMyThread.Create(false);
no exception occures.
Only when I start thread, that error occures. As obvious, the thread actually does nothing.

Code: [Select]

program test;
uses cthreads, classes, sysutils, p3_mysql;

type
  TMyThread = class(TThread)
  public
    constructor Create(aCreateSuspended: boolean);
    procedure Execute; override;
  end; 

constructor TMyThread.Create(aCreateSuspended: boolean);
begin
  FreeOnTerminate := true;
  inherited Create(aCreateSuspended);
end;

procedure TMyThread.Execute;
begin
 
end;


var
  fQuery: TP3MySql;
  mt: TMyThread;
begin

  fQuery := TP3MySql.create;
  fQuery.Connect('10.50.1.2', 'user', 'password');
  fQuery.ExecuteSQL('set names utf8', false);

  mt := TMyThread.Create(false);



fQuery.ExecuteSql('SELECT * FROM `p2_userbilling`.`user` WHERE (`id` = %d)', [9], true);
fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user` WHERE (`id_user` = %d)', [9], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user` WHERE (`id_user` = %d)', [9], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_forbidden_number` WHERE (`id_sms_user` = %d)', [1], true);

      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_sender` WHERE (`id_sms_user` = %d)', [1], true);
//      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_value` WHERE (`id_sms_user` = %d)', [1], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`gateway_settings` WHERE (`id` = %d)', [5], true);



end.


[Cyrax]

You might want to create a simple message pump/loop for your test program which calls CheckSynchronize periodically. Also you might want to test if adding cmem unit before cthreads unit solves anything.

[derek.john.evans]

2.6.36-gentoo-r5 + FPC 2.6.0 = working (all 5 programs works)
2.6.31-gentoo-r6 + FPC 2.2.2 = working (all 5 programs works)
3.16.0-4-amd64 Debian + 2.6.4+dfsg-4 = not working  (4 out of 5 not working)

I'm not up on the whole Linux distro compatibility issue, but, if a exe is working on 2 Linux distros, why cant you test the working exe on Debian?

Have you tried to standardize the version of FPC you compile with?

To me, it sounds like your mysql library is not playing nicely, so I'd check that all the versions are the same. I'd also try to run the queries in a GUI app.

I cant imagine a decent MySQL library component not working in a GUI with extra TThreads. ie: That would indicate, it needs to be replaced.

[rvk]

What happens if you do it like this:

Code: [Select]

program test;
uses cthreads, classes, sysutils, p3_mysql;

type
  TMyThread = class(TThread)
  public
    procedure Execute; override;
  end; 

procedure TMyThread.Execute;
begin
end;

var
  fQuery: TP3MySql;
  mt: TMyThread;
begin

  fQuery := TP3MySql.create;
  fQuery.Connect('10.50.1.2', 'user', 'password');
  fQuery.ExecuteSQL('set names utf8', false);

  mt := TMyThread.Create(false);
  mt.waitfor;
  mt.free;

      fQuery.ExecuteSql('SELECT * FROM `p2_userbilling`.`user` WHERE (`id` = %d)', [9], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user` WHERE (`id_user` = %d)', [9], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user` WHERE (`id_user` = %d)', [9], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_forbidden_number` WHERE (`id_sms_user` = %d)', [1], true);

      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_sender` WHERE (`id_sms_user` = %d)', [1], true);
//      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`sms_user_value` WHERE (`id_sms_user` = %d)', [1], true);
      fQuery.ExecuteSql('SELECT * FROM `p2_sms`.`gateway_settings` WHERE (`id` = %d)', [5], true);

end.
I still feel that you should try what Geepster and I are saying. If your programs work with FPC 2.6.0 and 2.2.2 on another Linux-distro, then try it with those on the new one too. (or easiest is using the executable from those distro like Geepster suggests) Or try the 2.6.4 version on the other distros.

If you can't strip down your problem-project, maybe you can build up another test-project, adding things which you do in your big project. If we don't have a test-project to play around with it's near impossible for us to find the problem.