Recent

Author Topic: Reusing objects and multi threading  (Read 3079 times)

BeniBela

  • Hero Member
  • *****
  • Posts: 702
    • homepage
Reusing objects and multi threading
« on: October 18, 2016, 12:58:19 pm »
So I noticed memory management and object initialization/cleanup can take quite some time, when there are many small objects. Thus it seems wise to just skip it by reusing the instances: When the object is freed, it is not actually freed, but added to a list of old objects. When a new object is created, it is not actually created, but taken from that list, unless it is empty:

Code: Pascal  [Select]
  1. var
  2.     commonValues: array[0..0] of record
  3.       size: integer;
  4.       cache: array[0..10000] of tobject;
  5.     end;
  6.  
  7. type TFasterObject = class(TObject)
  8.     class function newinstance : tobject;override;
  9.     procedure FreeInstance;override;
  10.     function classKind: integer; virtual;
  11. end;
  12.      TProcedureOfObject=procedure () of object;
  13. class function TFasterObject.newinstance: tobject;
  14. var k: integer;
  15.   size: Integer;
  16.   //hackMethod: TMethod;
  17. begin
  18.   k := classKind;
  19.   size := commonValues[k].size;
  20.   if size > 0 then begin
  21.     result := commonValues[k].cache[size-1];
  22.     dec(commonValues[k].size);
  23.     //this can be skipped by cleaning it up in the destructor
  24.     //hackMethod := tmethod(@CleanupInstance);
  25.     //hackMethod.Data := result;
  26.     //TProcedureOfObject(hackMethod)();
  27.     //InitInstance(result);
  28.   end else begin
  29.     result := inherited newinstance;
  30.   end;
  31. end;
  32.  
  33. procedure TFasterObject.FreeInstance;
  34. var
  35.   k: integer;
  36.   size: Integer;
  37. begin
  38.   k := classKind;
  39.   size := commonValues[k].size;
  40.   if (size < high(commonValues[k].cache)) then begin
  41.     commonValues[k].cache[size] := self;
  42.     inc(commonValues[k].size);
  43.   end else begin
  44.     //CleanupInstance;
  45.     FreeMem(Pointer(Self));
  46.   end;
  47. end;
  48.  
  49. function TFasterObject.classKind: integer;
  50. begin
  51.   result := 0; //unique id, overriden for every derived class
  52. end;
  53.  
  54.  

In fact, the program now runs 10% faster in valgrind, without any other change.

Unfortunately this is not thread safe. How do you make it thread safe? A full lock could be slower than the normal TObject. Is there a lock-free stack?

I would like to avoid a free list with a next pointer in each object, because that needs  unbounded space, if there are many objects, while this array has a fixed, small size, when caching only the first 10000 objects.
« Last Edit: October 18, 2016, 01:08:46 pm by BeniBela »

Thaddy

  • Hero Member
  • *****
  • Posts: 9439
Re: Reusing objects and multi threading
« Reply #1 on: October 18, 2016, 01:13:22 pm »
The standard memory manager in FPC is a piece of shit known to be optimizing for average single core computing.
Since the olden days you can also write your own memory manager or use cmem (which performs better in real life scenario's)
I never understood what they were drinking/smoking  when they wrote the memory manager... At least it is pluggable.

To answer your question: at least on Linux cmem is threadsafe afaik.
« Last Edit: October 18, 2016, 01:17:56 pm by Thaddy »
also related to equus asinus.

Fungus

  • Sr. Member
  • ****
  • Posts: 352
Re: Reusing objects and multi threading
« Reply #2 on: October 18, 2016, 01:14:48 pm »
I do not understand why you would complicate your coding by reusing objects in a multi-threaded environment. Every time you spawn a new thread you will create a new object anyway - so why not encapsulate the functionality of the cached objects in the thread objects themselves?

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7862
Re: Reusing objects and multi threading
« Reply #3 on: October 18, 2016, 02:03:53 pm »
I do not understand why you would complicate your coding by reusing objects in a multi-threaded environment. Every time you spawn a new thread you will create a new object anyway - so why not encapsulate the functionality of the cached objects in the thread objects themselves?

Thread creation is expensive, in many real world scenarios worker threads are pooled too

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7862
Re: Reusing objects and multi threading
« Reply #4 on: October 18, 2016, 02:04:29 pm »
The standard memory manager in FPC is a piece of shit known to be optimizing for average single core computing.
Since the olden days you can also write your own memory manager or use cmem (which performs better in real life scenario's)
I never understood what they were drinking/smoking  when they wrote the memory manager... At least it is pluggable.

To answer your question: at least on Linux cmem is threadsafe afaik.

Do you have a list of cases where the default memmanager is not threadsafe?

Fungus

  • Sr. Member
  • ****
  • Posts: 352
Re: Reusing objects and multi threading
« Reply #5 on: October 18, 2016, 02:17:02 pm »
Thread creation is expensive, in many real world scenarios worker threads are pooled too

Yes, but the question as I understand it is how to cache objects in a multi-threaded environment. In order to create such environment the threads MUST be created anyhow and that is why I'm wondering why anyone would not just encapsulate the functionality of the cached objects into the threads in order to spare the complexity of the caching.

If this is a case of one thread who handles multiple clients (or whatever) coming an going the sollution would be to create a cache with client handlers for each thread instead of creating a shared cache with multi-threaded access. This would solve the problem at hand.

BeniBela

  • Hero Member
  • *****
  • Posts: 702
    • homepage
Re: Reusing objects and multi threading
« Reply #6 on: October 18, 2016, 02:47:51 pm »
The standard memory manager in FPC is a piece of shit known to be optimizing for average single core computing.
Since the olden days you can also write your own memory manager or use cmem (which performs better in real life scenario's)
I never understood what they were drinking/smoking  when they wrote the memory manager... At least it is pluggable.

To answer your question: at least on Linux cmem is threadsafe afaik.

Unfortunately cmem was slower in this case

But reusing also avoids the VMT initialization, the RecordRTTI  cleanup call, ...


Yes, but the question as I understand it is how to cache objects in a multi-threaded environment. In order to create such environment the threads MUST be created anyhow and that is why I'm wondering why anyone would not just encapsulate the functionality of the cached objects into the threads in order to spare the complexity of the caching.

It is about caching all these objects. Every object usage in every function.

The functions do not know if there are threads, and you do not want to add another parameter to every function and constructor to pass a cache structure.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7862
Re: Reusing objects and multi threading
« Reply #7 on: October 18, 2016, 03:07:47 pm »
Thread creation is expensive, in many real world scenarios worker threads are pooled too

Yes, but the question as I understand it is how to cache objects in a multi-threaded environment. In order to create such environment the threads MUST be created anyhow and that is why I'm wondering why anyone would not just encapsulate the functionality of the cached objects into the threads in order to spare the complexity of the caching.

If you have a variable number of object per thread (e..g depending on query), this consumes more objects then to pool them centrally.
 
I routinely pool all big objects. (several MBs and more) in many of my longrunning apps.

Fungus

  • Sr. Member
  • ****
  • Posts: 352
Re: Reusing objects and multi threading
« Reply #8 on: October 18, 2016, 03:11:39 pm »
It is about caching all these objects. Every object usage in every function.

The functions do not know if there are threads, and you do not want to add another parameter to every function and constructor to pass a cache structure.

I cannot see how you would be able to do this without having an object cache which is protected by a critical section. This would (probably) be a performance drawback if impacted simultaneously by multiple threads.

Fungus

  • Sr. Member
  • ****
  • Posts: 352
Re: Reusing objects and multi threading
« Reply #9 on: October 18, 2016, 03:20:54 pm »
If you have a variable number of object per thread (e..g depending on query), this consumes more objects then to pool them centrally.

That would depend on the scenario. ThreadA could handle all ObjectA, ThreadB could handle all ObjectB and so on. This would not consume more objects than a shared cache/pool.
 
I routinely pool all big objects. (several MBs and more) in many of my longrunning apps.

I bet you only do this from one thread or you protect the big objects with a critical section :)