Passing object instance between linux processes

[Чебурашка]

I make these assumptions:

1. I have two executables both compiled with same version of fpc.
2. The two executables are ran in the same PC.
3. OS is linux (machine architecture is not important because of 2)

I have one object (instance of class) in the first executable. If I want to pass it to the second executable one approach is to serialize the object into a representation that can be transferred to the second executable, and then have the second executable to make the reverse process transforming the serialized representation back to a real object usable by second executable.

Since normally when I work within the same process, passing an object to a method or to another thread just means supplying the object variable (a pointer), and this is very fast, I would like to see If I can invent something similar for passing between processes. Of course this is normally impossible because each process has it's own memory space so that an object created in first process is not accessible to the second.

But I was thinking: if I ask OS to create a memory space shared between the two processes with shmem (I already did this records sharing very easily between two processes), and I am able to have my object to live inside that shared memory, then if second process received the pointer it actually receives the location where the real object lives, so it could access to the object completely...

Of course here I am absolutely not interested in managing the synchronization of access by the two process or the object lifecycle management, this seems quite trivialto me. But please let me know if instead also this task if difficult to implement.

To me the only very difficul part is to instruct the memory manager to create the objects so that they live completely inside the shmem area, I have really no idea how to do.

Is this all idea a silly one? Or can be senseful if one whats to share objects quickly between certain process that have the compatibility described at the beginning?

Thanks.

[marcov]

In theory it can work, but it is usually too cumbersome to be useful.

E.g. if the object containers other allocations (strings, dynamic arrays) you must take very special case that those are also in the shared memory space.

Occasionally you see it for an app exporting e.g. statistics or the last measured curve via shared mem, and another app operating on them.

All my experience with shared memory was on Windows though.

I've also seen application keeping their most crucial state machine information in the shared mem exported by another application.  It is a kind of pattern where one app (that has exported shared memory) watches over a different application. The monitored application often has some component (COM or hardware related) that is external and fragile.  If it stops responding, the guardian terminates and restarts it, and on restart it reinitializes to the last known state from shared mem.

[Чебурашка]

E.g. if the object containers other allocations (strings, dynamic arrays) you must take very special case that those are also in the shared memory space.

My idea was that such a mechanism could operate like this:

1. Process 1 starts and creates the shared memory SH
2. Like is now, Process 1 has a the default memory manager so that if one ask a memory allocation, the standard process memory is used
3. Process 1 has also another memory manager associated to SH
4. In some way it should be possible to use for one allocation the second mem manager, and this means that all the allocations triggered by the master allocation reside in SH
5. All further reallocations also have to live in SH (mmm this seems kinda complex)
6. SH is accessed by Process 2
7. Also Process 2 has an extra mem manager associated to SH
8. Pointer obtained from 4 is given to Process 2
9. Again also any reallocation made by Process 2 via the extra mem manager must guaranteed that this shared object entirely lives in SH

I don't really know if this is somehow possible...

[marcov]

Multiple memory managers have been  explored, but the problem is that said types are automatically allocated, so you can't select the mem manager.

Even if you can hack something together, usability would suffer really bad.

You can of course pre allocate some static string buffers, on the purely procedural pointer to char level. if it is only a few objects whose structure doesn't mutate (much), that is doable, and pretty much what happened in the examples that I gave.

[MarkMLl]

I've considered it in the context of a record (specifically, not a TP-style class or a Delphi-style object) etc. in shared memory, possibly using e.g. SIGUSR2 to synchronise changes (although a token via e.g. a unix-domain socket would probably be better). However I've never got round to actually coding it.

I suspect that you'd do better serialising it via e.g. XML: not necessarily in terms of performance but in terms of overall manageability. At this point I think it needs to be booted back to Marco et al. who undoubtedly knows more than I do about what already exists in FPC and/or the LCL.

MarkMLl

[Чебурашка]

I suspect that you'd do better serialising it via e.g. XML: not necessarily in terms of performance but in terms of overall manageability.

I was thinking that the advantages of my idea would be at least that that I'd avoid the following:

• The serialization process which consumes cpu/and memory
• The need of having (and maintaining) the code to make serialization (this is ofter source of errors)
• The deserialization process which consumes cpu/and memory
• The need of having (and maintaining) the code to make deserialization(this is ofter source of errors)

Considering tha serialization/deserializaion just result into recreating a remote exact copy... avoiding it would be very nice.

Of course what I have in mind is only suitable in certain cases, especially where the is a container of information which is created by a source and has to be shipped to a destination (that lives in another memeory space). And also in might be passed back, in case destiantion transforms itself into source and viceversa.

The fact that marcov says that multiple mem managers have already been considered but they aren't available tells me that this matter is maybe too complex/risky, so prob better forget about it and live safe. 

[cdbc]

Hi
I use "libfpcmemman.so" & "sharemem", I took them from the Winders section of "fpcsrc", tweaked it to run on Linux and even made it dyn-loading...
I use this in my plugin-framework. If I can share a memman(works just fine), then wouldn't it be possible for you to share other stuff, process <-> process??
I mean, include sharemem in both apps as the first item in uses in lpr, just to test, if you can share objects/memman(in this case), between processes, if that works, the road is open for you to explore 
Just my "Nickles Worth" // pun intended
Regards Benny

[Чебурашка]

Hi
I use "libfpcmemman.so" & "sharemem", I took them from the Winders section of "fpcsrc", tweaked it to run on Linux and even made it dyn-loading...
I use this in my plugin-framework. If I can share a memman(works just fine), then wouldn't it be possible for you to share other stuff, process <-> process??
I mean, include sharemem in both apps as the first item in uses in lpr, just to test, if you can share objects/memman(in this case), between processes, if that works, the road is open for you to explore 
Just my "Nickles Worth" // pun intended
Regards Benny

Thank you vm, I will look into this, as the externalization of certain operations has become very useful into my projects, to manage very smoothly many issues related to possibily failing operations.