* * *

Author Topic: Dynamic Load Package  (Read 1055 times)

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Dynamic Load Package
« on: May 25, 2018, 06:58:58 pm »
Hi again all !


I found a clue about how BPL loaded and unloaded. I hope it will be usefull...
It seem applicable for any application built using Delphi 7 and by Delphi (IDE) itself.


The first loaded BPL by *.exe will always is "rtl70.bpl"
It might be loaded as famous LoadLibrary
The first called procedure will be '@GetPackageInfoTable' or '@Rtl@@GetPackageInfoTable$qqrv',
it returns a 'PackageInfo' type; which is a pointer of array of unit info.

So, for load & unload dynamic package in runtime, it seem as simple now:
by calling PackageLoad & PackageUnload inside an application, anytime.



AFAIK current fpc (3.0.4) is (almost?) ready to do same thing..
I have read the /fpc/compiler/pmodules.pas+parser.pas, we just never has a "package rtl; " yet to build our own fpc rtl dynamic package.
What do you think?
« Last Edit: May 27, 2018, 04:26:21 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

ASerge

  • Hero Member
  • *****
  • Posts: 936
Re: Dynamic Load Package
« Reply #1 on: May 25, 2018, 07:17:58 pm »
I found a clue about how BPL loaded and unloaded. I hope it will be usefull...
And why would you do that?

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Re: Dynamic Load Package
« Reply #2 on: May 25, 2018, 07:40:46 pm »
I found a clue about how BPL loaded and unloaded. I hope it will be usefull...
And why would you do that?
Various reasons. from just curiosity, for killing spare-time, and mainly because its fun.
Nah, anyway I have tried since a week ago to build something like *.BPL (delphi true dynamic load package) using fpc/Lazarus,
and today I still failed.
I have think Sven Bart helped me[1] about dynamic package in fpc level, I just felt missing something in between.
I read the wiki a lot, like this one[3]

So, I switch my mindset, it said: how Delphi actually load the BPL? [2]
Then, I run PEDUMP.exe[4] of a bunch of *.BPL to see the section of imports & exported functions.
and the online tool too. [5]
so here I am.  8)


[1] http://forum.lazarus-ide.org/index.php/topic,41248.msg286331.html#msg286331
[2] http://forum.lazarus-ide.org/index.php/topic,41248.msg286075.html#msg286075
[3] http://wiki.freepascal.org/Packaging_System_and_dividing_FPC_-_Lazarus_into_packages
[4] www.wheaty.net/pedump.zip
[5] http://pedump.me
« Last Edit: May 25, 2018, 08:12:37 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

PascalDragon

  • Full Member
  • ***
  • Posts: 198
  • Compiler Developer
Re: Dynamic Load Package
« Reply #3 on: May 26, 2018, 05:09:13 pm »
It is already clear what needs to be done. The problem merely is time and to rework the RTL to support this correctly.

Also I hope that the code you posted isn't copied from Delphi. If it is however I ask you to edit your post and remove it.

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Re: Dynamic Load Package
« Reply #4 on: May 27, 2018, 05:43:15 pm »
.. I hope that the code you posted isn't copied from Delphi. If it is however I ask you to edit your post and remove it.
It was from Delphi's system.pas. Oops!?  :-[  I apologize. Now I did remove that.


It is already clear what needs to be done. The problem merely is time and to rework the RTL to support this correctly.
By trying read the fpc's compiler source code, yes it is clear. That was seem as something like chicken and egg problem for me.
-->Chicken will be only available from egg, and egg require chicken to be existed.


==>RTL (and another) dynamic-loading-package will be only available when is supported by compiler, and compiler requires RTL to be existed in dynamically-loaded-package shape.
It maybe inaccurate compared to the actual fpc compilers logic, since there are thousands of code to be understood carefully, and I didn't found any documentation about it unless that (possibly) outdated one.


So, I depend on a feeling that Delphi has more elegant trick when managing type/class/RTTI than fpc plan:
Quote from: PascalDragon link=topic=41248.msg286445#msg286445
When a dynamic package is loaded using LoadPackages() or unloaded with UnloadPackage() there are various house keeping tasks that the RTL needs to do:
- initialize or uninitialize units as needed
- insert new types into the public type list for correct support in the Rtti unit
- insert new resource strings into the internal resource string list for correct support ...
I think, the better place to register class is not in the global type list, but in each package itself.
So each package has it own list (container) of registered class/type. soon that list is integrated/registered to global list of class used by FindClass().

This way, when the module is being unloaded, it will be easy to destroy that list because the member of list is limited containing inside the package.


Otherwise, when all registered class are mixed in one global list, it would be hard for RTL/RTTI/resource to remove classes from a package being unloaded: which class/type/resource to be removed is relevant/belong to the package being unloaded?


----------
Another magic (elegant trick) of dynamic-loading-package:
Even Delphi uses all BPL on almost Delphi running/lifetime as dynamic linking library, it is not a must for application generated.
It mean that any BPL can also be used by compiler to built into static linking application.


However, above are my stupid humble opinion that usually just one or two step understanding achievement of tens step done by fpc teams. 8-) [/size][/font]
« Last Edit: May 27, 2018, 05:50:41 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

PascalDragon

  • Full Member
  • ***
  • Posts: 198
  • Compiler Developer
Re: Dynamic Load Package
« Reply #5 on: May 28, 2018, 02:28:52 pm »
.. I hope that the code you posted isn't copied from Delphi. If it is however I ask you to edit your post and remove it.
It was from Delphi's system.pas. Oops!?  :-[  I apologize. Now I did remove that.

Thank you.

It is already clear what needs to be done. The problem merely is time and to rework the RTL to support this correctly.
By trying read the fpc's compiler source code, yes it is clear. That was seem as something like chicken and egg problem for me.
-->Chicken will be only available from egg, and egg require chicken to be existed.


==>RTL (and another) dynamic-loading-package will be only available when is supported by compiler, and compiler requires RTL to be existed in dynamically-loaded-package shape.
It maybe inaccurate compared to the actual fpc compilers logic, since there are thousands of code to be understood carefully, and I didn't found any documentation about it unless that (possibly) outdated one.

Compiler and RTL work hand in hand, that's nothing new. Though here the main point is that the compiler needs to generate the meta information that the RTL requires for package loading.

So, I depend on a feeling that Delphi has more elegant trick when managing type/class/RTTI than fpc plan:
Quote from: PascalDragon link=topic=41248.msg286445#msg286445
When a dynamic package is loaded using LoadPackages() or unloaded with UnloadPackage() there are various house keeping tasks that the RTL needs to do:
- initialize or uninitialize units as needed
- insert new types into the public type list for correct support in the Rtti unit
- insert new resource strings into the internal resource string list for correct support ...
I think, the better place to register class is not in the global type list, but in each package itself.
So each package has it own list (container) of registered class/type. soon that list is integrated/registered to global list of class used by FindClass().

What I had written above is the simplified version and how it looks like for the user. The RTTI and resource string information is per unit and each package contains the information of its contained and required units.

FindClass() has nothing to do with packages. FindClass() only handles classes explicitely registered with RegisterClasses().

Another magic (elegant trick) of dynamic-loading-package:
Even Delphi uses all BPL on almost Delphi running/lifetime as dynamic linking library, it is not a must for application generated.
It mean that any BPL can also be used by compiler to built into static linking application.

You are wrong. If Delphi uses dynamic packages for an application it uses the dcp-files at least for those that are part of any used package, for all other units or if the application is not using packages at all, but directly the dcu-files. FPC is doing the same.

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus