Install Package w/o restart Lazarus?

[x2nie]

Hi gurus,

...
I wonder if Lazarus could used each installed package as an external library basis....
but it can not, currently.

so my question is:
1. Why? what is the problem to make Lazarus/FreePascal able to import class/component from external library?

2. Is there any plan to achieve that?


================

Background:

By my limited knowledge of Lazarus, all component installed in Lazarus will be compiled inside Lazarus IDE (lazarus.exe).
I don't know why this design was chosen, .. is it because FreePascal Compiler/Linker's limitation or maybe it because of another difficulty on compiling/linking imported external library (*.bpl / *.lpl) into a bigger binary (*.exe) ...?
Whatever the reason, in my humble opinion : I believe that we can solve & do that someday in future.


Anyway,
I found a demo somewhere in internet, that show loading a form from external library.
the form is not only shown, but it shown embeded in other form.
I imagine that we can use above similar technique for Lazarus designer: a component loaded from a library (*.bpl /*.lpl) and then placed in Lazarus form designer by such code:

Code: [Select]

TControl(LoadedComponent).parent := CurrentDesignerForm;

Another benefit of loading/storing lazarus compiled package into an external library is
there will no need for restarting lazarus when a desing-time-package installed.
though, I am fine of restarting lazarus; my machine is fast enough for wait Lazarus restart.


Bytheway,
A big chance if Lazarus could load a component from external library is:
* new kind of Lazarus Designer.
I imagine that the form designer is in true GTK (for example) appearance including GTK themes;
or in true QT appearance including QT toolbar (which is beautiful)
dropped to the form designer (TForm1) from Lazarus component palette.
In other word, the designer will in true selected widgetset configuration while the IDE is may in another widgetset.


It was done in Delphi7 wich is ech form designer is not a part of the IDE,
but the form designer is loaded from external library: so D7 can switch between GDI / QT, per project basis.


---------------


Back to topic, I wonder if it was possible ? and What are requirement for make it possible?


Any explanation, disagree, idea, complaint, proof or critique are welcome.
(sorry again for my bad english)


x2nie

[Mike.Cornflake]

This weekend, there is a conference in the Netherlands where Dynamic Packages for Lazarus are being demonstrated for the first time.  Guessing it'll be shortly after that that we hear something here :-)

[howardpc]

I wonder if Lazarus could used each installed package as an external library basis....
but it can not, currently.

so my question is:
1. Why? what is the problem to make Lazarus/FreePascal able to import class/component from external library?

2. Is there any plan to achieve that?

Mike has addressed point 2.
Your first point is discussed at length by Marco here: http://wiki.freepascal.org/packages
"What is the problem" might naively assume there is a just one problem, which if only it were solved would make everything perfect.

As Marco's article makes clear, the introduction of dynamic packages to FPC/Lazarus (and it seems a beta of the component package plug-in concept is just around the corner) brings its own problems that having a single statically-linked Lazarus IDE executable largely avoids.

One is the vast increase in complexity and maintainability for component developers (not to speak of the possible impact on FPC and Lazarus developers themselves), since versionitis (already a problem years ago for Delphi when it was a single Win32 platform RAD tool) means developers will need to keep multiple versions of FPC/Lazarus (and of their component packages) if their pluggable components are to be backwards compatible. They will never work with a trunk version. And although FPC has a slowish release cycle, Lazarus releases come pretty thick and fast.

All the while Delphi was a single-platform IDE its proprietary .bpl format could be adapted to work with successive Delphi versions. But even the difference between 32-bit and 64-bit dlls on that one Windows platform proved a stumbling block to dynamic packaging - for a long time after FPC could target Win64, Delphi could not.

Both FPC and Lazarus are truly cross-platform. The Lazarus IDE on Mac OS X is a native application. To use 'pluggable' dll/so (both 32-bit and 64-bit) on the many platforms FPC/Lazarus supports would be close to a miracle. Apparently it can be done. Whether the consequent increase in complexity/maintainability and steeper learning curve for developers is worth it seems to me to be an open question.

[x2nie]

This weekend, there is a conference in the Netherlands where Dynamic Packages for Lazarus are being demonstrated for the first time.  Guessing it'll be shortly after that that we hear something here :-)

Wow !!!   that's would be very cool !
Who told you that?




Hi, anyway,
your reply inspiring me for a keyword while googling. using "Dynamic Packages for Lazarus".
And it is going into more deep articles:
http://wiki.freepascal.org/packages
http://wiki.freepascal.org/Lazarus_Packages
http://wiki.freepascal.org/Packaging_System_and_dividing_FPC_-_Lazarus_into_packages


Maybe the most amazing part of Lazarus Package design is in this paragraph:
http://wiki.freepascal.org/packages#Lazarus_and_binary_packages

Lazarus and binary packages
If we keep the following paragraph in mind, lets make some reasonable assumptions. To be conservative we limit us to two releases annually (e.g. a release and an update to it by Lazarus every year).

• Rule of thumb, the most recent versions and the versions right before that are the most used versions (because FPC users generally do update, but slowly due to e.g. Linux distributions that still come with older versions). Add another version for Debian/Ubuntu that seem to lag more heavily than the others, and which is too popular to be ignored.

• Say that of the 20 possible platforms, we support the top tier platforms only, and lets be rigorous and strip FreeBSD and not primary architectures (like Linux on PowerPC), then we keep Windows, Linux and OS X.

• However Windows and Linux these come in two architectures: 32 and 64-bit. Since the slightly "odd" platforms are a major FPC attraction, you can't discount them all. Limit yourself to Linux/32 and Win32 and you might as well use Delphi and Kylix.

• OS X even in five. (i386,x86_64 in the near future,powerpc,powerpc64 and ARM for IPhone)

• Windows also means WINCE. At least the ARM (pocketPC compatible) version. We marginalize wince versioning for now.

• So then we come to 10 platforms in 4 versions : for every release of a binary component 40 binary componentlibs need to be built. While you can try to clip this here and there a bit (e.g. stop OS X PPC somewhere soon), you occassionally also might have to support a bit more versions. FPC 2.4.4 is still occasionally seen in business use currently, and 2.6.0, 2.6.2 and 2.6.4 followed it. That's just major releases, not counting Lazarus respins and updates that might not be 100% compatible.And then there is of course a lot of "crack" users using 2.5.1/2.7.1 in 2500 revision/year, but you can assume they know to compile their own. OTOH if it is a binary only trial version, you can't.Note that this means that we also already skip the Tier two platform range that might still be interesting. FreeBSD (an occasional server platform) and a lot of embedded linux (e.g. on ARM). Also we don't include the gaming platforms (NDS, GBA, XBOX), assume only one OS X version per arch etc.So I think the number of 40 builds might still stand. Think also what this (40 binary builds) means for the support burden. Contrary to the 4-5 the avg Delphi component builder does. Some might be crosscompilable, but some (like the 5 OS X targets) will be harder. Moreover crosscompiling or VMs don't really absolve you from testing on the target.Yet another problem is the fragmented nature of Linux, which till now we considered whole. Do you really want to build a set of .rpms,.debs,.ipk's for each and every release? OS X and WinCE also have potential target platform versioning issues.I hope that I at least made you think twice about the practicalities of binary distribution, and why it isn't as simple as copying the Delphi "binary packages" model. "Dorothy, I think we are not in Kansas anymore".

Whooaa ~~...! 40 variant should be supported for each binary package.? If we add the widgetset number into this complexity, it maybe twice or three times of 40.?


No way! Let's then get out from that assumption, we got it:
 * It is okay  for compiling from the source code (not from binary package), or from intermediate code (*.ppu / *.obj). I mean we don't require the whole existence of 40 varian of each supported platform.
* Specially for designtime and form designer, we only need the current platform. For example, in Windows we only need windows-binary-packages.
* Even for each supported widgetset, we might only require (at least) one: the same widgetset as IDE compiled with.
* When user switch to other widgetset, Lazarus might reload the form designer of related binary-package per Form basis;
   if there available then lazarus load them, if they not exist (or found an obsolete binary-package) then Lazarus ask user for compiling the required binaries-packages.
* essentially, Lazarus would use binary-packages efficiently : compile only the needed, not compile the whole supported platforms.


What do you think?

[howardpc]

This weekend, there is a conference in the Netherlands where Dynamic Packages for Lazarus are being demonstrated for the first time.  Guessing it'll be shortly after that that we hear something here :-)

Wow !!!   that's would be very cool !
Who told you that?

See here: http://www.blaisepascal.eu/DucthPascon/Info_Laz_2014.php

[BigChimp]

What do you think?

Wait for the actual specifications of the binary package system and future plans

[Cyrax]

See this : http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/svenbarth/packages/?sortby=date&view=log

[marcov]

That remark about 40 packages is if you want to do it decently. Probably package vendors (that don't want source based evaluation versions, no matter what) simply will chose to support a minimum of versions and platforms, and dictate the platform and version to use.

So I think the commercial entities will simply put up versions for 32-bit Windows and the easiest to support OS X in the last stable when they released.  Linux would be in demand too, but then widgetset already complicates, as does the fact that people tend to use the 3rd party built FPC+Lazarus that comes with the distribution, instead of a well defined centrally released  (and thus ppu compatible) one.

I think for the average user, dynamic packages will only mean a slower starting lazarus, except for the occasional trial package. (for which they might have to maintain a second install)

I don't know if that "slowing" effect will be measurable,  so don't take that as that Lazarus will be as slow starting as Delphi, there might be other factors there too (.NET initialization OTA plugins etc)

[marcov]

What do you think?

Wait for the actual specifications of the binary package system and future plans

Sven is working on it, and it won't be merged till trunk till the next major branch (2.8 or 3.0 whatever it will be called) is branched off.

Apparently with this demonstration Sven has reached some milestone, but we haven't heard much on core about that either.

The same goes for the Extended RTTI (D2010+) stuff on the agenda of that Dutch meeting, that is in a branch that is post-3.0

[x2nie]

I think for the average user, dynamic packages will only mean a slower starting lazarus,
...
I don't know if that "slowing" effect will be measurable,  so don't take that as that Lazarus will be as slow starting as Delphi, there might be other factors there too (.NET initialization OTA plugins etc)

Maybe, but maybe the opposite too: Lazarus would start faster. Though I don't know the reason of it would be slower.


When lazarus package is compared as Photoshop Filter (effect) plugins, photoshop doesn't need to really load any plugin on starting.
Well, I don't really know how PS load its plugin; but I have the GraphicMagic Pro source code which is approximately has similar amount of Filter plugin.
How we can do lightning fast plugins scanning/loading?

• Don't actually load the plugins !
• Read the meta-plugin. Meta plugins is an TIniFile.
• for each plugin has TIniFile's "Section"
• each section consist of plugin name (for MenuItem.Caption), plugin path (for MenuItem.parent), etc
• the only need to check is : file-exist + file.date_modified. if the date_modified between the real file and meta plugin is different, then the real-plugin-loading is occur. If file not exist, then MenuItem.Enabled = False;
• in Photoshop, there is a "Scan filters" menu to update the whole metadata and deletes invalid plugin's meta-data.
• Registering a new plugin/package is usually done manual.
• Loading the real package/plugin is on demand: load when only needed.

If you would, replacing the TIniFile with binary-record-typed-file would increase the speed.
Other benefit of load plugin this way is : reduce RAM amount required by host application (which is Lazarus).
==================

I said (above): I don't know the reason why dynamic package would make Lazarus start slower.

[x2nie]

1. Why? what is the problem to make Lazarus/FreePascal able to import class/component from external library?
...

...
"What is the problem" might naively assume there is a just one problem, which if only it were solved would make everything perfect.

As Marco's article makes clear, the introduction of dynamic packages to FPC/Lazarus (and it seems a beta of the component package plug-in concept is just around the corner) brings its own problems that having a single statically-linked Lazarus IDE executable largely avoids.

One is the vast increase in complexity and maintainability for component developers (not to speak of the possible impact on FPC and Lazarus developers themselves), since versionitis (already a problem years ago for Delphi when it was a single Win32 platform RAD tool) means developers will need to keep multiple versions of FPC/Lazarus (and of their component packages) if their pluggable components are to be backwards compatible. They will never work with a trunk version. And although FPC has a slowish release cycle, Lazarus releases come pretty thick and fast.

Oh I got it!
You might want to say: one of the problem is the dynamic-packages should only be recognized by the same version of FPC only.
I never predict it in before. I didn't think about it that far.

Both FPC and Lazarus are truly cross-platform. The Lazarus IDE on Mac OS X is a native application. To use 'pluggable' dll/so (both 32-bit and 64-bit) on the many platforms FPC/Lazarus supports would be close to a miracle. Apparently it can be done. Whether the consequent increase in complexity/maintainability and steeper learning curve for developers is worth it seems to me to be an open question.

Hey, I can say: don't worry!
You might only worry if the dinamic-packages installed in lazarus were shipped without source-code; aren't you?
When the binary of dinamic-package is not recognized by Lazarus, Lazarus could notice user for recompiling them from source-code; no mater in which platform Lazarus run.


I can imagine the best Lazarus could have: not shipped with any dinamic package at all!
if so, how lazarus can work with dinamic-package? the answer: compile them on the very beginning (first running after lazarus installed).


The solution of "versionistis" of dinamic-package is: recompile (if required) from source code. It don't need your attention, it would be done automatically (like Delphi does as always).


I think, the third'party package that aren't shipped with source-code is only my 5 cent. It wouldn't affect for Lazarus distributed codes.


What do you think ?

[howardpc]

What do you think ?

Until people have an opportunity to test out whatever Sven, Joost and others have come up with (and will be demonstrating shortly in Leiden) it is hard to say. This is all just theoretical so far.

[x2nie]

What do you think ?

Until people have an opportunity to test out whatever Sven, Joost and others have come up with (and will be demonstrating shortly in Leiden) it is hard to say. This is all just theoretical so far.

  Well, I then have no more words. ~


I wish that event in Leiden a big success !!.  Congratulation FPC+Lazarus.

[marcov]

I don't know if that "slowing" effect will be measurable,  so don't take that as that Lazarus will be as slow starting as Delphi, there might be other factors there too (.NET initialization OTA plugins etc)

Maybe, but maybe the opposite too: Lazarus would start faster. Though I don't know the reason of it would be slower.

Packages have overhead, so no, it would definitely be slower. However I don't know the magnitude. It might not be noticable.

When lazarus package is compared as Photoshop Filter (effect) plugins, photoshop doesn't need to really load any plugin on starting.

Well, it does, doesn't it? Since it has to show it on the component bar, and they must be registered to open a form that uses them.  There are no metaplugins yet.

[x2nie]

What do you think ?

Until people have an opportunity to test out whatever Sven, Joost and others have come up with (and will be demonstrating shortly in Leiden) it is hard to say. This is all just theoretical so far.

Just curious, what's summary of Dynamic Packaging on Leiden?

See this : http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/svenbarth/packages/?sortby=date&view=log

There was seem no longer progress on that repository.