Recent

Author Topic: Packaging lazarus-based software for Linux distributions - 3rd party components  (Read 8184 times)

FangQ

  • Full Member
  • ***
  • Posts: 134
I am creating deb and rpm packages for a lazarus-based GUI program, with a hope that I will upload those later to debian/ubuntu/fedora for wide distribution.

the difficulty I am facing is that my GUI program uses a 3rd party component, GLScene, for 3d rendering. if it does not, I could have just added lazarus-ide as Build-Depends (in the case deb), and call lazbuild myapp.lpi to build it on the server side. But I don't know how to deal with GLScene. It is not one of the standard packages, and installing it requires recompiling lazarus-ide.

I bet many people here write Linux/cross-platform apps with lazarus, and perhaps some of you are releasing in linux repositories. is there any hope that I can package such application?

it would have been nice if lazarus can install components without recompiling (in the binary forms) - this way, people can just create packages for 3rd party components and management via package systems (apt/yum/dnf). is there such plan?

Chronos

  • Full Member
  • ***
  • Posts: 240
    • PascalClassLibrary
I have few own application packed to deb packages for Ubuntu and they don't need to recompile Lazarus IDE itself. You would need to recompile Lazarus only in case of design-time packages and if you want to use components in Lazarus IDE in design time. But usually if you build application which requires some packages using lazbuild from terminal you don't need to recompile Lazarus IDE. It just needs to know where packages are located. I usually place copy of source of packages with specific version to each application so it can be distributed and built easily even they have different versions of used packages.

Here you can find tips how to let Launchpad build your project automatically if its source changes:
http://wiki.freepascal.org/Publish_project_on_Launchpad

Practical example: http://svn.zdechov.net/AcronymDecoder/trunk/Install/deb/
« Last Edit: December 21, 2018, 09:24:02 pm by Chronos »

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
it would have been nice if lazarus can install components without recompiling (in the binary forms) - this way, people can just create packages for 3rd party components and management via package systems (apt/yum/dnf). is there such plan?

People dream about this all the time, but such scheme will be terribly limited. Contrary to windows, where nearly everybody uses the release built distribution, linux packages are often build with custom built FPC's and lazaruses.

However such packages only work with the exact same build, so you would still have to have a sdlscene.bpl for every possible Linux lazarus out there (and that are many). So while it is a theoretical possibility, I don't expect it to be too practical, not as practical as people expect from e.g. Delphi.

Linux simply isn't big on binary distribution. There is a reason why scripting languages are so popular on Linux, binary distribution is a mess
« Last Edit: September 04, 2018, 11:12:07 am by marcov »

Thaddy

  • Hero Member
  • *****
  • Posts: 14201
  • Probably until I exterminate Putin.
Not to mention the fact that you would need to create packages for every major Linux flavor because the package format and package manager differ between different families.

Now playing AC/DC: If you want blood you got it.
Specialize a type, not a var.

taazz

  • Hero Member
  • *****
  • Posts: 5368
it would have been nice if lazarus can install components without recompiling (in the binary forms) - this way, people can just create packages for 3rd party components and management via package systems (apt/yum/dnf). is there such plan?

People dream about this all the time, but such scheme will be terribly limited. Contrary to windows, where nearly everybody uses the release built distribution, linux packages are often build with custom built FPC's and lazaruses.

However such packages only work with the exact same build, so you would still have to have a sdlscene.bpl for every possible Linux lazarus out there (and that are many). So while it is a theoretical possibility, I don't expect it to be too practical, not as practical as people expect from e.g. Delphi.

Linux simply isn't big on binary distribution. There is a reason why scripting languages are so popular on Linux, binary distribution is a mess
with out recompile the lazarus IDE its ok to recompile the packages or if you are on the release just install with out recompile.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

sash

  • Sr. Member
  • ****
  • Posts: 366
the difficulty I am facing is that my GUI program uses a 3rd party component

So what is exact problem?
It seems like you're trying to distribute binary (already compiled) application. In that case the only dependencies you should care are system dynamic libraries which could be handled as package dependency in respective package manager format.

* For automated builds (lazbuild) you don't need package to be installed in IDE, it is enough for compiler to setup path for used units.
« Last Edit: September 04, 2018, 08:41:31 pm by sash »
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

FangQ

  • Full Member
  • ***
  • Posts: 134
I have few own application packed to deb packages for Ubuntu and they don't need to recompile Lazarus IDE itself. You would need to recompile Lazarus only in case of design-time packages and if you want to use components in Lazarus IDE in design time. But usually if you build application which requires some packages using lazbuild from terminal you don't need to recompile Lazarus IDE. It just needs to know where packages are located. I usually place copy of source of packages with specific version to each application so it can be distributed and built easily even they have different versions of used packages.

Here you can find tips how to let Launchpad build your project automatically if its source changes:
http://wiki.freepascal.org/Publish_project_on_Launchpad

Practical example: http://svn.zdechov.net/svn/AcronymDecoder/trunk/Install/deb/

thanks, I did not know I can compile the 3rd party component locally without needing to recompile the ide.

by the way, what command to use to build a component? lazbuild *.lpk? I will do some experiment on my side.

FangQ

  • Full Member
  • ***
  • Posts: 134
So what is exact problem?
It seems like you're trying to distribute binary (already compiled) application. In that case the only dependencies you should care are system dynamic libraries which could be handled as package dependency in respective package manager format.

* For automated builds (lazbuild) you don't need package to be installed in IDE, it is enough for compiler to setup path for used units.

no, I wasn't trying to package binary files - which won't work for Linux distributions due to the library versions. At least from my outdated (10 yr ago) packaging experience with Fedora, I need to upload the source rpm to the automated packaging system (it was bodhi at that time), request the server to spawn a virtual machine, in the virtual machine, download & install all needed Build-depend packages, and then compile the binaries from source package for a particular release, and generate a binary rpm. If everything works out, your binary package will be sent to all users of that release. You can specify multiple releases so the system will automatically create binary packages for you. Not sure if things still run the same way as it was before.

As you can see, the binary packages are completely created on the packaging server. that means I need to tell the packaging server everything needed to compile my binary from source. if it was vanilla lazarus app without 3rd party components, I know lazbuild will take care of it, and everything should work. I did not know I could also compile an 3rd party component as part of the source, and tell lazbuild (on the server) where to find it.

Right now, I think the first thing I need to figure out is how to compile glscene on-the-fly using lazbuild. the next thing is how to tell lazbuild to link my code with the compiled glscene libraries/object files.

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Right now, I think the first thing I need to figure out is how to compile glscene on-the-fly using lazbuild. the next thing is how to tell lazbuild to link my code with the compiled glscene libraries/object files.

I think the best example you can find of how to build a complex application with packages ... is Lazarus itself. Just open the source of lazbuild and study the Makefiles. Building your app. will probably be simpler than that:)
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

FangQ

  • Full Member
  • ***
  • Posts: 134
People dream about this all the time, but such scheme will be terribly limited. Contrary to windows, where nearly everybody uses the release built distribution, linux packages are often build with custom built FPC's and lazaruses.

However such packages only work with the exact same build, so you would still have to have a sdlscene.bpl for every possible Linux lazarus out there (and that are many). So while it is a theoretical possibility, I don't expect it to be too practical, not as practical as people expect from e.g. Delphi.

Linux simply isn't big on binary distribution. There is a reason why scripting languages are so popular on Linux, binary distribution is a mess

dynamically generating binary packages from source packages has never been an issue for most linux distributions (despite the large number flavors and versions). that's exactly why people invented systems like bodhi/koji/launchpad to do automated packaging. as long as the binary package was compiled on the same release (say Ubuntu 14.04 or 16.04) as the lazarus binary, I don't see why lazarus-ide can not accept a pre-compiled binary as a standard way to add new components.  The automated packaging server will ensure that lazarus and 3rd party components are compiled by the same set of libraries on the same release version; admitted that a single Linux distribution have multiple releases, but from a package maintainer perspective, all you need to do is to push your source package to different versions, and it will be compiled and released for you. so, the distribution and library version control, despite more complicated than windows, is transparent to the users.

from the above discussions, it looks like fpc and lazbuild can already handle 3rd party components in a modular and dynamic fashion; the inflexiblity of adding components by forcing users to recompile IDE is rather a limitation of the lazarus-ide design. What I was hoping to have was to make lazarus-ide to offer the same flexibility as lazbuild, by dynamically linking with pre-compiled 3rd party components.

a good example is octave. you can install octave GUI, you can also install other octave toolboxes, such as octave-image or octave-signal; these component packages are either in the source-form or precompiled mex files, but whatever form, they can be automatically recognized and invoked in the octave program. that's what hope to see for lazarus-ide.

taazz

  • Hero Member
  • *****
  • Posts: 5368
from the above discussions, it looks like fpc and lazbuild can already handle 3rd party components in a modular and dynamic fashion; the inflexiblity of adding components by forcing users to recompile IDE is rather a limitation of the lazarus-ide design.
true, then again only if you need the components in the ide.

What I was hoping to have was to make lazarus-ide to offer the same flexibility as lazbuild, by dynamically linking with pre-compiled 3rd party components.
a good example is octave. you can install octave GUI, you can also install other octave toolboxes, such as octave-image or octave-signal; these component packages are either in the source-form or precompiled mex files, but whatever form, they can be automatically recognized and invoked in the octave program. that's what hope to see for lazarus-ide.
You already have that its called Online Package manager, and the fun thing is I do not have to conform  to all that pesky linux rituals.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

sash

  • Sr. Member
  • ****
  • Posts: 366
no, I wasn't trying to package binary files - which won't work for Linux distributions due to the library versions.

First, and generally, it will not work, only if your build system has more recent libraries then those on distribution target. I'm building binaries on Debian which are working on OpenSuse and of course on Ubuntu, without any special efforts. You simply need to be sure your let' say libglxxx.so.xxx on build system is not newer than suggested target. In case target machine has outdated system, you could fix it with package dependencies (don't know about others, but it works with .deb/apt environment).

Right now, I think the first thing I need to figure out is how to compile glscene on-the-fly using lazbuild. the next thing is how to tell lazbuild to link my code with the compiled glscene libraries/object files.

You don't need `to compile glscene`. You (compiler) need some particular units (with respective dependencies) from there.

Lazbuild just needs a Lazarus project. Paths for sources are stored (by default) in relative format there.
All you have to do is to maintain the same (as on development workstation) directory structure on a build machine.
Not to mention you could use Build modes (Debug/Release/Whatever) and redefine paths there, individually for each mode.




Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
dynamically generating binary packages from source packages has never been an issue for most linux distributions (despite the large number flavors and versions). that's exactly why people invented systems like bodhi/koji/launchpad to do automated packaging.

Sigh. Yes, I know the theory. But let's examine what there is now: it seems the default lazarus package distributed by Debian/unbuntu can't build itself after 15 years of package work (20+ in the case of FPC on Debian).

Which is why most people use Lazarus generated debs, downloaded from the site.

This means by the most majorly used distros (Ubuntu,Debian) this will already horribly fail due to two different lazarus builds in roulation (per distro-version).

Does it then really make sense to dream of the next step, and implement gigantic major features like binary packages?

When do you expect the distros to have it working ? By 2100?

Anyway,

yes, binary packages are worked on (though probably not in the next major version, so this is years away), but I think the main goal is plug-in architectures in your own applications.

For lazarus IDE, it is more mixed, it will of course speed up installing packages, but at the same time it will significantly delay IDE startup (most of the splash screen time of Delphi is packages initialization, and their implementation is probably faster than ours).

I wouldn't be surprised if whole swaths of more advanced users will avoid a package based IDE because of that.

It also won't make anything easier, since both contrary to delphi, there will be version mismatches due to independent building. That remains the same for source and binary. (except that with source you get a hint what is wrong with an error in a line. With binary packages you get either a message that doesn't say much ('package incompatlbe, crc doesn't match") or a fat GPF.

« Last Edit: September 24, 2018, 11:05:32 am by marcov »

FangQ

  • Full Member
  • ***
  • Posts: 134
First, and generally, it will not work, only if your build system has more recent libraries then those on distribution target. I'm building binaries on Debian which are working on OpenSuse and of course on Ubuntu, without any special efforts. You simply need to be sure your let' say libglxxx.so.xxx on build system is not newer than suggested target. In case target machine has outdated system, you could fix it with package dependencies (don't know about others, but it works with .deb/apt environment).

You don't need `to compile glscene`. You (compiler) need some particular units (with respective dependencies) from there.

Lazbuild just needs a Lazarus project. Paths for sources are stored (by default) in relative format there.
All you have to do is to maintain the same (as on development workstation) directory structure on a build machine.
Not to mention you could use Build modes (Debug/Release/Whatever) and redefine paths there, individually for each mode.

thanks for the comment. I opened the .lpi file and from the section named "Units", I found a number of GLScene Units that was referred using its absolute path, and I copied those in a subfolder in my project's folder, and changed the lpi Units section and replaced the path with the relative local path

https://github.com/fangq/mcx/commit/9f18ccb2049d73ba1df0d4dd6e198271172474e1#diff-1eaacebea919f98b6105117358e60a44

I was able to compile my project on a system with a freshly installed lazarus 1.8.3, but I do not see any of the GLScene units (*.pas) were compiled in the command line output, lazbuild also emit the following warning:

Code: Pascal  [Select][+][-]
  1. .../mcxstudio.lpr(10,10) Hint: (5023) Unit "GLScene_RunTime" not used in mcxstudio

I did not copy the GLScene_RunTime.lpk file to my project.

after compilation, the binary can be launched, but when I click on the menu to show the GLscene window, it gave me an access violation error.  This does not happen in my lazarus-ide compiled binary.

any thing I missed?

to reproduce this issue, you may use the below commands:

Code: Pascal  [Select][+][-]
  1. git clone https://github.com/fangq/mcx.git
  2. cd mcx
  3. git checkout etherdome
  4. cd mcxstudio
  5. lazbuild --build-mode=release mcxstudio.lpi

then run the GUI at debug/mcxstudio. The access violation error happens when one select the toolbar Plot\Preview.

FangQ

  • Full Member
  • ***
  • Posts: 134
Sigh. Yes, I now the theory. But let's examine what there is now: it seems the default lazarus package distributed by Debian/unbuntu can't build itself after 15 years of package work (20+ in the case of FPC on Debian).

Which is why most people use Lazarus generated debs, downloaded from the site.

This means by the most majorly used distros (Ubuntu,Debian) this will already horribly fail due to two different lazarus builds in roulation (per distro-version).

Does it then really make sense to dream of the next step, and implement gigantic major features like binary packages?

When do you expect the distros to have it working ? By 2100?

Anyway,

yes, binary packages are worked on (though probably not in the next major version, so this is years away), but I think the main goal is plug-in architectures in your own applications.

For lazarus IDE, it is more mixed, it will of course speed up installing packages, but at the same time it will significantly delay IDE startup (most of the splash screen time of Delphi is packages initialization, and their implementation is probably faster than ours).

I wouldn't be surprised if whole swaths of more advanced users will avoid a package based IDE because of that.

It also won't make anything easier, since both contrary to delphi, there will be version mismatches due to independent building. That remains the same for source and binary. (except that with source you get a hint what is wrong with an error in a line. With binary packages you get either a message that doesn't say much ('package incompatlbe, crc doesn't match") or a fat GPF.

I do share your pain and making a single Linux binary running on all versions of Linux distros would be awesome.

For simple (C) programs, I was able to produce statically linked binaries that has better portability, but on some platforms, it remains failed with the incompatible GLIBC_xx flags or GOMP_xx flags, despite that I've statically linked these libraries. in those occasions, I felt extremely frustrated and even angry, like you felt.

I think the major issue is that the core system libraries (like glibc) does not have a standardized core, or has a modular design that aims for high portability. the developers often make least scrutinized decisions that often lead to breakage of backward compatibility. The pursing of latest features (like the c++17 standards) often outweigh the compatibility of existing and legacy features in their decision-making. If the gcc team were run by a company, breaking backward compatibility equals to losing existing customers, that's a penalty that is big enough for them to pay attention to it, but this does not seem to apply to the open-source developer teams.

 

TinyPortal © 2005-2018