Forum > Third party
Lazarus Package Manager - CLI package manager for buildsystems
Warfley:
Hi,
as part of a project to build a fully automated building toolchain for lazarus projects, I needed a package manager for lazarus which is capable of loading OPM packages. So I've wrote one. And as some of you might also need something similar, here it is: https://github.com/Warfley/LazarusPackageManager.
What it does:
This little script (or set of scripts to be more accurate) allows for downloading and installing packages to any lazarus installation. It can search the OPM for packages, install packages via HTTP download, GIT or SVN.
It was build for being usable within docker, so there is also an example with a docker image for a project using DCP-Crypt
How to use:
Simple, assuming the lpm script is in the path:
--- Code: Bash [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---$> lpm update # synchronize OPM list$> lpm lazarus add 2.0.6 /developer/lazarus/2.0.6 # add lazarus installation so it knows where to install$> lpm install 2.0.6 "Synapse 40.1" # install synapse from OPM to lazarus$> lpm direct-download indy https://packages.lazarus-ide.org/Indy10.zip # manual download of a package (even though this is also in OPM)$> lpm install 2.0.6 indy # install just downloaded package$> lazbuild ... # now after installing the packages we can use lazbuild to simply build our projectsFor simplicity there is also the build command, which will automatically try to download any missing packages from OPM:
--- Code: Bash [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---$> lpm update # synchronize OPM list$> lpm lazarus add 2.0.6 /developer/lazarus/2.0.6 # add lazarus installation so it knows where to install$> lpm build 2.0.6 project.lpi # add --yes or -y to not get asked to install new packages
What it does not:
* Resolve dependencies (todo)
* Rebuild Lazarus (this is deploy only, not intended for development IDE's, use OPM instead)
* Remove packages (the main target are throwaway docker builds, so no need to remove these afterwards)
* Install local packages (i.e. packages located in lazarus/components) (is this required?)
Requirements:
* python3
* unzip (if you want to use the OPM or download zip archives)
* tar (if you want to download tar archives)
* gzip (if you want to download tar.gz archives)
* bzip2 (if you want to download tar.bz archives)
* git (if you want to download git repositories)
* svn (if you want to download svn repositories)
* lazarus (obviously)
* linux (I'm not going to test this on windows, I hate python under windows)
You might wonder why I wrote this in Python and not in Lazarus, thats because python has a really nice argument parser for CLI programs, and also this script is so small that I don't really need the structure of pascal :D
soerensen3:
That's good news! I will try that later!
for installing local packages there is already lazbuild. However it lacks the feature of removing packages as well. This would be handy sometimes if you installed a package and broke your lazarus this way, because the package makes it crash.
valdir.marcos:
--- Quote from: Warfley on April 03, 2020, 03:08:58 am ---Hi,
as part of a project to build a fully automated building toolchain for lazarus projects, I needed a package manager for lazarus which is capable of loading OPM packages. So I've wrote one. And as some of you might also need something similar, here it is: https://github.com/Warfley/LazarusPackageManager.
What it does:
This little script (or set of scripts to be more accurate) allows for downloading and installing packages to any lazarus installation. It can search the OPM for packages, install packages via HTTP download, GIT or SVN.
It was build for being usable within docker, so there is also an example with a docker image for a project using DCP-Crypt
How to use:
Simple, assuming the lpm script is in the path:
--- Code: Bash [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---$> lpm update # synchronize OPM list$> lpm lazarus add 2.0.6 /developer/lazarus/2.0.6 # add lazarus installation so it knows where to install$> lpm install 2.0.6 "Synapse 40.1" # install synapse from OPM to lazarus$> lpm direct-download indy https://packages.lazarus-ide.org/Indy10.zip # manual download of a package (even though this is also in OPM)$> lpm install 2.0.6 indy # install just downloaded package$> lazbuild ... # now after installing the packages we can use lazbuild to simply build our projects
What it does not:
* Resolve dependencies (todo)
* Rebuild Lazarus (this is deploy only, not intended for development IDE's, use OPM instead)
* Remove packages (the main target are throwaway docker builds, so no need to remove these afterwards)
* Install local packages (i.e. packages located in lazarus/components) (is this required?)
I currently tested this with Synapse and DCP-Crypt, two packages which are runtime only. While I think it should also work with design time packages, I don't know yet (I don't have any test projects), so if you want to try it, feedback would be appriceated
Requirements:
* python3
* unzip (if you want to use the OPM or download zip archives)
* tar (if you want to download tar archives)
* gzip (if you want to download tar.gz archives)
* bzip2 (if you want to download tar.bz archives)
* git (if you want to download git repositories)
* svn (if you want to download svn repositories)
* lazarus (obviously)
* linux (I'm not going to test this on windows, I hate python under windows)
You might wonder why I wrote this in Python and not in Lazarus, thats because python has a really nice argument parser for CLI programs, and also this script is so small that I don't really need the structure of pascal :D
--- End quote ---
@Warfley
That's good news!
First, I appreciate and thank for you effort.
Second, just out of curiosity.
Would it be harder to use traditional Bash shell script instead of Python since you were already on Linux?
https://en.wikipedia.org/wiki/Shell_script
https://en.wikipedia.org/wiki/Bash_(Unix_shell)
--- Quote from: soerensen3 on April 03, 2020, 11:09:58 am ---That's good news! I will try that later!
for installing local packages there is already lazbuild. However it lacks the feature of removing packages as well. This would be handy sometimes if you installed a package and broke your lazarus this way, because the package makes it crash.
--- End quote ---
Very well observed. Thanks.
Warfley:
--- Quote from: valdir.marcos on April 03, 2020, 02:41:04 pm ---Second, just out of curiosity.
Would it be harder to use traditional Bash shell script instead of Python since you were already on Linux?
https://en.wikipedia.org/wiki/Shell_script
https://en.wikipedia.org/wiki/Bash_(Unix_shell)
--- End quote ---
Yeah, it would be much harder. I'm actually quite a fan of bash scripts, but json parsing alone can give you a real headache, and these scripts actually only do 3 things: 1. parse json, 2. download files, 3. call lazbuild. While bash has a slight edge in ease for calling programs (and therefore downloading thanks to wget, curl or lynx), it is not much easier to weigh out the pain it would be to handle complex datastructures with it.
Python on the other hand supports OOP, which makes handling datastructures represented by json rather easy. Example to downlad a tar.gz and unzip it:
--- Code: Bash [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---# bashwget -O- url | tar -xz# pythonwith request.open(url) as req: p = Popen(["tar", "-xz"], stdin=req) p.communicate()Sure python is a little more complicated about this, but after all it's rather easy. Then when it comes to parsing json:
--- Code: Python [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---serialized = json.load(req)for key, value in serialized.items(): ...While in bash I would need to either write some really dreadful sed queries or use specified tools and first convert the json to a string array which i can than iterate, and here python wins hands down
--- Quote from: soerensen3 on April 03, 2020, 11:09:58 am ---That's good news! I will try that later!
for installing local packages there is already lazbuild. However it lacks the feature of removing packages as well. This would be handy sometimes if you installed a package and broke your lazarus this way, because the package makes it crash.
--- End quote ---
Yeah I use internally simply lazbuilds -add-package-link to register the packages (I currently don't know if this is enough for design time packages, but from my understanding lazarus should build them with the project as long as they are registered), and the fact that my tool can't remove packages also relies on the fact that lazbuild can't do this.
When I break my lazarus with new components I usually use make to rebuild it withouth the components (simple make all) and then can start it and uninstall these components before rebuilding it from the IDE. This is especially funny when I'm using it on my raspi, because I don't use a DE on it and only connect via SSH, so to install packages I can easiely use lazbuild but for uninstalling I need to use X11 forwarding.
I really like lazbuild and think it's great to have a CLI interface for building lazarus projects, but it's missing some functionality.
I am also toying with the Idea to make this thing fully OPM compatible (i.e. using OPM configurations and locations) and integrate it directly into Lazbuild, I was looking into it, and decided that for now this is to complicated for what I want to accomplish, but is on my todo list for the future. The goal is to simply do something like calling "lazbuild opm install Indy10" or so.
But as I said, my current goal is to create a toolchain for building lazarus projects within docker, and a python script is more than enough for now.
JuhaManninen:
--- Quote from: Warfley on April 03, 2020, 06:25:00 pm ---I am also toying with the Idea to make this thing fully OPM compatible (i.e. using OPM configurations and locations) and integrate it directly into Lazbuild, I was looking into it, and decided that for now this is to complicated for what I want to accomplish, but is on my todo list for the future. The goal is to simply do something like calling "lazbuild opm install Indy10" or so.
--- End quote ---
Actually it should integrate directly into the Lazarus package system. The goal is that the different package sources could integrate seamlessly together. Meaning that a missing dependency of a local user installed package could be resolved by an online package, or vice versa.
Please see unit PackageLinkIntf in package BuildIntf (used to be in IdeIntf).
The package system knows 3 types of packages: Global, Online and User.
Global means a package in Lazarus sources. They are found based on .lpl files.
Online comes through OPM.
User installed packages can be anywhere in file system and their location is remembered by the IDE and LazBuild.
Unfortunately the online packages don't yet integrate fully. The internal data structure does not have enough data for solving dependencies. OPM uses its own logic from outside but it is not part the internal package system.
If somebody wants to improve the situation, please do.
Another thing is that the package system should not depend on LCL. My goal is to remove LCL dependency from LazBuild but that goal is far far away ...
Splitting BuildIntf from IdeIntf was one step but it does not help by itself. It provides interface for units which still have the LCL dependency.
The problem is now solved with a NoGUI widgetset hack which prevents pulling in the actual GUI libs.
Navigation
[0] Message Index
[#] Next page