Lazarus

Announcements => Third party => Topic started by: Gustavo 'Gus' Carreno on April 12, 2021, 08:22:49 pm

Title: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Gustavo 'Gus' Carreno on April 12, 2021, 08:22:49 pm
Hi there,

First of all, full disclosure, I'm far from being a JavaScript or TypeScript developer.
Second of all, because I'm such a developer, I had to develop this by "feeling in the dark".

And it really shows in the amount of tini-tyni commits I've done, while "feeling in the dark".

Ok, whit that said, I want to present setup-lazarus (https://github.com/gcarreno/setup-lazarus) a GitHub Action, listed in the GitHub Marketplace.

Main features are:

I've also made a repository where I test the Action in a Lazarus project: lazarus-with-github-actions (https://github.com/gcarreno/lazarus-with-github-actions)

I have a love/hate relationship with being in the spotlight, love because I'm human and have an ego, hate because I hate having my head explode due to ego.

So I would be remiss if I didn't mention that @salvadorbs didn't like my implementation, because it lacked cross-compilation and did another solution.

He decided to use @DonAlfredo's(Reinero?) fpcup / fpcuplaz combination.
He decided to code it all up in a command line kinda way.

Yes. My workflow is in https://github.com/salvadorbs/Asuite/blob/develop/.github/workflows/main.yml. But beware, the workflow is not entirely my doing. I have the idea (and many parts of the workflow) from @Olly (and its workflow file https://github.com/ollydev/Simba/blob/simba1400/.github/workflows/build.yml ): https://forum.lazarus.freepascal.org/index.php/topic,34645.msg389924.html#msg389924

If I ever finish the main 2 issues of setup-lazarus(dependency on OPM and mac support), I'll maybe try and put this idea in TypeScript form.

Cheers,
Gus

PS: @trev Dunno if this belongs in Third Party. If not, could you kindly move my sorry ass? MTIA!!
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Warfley on April 14, 2021, 04:49:50 pm
A year ago I built something similar for gitlab, the core of it is this python script: Link (https://github.com/Warfley/LazarusPackageManager), it already solves the dependency issue and the multipackage packages (sounds kinda weird). Also has some neat functions like automatically detecting which packages a project requires, so maybe give it a look.

I don't know much about how github actions work, but for gitlab I used Linux docker images, which contain the Lazarus installation (Link (https://github.com/Warfley/LPMDocker)) including a cross compiler for Windows, build using fpclazup.
This has the advantage of the docker images being prebuilt, so the lazarus and the cross compiler installation don't take up runtime during the CI/CD. If you are more interested in how this works, at least on gitlab, you can checkout my gitlab example: Link (https://gitlab.com/Warfley/lazarusgitlabci), but I have no idea how this would look on github.

I thought about porting this apporach to github actions, but tbh I mostly use gitlab anyway and github actions are to complicated for me xD. But it's nice to see that others also work on devops stuff for Lazarus.

Edit: just seen that I never commited the dependency resolution in my python script, so what I claimed earlier isn't really true. Maybe I can find it on my old laptop, but seems like this commit is lost, but maybe I will update this script some time, lately I've been using this again and I found a few things I wanted to add anyway.
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: El Salvador on April 14, 2021, 07:33:07 pm
Hi Gus!  :D

So I would be remiss if I didn't mention that @salvadorbs1 didn't like my implementation, because it lacked cross-compilation and did another solution.
Your solution is valid, but it is not suitable for my project. That's all. For example, your solution lacks cross-compilation for win64 (if I remember correctly, but I have a bad memory  :-[ ).

Quote
He decided to only use the Ubuntu environment.
Nope. I use both windows and ubuntu. In particular one Windows 64bit and three Ubuntu (so I can build easily for GTK2, GTK3 and QT).

Quote
He decided to use @DonAlfredo's(Reinero?) old fpcup.
Yep, but it is not old. @DonAlfredo keeps the codebase aligned with the counterpart FpcUpDeluxe. Why fpcup (with docker option)? Well, for these reasons:
- I can easily install Lazarus (and FPC) patches in my lazarus build. Yes, I need it to restore a feature in windows that was removed in Lazarus 2.2.
- Since OPM doesn't support install packages of a particular version, I decided to directly put third party packages as git submodules into my project's repository. So, I add these packages using lazbuild.exe (with param --skip-dependencies)

Slow lazarus build isn't a big deal because I can cache the entire Lazarus directory using github actions/cache (no cache 13minutes, cache only 3minutes).

Quote
He decided to code it all up in a command line kinda way.
Yes. I don't have time to learn how to develop Github Actions (and typescript).  :D

Quote
@SalvadorBS: If it's you, could you please gimme a link to your workflow.yaml file, so I can link it here? Thanks!!
Yes. My workflow is in https://github.com/salvadorbs/Asuite/blob/develop/.github/workflows/main.yml. But beware, the workflow is not entirely my doing. I have the idea (and many parts of the workflow) from @Olly (and its workflow file https://github.com/ollydev/Simba/blob/simba1400/.github/workflows/build.yml ): https://forum.lazarus.freepascal.org/index.php/topic,34645.msg389924.html#msg389924
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Gustavo 'Gus' Carreno on April 15, 2021, 07:36:30 am
Hey Warfley,

A year ago I built something similar for gitlab, the core of it is this python script: Link (https://github.com/Warfley/LazarusPackageManager), it already solves the dependency issue and the multipackage packages (sounds kinda weird). Also has some neat functions like automatically detecting which packages a project requires, so maybe give it a look.

I'll have a look into that repo.
I have to be perfectly honest and say that it's mainly due to laziness that I haven't done the dependency and multipackage packages(it does sound funny) and nothing else.
So maybe looking at someone else's code may flicker a spark, hopefully :)

I don't know much about how github actions work, but for gitlab I used Linux docker images, which contain the Lazarus installation (Link (https://github.com/Warfley/LPMDocker)) including a cross compiler for Windows, build using fpclazup.
This has the advantage of the docker images being prebuilt, so the lazarus and the cross compiler installation don't take up runtime during the CI/CD. If you are more interested in how this works, at least on gitlab, you can checkout my gitlab example: Link (https://gitlab.com/Warfley/lazarusgitlabci), but I have no idea how this would look on github.

Under GitHub I'm not sure of the virtualization software they use.
What you're given access is a Windows/Ubuntu/macOS machine with some stuff already installed.
Mainly linux tools to aid in installing other software. And even that I'm not sure since I didn't dig into the docs for Actions.
I looked at some simple TypeScript code and then hacked my way to the current incarnation.

I thought about porting this apporach to github actions, but tbh I mostly use gitlab anyway and github actions are to complicated for me xD. But it's nice to see that others also work on devops stuff for Lazarus.

GitHub actions are not complicated at all, it's just a YAML file to configure the triggers and list other actions you can use, like the necessary action to clone the project.
But you don't have to use any fancy stuff,  cuz on that same YAML you can just use plain old bash scripting, or call python, etc, from that YAML.

Edit: just seen that I never commited the dependency resolution in my python script, so what I claimed earlier isn't really true. Maybe I can find it on my old laptop, but seems like this commit is lost, but maybe I will update this script some time, lately I've been using this again and I found a few things I wanted to add anyway.

You naughty, naughty boy!!! ;) Giving me hope in continuing to be a lazy bastard, and then you yank the rug ritght  under me feet, AHHH THE DESPAIR ;)

I would really thank you if you could find even the smallest of implementations!!!

Cheers,
Gus
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Gustavo 'Gus' Carreno on April 15, 2021, 07:57:01 am
Hey Salvador,

Your solution is valid, but it is not suitable for my project. That's all. For example, your solution lacks cross-compilation for win64 (if I remember correctly, but I have a bad memory  :-[ ).

That is no longer the case, and it's because of you that I found out that the Windows machines are actually 64b and not 32b as I, wrongly, assumed.
At the moment the Windows machines are downloading the 64b version of Lazarus+FPC and using that to compile.

What I don't have is the ability to specify any cross-compilation.
Which isn't needed if you're only interested in the CI(Continuous Integration) aspect.
If you want to cross-compile, package(zip, tar.gz) and then upload to a release, well, that will need some thought as it's more in the realm of CD(Continuous Distribution)
I guess I can mess around a bit with the cross packages provided at sourceforge(?) and get it done, but that will have to wait until I get dependency and multipackage packages(still sounds funny) done and also after making it macOS enabled.

Nope. I use both windows and ubuntu. In particular one Windows 64bit and three Ubuntu (so I can build easily for GTK2, GTK3 and QT).

Ooopps, sorry, mea culpa. I still think my memory is any good and don't confirm my facts...

Yep, but it is not old. @DonAlfredo keeps the codebase aligned with the counterpart FpcUpDeluxe. Why fpcup (with docker option)? Well, for these reasons:

Yeap, my fault again. I agree that the word OLD was misused.
What I shoulda said was: Came before fpcupdeluxe. That's more accurate than old.

Slow lazarus build isn't a big deal because I can cache the entire Lazarus directory using github actions/cache (no cache 13minutes, cache only 3minutes).

Humm, that's a nice thing to have in mind :)

Yes. I don't have time to learn how to develop Github Actions (and typescript).  :D

Yeah, can relate to this one. It was a rather steep learning curve to get the Action to it's current incarnation :)

Quote
@SalvadorBS: If it's you, could you please gimme a link to your workflow.yaml file, so I can link it here? Thanks!!
Yes. My workflow is in https://github.com/salvadorbs/Asuite/blob/develop/.github/workflows/main.yml. But beware, the workflow is not entirely my doing. I have the idea (and many parts of the workflow) from @Olly (and its workflow file https://github.com/ollydev/Simba/blob/simba1400/.github/workflows/build.yml ): https://forum.lazarus.freepascal.org/index.php/topic,34645.msg389924.html#msg389924

Thanks Salvador, I'll update my post with this and all the corrections you posted.

Cheers,
Gus
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: olly on April 15, 2021, 05:54:03 pm
Your action is great. It's very simple and Lazarus can be installed in one yml line. If you just want a quick compile test on windows using the latest Lazarus release it's perfect.

Though if you want a lot more customizable action, fplazup is the way to go.

Thanks to DonAlfredo for bug fixing, and adding docker mode.
Thanks to FPC/Lazarus developers fixing random LazBuild crashes.

I have yet to have FPC & Laz installation fail in a long time thanks to above!

Now, we wait on some features from Github Actions team (option to store cache even if job fails, option to import and run yml from an external repo)

Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Gustavo 'Gus' Carreno on April 27, 2021, 05:25:08 am
Hi all,

Due to the herculean effort from Levi (https://github.com/leviable) the Action now supports macOS.
He must be commended because he spent a whole day chasing macOS-isms, but in the end he was triumphant!! :D

Unfortunately with some limitations:

And because of that you'll have to have some conditionals on your .github/workflows/workflow.yaml file:
Code: YAML  [Select][+][-]
  1.     - name: Build the Main Application
  2.       if: ${{ matrix.operating-system != 'macos-latest' }}
  3.       run: lazbuild -B "src/lazaruswithgithubactions.lpi"
  4.     - name: Build the Main Application (macOS)
  5.       if: ${{ matrix.operating-system == 'macos-latest' }}
  6.       run: lazbuild -B --ws=cocoa "src/lazaruswithgithubactions.lpi"
  7.  

This would be more elegant with a else: but I'm not sure that exists, but will have to check :)

Cheers,
Gus
Title: Re: GitHub Action to setup Lazarus+FPC and/or OPM Packages
Post by: Gustavo 'Gus' Carreno on January 06, 2022, 10:38:57 pm
Hey Y'all,

After chasing my tail for more time that I care to admit, and about 13 releases I'm proud to announce that setup-lazarus is now able to install Lazarus 2.2.0 / FPC 3.2.2 on the GitHub runners running windows-latest, ubuntu-latest and macos-latest.


But, yeah, after much feeling in the dark to get it done, I've solved all the issues and it's now working as you can attest from the project that uses it to demonstrate how it's done: https://github.com/gcarreno/lazarus-with-github-actions/actions/runs/1664946821

Cheers,
Gus
TinyPortal © 2005-2018