Recent

Author Topic: FreePascal/Lazarus should position itself as a viable container language  (Read 1399 times)

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Been awhile since I made a new post, been rather busy as of late.  Anyways, I took a moment to see how easy and convenient it is to run both an FPC compile and execution into a Docker container.  I am very happy to say, that a simple console program was able to effortlessly compile inside a Docker container, and I was able to generate a new container using alpine as the base to create what is an extremely small image of just 5.7MB!  This means that FreePascal programs seem to work fine with the limited library support of a base alpine install.  Currently, one of the more popular Docker container languages for compiled code is GoLang, if one wants to generate tiny, efficient, and super scalable containers in a production environment.  I will provide below a working Dockerfile which will generate a container which will allow one to compile FreePascal programs using the latest compiler, and another Dockerfile to display how easy it is to containerize a simple FreePascal program.  It would be really awesome to see native Docker image creation right from the Lazarus IDE.

Code: Text  [Select]
  1. FROM debian:stretch
  2.  
  3. RUN apt-get update && apt-get install -y build-essential
  4.  
  5. ADD fpc-3.0.4.x86_64-linux /tmp/fpc-install
  6.  
  7. RUN cd /tmp/fpc-install && sh install.sh && cd / && rm -rf /tmp/fpc-install
  8.  

Build the above using
Code: Text  [Select]
  1. docker build -t fpc:3.0.4 .

To use this Dockerfile, I did need to modify the install.sh to support an unintended installation.  I also removed doc-pdf and demos, as that isn't needed during a build pipeline.  I am planning on expanding on this to add Lazarus building support using lazbuild.  I also use Debian Stretch as the building environment due to it's extensive library and overall development support.

Here's a Dockerfile to show how this can be used to build a docker image with a compiled final product bound for a production server in the cloud.

Code: Text  [Select]
  1. FROM fpc:3.0.4 AS builder
  2.  
  3. ADD hello.pas /root
  4.  
  5. RUN cd /root && fpc hello.pas
  6.  
  7. FROM alpine:latest
  8.  
  9. COPY --from=builder /root/hello /root/hello
  10.  
  11. ENTRYPOINT ["/root/hello"]
  12.  

I am planning on pushing up the fpc:3.0.4 docker image, and later on, a docker image capable of building Lazarus programs.

Docker is also a nice alternative to cross-compiling on the same arch.  I am planning on also releasing docker images such as this which will allow one to effortlessly cross-compile to a Raspberry Pi using the same method shown above.

https://ownyourbits.com/2018/06/27/running-and-building-arm-docker-containers-in-x86/

julkas

  • Hero Member
  • *****
  • Posts: 520
  • KISS principle / Lazarus 2.0.6 / FPC 3.0.4
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #1 on: December 13, 2019, 07:20:05 pm »
@kveroneau +1.
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
  mov rax, a
  mov rdx, b
  mul rdx
  mov [clo], rax
  mov [chi], rdx
end;

mr-highball

  • Full Member
  • ***
  • Posts: 103
-Highball

lucamar

  • Hero Member
  • *****
  • Posts: 2264
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #3 on: December 13, 2019, 08:01:12 pm »
[...] I did need to modify the install.sh to support an unintended installation.

Funny typo! Guess you meant "unattended", huh?  :D

Sorry but it made me ROFL. :-[

(And thanks; I very much needed that laugh today)
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.4/2.0.6  - FPC 3.0.4 on:
(K|L)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #4 on: December 13, 2019, 09:21:21 pm »
Definitely seeing some interest in this here.  As a result, this weekend I will publish into Docker Hub several Docker images, a FPC 3.0.4 for both amd64 and arm(Raspberry Pi), the latter will be a Docker image which will run on your powerful AMD64 development workstation, but compile Raspberry Pi compatible ARM binary images.  I will also setup a GitHub with how these build images are put together, along with an example project and Dockerfile to demonstrate it.  I will post the appropriate links here when they are ready.

For those curious on how to make an ARM FPC builder image, here's a Dockerfile you can use:
Code: Text  [Select]
  1. FROM arm32v7/debian:stretch
  2.  
  3. COPY qemu-arm-static /usr/bin/
  4.  
  5. RUN apt-get update && apt-get install -y build-essential
  6.  
  7. ADD fpc-3.0.4.arm-linux /tmp/fpc-install
  8.  
  9. RUN cd /tmp/fpc-install && sh install.sh && cd / && rm -rf /tmp/fpc-install
  10.  

This will run on your AMD64 Linux workstation.  Example run:

Code: Text  [Select]
  1. $ docker run -it --rm -v `pwd`:/mnt fpc:3.0.4-arm
  2. root@8d4481a9354e:/# cd mnt/
  3. root@8d4481a9354e:/mnt# fpc hello.pas
  4. Free Pascal Compiler version 3.0.4 [2017/10/11] for arm
  5. Copyright (c) 1993-2017 by Florian Klaempfl and others
  6. Target OS: Linux for ARMHF
  7. Compiling hello.pas
  8. Assembling hello
  9. Linking hello
  10. /usr/bin/ld: warning: link.res contains output sections; did you forget -T?
  11. 5 lines compiled, 0.6 sec
  12. root@8d4481a9354e:/mnt# ./hello
  13. Hello World from a container!
  14. root@8d4481a9354e:/mnt# exit
  15. kveroneau@dockerdev:~/fpc/test$ file hello
  16. hello: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.0.0, stripped
  17.  

Enjoy!
« Last Edit: December 13, 2019, 11:10:02 pm by kveroneau »

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #5 on: December 14, 2019, 02:15:40 am »
Creating a new post instead of modifying my previous one as I did earlier as I want everybody watching the thread to receive this awesome update!

So, instead of creating these docker images this weekend, I created them this evening!  So surprise everyone!  I will soon create a new GitHub repo and place the code to generate these builder images there.  You can use these images as explained above.

https://hub.docker.com/r/kveroneau/fpc/tags

I have pushed both an amd64 and a arm32v7.  I also took the additional time to scp the arm32v7 image to my Raspberry Pi and ran it there successfully!

This can definitely be a replacement for cross-compiling to a Raspberry Pi, and I know I will be using this for that exact purpose.  A Lazarus compatible image will be available soon.  If you have any Docker related questions, feel free to ask me.  My job title is officially a Development Operations Specialist, so I specialize in build and deployment type of tasks.  Docker can be easily integrated into a CI pipeline, so if you do need to target the Raspberry Pi, in your CI build tool, you can perform a build using the following command line(keeping a nice clean room build):

Code: [Select]
docker run --rm -v `pwd`:/mnt kveroneau/fpc:3.0.4-arm32v7 fpc /mnt/myapp.pas
Although I'd recommend adding additional customizations to the image by creating a new Dockerfile to inherit the one I built, and add any additional libraries required.  For all intensive purposes this is a very minimal build environment.

Look forward to additional FPC/Lazarus docker developments from me.

PascalDragon

  • Hero Member
  • *****
  • Posts: 894
  • Compiler Developer
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #6 on: December 14, 2019, 11:00:49 am »
It would be really awesome to see native Docker image creation right from the Lazarus IDE.
Well, the IDE is extendable. As you seem to be quite interested in this topic, why don't you try to create an extension for the IDE that allows to do this from within the IDE? You can find some information here. :)

guest64953

  • Guest
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #7 on: December 14, 2019, 05:59:29 pm »
Additional information: Eclipse IDE already support running code inside docker container.

guest64953

  • Guest
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #8 on: December 16, 2019, 05:14:31 am »
@kveroneau: What about Buster version, man? Stretch is oldstable now  :)

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #9 on: December 18, 2019, 06:58:18 pm »
@kveroneau: What about Buster version, man? Stretch is oldstable now  :)
My idea was to support the lowest common denominator, thus if you need to build a binary image using these Docker images, they should work in Stretch, and anything higher.  Not everyone has upgraded to Buster, all of my Raspberry Pis and Linux machines I run still use Stretch for the time being.

If you need a specific library version in Buster, you can generate a compatible Dockerfile pretty easily by swapping out FROM debian:stretch with FROM debian:buster in the Dockerfile, and build it.

If there is more demand for a premade Buster build image, I will create one alongside the stretch one and push it into Docker Hub as well.  I am currently more focused on creating a working Lazarus building Docker image, to allow for easy porting of Lazarus built programs to an ARM-based device such as a Raspberry Pi and others.

Does anyone know if .so libraries generated using the binutils on a Raspberry Pi can be used on Android with either no or some modifications?  It'd be nice to have a Docker image for FPC/Lazarus which can actually build Android APKs without needing to download and install the entire Android SDK and NDK onto a host machine.

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #10 on: December 19, 2019, 03:56:44 am »
Managed to get a Lazarus docker image up and running before the weekend.

You can find both an amd64 and an arm32v7 image here in Docker Hub: https://hub.docker.com/r/kveroneau/lazarus

I downloaded and detarred the Lazarus installer from the website, and here is the Dockerfile to create it:

Code: Text  [Select]
  1. FROM kveroneau/fpc:3.0.4-amd64
  2.  
  3. ADD lazarus /usr/src/lazarus
  4.  
  5. RUN apt-get update && apt-get install -y libgdk-pixbuf2.0-dev libx11-dev libgtk2.0-dev libglib2.0-dev libpango1.0-dev libcairo2-dev libatk1.0-dev  && cd /usr/src/lazarus && make bigide
  6.  
  7. ENV PATH /usr/src/lazarus:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  8.  
  9. RUN mkdir /root/.lazarus
  10.  
  11. COPY environmentoptions.xml /root/.lazarus/
  12.  

It can be tested by running something like this

Code: Text  [Select]
  1. $ docker run -it --rm -v `pwd`:/mnt kveroneau/lazarus:2.0.6-amd64
  2. root@ed782d61779c:/# cd mnt/tests/
  3. root@ed782d61779c:/mnt/tests# ls
  4. testproj.lpi  testproj.lpr  testproj.lps
  5. root@ed782d61779c:/mnt/tests# lazbuild testproj.lpr
  6. Hint: (lazarus) [RunTool] "/usr/bin/fpc" "-iWTOTP"
  7. Hint: (lazarus) [RunTool] "/usr/bin/fpc" "-va" "compilertest.pas"
  8. TProject.DoLoadStateFile Statefile not found: /mnt/tests/lib/x86_64-linux/testproj.compiled
  9. Info: (lazarus) Execute Title="Compile Project, Target: testproj"
  10. Info: (lazarus) Working Directory="/mnt/tests/"
  11. Info: (lazarus) Executable="/usr/bin/fpc"
  12. Info: (lazarus) Param[0]="-B"
  13. Info: (lazarus) Param[1]="-MObjFPC"
  14. Info: (lazarus) Param[2]="-Scghi"
  15. Info: (lazarus) Param[3]="-Cg"
  16. Info: (lazarus) Param[4]="-O1"
  17. Info: (lazarus) Param[5]="-g"
  18. Info: (lazarus) Param[6]="-gl"
  19. Info: (lazarus) Param[7]="-l"
  20. Info: (lazarus) Param[8]="-vewnhibq"
  21. Info: (lazarus) Param[9]="-Fi/mnt/tests/lib/x86_64-linux"
  22. Info: (lazarus) Param[10]="-Fu/mnt/tests/"
  23. Info: (lazarus) Param[11]="-FU/mnt/tests/lib/x86_64-linux/"
  24. Info: (lazarus) Param[12]="-FE/mnt/tests/"
  25. Info: (lazarus) Param[13]="-o/mnt/tests/testproj"
  26. Info: (lazarus) Param[14]="testproj.lpr"
  27. Hint: (11030) Start of reading config file /etc/fpc.cfg
  28. Hint: (11031) End of reading config file /etc/fpc.cfg
  29. Free Pascal Compiler version 3.0.4 [2017/10/03] for x86_64
  30. Copyright (c) 1993-2017 by Florian Klaempfl and others
  31. /usr/bin/ld: warning: /mnt/tests/link.res contains output sections; did you forget -T?
  32. (1002) Target OS: Linux for x86-64
  33. (3104) Compiling testproj.lpr
  34. (9015) Linking /mnt/tests/testproj
  35. (1008) 77 lines compiled, 0.4 sec
  36. (1022) 2 hint(s) issued
  37.  

This base Lazarus image can be used to extend to various Lazarus packages, such as weblaz, brooks, among others.  I will be creating a weblaz builder image soon, which will allow the easy building of Weblaz programs to either AMD64 or the Raspberry Pi via ARM32v7.

As of this posting, I am still in the process of building the AM32v7 image, but by the time you see this post, it should be up and ready to pull.

I should also note that these docker images are still pretty experimental, and I have only tested them with console-based applications.  I will be doing more extensive tests in the future and updating the Docker images appropriately when I find bugs and issues with either compilation or runtime.  If you find any issues, please post a reply into this thread and I will do my best to get it resolved.  Please let me know of any libraries you might be using so that I can properly assist.

I have prepared a GitHub repository with all the Dockerfiles and modifications to install.sh I made to get this all working.

https://github.com/kveroneau/fpc-docker
« Last Edit: December 19, 2019, 04:34:46 am by kveroneau »

PascalDragon

  • Hero Member
  • *****
  • Posts: 894
  • Compiler Developer
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #11 on: December 19, 2019, 09:27:23 am »
Does anyone know if .so libraries generated using the binutils on a Raspberry Pi can be used on Android with either no or some modifications?  It'd be nice to have a Docker image for FPC/Lazarus which can actually build Android APKs without needing to download and install the entire Android SDK and NDK onto a host machine.
Can't say for C libraries, but for FPC no, they are not. By default on Linux the RTL directly calls the kernel whenever possible, while on Android the C library is used. Also the library initialization behaves a bit differently. That's why FPC has a separate Android target after all.

kveroneau

  • Jr. Member
  • **
  • Posts: 83
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #12 on: December 19, 2019, 04:18:27 pm »
Does anyone know if .so libraries generated using the binutils on a Raspberry Pi can be used on Android with either no or some modifications?  It'd be nice to have a Docker image for FPC/Lazarus which can actually build Android APKs without needing to download and install the entire Android SDK and NDK onto a host machine.
Can't say for C libraries, but for FPC no, they are not. By default on Linux the RTL directly calls the kernel whenever possible, while on Android the C library is used. Also the library initialization behaves a bit differently. That's why FPC has a separate Android target after all.
Hmm, a compiled FPC binary I made using my lazarus:2.0.6-arm32v7 ran with zero issues inside Termux(A Linux console environment for Android).  I'm not sure why Lazarus built a statically compiled binary, but I think that helped.  I was honestly surprised it ran, but this proves that static binaries of console programs can run on Android through Termux.  My Android is a Samsung Note 9 using Linux kernel 4.9.112.  Although this compatibility thing could be a result of a Samsung's efforts to run a native Linux GUI desktop through their DeX, as Linux on DeX is using stock Ubuntu DEB packages from the official Ubuntu repo.  So, this may not work on all Androids depending on the Kernel API.  I'm sure FPC binaries, even GUI programs would run through Linux on DeX, but alas Samsung is dropping support fort Linux on DeX, which may mean that the next Android Linux Kernel they ship might be locked back down to require using the C libraries again.

Termux has a FreePascal compiler available in it's repository, so you can compile most FPC console programs directly on your phone using Termux.  However, these compiled binaries generated are actually LSB shared objects, when I check the binary using the standard file command, so they are being loaded by Android and/or Termux differently than a standard ELF binary.  This may actually limit what the FPC program can do when copied over.  All I tested was a simple WriteLn in a default TCustomApplication class.  Networking, for example might not work potentially through Termux if the Android C libraries are not called.

https://play.google.com/store/apps/details?id=com.termux&hl=en

https://www.linuxondex.com/

PascalDragon

  • Hero Member
  • *****
  • Posts: 894
  • Compiler Developer
Re: FreePascal/Lazarus should position itself as a viable container language
« Reply #13 on: December 20, 2019, 09:21:42 am »
Does anyone know if .so libraries generated using the binutils on a Raspberry Pi can be used on Android with either no or some modifications?  It'd be nice to have a Docker image for FPC/Lazarus which can actually build Android APKs without needing to download and install the entire Android SDK and NDK onto a host machine.
Can't say for C libraries, but for FPC no, they are not. By default on Linux the RTL directly calls the kernel whenever possible, while on Android the C library is used. Also the library initialization behaves a bit differently. That's why FPC has a separate Android target after all.
Hmm, a compiled FPC binary I made using my lazarus:2.0.6-arm32v7 ran with zero issues inside Termux(A Linux console environment for Android).  I'm not sure why Lazarus built a statically compiled binary, but I think that helped.  I was honestly surprised it ran, but this proves that static binaries of console programs can run on Android through Termux.  My Android is a Samsung Note 9 using Linux kernel 4.9.112.  Although this compatibility thing could be a result of a Samsung's efforts to run a native Linux GUI desktop through their DeX, as Linux on DeX is using stock Ubuntu DEB packages from the official Ubuntu repo.  So, this may not work on all Androids depending on the Kernel API.  I'm sure FPC binaries, even GUI programs would run through Linux on DeX, but alas Samsung is dropping support fort Linux on DeX, which may mean that the next Android Linux Kernel they ship might be locked back down to require using the C libraries again.

It can work (and in most cases does work) however Google does not guarantee that the kernel API stays stable (unlike what the Linux developers care for), thus why we link against the C library with target Android unlike the static linking with target Linux.

Additionally there are subtle differences between a normal Linux distribution and Android, e.g. the used widestring manager, retrieving the home directory (plus Writeln can be redirected to the Android log). Just look for the ANDROID define in the RTL to see the differences.