Recent

Author Topic: Call to libc.so methods.  (Read 2215 times)

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Call to libc.so methods.
« Reply #15 on: August 12, 2022, 03:46:58 pm »
OK, I understand that, at the moment, fpc still needs libc.so for some rtl units.
dynlibs, cthreads, clocale being the main ones.


Ha, perfect ( and I have already a version of dynlibs, cthreads, clocale full no-link-to-clib.so working  ;) )

So that means you have implemented a thread manager based on the Clone syscall and related ones for critical sections and events that supports multiple platforms? Or how exactly did you duplicate the functionality provided by cthreads?

You know, surely, that a fpc application compiled on last Ubuntu or Debian distro does not run on last ArchLinux and Fedora distros.
But dynamically loading won't solve that. Linux simply doesn't try to have cross distro (and cross version) compatibility of binaries.

Hum, I did try a demo app compiled using external libc and the same with loadlibrary(libc.so) and the one using "external" refuse to run in other distros and the one with loadlibrary() runs perfectly on all distros.  But of course for a personal application that you want to use on same system that you used to compile it it is not a problem.
( You know my obsession about OOTB (Out-Of-The-Box)  :-[)

Only because it works in your specific test case that does not mean that it will work for a general case.

Fred vS

  • Hero Member
  • *****
  • Posts: 3158
    • StrumPract is the musicians best friend
Re: Call to libc.so methods.
« Reply #16 on: August 12, 2022, 04:07:47 pm »
But what about dynamically load libc.so with dynlbs ( dynlibs.pas himself needs a little change to not link to libc.so, I try it and did not find any problem) ?
DynLibs (or more precisely unit dl) requires the C library, because it is very likely that the user is loading a library that requires the C library as well and thus the application needs to correctly initialize the C library from the start, not to mention that libdl does depend on libc anyway. This is also mentioned in unit dl:
Code: Pascal  [Select][+][-]
  1. {$if defined(linux)}
  2.     { if libc is not linked explicitly, FPC might chose the wrong startup code, as
  3.       libdl depends on libc on linux, this does not hurt }
  4.     {$linklib c}
  5. {$endif}

Hello Sven.
Of course I did see it and did try to remove  {$linklib c} and in place did a intlibc dynamically ( so no more link to libc ) and it works perfectly.
If you want I can give a demo.

You know, surely, that a fpc application compiled on last Ubuntu or Debian distro does not run on last ArchLinux and Fedora distros.

This because Ubuntu or Debian distros use libc.so.6 version 2.31 and ArchLinux and Fedora use libc.so.6 version 2.34. (and vice-versa)
And if you link libc.so, the exact same sub version is needed, it is not the case with dynlibs.
You need to compile on the older system then it will also run on the newer one. This is also the case for C applications and by design.

Sorry Sven, I apologize a lot, but it is not right, you may try, compile a fpc app that link to libc.so on last Ubuntu, it will fail to run on last ArchLinux and Fedora because sub-version are not identical.

May I ask what must be done if a new unit is added in the rtl and if that unit is used by a other unit of the rtl?
I did try, only adding the unit and recompile fpc, the compiler compiles ok but when compiling the new unit needed by the other rtl unit, the compilation fail with this:

What exactly did you do? Your description can mean anything...

It is for the new fpc-dynload-libc.so game.
I did begin with the first unit that link to libc.so = /fpcsrc/3.2.2/rtl/linux/si_c.pp
In si_c.inc, removed all use of "external"

Code: Pascal  [Select][+][-]
  1. {var // all removed
  2.   libc_environ: pchar; external name '__environ';
  3.   libc_fpu_control: word; external name '__fpu_control';
  4.   libc_init_proc: procedure; external name '_init';
  5.   libc_fini_proc: procedure; external name '_fini';
  6. procedure libc_atexit; external name '__libc_atexit';
  7. procedure libc_exit; external name '__libc_exit';
  8. procedure libc_init; external name '__libc_init';
  9. procedure libc_setfpucw; external name '__setfpucw';
  10. procedure libc_start_main; external name '__libc_start_main';
  11. }
  12. var
  13.   fpc_ret,fpc_ret_rbp : pointer;

And in unit si_c added: mselibc:

Code: Pascal  [Select][+][-]
  1. unit si_c;
  2.  
  3. interface
  4.  
  5. uses
  6. mselibc;

That mselibc.pas has the removed procedures declared in si_c.inc.
And dont have any unit in "uses" section.

Now what to do?

I have copy mselibc.pas in /fpcsrc/3.2.2/rtl/linux/ because otherwise si_c does not find it.
But after this there was a error because mselibc.pas did not find objpas unit, so I copy that unit into /fpcsrc/3.2.2/rtl/linux/.
At the end all units where found and it compiles ok till that internal-error:

I know perfectly that the way to add units is absolutely not the right way, so, concretely, what must I do for that mselibc.pas is accepted in rtl?
« Last Edit: August 12, 2022, 04:17:27 pm by Fred vS »
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Call to libc.so methods.
« Reply #17 on: August 12, 2022, 04:41:29 pm »
Of course I did see it and did try to remove  {$linklib c} and in place did a intlibc dynamically ( so no more link to libc ) and it works perfectly.

Again, only because your specific example works, does not mean that this will work in general. It depends highly on what other libraries are loaded and what they do (e.g. if you load a library that accesses something in the C library that's only initialized when the initialization of the C library is executed which is only the case when FPC uses the startup file for libc you might get a crash at best or unexplainable behaviour at worst).

You know, surely, that a fpc application compiled on last Ubuntu or Debian distro does not run on last ArchLinux and Fedora distros.

This because Ubuntu or Debian distros use libc.so.6 version 2.31 and ArchLinux and Fedora use libc.so.6 version 2.34. (and vice-versa)
And if you link libc.so, the exact same sub version is needed, it is not the case with dynlibs.
You need to compile on the older system then it will also run on the newer one. This is also the case for C applications and by design.

Sorry Sven, I apologize a lot, but it is not right, you may try, compile a fpc app that link to libc.so on last Ubuntu, it will fail to run on last ArchLinux and Fedora because sub-version are not identical.

And you think I didn't try that before answering? I compiled an example application that links against the C library on Ubuntu with C library 2.31 and can successfully run it on an Arch with 2.33 as well as an Arch with 2.35. Same with a C program.

I know perfectly that the way to add units is absolutely not the right way, so, concretely, what must I do for that mselibc.pas is accepted in rtl?

You need to set up rules for your mselibc unit in the rtl/inc/linux/Makefile.fpc (use e.g. unixcp as a base) and then regenerate the rtl/inc/linux/Makefile using fpcmake -Tall inside the rtl/inc/linux directory. Then you should be able to build the RTL again.

Though speaking of the dependency to ObjPas: you shouldn't use $mode objfpc or $mode delphi inside your mselibc as that will result in the conclusion of the ObjPas unit and thus any application that does not need Object Pascal will pull in the Object Pascal code nevertheless.

Fred vS

  • Hero Member
  • *****
  • Posts: 3158
    • StrumPract is the musicians best friend
Re: Call to libc.so methods.
« Reply #18 on: August 12, 2022, 04:57:12 pm »
Sorry Sven, I apologize a lot, but it is not right, you may try, compile a fpc app that link to libc.so on last Ubuntu, it will fail to run on last ArchLinux and Fedora because sub-version are not identical.

And you think I didn't try that before answering? I compiled an example application that links against the C library on Ubuntu with C library 2.31 and can successfully run it on an Arch with 2.33 as well as an Arch with 2.35. Same with a C program.

Mama mia, I got that info from a other forum, I trusted it without trying my self.
I feel very bad and so sad that I may never trust in anything before try it by my self.

And the way that I affirmed that you was wrong, lamentable.  :'(

OK, all that topic has no reason anymore, too much trick to do and the easy solution is to use a older libc.so for compilation.

Sorry Sven, really sorry.for all the noise again.

I know perfectly that the way to add units is absolutely not the right way, so, concretely, what must I do for that mselibc.pas is accepted in rtl?

You need to set up rules for your mselibc unit in the rtl/inc/linux/Makefile.fpc (use e.g. unixcp as a base) and then regenerate the rtl/inc/linux/Makefile using fpcmake -Tall inside the rtl/inc/linux directory. Then you should be able to build the RTL again.

Though speaking of the dependency to ObjPas: you shouldn't use $mode objfpc or $mode delphi inside your mselibc as that will result in the conclusion of the ObjPas unit and thus any application that does not need Object Pascal will pull in the Object Pascal code nevertheless.

OK thanks (but I loose all fun).
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Call to libc.so methods.
« Reply #19 on: August 12, 2022, 05:10:38 pm »
Sorry Sven, I apologize a lot, but it is not right, you may try, compile a fpc app that link to libc.so on last Ubuntu, it will fail to run on last ArchLinux and Fedora because sub-version are not identical.

And you think I didn't try that before answering? I compiled an example application that links against the C library on Ubuntu with C library 2.31 and can successfully run it on an Arch with 2.33 as well as an Arch with 2.35. Same with a C program.

Mama mia, I got that info from a other forum, I trusted it without trying my self.
I feel very bad and so sad that I may never trust in anything before try it by my self.

To be fair there was quite some noise regarding this topic so it's a bit hard to keep track of what works and what does not. But yes, testing oneself is an important part with such topics ;)

And the way that I affirmed that you was wrong, lamentable.  :'(

Thankfully I'm not easily offended. :P

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Call to libc.so methods.
« Reply #20 on: August 12, 2022, 05:46:26 pm »
To be fair there was quite some noise regarding this topic so it's a bit hard to keep track of what works and what does not. But yes, testing oneself is an important part with such topics ;)

I've seen it happen on an ALSA library interface that explicitly branded itself with the library version during compilation... but I think we have to consider that type of self-harm to be an aberration.

Apart from that libgtkX and/or the Qt interface can be a problem, but the timespan is probably rather longer than most people here would consider relevant: there's only so many versions of non-current library that can be installed before hitting some detail relating to the kernel version, X11, udev or (I suspect) systemd.

And all of this affects other large bodies of software, e.g. Firefox, at least as badly as it affects FPC/Lazarus.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Fred vS

  • Hero Member
  • *****
  • Posts: 3158
    • StrumPract is the musicians best friend
Re: Call to libc.so methods.
« Reply #21 on: August 12, 2022, 07:40:09 pm »
Though speaking of the dependency to ObjPas: you shouldn't use $mode objfpc or $mode delphi inside your mselibc as that will result in the conclusion of the ObjPas unit and thus any application that does not need Object Pascal will pull in the Object Pascal code nevertheless.

Just for the (last) fun and to finalize all the (hard) work done ( I try it only for si_c.pp then I stop ).

So mselibc.pas should not use {$mode objFPC}  (aaargh, it uses it at the moment).
Does it exist a easy trick to convert code using  {$mode objFPC} into {$mode FPC}.
For example result is not allowed for {$mode FPC}, what must it use?

 :)
[EDIT] Ok, I get it, it was simple, a typical {$mode FPC} function is like this:

Code: Pascal  [Select][+][-]
  1. {$mode FPC}
  2.  
  3. function myfpcmode() : integer;
  4. begin
  5. myfpcmode := 5 ; // in place of reurn
  6. end

All is converted, only compilation fails with array of const

Code: Pascal  [Select][+][-]
  1. function sscanf(__s:Pchar; __format:Pchar; args:array of const):longint;cdecl;external clib name 'sscanf';

I replaced with array of char to make it compile but what should I use?
« Last Edit: August 12, 2022, 09:18:42 pm by Fred vS »
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

Fred vS

  • Hero Member
  • *****
  • Posts: 3158
    • StrumPract is the musicians best friend
Re: Call to libc.so methods.
« Reply #22 on: August 12, 2022, 09:57:36 pm »
You need to set up rules for your mselibc unit in the rtl/inc/linux/Makefile.fpc (use e.g. unixcp as a base) and then regenerate the rtl/inc/linux/Makefile using fpcmake -Tall inside the rtl/inc/linux directory. Then you should be able to build the RTL again.

Though speaking of the dependency to ObjPas: you shouldn't use $mode objfpc or $mode delphi inside your mselibc as that will result in the conclusion of the ObjPas unit and thus any application that does not need Object Pascal will pull in the Object Pascal code nevertheless.

Like always, perfectly explained, with this in makefile.fpc:

Code: Pascal  [Select][+][-]
  1. mselibc$(PPUEXT) : $(UNIXINC)/mselibc.pas $(SYSTEMUNIT) $(PPUEXT) baseunix$(PPUEXT) dynlibs$(PPUEXT)
  2.         $(COMPILER) $(UNIXINC)/mselibc.pas

fpc compiles + works using mselibc.pas for ci_c.pp (and works also using mse_dynlibc define to dynamically load libc.so vs external-link).

Nice combat ( I agree totally useless ).

In attachment the mselibc.pas in FPC-mode very complete and tested with choice to use "external + link clib" or "dynamic-load" way.

[EDIT] Attachment was updated.

Many thanks for your patience.

Fre;D
« Last Edit: August 13, 2022, 08:17:20 pm by Fred vS »
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Call to libc.so methods.
« Reply #23 on: August 13, 2022, 06:47:21 pm »
So mselibc.pas should not use {$mode objFPC}  (aaargh, it uses it at the moment).
Does it exist a easy trick to convert code using  {$mode objFPC} into {$mode FPC}.
For example result is not allowed for {$mode FPC}, what must it use?

Well, you could also enable the Result independently by using $modeswitch result. ;)

All is converted, only compilation fails with array of const

Code: Pascal  [Select][+][-]
  1. function sscanf(__s:Pchar; __format:Pchar; args:array of const):longint;cdecl;external clib name 'sscanf';

I replaced with array of char to make it compile but what should I use?

The correct solution for non-ObjPas modes is the following:

Code: Pascal  [Select][+][-]
  1. function sscanf(__s:Pchar; __format:Pchar):longint;cdecl;varargs;external clib name 'sscanf';

You then need to call the function like in C however instead of using square brackets:

Code: Pascal  [Select][+][-]
  1. sscanf('1234', '%d', @intvar);

Fred vS

  • Hero Member
  • *****
  • Posts: 3158
    • StrumPract is the musicians best friend
Re: Call to libc.so methods.
« Reply #24 on: August 13, 2022, 08:13:34 pm »
...
For example result is not allowed for {$mode FPC}, what must it use?

Well, you could also enable the Result independently by using $modeswitch result. ;)
Ha, nice to know this (but a few late  ;) )

All is converted, only compilation fails with array of const
Code: Pascal  [Select][+][-]
  1. function sscanf(__s:Pchar; __format:Pchar; args:array of const):longint;cdecl;external clib name 'sscanf';
I replaced with array of char to make it compile but what should I use?

The correct solution for non-ObjPas modes is the following:
Code: Pascal  [Select][+][-]
  1. function sscanf(__s:Pchar; __format:Pchar):longint;cdecl;varargs;external clib name 'sscanf';

Perfect, updated mselibc.pas in attachment with all the "array of const" fixed.

Thanks Sven!

Fre;D
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

 

TinyPortal © 2005-2018