Recent

Author Topic: FreeBSD still needs 14+ year old system calls  (Read 299 times)

trev

  • Full Member
  • ***
  • Posts: 194
FreeBSD still needs 14+ year old system calls
« on: October 10, 2019, 08:21:09 am »
I've been trying to track down how/where/when/why the Free Pascal compiler uses 14+ year old FreeBSD system calls which necessitates compiling the FreeBSD kernel with COMPAT_FreeBSD6 (which in turn requires COMPAT_FreeBSD7) kernel options.

https://wiki.freepascal.org/FreeBSD#COMPAT_requirement_of_the_port suggests that the requirement was dropped with version FP 2.2.4 and "and the value of .long line into cprt0.as file is changed to ${OSVERSION} automatically."

Looking at the source for /usr/local/share/fpc-source-3.0.4/rtl/freebsd/x86_64/*as files, reveals:

Code: [Select]
# FreeBSD  ELF startup code for Free Pascal for dynamical linking to libc.
#
# To avoid needing a "COMPAT" system, patch the constant following the
# "FreeBSD" field in the abitag to the relevant ABI number
# Some versions of the "file" command support
# visualizing this constant. If not use elfdump, or patch
# automatically using the ../i386/identpatch.sh script
#
# some typical values:
#
# freebsd 5.4 504000
# freebsd 6.3 (prerelease) : 603100
# freebsd 7.0 700055
# freebsd 8.0 800500  (-stable)
# FreeBSD 9.0 900044

abitag:
        .long   8
        .long   4
        .long   1
        .string "FreeBSD"
        .long   900044

900044 is FreeBSD 9.0-RELEASE which was end -of-life in March 2013.

Adding to the mystery, the .note section from the ppcx64 which calls compat6.mmap is:

Code: [Select]
entry: 3
        sh_name: .note.ABI-tag
        sh_type: SHT_PROGBITS
        sh_flags: SHF_ALLOC
        sh_addr: 0x6b4160
        sh_offset: 2834784
        sh_size: 24
        sh_link: 0
        sh_info: 0
        sh_addralign: 4
        sh_entsize: 0

I've reached the end of my limited understanding of the issue. Does anyone else have any further ideas on how to eliminate this dependency?
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7430
Re: FreeBSD still needs 14+ year old system calls
« Reply #1 on: October 10, 2019, 10:06:31 am »
I'm (was?) the FreeBSD maintainer, but last summer the last FreeBSD system was turned off, so I'm without production FreeBSD systems for the first time in 25+ years, and with that went most of my motivation. That said, not much new development happened after 2015 anyway, and even in 2010-2015 it already more maintenance than new work.

After 2015 I mostly kept what was there compiling/working with FreeBSD11 if you switched to gld after that. Some changes in FPC to the Linux port were not followed for the freebsd port (like the pascal startup code, initially because it seemed to need working with weak symbols). OpenBSD was recently updated to use this.  The releases all worked out of the box with stock kernels up to FreebSD11

Nowadays most non llvm versions are out of support or nearly out of support, and all my release building VMs are outdated, and the port faces some non-trivial work. Anyway some points:

1. The ELF abi tag was generated for 32-bit by identpatch.sh, as the url says. I had a toplevel script that automated things a bit.

However with the retooling of freebsd that broke, like pretty much everything else.  I think it never worked for 64-bit releases, but I just copied the 32-bit output to the 64-bit file.

Basically the talk of COMPAT_ in that link is just about the abitag not being updated, and having to brandelf binaries. It is not related to the MMAP thing.

2. FPC does not do anything special, and just calls syscall 197 to do mmap. The call is fpmmap in rtl/bsd/ossysc.inc. Maybe the linker or the ports system detects this and does something special. That said, some syscalls like mmap and ftruncate() are special due to their 64-bit offset use, and probably change more often than others. Sometimes workarounds for other (non FPC) problems get confused by  the fact that simple FPC programs don't link to libc.

3. The OS specific parts of a *nix FPC port are:
    - a custom startup code.  (prt0.as for non libc linking programs, cprt0.as for libc linking programs)
    - a large number of constants. syscall numbers and the usual *nix constants and some internals like ioctl values.
   -  likewise for structures.  stat,statfs and a few others.
   -  Sometimes the syscalls themselves are special (and not generic *nix or *bsd).
   -  termios and similar bits are usually OS specific too.
   -  how the linker is called.
   -  creating libgdb and linking it with the textmode IDE.  FPC 3.2+ won't do that anymore though, so that drops off the list.

Probably all need review, since 2010 only things that break have been looked at. Some very old notes (2003-2005 era, the newer date in the PDF were some minor changes) are here.

There was an option to compile FPC to use libc calls instead of syscalls (pass OPT="-dFPC_USE_LIBC"), but the exact state is unknown, it is not used much. (only solves going through the list of syscalls once every 5 years or so, the other points still must be done, and might even complicate)

Code: [Select]
// rtl/freebsd/sysnr.inc

 syscall_nr_mmap                        = 197;

// rtl/bsd/ossysc.inc
Function Fpmmap(start:pointer;len:size_t;prot:cint;flags:cint;fd:cint;offst:off_t):pointer; [public, alias:  'FPC_SYSC_MMAP'];

begin
 {$ifdef CPU64}
  Fpmmap:=pointer(ptruint(do_syscall(TSysParam(syscall_nr_mmap),TSysParam(Start),TSysParam(Len),TSysParam(Prot),TSysParam(Flags),TSysParam(fd),0,TSysParam(offst))));
{$else}
 Fpmmap:=pointer(ptruint(do_syscall(syscall_nr_mmap,TSysParam(Start),Len,Prot,Flags,fd,0,
         {$ifdef FPC_BIG_ENDIAN}    hi(offst),lo(offst){$endif}
         {$ifdef FPC_LITTLE_ENDIAN} lo(offst),hi(offst){$endif}
         )));
{$endif}
end;
« Last Edit: October 10, 2019, 10:09:29 am by marcov »

trev

  • Full Member
  • ***
  • Posts: 194
Re: FreeBSD still needs 14+ year old system calls
« Reply #2 on: October 11, 2019, 08:02:01 am »
Ahah!

https://github.com/freebsd/freebsd/blob/master/sys/kern/syscalls.master

Code: [Select]
197 AUE_MMAP COMPAT6 {
void *mmap(
    _In_ void *addr,
    size_t len,
    int prot,
    int flags,
    int fd,
    int pad,
    off_t pos
);
}

Instead, we need syscall 477:

Code: [Select]
477 AUE_MMAP STD {
void *mmap(
    _In_ void *addr,
    size_t len,
    int prot,
    int flags,
    int fd,
    off_t pos
);
}

Thanks for the detailed explanation.
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

trev

  • Full Member
  • ***
  • Posts: 194
Re: FreeBSD still needs 14+ year old system calls
« Reply #3 on: October 12, 2019, 03:40:07 am »
I thought the solution would be easy, but alas, it's not.

I updated the rtl/freebsd/sysnr.inc to replace the old system call (197) with the new system call (477) and tried recompiling. Well, the ppc1 bootstrap compiler looks good, no COMAT6 calls, but it then bombs out thus:

Code: [Select]
/bin/mkdir -p x86_64/units/x86_64-freebsd
/usr/local/share/fpc-source-3.0.4/compiler/ppc1 -Ur -Xs -O2 -n -Fux86_64 -Fusystems -Fu/usr/local/share/fpc-source-3.0.4/rtl/units/x86_64-freebsd -Fix86_64 -FE. -FUx86_64/units/x86_64-freebsd -Cg -dRELEASE     -dx86_64 -dGDB -dBROWSERLOG -Fux86 -Sew pp.pas
/bin/sh: %%LDPATH%%: not found
pp.pas(247,1) Error: Error while linking
pp.pas(247,1) Fatal: There were 1 errors compiling module, stopping
Fatal: Compilation aborted

No luck tracking down the %%LDPATH%% variable. Ideas?
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7430
Re: FreeBSD still needs 14+ year old system calls
« Reply #4 on: October 12, 2019, 12:50:24 pm »
At the very least you need to comment the "0" after FD in ossysc.inc