Lazarus

Free Pascal => Unix => Topic started by: Pascal on October 24, 2018, 08:49:22 am

Title: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 08:49:22 am
My app uses SQLite3Conn. When cross compiling to powerpc64-aix it tries to link libsqlite3.a although only libsqlite3.so
is availabel. How can i enable dynamic linking here?
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 09:06:37 am
I added -XD to "Custom Options" but linker still complains:

Code: Text  [Select]
  1. Hint: (11030) Start of reading config file C:\freepascal\fpc\trunk\bin\i386-win32\fpc.cfg
  2. Hint: (11031) End of reading config file C:\freepascal\fpc\trunk\bin\i386-win32\fpc.cfg
  3. Free Pascal Compiler version 3.3.1 [2018/10/24] for powerpc64
  4. Copyright (c) 1993-2018 by Florian Klaempfl and others
  5. (1002) Target OS: AIX for PowerPC64
  6. (3104) Compiling susMake.lpr
  7. (9009) Assembling susmake
  8. (9015) Linking C:\freepascal\projects\susMake\susmake
  9. C:\freepascal\fpc\trunk\bin\i386-win32\powerpc64-aix-ld.exe: cannot find -lsqlite3
  10. C:\freepascal\projects\susMake\susMake.lpr(30,44) Error: (9013) Error while linking
  11. C:\freepascal\projects\susMake\susMake.lpr(30,44) Fatal: (10026) There were 1 errors compiling module, stopping
  12. Fatal: (1018) Compilation aborted
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 10:10:56 am
I also tried to add option "-dy", "-Bdynamic" to linker. No success.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: marcov on October 24, 2018, 10:44:21 am
Did you double check that the files are for the correct architecture (IOW power64 ?)
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 10:49:10 am
Did you double check that the files are for the correct architecture (IOW power64 ?)
Don't know. libsqlite3.so could by 32bit. How can i check this?
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 11:14:28 am
Tried powerpc instead of powerpc64:

Code: Text  [Select]
  1. Hint: (11030) Start of reading config file C:\freepascal\fpc\trunk\bin\i386-win32\fpc.cfg
  2. Hint: (11031) End of reading config file C:\freepascal\fpc\trunk\bin\i386-win32\fpc.cfg
  3. Free Pascal Compiler version 3.3.1 [2018/10/24] for powerpc
  4. Copyright (c) 1993-2018 by Florian Klaempfl and others
  5. (1002) Target OS: AIX for PowerPC
  6. (3104) Compiling susMake.lpr
  7. (9009) Assembling susmake
  8. (9015) Linking C:\freepascal\projects\susMake\susMake
  9. C:\freepascal\fpc\trunk\bin\i386-win32\powerpc-aix-ld.exe: cannot find -lsqlite3
  10. C:\freepascal\projects\susMake\susMake.lpr(30,44) Error: (9013) Error while linking
  11. C:\freepascal\projects\susMake\susMake.lpr(30,44) Fatal: (10026) There were 1 errors compiling module, stopping
  12. Fatal: (1018) Compilation aborted

So nothing changed here.

If i provide a libsqlite3.a in C:\freepascal\aixroot\lib it links in both, 32bit and 64bit.

So the linker does not seem to get informed that it shoul link dynamic!
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: marcov on October 24, 2018, 11:23:03 am
Pass -s and check the generated ppas.sh/.bat and link.res to see what parameters the linker gets.

If you back up some files, you can do various linking attempts by running ppas.bat without rebuilding everything every time
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Thaddy on October 24, 2018, 12:15:34 pm
Sqlite is one of the few large opensource projects for which you can say: if in doubt, compile it yourself.
Sqlite3 is very easy to compile yourself for your platform.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 12:45:24 pm
Pass -s and check the generated ppas.sh/.bat and link.res to see what parameters the linker gets.

If you back up some files, you can do various linking attempts by running ppas.bat without rebuilding everything every time

Thanks for the hints!

I could see that -Xc does nothing here. I used -k-dy and linker gets called with -dy but it is not willing to link libsqlite3.so neither in 32bit nor 64bit.

If i use --verbose i can see that linker only tries to find "libsqlite3.a".
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: marcov on October 24, 2018, 12:58:22 pm
It is possible that cross linkers can't shared link, or refuses to do so if some global resource (like ld.so.conf) is not found.

It is also possible that certain default paths are set. Maybe the linker makes a difference between shared and static linking directories.

Anyway, it is clearly not a FPC problem, but a linker one. Searching AIX maillists looking for crosscompiling or simply "windows aix crosscompile" seems the best option.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 24, 2018, 03:20:24 pm
Please try this.

I have uploaded new binutils.
https://github.com/newpascal/fpcupdeluxe/releases/download/wincrossbins_v1.0/powerpc-aix.zip
Just copy them over your existing ones.

Code: Pascal  [Select]
  1. program project1;
  2. {$L C:\fpclazbydeluxe\trunk\cross\lib\powerpc64-aix\libsqlite3.so}
  3. //{$L libsqlite3.so}
  4.  
  5. uses ctypes;
  6.  
  7. var
  8.   sqlite3_libversion: function(): pansichar; cdecl;
  9.   s:pansichar;
  10.   SQLiteLibraryHandle: TLibHandle;
  11.  
  12. procedure LoadAddresses(LibHandle: TLibHandle);
  13. begin
  14.   pointer(sqlite3_libversion) := GetProcedureAddress(LibHandle,'sqlite3_libversion');
  15. end;
  16.  
  17. begin
  18.   SQLiteLibraryHandle := LoadLibrary('libsqlite3.so');
  19.   LoadAddresses(SQLiteLibraryHandle);
  20.   writeln('hello !');
  21.   s:=sqlite3_libversion();
  22.   writeln(s);
  23. end.

The above does cross-compile. But if it works on AIX you have to test yourself !
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 03:56:46 pm
Please try this.

I have uploaded new binutils.
https://github.com/newpascal/fpcupdeluxe/releases/download/wincrossbins_v1.0/powerpc-aix.zip
Just copy them over your existing ones.

Code: Pascal  [Select]
  1. program project1;
  2. {$L C:\fpclazbydeluxe\trunk\cross\lib\powerpc64-aix\libsqlite3.so}
  3. //{$L libsqlite3.so}
  4.  
  5. uses ctypes;
  6.  
  7. var
  8.   sqlite3_libversion: function(): pansichar; cdecl;
  9.   s:pansichar;
  10.   SQLiteLibraryHandle: TLibHandle;
  11.  
  12. procedure LoadAddresses(LibHandle: TLibHandle);
  13. begin
  14.   pointer(sqlite3_libversion) := GetProcedureAddress(LibHandle,'sqlite3_libversion');
  15. end;
  16.  
  17. begin
  18.   SQLiteLibraryHandle := LoadLibrary('libsqlite3.so');
  19.   LoadAddresses(SQLiteLibraryHandle);
  20.   writeln('hello !');
  21.   s:=sqlite3_libversion();
  22.   writeln(s);
  23. end.

The above does cross-compile. But if it works on AIX you have to test yourself !

For me too! But the executable is unusable. See output:
Code: Text  [Select]
  1. exec(): 0509-036 Das Programm unixtest kann wegen der folgenden Fehler nicht geladen werden:
  2.         0509-150   Das abhängige Modul C:\freepascal\aixroot/lib/libsqlite3.so konnte nicht geladen werden.
  3.         0509-022 Das Modul C:\freepascal\aixroot/lib/libsqlite3.so kann nicht geladen werden.
  4.         0509-026 Systemfehler: Eine Datei oder ein Verzeichnis des Pfadnamens ist nicht vorhanden.
  5.  
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 24, 2018, 03:59:37 pm
Use this one:
{$L libsqlite3.so}
(without the path)
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 06:04:53 pm
Use this one:
{$L libsqlite3.so}
(without the path)

I did! Same result. The linker adds "C:\freepascal\aixroot\lib\".

I also tried /usr/lib/libsqlite3.so but that doesn't link!  %)
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: marcov on October 24, 2018, 06:17:36 pm
I believe that is what -Xr is for.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 24, 2018, 08:26:39 pm
I believe that is what -Xr is for.

Makes sense. I'll try that tomorrow.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 12:42:21 pm
-Xr does not help here.

The linker always adds the complete path where it found the lib to the executable. Instead it should just leave the library name (without path).

@DonAlfredo: Can you check the linker sources if there is a way to suppress this behaviour?
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 25, 2018, 12:51:04 pm
Would you mind adding this into your fpc.cfg ?
(changing the paths towards your setup naturally)

Code: Pascal  [Select]
  1. #IFDEF FPC_CROSSCOMPILING
  2. #IFDEF AIX
  3. #IFDEF CPUPOWERPC64
  4. # Inserted by fpcup 25-10-2018 09:30:42
  5. -FDC:\fpclazbydeluxe\trunk\cross\bin\powerpc-aix\
  6. -XPpowerpc-aix-
  7. -Xd
  8. -FlC:\fpclazbydeluxe\trunk\cross\lib\powerpc64-aix\
  9. -XRC:\fpclazbydeluxe\trunk\cross\lib\powerpc64-aix
  10. -Xr/usr/lib
  11. #ENDIF
  12. #ENDIF
  13. #ENDIF
  14.  

Ps:
Hacking through the linker sources is not my favorite way of spending precious programming time. The hacks until now were not that hard, but preventing a basic functionality of the linker will not be easy. And besides, it should not be necessary: cross-compiling should work on AIX as it works on other Linux and Unix systems. Meaning: we have to dig deeper ... ;-)
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 01:34:54 pm
Again: Many thanks for your efforts!

I changed my fpc.cfg:
Code: Pascal  [Select]
  1. #IFDEF FPC_CROSSCOMPILING
  2. #IFDEF AIX
  3. #IFDEF CPUPOWERPC64
  4. # Inserted by fpcup 25-10-2018 09:30:42
  5. -FDC:\freepascal\fpc\binutils\i386-win32\
  6. -XPpowerpc-aix-
  7. -Xd
  8. -FlC:\freepascal\cross\aix\
  9. -XRC:\freepascal\cross\aix\
  10. -Xr/usr/lib
  11. #ENDIF
  12. #ENDIF
  13. #ENDIF

And removed all custom options from project/mode.

Result of run:
Code: Text  [Select]
  1. exec(): 0509-036 Das Programm tools/unixtest kann wegen der folgenden Fehler nicht geladen werden:
  2.         0509-150   Das abhängige Modul C:\freepascal\cross\aix\/lib/libsqlite3.so konnte nicht geladen werden.
  3.         0509-022 Das Modul C:\freepascal\cross\aix\/lib/libsqlite3.so kann nicht geladen werden.
  4.         0509-026 Systemfehler: Eine Datei oder ein Verzeichnis des Pfadnamens ist nicht vorhanden.

Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 01:39:42 pm
With option "-s" i can see the linker call:
Code: Text  [Select]
  1. C:\freepascal\fpc\trunk\bin\i386-win32\powerpc64-aix-ld.exe -bpT:0x10000000 -bpD:0x20000000 -btextro   -L. -o C:\freepascal\projects\unixtest\unixtest C:\freepascal\projects\unixtest\link.res -b64 --sysroot=C:\freepascal\cross\aix\

"-Xr/usr/lib" ssems to have no effect here.
I tried "-k-rpath-link /usr/lib" and "-k-rpath /usr/lib". Both have no effet on the binary.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 25, 2018, 01:45:11 pm
Did you do a Clean and Build ?
But I guess you did ... ;-)

Now I am (nearly) out of ideas ! Few remain.

It would be nice if I could get access to an (or yours) AIX server through SSH. Makes testing easier.

Second, I am preparing a new release of fpcupdeluxe. It will build a cross-compiler for AIX. With settings I know about. I would like you to try to use that to install FPC and Lazarus, and the AIX cross-compiler. But please be patient. The release will take another couple of hours. Will inform you !

Ps:
To add linker options, do:
-k-rpath-link -k/usr/lib
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 02:02:31 pm
Did you do a Clean and Build ?
But I guess you did ... ;-)

Now I am (nearly) out of ideas ! Few remain.

It would be nice if I could get access to an (or yours) AIX server through SSH. Makes testing easier.

Second, I am preparing a new release of fpcupdeluxe. It will build a cross-compiler for AIX. With settings I know about. I would like you to try to use that to install FPC and Lazarus, and the AIX cross-compiler. But please be patient. The release will take another couple of hours. Will inform you !

Ps:
To add linker options, do:
-k-rpath-link -k/usr/lib

We have to somehow leave out -XR. The problem we have then is that "crt0_64.o" is not found as it is added to the link.res as absolute path ("\lib\crt0_64.o") so it will not use the seach pathes. Were does this absolute path come from?

With symlink from /lib to /freepascal/cross/aix/lib i got it linking but the resolved path is still added to the binary.

Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 03:09:48 pm
I removed the absolute path to crt0_64.o from compiler/systems/t_aix.pas.
The object gets found now by the linker but the binary still includes the complete path to libsqlite3.so.

Code: Text  [Select]
  1. exec(): 0509-036 Das Programm tools/unixtest kann wegen der folgenden Fehler nicht geladen werden:
  2.         0509-150   Das abhängige Modul C:\freepascal\cross\aix\lib\/libsqlite3.so konnte nicht geladen werden.
  3.         0509-022 Das Modul C:\freepascal\cross\aix\lib\/libsqlite3.so kann nicht geladen werden.
  4.         0509-026 Systemfehler: Eine Datei oder ein Verzeichnis des Pfadnamens ist nicht vorhanden.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 25, 2018, 04:15:52 pm
New try, with all standard options.

Please add this into the linker options.
-rpath /usr/lib
(see included picture)

The -Xr option is ignored by FPC for AIX ... do not ask why.

If I look at the generated binary, the /usr/lib library searchpatch is added !
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 04:20:39 pm
New try, with all standard options.

Please add this into the linker options.
-rpath /usr/lib
(see included picture)

The -Xr option is ignored by FPC for AIX ... do not ask why.

If I look at the generated binary, the /usr/lib library searchpatch is added !

I did that already and i also could see "/usr/lib" in the binary but right after this string there is the found path of libsqlite3.so in the binary ("C:\freepascal\cross\aix\lib").
I found no way to get it out of the binary!

This drives me nuts  %)
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 25, 2018, 04:25:51 pm
I see something different. There is a zero between the path and the library. Meaning two different strings. Its not an absolute path.
See picture.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 04:28:17 pm
I see something different. There is a zero between the path and the library. Meaning two different strings. Its not an absolute path.
See picture.

Yes, right but system puts the final string together from path-string and pathdelimiter and library-string  :(
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 25, 2018, 06:19:19 pm
The linker should omit the pathes in the binary if -rpath is specified!

Is there an option for that or for omitting the pathes to the shared libs in general?
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 26, 2018, 08:40:43 am
While trying to link the static libsqlite3.a the linker refused to link a 64 bit version and only accepts the 32 bit version.
If this 64 bit binary finds the 32 bit libsqlite3.a on the system it loads and starts.
Strange!
Why does the linker not accept the 64 bit static libs?
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: DonAlfredo on October 26, 2018, 09:07:17 am
The 64bit version is only a link towards the 32bit version.
The libs are suitable for boths architectures.
Title: Re: libsqlite3.so vs libsqlite3.a
Post by: Pascal on October 26, 2018, 09:38:18 am
Yes, i just realized the same! Sorry  :-[