Lazarus

Free Pascal => Unix => Topic started by: mm7 on June 05, 2020, 02:45:23 pm

Title: FPC320 lib for SSL
Post by: mm7 on June 05, 2020, 02:45:23 pm
There are know issues with libssl v1.1 in FPC304.
I've decided to work it around by building my little library in FPC320 and then use it from program in FPC304.

Code: Pascal  [Select][+][-]
  1. library freehttpslib;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.   Classes, SysUtils,
  6.   fphttpclient, fpopenssl, OpenSSL, ssockets, opensslsockets,
  7.   strutils;
  8.  
  9. function https_get(url:string):RawByteString; cdecl;
  10. var Client: TFPHttpClient;
  11. begin
  12.    Result := '';
  13.    InitSSLInterface;
  14.    Client := TFPHttpClient.Create(nil);
  15.    try
  16.      Client.AddHeader('User-Agent','Mozilla/5.0 (compatible; fpweb)');
  17.      Client.AllowRedirect := true;
  18.      //URL:='https://google.com/';
  19.      Result := Client.Get(URL);
  20.    finally
  21.      Client.Free;
  22.    end;
  23. end;
  24.  
  25. const
  26.   Prefix = {$if defined(unix)}'lib'
  27.            {$elseif freebsd}'lib'
  28.            {$else}''
  29.            {$endif};
  30. exports
  31.   https_get name Prefix+'https_get';
  32. end.
  33.  
The library is built fine.

Program
Code: Pascal  [Select][+][-]
  1. program project1;
  2. {$mode objfpc}{$H+}
  3.  
  4. {$IF FPC_FULLVERSION < 32000}
  5. function libhttps_get(url:string):RawByteString; cdecl; external 'libfreehttpslib.so';
  6. {$endif}
  7.  
  8. var R: RawByteString;
  9. begin
  10.   R := libhttps_get('https://google.com');
  11.   writeln(R);
  12. end.
  13.  
The program is built fine.

But it fails when I run it
Code: Pascal  [Select][+][-]
  1. mark@ustudio:~/MyProjects/fpc/https32$ ldd project1
  2.         linux-vdso.so.1 (0x00007fff2a71e000)
  3.         libfreehttpslib.so => /usr/local/lib/libfreehttpslib.so (0x00007f43b6fe6000)
  4.         /lib64/ld-linux-x86-64.so.2 (0x00007f43b6f3f000)
  5.         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f43b6d3b000)
  6.         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f43b694a000)
  7. mark@ustudio:~/MyProjects/fpc/https32$ ./project1
  8. Inconsistency detected by ld.so: dl-minimal.c: 126: realloc: Assertion `ptr == alloc_last_block' failed!
  9.  

why?
Title: Re: FPC320 lib for SSL
Post by: Cyrax on June 05, 2020, 02:59:31 pm
Use PChar type instead of String type.
Title: Re: FPC320 lib for SSL
Post by: mm7 on June 05, 2020, 03:29:56 pm
just tryed. Same error
Inconsistency detected by ld.so: dl-minimal.c: 126: realloc: Assertion `ptr == alloc_last_block' failed!
Title: Re: FPC320 lib for SSL
Post by: Cyrax on June 05, 2020, 05:32:59 pm
Run your program under strace utility and see what your program really doing when its running.
Title: Re: FPC320 lib for SSL
Post by: Cyrax on June 05, 2020, 05:37:55 pm
See this stackoverflow post : https://stackoverflow.com/questions/59956996/problem-loading-a-library-with-ffi-in-php-7-4/59994038#59994038
Title: Re: FPC320 lib for SSL
Post by: mm7 on June 05, 2020, 06:08:46 pm
After the program found all required libs, and even has opened connection to google, it has failed.
Code: Pascal  [Select][+][-]
  1. ......
  2. connect(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("172.217.165.14")}, 16) = -1 EINPROGRESS (Operation now in progress)
  3. select(4, NULL, [3], NULL, {tv_sec=3, tv_usec=0}) = 1 (out [3], left {tv_sec=2, tv_usec=989744})
  4. getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
  5. fcntl(3, F_GETFL)                       = 0x802 (flags O_RDWR|O_NONBLOCK)
  6. fcntl(3, F_SETFL, O_RDWR)               = 0
  7. futex(0x7f2075956810, FUTEX_WAKE_PRIVATE, 2147483647) = 0
  8. futex(0x7f2075956804, FUTEX_WAKE_PRIVATE, 2147483647) = 0
  9. futex(0x7f20759568d8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
  10. mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2075ff5000
  11. writev(2, [{iov_base="Inconsistency detected by ld.so:"..., iov_len=33}, {iov_base="dl-minimal.c", iov_len=12}, {iov_base=": ", iov_len=2}, {iov_base="126", iov_len=3}, {iov_base=": ", iov_len=2}, {iov_base="realloc", iov_len=7}, {iov_base=": ", iov_len=2}, {iov_base="Assertion `", iov_len=11}, {iov_base="ptr == alloc_last_block", iov_len=23}, {iov_base="' failed!\n", iov_len=10}], 10Inconsistency detected by ld.so: dl-minimal.c: 126: realloc: Assertion `ptr == alloc_last_block' failed!
  12. ) = 105
  13. exit_group(127)                         = ?
  14. +++ exited with 127 +++
  15.  

I suspect something is wrong with memory/stack etc...
But I am not sure how to fix it.

the library was compiled with 0 No optimization, Relocatable -WR, Heap size 0, Stack size 0, All checks and assertions, and debug info.
Title: Re: FPC320 lib for SSL
Post by: Fred vS on June 05, 2020, 06:57:11 pm
Hello.

Using cmem + cthreads is working here:

The library:

Code: Pascal  [Select][+][-]
  1. library freehttpslib;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.  cmem,
  6.  {$ifdef unix}
  7.   cthreads, {$endif}
  8.   Classes, SysUtils,
  9.   fphttpclient, fpopenssl, OpenSSL, ssockets, opensslsockets,
  10.   strutils;
  11.  
  12. function https_get(url:string):RawByteString; cdecl;
  13. var Client: TFPHttpClient;
  14. begin
  15.    Result := '';
  16.    InitSSLInterface;
  17.    Client := TFPHttpClient.Create(nil);
  18.    try
  19.      Client.AddHeader('User-Agent','Mozilla/5.0 (compatible; fpweb)');
  20.      Client.AllowRedirect := true;
  21.      //URL:='https://google.com/';
  22.      Result := Client.Get(URL);
  23.    finally
  24.      Client.Free;
  25.    end;
  26. end;
  27.  
  28. const
  29.   Prefix = {$if defined(unix)}'lib'
  30.            {$else}''
  31.            {$endif};
  32. exports
  33.   https_get name Prefix+'https_get';
  34. end.


The program:

Code: Pascal  [Select][+][-]
  1. program project1;
  2. {$mode objfpc}{$H+}
  3.  
  4.   uses
  5.   cmem
  6.   {$ifdef unix}
  7.   ,cthreads {$endif}
  8.   ;
  9.  
  10. {$IF FPC_FULLVERSION < 32000}
  11. function libhttps_get(url:string):RawByteString; cdecl; external 'libfreehttpslib.so';
  12. {$endif}
  13.  
  14. var R: RawByteString;
  15. begin
  16.   R := libhttps_get('https://google.com');
  17.   writeln(R);
  18. end.


Result:

Code: Pascal  [Select][+][-]
  1. fred@fiens ~/freehttps> ./project1


Quote
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="es"><head><meta content="Google.es permite acceder a la información mundial en castellano, catalán, gallego, euskara e inglés." name="description"><meta content="noodp" name="robots"><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="UXEocwoZoPbp5q2TG2qhrA==">(function()

... // too much char for Lazarus forum...

\x22,\x22psrl\x22:\x22Eliminar\x22,\x22sbit\x22:\x22Buscar por imagen\x22,\x22srch\x22:\x22Buscar con Google\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x22,\x22refpd\x22:true,\x22rfs\x22:[],\x22sbpl\x22:16,\x22sbpr\x22:16,\x22scd\x22:10,\x22stok\x22:\x22lawobTB0fafCFjQoIvT26wfgiaw\x22,\x22uhde\x22:false}}';google.pmc=JSON.parse(pmc);})();</script>        </body></html>

Title: Re: FPC320 lib for SSL
Post by: mm7 on June 05, 2020, 07:10:17 pm
Thanks Cyrax!

after I've added -Cn to Custom options, then edited link.res
moving INPUT(/lib64/ld-linux-x86-64.so.2) after all other INPUTs, I was able to run program successfully.
It is quite a hacky way, but it does the trick.

Code: Pascal  [Select][+][-]
  1. )
  2. INPUT(
  3. -ldl
  4. )
  5. GROUP(
  6. -lc
  7. )
  8. INPUT(
  9. /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o
  10. /usr/lib/x86_64-linux-gnu/crtn.o
  11. )
  12. INPUT(
  13. /lib64/ld-linux-x86-64.so.2
  14. )
  15. SECTIONS
  16. {
  17.   .fpcdata           :
  18.   {
  19.     KEEP (*(.fpc .fpc.n_version .fpc.n_links))
  20.   }
  21.   .threadvar : { *(.threadvar .threadvar.* .gnu.linkonce.tv.*) }
  22. }
  23. INSERT AFTER .data;
  24.  


Thanks Fred!
I like your approach more because it does not require to hack into the build process. Excellent!
Also it can use String.

Title: Re: FPC320 lib for SSL
Post by: Fred vS on June 05, 2020, 07:11:51 pm
Re-hello.

You even dont need to use cmem but cthreads is needed in Unix OS.
Title: Re: FPC320 lib for SSL
Post by: argb32 on June 06, 2020, 02:02:06 am
Did you know that there is a pascal implementation of TLS? I.e. without the need of openssl or other external library:
https://github.com/fundamentalslib/fundamentals5/tree/master/Source/TLS
TinyPortal © 2005-2018