Recent

Author Topic: [SOLVED] Attempt to dereference a generic pointer on Linux but not on Windows  (Read 7431 times)

Gizmo

  • Hero Member
  • *****
  • Posts: 800
Folks

Last night I was up into the early hours trying to debug a mysterious issue.

I am using a fantastic well established C library in one of my projects using Lazarus 2.0.12 and FPC 3.2.0. One of the function calls of that library has the following C based declaration :

(https://github.com/libyal/libewf-legacy/blob/d35048cd97d2e192708988fc1424cd2ca2a00e76/include/libewf.h.in#L1589)
Code: C  [Select][+][-]
  1. int libewf_handle_get_utf8_hash_value(
  2.      libewf_handle_t *handle,
  3.      const uint8_t *identifier,
  4.      size_t identifier_length,
  5.      uint8_t *utf8_string,
  6.      size_t utf8_string_size,
  7.      libewf_error_t **error );

So it expects something like :

Code: C  [Select][+][-]
  1. libewf_handle_get_utf8_hash_value( handle, (uint8_t *) "MD5", 3, value, value_size, error )
(https://github.com/libyal/libewf-legacy/blob/main/libewf/libewf_metadata.c#L5084)

My implementation (which has only been slightly updated over the years from the original author of the wrapper) works flawlessly...on Windows. As follows :

Code: Pascal  [Select][+][-]
  1. function TLibEWF.libewf_GetHashValue(identifier:ansistring;var value:ansistring) : integer;
  2. var
  3.   err:pointer;
  4.   p:pansichar;
  5.   l:tsize;
  6. begin
  7.   err:=nil;
  8.   Result:=-1;
  9.   if fLibHandle<>0 then
  10.   begin
  11.   //if LIBEWF_VERSION='V1' then ...;
  12.   getmem(p,255);
  13.   if LIBEWF_VERSION='V2' then
  14.  
  15.   Result:=flibewfhandlegetutf8hashvalue(fCurEWFHandle,   // Holds the handle address of the file
  16.                                           pansichar(identifier),  // identifier shows 'MD5'
  17.                                           length(identifier),       // returns 3
  18.                                           p,                             // Holds the string of the hash itself
  19.                                           l,                              // Holds the digit 1
  20.                                           @err);                      // Nothing
  21.   if result=1 then value:=strpas(p);                      // Result does equal 1, and value now equals hash string
  22.   FreeMemory(p);
  23.   end;
  24. end;        

On Linux, however, everything is the same except p and l and err.

Code: Pascal  [Select][+][-]
  1. p = pchar($00007FFFEDA33C20) #152'7'#163#237#255#127, (p)^ = 152 #152
  2. l = 3902020624
  3. err = (err)^ = Attempt to dereference a generic pointer.
  4.  

I've asked the developer of the library who suspects there to be some issue between the interfacing of Pascal and the library and a parameter of the function call that might have a different bit size in Pascal then the library expects?

Can anyone see where I am going wrong here? Or who could explain why it works perfectly on Windows returning 1, but returns -1 on Linux?
« Last Edit: May 22, 2021, 04:46:53 pm by Gizmo »
Lazarus 2.0.12 and fpc 3.2.0 - Linux Mint 19 LTS, Windows 10 64 and Mac OSX Big Sur
Useful Page to remember : http://wiki.freepascal.org/Cross_compiling#From_Linux_x64_to_Linux_i386

MarkMLl

  • Hero Member
  • *****
  • Posts: 3064
Are you sure that the problem's where you think it is? The reason I ask is this:

Code: Pascal  [Select][+][-]
  1. begin
  2.   //if LIBEWF_VERSION='V1' then ...;
  3.   getmem(p,255);
  4.   if LIBEWF_VERSION='V2' then
  5.  

if your ... means that you've actually got v1-specific code in there then since the version check is being done at runtime then you're actually trying to compile both the v1- and v2-specific calls.

You need something like

Code: Pascal  [Select][+][-]
  1. {$define LIBEWF_VERSION_V2 }
  2.  
  3. {$ifndef LIBEWF_VERSION_V2 }
  4. ...
  5. {$else }
  6. ...
  7. {$endif LIBEWF_VERSION_V2 }
  8.  

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 3324
  • Compiler Developer
Can anyone see where I am going wrong here? Or who could explain why it works perfectly on Windows returning 1, but returns -1 on Linux?

How is the flibewfhandlegetutf8hashvalue declared?

Gizmo

  • Hero Member
  • *****
  • Posts: 800
Here are the declaration areas that I think are relevant. Hopefully it helps you help me?
Code: Pascal  [Select][+][-]
  1. type
  2.  TINT16 = ShortInt; // was short
  3.   TUINT16 = word;
  4.   TUINT8 = byte;
  5.   PLIBEWFHDL = pointer;
  6.   TSIZE = longword;
  7.   TSIZE64 = int64;
  8.   PSIZE64 = ^int64;
  9.   TARRPCHAR = array of pansiChar;
  10.   PARRPCHAR = ^TARRPCHAR;    
  11. ...
  12.  Tlibewfhandlegetutf8hashvalue     = function(handle : PLIBEWFHDL;identifier:pansichar; identifier_length:TSIZE; utf8_string:pansichar; utf8_string_length:TSIZE; error:pointer) : integer; cdecl;
  13.  
  14. ...TLibEWF = class(TObject)
  15.   private                
  16. ...
  17.   public
  18.     function libewf_GetHashValue(identifier:ansistring;var value:ansistring) : integer;
  19.  
  20. constructor TLibEWF.create();    
  21. if fLibHandle<>nilhandle then  
  22. ...
  23.  @flibewfhandlegetutf8hashvalue     :=GetProcAddress(fLibHandle,'libewf_handle_get_utf8_hash_value');
  24. ...

Mark - thanks for your comment though there is actually nothing after that V1 thing. It just forgot to strip it out.
Lazarus 2.0.12 and fpc 3.2.0 - Linux Mint 19 LTS, Windows 10 64 and Mac OSX Big Sur
Useful Page to remember : http://wiki.freepascal.org/Cross_compiling#From_Linux_x64_to_Linux_i386

Zvoni

  • Hero Member
  • *****
  • Posts: 698
Errrr..... the last parameter is a Pointer to Pointer according to the c-declaration in the first post

What's the underlying datatype of libewf_error_t?
Fixed-size integer or something? like an error code?

Usually a Pointer to pointer is an array in c

Have you tried casting it to the correct type before dereferencing it?
IIRC, you cannot dereference a "raw" pointer
« Last Edit: May 20, 2021, 03:15:58 pm by Zvoni »
One System to rule them all, One IDE to find them,
One Code to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
People call me crazy, because i'm jumping out of perfectly fine aircraft

Gizmo

  • Hero Member
  • *****
  • Posts: 800
Zvoni - I am struggling to find what error actually is. Nearest I can find is this :

https://github.com/libyal/libewf-legacy/blob/main/libewf/libewf_error.h

On the "pointer to pointer" point, my understanding, which may be wrong, is I have a var called err that is declared as a pointer :

err : pointer

And then in the call I use @err which means "point to the pointer for error". Is that not correct?

(though, AFAIK, its not the error return that is the problem)
Lazarus 2.0.12 and fpc 3.2.0 - Linux Mint 19 LTS, Windows 10 64 and Mac OSX Big Sur
Useful Page to remember : http://wiki.freepascal.org/Cross_compiling#From_Linux_x64_to_Linux_i386

Zvoni

  • Hero Member
  • *****
  • Posts: 698
Line 29
https://bitbucket.ki.agh.edu.pl/projects/FSLIB/repos/libewf/browse/include/libewf/error.h
It's a TypeDef for IntPtr_t --> Pointer to Fixed Size Integer?

You could try dereferencing it this way:

Writeln(PInteger(err)^);

Or just declare err as PInteger

EDIT: If that IntPtr_t is already a Pointer to Integer, wouldn't that mean you actually have to pass a Pointer To Pointer To Pointer as the third argument?
Type already a pointer plus twice '**'
In that case it would mean that err is a PPInteger

EDIT2: What is actually your error-message you get on Linux (from the compiler?)
« Last Edit: May 20, 2021, 04:31:01 pm by Zvoni »
One System to rule them all, One IDE to find them,
One Code to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
People call me crazy, because i'm jumping out of perfectly fine aircraft

Gizmo

  • Hero Member
  • *****
  • Posts: 800
I tried switching err to PInteger. But the return result is still -1 (on Linux).

As PInteger, debugger reports :
err = $000000000155F3F0
(err)^ = Attempt to dereference a generic pointer.

As Pointer, debugger reports :
err = $00000000015D8CD0
(err)^ = Attempt to dereference a generic pointer.

This is also stated in original post for info.

Either way, the point here is the code works on Windows. But it doesn't on Linux, or rather, it runs on Linux but doesn't return values on Linux that it should, and does, on Windows. The err var is not the problem, I don't believe. The problem is that var p and l are having values put in them that are invalid...on Linux. But perfectly fine, on Windows.

Using the libraries error reporting code I was able to generate this message data :

Code: [Select]
libuna_unicode_character_copy_to_utf8: UTF-8 string too small.
libuna_utf8_string_with_index_copy_from_utf8_stream: unable to copy Unicode character to UTF-8.
libfvalue_string_copy_to_utf8_string_with_index: unable to copy UTF-8 stream to UTF-8 string.
libfvalue_value_copy_to_utf8_string_with_index: unable to copy instance to UTF-8 string.
libfvalue_value_copy_to_utf8_string: unable to copy value: 0 to UTF-8 string.
libewf_handle_get_utf8_hash_value: unable to copy hash value to UTF-8 string.

which, given that p is complaining about being unable to be dereferenced, and l is a massive longword value instead of just 1, makes a degree of sense. I still don't see where the problem is, though, IRO Linux itself.
« Last Edit: May 20, 2021, 05:16:02 pm by Gizmo »
Lazarus 2.0.12 and fpc 3.2.0 - Linux Mint 19 LTS, Windows 10 64 and Mac OSX Big Sur
Useful Page to remember : http://wiki.freepascal.org/Cross_compiling#From_Linux_x64_to_Linux_i386

Zvoni

  • Hero Member
  • *****
  • Posts: 698
What‘s the Longword has to do with your error-message?
I‘d rather look at your PAnsichar
One System to rule them all, One IDE to find them,
One Code to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
People call me crazy, because i'm jumping out of perfectly fine aircraft

tetrastes

  • Full Member
  • ***
  • Posts: 127
Here are the declaration areas that I think are relevant. Hopefully it helps you help me?
Code: Pascal  [Select][+][-]
  1. type
  2.  TINT16 = ShortInt; // was short
  3. ...
  4.    TSIZE = longword;
  5.  ...
  6.  Tlibewfhandlegetutf8hashvalue     = function(handle : PLIBEWFHDL;identifier:pansichar; identifier_length:TSIZE; utf8_string:pansichar; utf8_string_length:TSIZE; error:pointer) : integer; cdecl;
  7.  
  8. ....

What is the bitness at Linux and Windows?
Your TSIZE (used for size_t in C) is for 32 bit. At 64-bit systems size_t is 64 bit.

Also note that your TINT16 is 8-bit, is it intentionally?

Gizmo

  • Hero Member
  • *****
  • Posts: 800
Re: Attempt to dereference a generic pointer on Linux but not on Windows
« Reply #10 on: May 22, 2021, 04:46:44 pm »
Zvoni

That PAnsiChar was certainly the culprit.

After several more days of wrestling, I went to the developer of the library directly who very kindly, and simply, and almost immediately pointed out that all I had to do was change the p variable to string!

So
Code: Pascal  [Select][+][-]
  1. var
  2.   p :string
  3. begin
  4.   SetLength(p, 512);
  5.   l := 512
  6. ...
  7. end;
  8.  

and it then works. The explanation being that Linux Unicode (wchar_t) strings are typically UTF-32 and on Windows UTF-16 little-endian. Since UTF-8 is typically a narrow string (char) he opts to treat these strings a byte strings/raw buffers (no encoding). And as they dont contain non ASCII chars in themselves, a normal string suffices in FPC.

Only a week of my life spent wrestling with that, and what an utter fool I feel like now. But hey ho. Solved. Thanks for all the help guys.
Lazarus 2.0.12 and fpc 3.2.0 - Linux Mint 19 LTS, Windows 10 64 and Mac OSX Big Sur
Useful Page to remember : http://wiki.freepascal.org/Cross_compiling#From_Linux_x64_to_Linux_i386

 

TinyPortal © 2005-2018