[SOLVED] Attempt to dereference a generic pointer on Linux but not on Windows

[Gizmo]

Folks

Last night I was up into the early hours trying to debug a mysterious issue.

I am using a fantastic well established C library in one of my projects using Lazarus 2.0.12 and FPC 3.2.0. One of the function calls of that library has the following C based declaration :

(https://github.com/libyal/libewf-legacy/blob/d35048cd97d2e192708988fc1424cd2ca2a00e76/include/libewf.h.in#L1589)

Code: C  [Select][+][-]

1. int libewf_handle_get_utf8_hash_value(
2.      libewf_handle_t *handle,
3.      const uint8_t *identifier,
4.      size_t identifier_length,
5.      uint8_t *utf8_string,
6.      size_t utf8_string_size,
7.      libewf_error_t **error );

So it expects something like :

Code: C  [Select][+][-]

1. libewf_handle_get_utf8_hash_value( handle, (uint8_t *) "MD5", 3, value, value_size, error )

(https://github.com/libyal/libewf-legacy/blob/main/libewf/libewf_metadata.c#L5084)

My implementation (which has only been slightly updated over the years from the original author of the wrapper) works flawlessly...on Windows. As follows :

Code: Pascal  [Select][+][-]

1. function TLibEWF.libewf_GetHashValue(identifier:ansistring;var value:ansistring) : integer;
2. var
3.   err:pointer;
4.   p:pansichar;
5.   l:tsize;
6. begin
7.   err:=nil;
8.   Result:=-1;
9.   if fLibHandle<>0 then
10.   begin
11.   //if LIBEWF_VERSION='V1' then ...;
12.   getmem(p,255);
13.   if LIBEWF_VERSION='V2' then
14.  
15.   Result:=flibewfhandlegetutf8hashvalue(fCurEWFHandle,   // Holds the handle address of the file
16.                                           pansichar(identifier),  // identifier shows 'MD5'
17.                                           length(identifier),       // returns 3
18.                                           p,                             // Holds the string of the hash itself
19.                                           l,                              // Holds the digit 1
20.                                           @err);                      // Nothing
21.   if result=1 then value:=strpas(p);                      // Result does equal 1, and value now equals hash string
22.   FreeMemory(p);
23.   end;
24. end;        

On Linux, however, everything is the same except p and l and err.

Code: Pascal  [Select][+][-]

1. p = pchar($00007FFFEDA33C20) #152'7'#163#237#255#127, (p)^ = 152 #152
2. l = 3902020624
3. err = (err)^ = Attempt to dereference a generic pointer.
4.  

I've asked the developer of the library who suspects there to be some issue between the interfacing of Pascal and the library and a parameter of the function call that might have a different bit size in Pascal then the library expects?

Can anyone see where I am going wrong here? Or who could explain why it works perfectly on Windows returning 1, but returns -1 on Linux?

[MarkMLl]

Are you sure that the problem's where you think it is? The reason I ask is this:

Code: Pascal  [Select][+][-]

1. begin
2.   //if LIBEWF_VERSION='V1' then ...;
3.   getmem(p,255);
4.   if LIBEWF_VERSION='V2' then
5.  

if your ... means that you've actually got v1-specific code in there then since the version check is being done at runtime then you're actually trying to compile both the v1- and v2-specific calls.

You need something like

Code: Pascal  [Select][+][-]

1. {$define LIBEWF_VERSION_V2 }
2.  
3. {$ifndef LIBEWF_VERSION_V2 }
4. ...
5. {$else }
6. ...
7. {$endif LIBEWF_VERSION_V2 }
8.  

MarkMLl

[PascalDragon]

Can anyone see where I am going wrong here? Or who could explain why it works perfectly on Windows returning 1, but returns -1 on Linux?

How is the flibewfhandlegetutf8hashvalue declared?

[Gizmo]

Here are the declaration areas that I think are relevant. Hopefully it helps you help me?

Code: Pascal  [Select][+][-]

1. type
2.  TINT16 = ShortInt; // was short
3.   TUINT16 = word;
4.   TUINT8 = byte;
5.   PLIBEWFHDL = pointer;
6.   TSIZE = longword;
7.   TSIZE64 = int64;
8.   PSIZE64 = ^int64;
9.   TARRPCHAR = array of pansiChar;
10.   PARRPCHAR = ^TARRPCHAR;    
11. ...
12.  Tlibewfhandlegetutf8hashvalue     = function(handle : PLIBEWFHDL;identifier:pansichar; identifier_length:TSIZE; utf8_string:pansichar; utf8_string_length:TSIZE; error:pointer) : integer; cdecl;
13.  
14. ...TLibEWF = class(TObject)
15.   private                
16. ...
17.   public
18.     function libewf_GetHashValue(identifier:ansistring;var value:ansistring) : integer;
19.  
20. constructor TLibEWF.create();    
21. if fLibHandle<>nilhandle then  
22. ...
23.  @flibewfhandlegetutf8hashvalue     :=GetProcAddress(fLibHandle,'libewf_handle_get_utf8_hash_value');
24. ...

Mark - thanks for your comment though there is actually nothing after that V1 thing. It just forgot to strip it out.

[Zvoni]

Errrr..... the last parameter is a Pointer to Pointer according to the c-declaration in the first post

What's the underlying datatype of libewf_error_t?
Fixed-size integer or something? like an error code?

Usually a Pointer to pointer is an array in c

Have you tried casting it to the correct type before dereferencing it?
IIRC, you cannot dereference a "raw" pointer

[Gizmo]

Zvoni - I am struggling to find what error actually is. Nearest I can find is this :

https://github.com/libyal/libewf-legacy/blob/main/libewf/libewf_error.h

On the "pointer to pointer" point, my understanding, which may be wrong, is I have a var called err that is declared as a pointer :

err : pointer

And then in the call I use @err which means "point to the pointer for error". Is that not correct?

(though, AFAIK, its not the error return that is the problem)

[Zvoni]

Line 29
https://bitbucket.ki.agh.edu.pl/projects/FSLIB/repos/libewf/browse/include/libewf/error.h
It's a TypeDef for IntPtr_t --> Pointer to Fixed Size Integer?

You could try dereferencing it this way:

Writeln(PInteger(err)^);

Or just declare err as PInteger

EDIT: If that IntPtr_t is already a Pointer to Integer, wouldn't that mean you actually have to pass a Pointer To Pointer To Pointer as the third argument?
Type already a pointer plus twice '**'
In that case it would mean that err is a PPInteger

EDIT2: What is actually your error-message you get on Linux (from the compiler?)

[Gizmo]

I tried switching err to PInteger. But the return result is still -1 (on Linux).

As PInteger, debugger reports :
err = $000000000155F3F0
(err)^ = Attempt to dereference a generic pointer.

As Pointer, debugger reports :
err = $00000000015D8CD0
(err)^ = Attempt to dereference a generic pointer.

This is also stated in original post for info.

Either way, the point here is the code works on Windows. But it doesn't on Linux, or rather, it runs on Linux but doesn't return values on Linux that it should, and does, on Windows. The err var is not the problem, I don't believe. The problem is that var p and l are having values put in them that are invalid...on Linux. But perfectly fine, on Windows.

Using the libraries error reporting code I was able to generate this message data :

Code: [Select]

libuna_unicode_character_copy_to_utf8: UTF-8 string too small.
libuna_utf8_string_with_index_copy_from_utf8_stream: unable to copy Unicode character to UTF-8.
libfvalue_string_copy_to_utf8_string_with_index: unable to copy UTF-8 stream to UTF-8 string.
libfvalue_value_copy_to_utf8_string_with_index: unable to copy instance to UTF-8 string.
libfvalue_value_copy_to_utf8_string: unable to copy value: 0 to UTF-8 string.
libewf_handle_get_utf8_hash_value: unable to copy hash value to UTF-8 string.

which, given that p is complaining about being unable to be dereferenced, and l is a massive longword value instead of just 1, makes a degree of sense. I still don't see where the problem is, though, IRO Linux itself.

[Zvoni]

What‘s the Longword has to do with your error-message?
I‘d rather look at your PAnsichar

[tetrastes]

Here are the declaration areas that I think are relevant. Hopefully it helps you help me?

Code: Pascal  [Select][+][-]

1. type
2.  TINT16 = ShortInt; // was short
3. ...
4.    TSIZE = longword;
5.  ...
6.  Tlibewfhandlegetutf8hashvalue     = function(handle : PLIBEWFHDL;identifier:pansichar; identifier_length:TSIZE; utf8_string:pansichar; utf8_string_length:TSIZE; error:pointer) : integer; cdecl;
7.  
8. ....

What is the bitness at Linux and Windows?
Your TSIZE (used for size_t in C) is for 32 bit. At 64-bit systems size_t is 64 bit.

Also note that your TINT16 is 8-bit, is it intentionally?

[Gizmo]

Zvoni

That PAnsiChar was certainly the culprit.

After several more days of wrestling, I went to the developer of the library directly who very kindly, and simply, and almost immediately pointed out that all I had to do was change the p variable to string!

So

Code: Pascal  [Select][+][-]

1. var
2.   p :string
3. begin
4.   SetLength(p, 512);
5.   l := 512
6. ...
7. end;
8.  

and it then works. The explanation being that Linux Unicode (wchar_t) strings are typically UTF-32 and on Windows UTF-16 little-endian. Since UTF-8 is typically a narrow string (char) he opts to treat these strings a byte strings/raw buffers (no encoding). And as they dont contain non ASCII chars in themselves, a normal string suffices in FPC.

Only a week of my life spent wrestling with that, and what an utter fool I feel like now. But hey ho. Solved. Thanks for all the help guys.