Recent

Author Topic: Issue printing RawByteString in Python (DLL, ctypes)  (Read 14190 times)

codehorror

  • New member
  • *
  • Posts: 7
Issue printing RawByteString in Python (DLL, ctypes)
« on: April 02, 2024, 08:25:01 pm »
I'm writing a DLL that is consumed by a Python application.

I have a function that makes a GET request using TFPHttpClient. The function returns a pointer to a RawByteString:

Code: Pascal  [Select][+][-]
  1. PResponseString = ^RawByteString;
  2. function Home(Client: PFPHttpClient): PResponseString; stdcall;
  3.   begin
  4.     New(Result);
  5.     try
  6.       Result^ := Client^.Get('https://www.example.com');
  7.     except
  8.       on E: Exception do
  9.       begin
  10.         OutputDebugString(PChar('Exception: ' + E.Message));
  11.         Dispose(Result);
  12.         Result := nil;
  13.       end;
  14.     end;
  15.   end;        
  16.  

I can call the function no problem. But the response is confusing me. Here's my Python code:

Code: Python  [Select][+][-]
  1. from ctypes import c_bool, CDLL, c_void_p, c_char_p
  2. import os
  3. import sys
  4.  
  5. DLL_NAME = "dllproj"
  6. dll_name = os.path.join(
  7.     os.path.abspath(os.path.dirname(__file__)), DLL_NAME + ".dll"
  8. )
  9.  
  10. dll = CDLL(dll_name)
  11.  
  12. CreateClient = dll.CreateFBClient
  13. CreateClient.argtypes = [c_char_p, c_char_p]
  14. CreateClient.restype = c_void_p
  15.  
  16. Home = dll.Home
  17. Home.argtypes = [c_void_p]
  18. Home.restype = c_char_p
  19.  
  20. IsConnected.argtypes = [c_void_p]
  21. IsConnected.restype = c_bool
  22.  
  23. DestroyClient = dll.DestroyFBClient
  24. DestroyClient.argtypes = [c_void_p]
  25.  
  26. # This is odd having to do that, need to find better way to do this
  27. cuser = "7testing"
  28. x = "4test"
  29.  
  30. client = CreateClient(
  31.     cuser.encode("ascii"),
  32.     x.encode("ascii")
  33. )
  34.  
  35. response = Home(client)
  36. print(response)
  37. DestroyClient(client)
  38.  

Note: I have to insert the length of the string for cuser and x because strings in pascal uses 0 byte to store length.

When I print the response I get the following:

Code: Pascal  [Select][+][-]
  1. b'\x88\x17\xd8\x01#\x02'
  2.  

Which is confusing, because when I debug the DLL in Lazarus I can see the result contains what I expect - the HTML source code (see attatchment). So where is the rest of the bytes?

I've tried using decode in Python using: ascii, UTF-8, and UTF-16.

Code: Pascal  [Select][+][-]
  1. print(response.decode("utf-8"))
  2.  

Code: Pascal  [Select][+][-]
  1. UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 3: invalid start byte
  2.  

Any advice, or ideas?
« Last Edit: April 02, 2024, 08:50:23 pm by codehorror »

Darie

  • New Member
  • *
  • Posts: 31
The sphere of knowledge is in continuous expansion. So is the contact of it's points with the unknown --Blaise Pascal

tetrastes

  • Sr. Member
  • ****
  • Posts: 493
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #2 on: April 02, 2024, 09:56:20 pm »
I doubt that python understands AnsiStrings. Try smth like this:
Code: Pascal  [Select][+][-]
  1. function Home(Client: PFPHttpClient): PChar; stdcall;
  2.   begin
  3.     try
  4.       Result := PChar(Client^.Get('https://www.example.com'));
  5.     except
  6.       on E: Exception do
  7.       begin
  8.         OutputDebugString(PChar('Exception: ' + E.Message));
  9.         Result := nil;
  10.       end;
  11.     end;
  12.   end;

Code: Python  [Select][+][-]
  1. # This is odd having to do that, need to find better way to do this
  2. cuser = "7testing"
  3. x = "4test"
  4.  

Note: I have to insert the length of the string for cuser and x because strings in pascal uses 0 byte to store length.

Any advice, or ideas?
Advice is to read about pascal strings.

codehorror

  • New member
  • *
  • Posts: 7
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #3 on: April 02, 2024, 10:50:40 pm »
I doubt that python understands AnsiStrings. Try smth like this:
Code: Pascal  [Select][+][-]
  1. function Home(Client: PFPHttpClient): PChar; stdcall;
  2.   begin
  3.     try
  4.       Result := PChar(Client^.Get('https://www.example.com'));
  5.     except
  6.       on E: Exception do
  7.       begin
  8.         OutputDebugString(PChar('Exception: ' + E.Message));
  9.         Result := nil;
  10.       end;
  11.     end;
  12.   end;

Code: Python  [Select][+][-]
  1. # This is odd having to do that, need to find better way to do this
  2. cuser = "7testing"
  3. x = "4test"
  4.  

Note: I have to insert the length of the string for cuser and x because strings in pascal uses 0 byte to store length.

Any advice, or ideas?
Advice is to read about pascal strings.

Thanks I'll give this a try now :)

Regarding Pascal strings. I gave this a read (https://wiki.freepascal.org/Character_and_string_types), and noticed the first element of a string is the size of the string (if no length set, first two bytes, the other for ref count). But Python doesn't understand this. That's why I placed the length of each string as the first character in cuser and x.

codehorror

  • New member
  • *
  • Posts: 7
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #4 on: April 02, 2024, 10:54:41 pm »
That worked! I struggled for a good 4 hours with that.

I think I should be able to use PChar to avoid the string length problem I have.

Code: Pascal  [Select][+][-]
  1. function CreateFBClient(CUser: PChar; Xs: PChar): PFPHttpClient; stdcall;
  2.  

Edit: Yup that works too.
« Last Edit: April 02, 2024, 11:12:15 pm by codehorror »

tetrastes

  • Sr. Member
  • ****
  • Posts: 493
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #5 on: April 02, 2024, 11:48:30 pm »
Regarding Pascal strings. I gave this a read (https://wiki.freepascal.org/Character_and_string_types), and noticed the first element of a string is the size of the string ...
That's why I placed the length of each string as the first character in cuser and x.
This is only for ShortStrings. And you didn't place the length. You placed characters '7' and '4' (i.e. bytes $37 and $34, instead of $07 and $04).

codehorror

  • New member
  • *
  • Posts: 7
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #6 on: April 03, 2024, 12:10:10 am »
Regarding Pascal strings. I gave this a read (https://wiki.freepascal.org/Character_and_string_types), and noticed the first element of a string is the size of the string ...
That's why I placed the length of each string as the first character in cuser and x.
This is only for ShortStrings. And you didn't place the length. You placed characters '7' and '4' (i.e. bytes $37 and $34, instead of $07 and $04).

Oh, yes I see where I went wrong there. Thank you for your help and advice it's much appreciated.

Thaddy

  • Hero Member
  • *****
  • Posts: 14625
  • Sensorship about opinions does not belong here.
Re: Issue printing RawByteString in Python (DLL, ctypes)
« Reply #7 on: April 03, 2024, 11:50:16 am »
Since rawbytestring is essentially a Tbytes in the context of this  thread, consider this simplification:
Code: Pascal  [Select][+][-]
  1. {$mode objfpc}{$modeswitch typehelpers}{$H+}
  2. uses
  3.   sysutils;
  4.  
  5. type
  6.    TByteHelper = type helper for Tbytes
  7.      function ToString:String;
  8.    end;
  9.    
  10.    function TbyteHelper.ToString:string;
  11.    begin
  12.      setstring(Result, PChar(Self), length(self));
  13.    end;  
  14.    
  15.    operator := (const a:Tbytes):string;
  16.    begin
  17.      setstring(Result, PChar(a), length(a));
  18.    end;
  19.  
  20. var
  21.   t:Tbytes = (65,66,67,68);
  22.   s:string;
  23. begin
  24.   writeln(t.ToString);
  25.   s := t;
  26.   writeln(s);
  27. end.
It might help in your case.
There is one caveat: maybe manually add a #0 or #0#0 for the Python interface. I can't test this right now.
« Last Edit: April 03, 2024, 11:56:03 am by Thaddy »
bitrate is always calculated like this:sample rate * bitdepth * number of channels.

 

TinyPortal © 2005-2018