Converting Hex string to binary string

[Frogfather]

Morning!

I've woken up this morning with an irresistible urge to convert hex strings to binary.  This may of course have something to do with today's https://adventofcode.com/2021/day/16 puzzle

I've used the Hex to bin method in this earlier forum post: https://forum.lazarus.freepascal.org/index.php/topic,32612.msg210421.html#msg210421

Code: Pascal  [Select][+][-]

1. procedure TmainForm.day16part1;
2. var
3.   HValue,BValue:PChar;
4.   BBufSize:Integer;
5.   inStr,outStr:String;
6.   output:integer;
7.  
8.   function HexToNibble(const hexvalue: char; out nibble: byte): boolean;inline;
9.   begin
10.     if hexvalue IN ['0'..'9'] then
11.       nibble:=((ord(hexvalue)) and 15)
12.     else if char(byte(hexvalue) or $20) IN ['a'..'f'] then
13.       nibble:=((ord(hexvalue)+9) and 15)
14.     else
15.       Exit(False);
16.     Result := True;
17.   end;
18.  
19.   function HexToBin(HexValue, BinValue: PChar; BinBufSize: Integer): Integer;
20.   var
21.     i,j : integer;
22.     h,l : byte;
23.   begin
24.     i:=binbufsize;
25.     while (i>0) do
26.     begin
27.       if not HexToNibble(hexvalue^, h) then
28.         break;
29.       inc(hexvalue);
30.  
31.       if not HexToNibble(hexvalue^, l) then
32.         break;
33.       j := l + (h shl 4);
34.       inc(hexvalue);
35.  
36.       binvalue^:=chr(j);
37.       inc(binvalue);
38.       dec(i);
39.     end;
40.     result:=binbufsize-i;
41.   end;
42.  
43. //Above is code from the forum post. Below is my code.
44. begin
45.   inStr:='EE00D40C823060';
46.   outStr:='';
47.   HValue:=@inStr[1];
48.   BValue:=@outStr;
49.   BBufSize:=length(inStr)*4;
50.   output:=HexToBin(HValue,bValue,bBufSize);
51.   lbResults.items.add('input string '+inStr);
52.   lbResults.items.add('output string '+outStr);
53. end;  
54.  

Unfortunately this doesn't work - throws an unhelpful 'bad access' error.
The main problem here is I don't really understand what's going on and therefore don't know how to resolve it. If anyone has a few minutes to supply a dummies guide to pointers I'd be most grateful!

Cheers

John C

[marcov]

WHere do you allocate room for characters in outstr? You set it to empty, which is 0 chars.

[Frogfather]

Hi Marcov,

Thanks for the quick reply.

I told you I didn't fully understand what I was doing 

I've tried setLength(outStr, length(instr)*4) (on the assumption that each hex character will get converted to 4 binary characters)  but outstr is still empty after calling the HexToBin method.
Also tried just setting it to a long line of spaces with the same result.

Also I still get a 'bad access' error although this seems to happen after the method has finished (i.e. I can get to line 51 without error but continuing causes the error to be thrown. This also happens if I delete the two logging lines)

[avra]

Not directly hex2bin, but FPC trunk now supports for example SomeOrdTypeVar.ToBinString, SomeOrdTypeVar.Bits[3] and SomeOrdTypeVar.Nibbles[1] (with customizable localized formats when new syshelpers unit is used).
https://gitlab.com/freepascal.org/fpc/source/-/issues/39268

Actually, everything from BitHelpers is now part of FreePascal.

[bytebites]

Code: Pascal  [Select][+][-]

1.   HValue:=pchar(inStr);
2.   BValue:=pchar(outStr);  

[BobDog]

Windows, perhaps Linux using 'c' instead of 'msvcrt.dll'

Code: Pascal  [Select][+][-]

1.  
2.  function  ultoa(T:longint;F:pointer;sz:integer):pchar ;cdecl external 'msvcrt.dll' name '_ultoa';
3.  function strtoul (s:pchar;b:pointer;L: longint):longint ;cdecl external 'msvcrt.dll' name 'strtoul';
4.  
5.  
6.  function intToBase(N:longint;_base:byte):pchar;
7.  var
8.    s:ansistring;
9.     buffer:pchar;
10.     begin
11.     setlength(s,50);
12.     buffer:=@s[1];
13.     ultoa(n,buffer,_base);
14.     exit(buffer)
15. end;
16.  
17.  function intFromBase(s:pchar;_base:byte):longint ;
18.     begin
19.     exit(strtoul(s,nil,_base));
20. end;
21.  
22. function hextobin(hx:ansistring):ansistring;
23. var ans:ansistring;
24. var t:longint;
25. begin
26. t:=intfrombase(pchar(hx),16);
27. ans:=inttobase(t,2);
28. exit(ans);
29.  
30. end;
31.  
32. var
33. num,i:longint;
34. hx:ansistring;
35. ans:ansistring;
36. begin
37. for i:=1 to 10 do
38. begin
39. num:=random(2147483647);
40. writeln('number ',num);
41. hx:=hexstr(num,8);
42. writeln('hex of number ',hx);
43. ans:=hextobin(hx);
44. writeln('Bin from hex ',ans);
45. writeln('check  ',intfrombase(pchar(ans),2));
46. writeln;
47. end;
48. writeln('Press return to end . . .');
49. readln;
50.  
51. end.
52.  

[Frogfather]

Thanks all for the very quick and detailed replies.

For the purposes of the puzzle I've written my own method, but will certainly take a look at the examples posted later.

Cheers

John C

[Remy Lebeau]

I've woken up this morning with an irresistible urge to convert hex strings to binary.

Are you aware that FreePascal has a HexToBin() function for that exact purpose?

[winni]

Hi!

Even simpler since the very early Pascal days:

Code: Pascal  [Select][+][-]

1. var i,error : integer;
2. ...
3.   val ('$FF',i,error);  

Winni

[jollytall]

Well, maybe a bit of overkill, but my longmath unit (https://forum.lazarus.freepascal.org/index.php/topic,20586.msg425903.html#msg425903) can handle any base between 2 and 16 in any length, even making calculations with them. A simple line can work:
s := '%1' * '$123456789ABCDEF'; returning:
s = '100100011010001010110011110001001101010111100110111101111'

'%1' is simply 1, but the input is in base two, indicating that the result's natural format shall also be base 2 and '$123..' is a hexa number. The product is the same number, but because it is now natively base 2, when put in a string with the := assign operator, it gives a base2 string.

[winni]

Hi!

This can be done with only the system functions:

Code: Pascal  [Select][+][-]

1. var err : integer;
2.      i,k : Int64;
3.      s : string;
4.    ...
5.    val ('%1',i,err);
6.    val ('$123456789ABCDEF',k,err);
7.    s :=  BinStr(i*k,64);          
8.  

There are so many powerful functions in the system unit - often the stuff in the multiple units is redundand.

Winni

[Frogfather]

I've woken up this morning with an irresistible urge to convert hex strings to binary.

Are you aware that FreePascal has a HexToBin() function for that exact purpose?

Hi Remy,

Yes, I was having some difficulty getting it to produce the correct output (or indeed any output) probably as I've mentioned earlier, due to my not quite understanding it. For the purposes of Advent of Code I've written my own which does what I want, but once I have some more free time (ha!) I'll try to replace it with the built in method.

Cheers
John C

[BobDog]

Hello jollytall.
converting from base to base where the base is a power of two follows an easy pattern.
You step through 2^0(=1) from hex to bin
and 2^4(=16) from bin to hex.
inttobin and inttohex can be used with these steps.
The answer is accumulative for large strings.
Could you check the larger number with your library?

Code: Pascal  [Select][+][-]

1. uses
2. strutils,sysutils;
3.  
4. function TrimLeadingZeros(const S: string): string;
5. var
6.   I, L: Integer;
7. begin
8.   L:= Length(S);
9.   I:= 1;
10.   while (I < L) and (S[I] = '0') do Inc(I);
11.   exit (Copy(S, I));
12. end;
13.  
14. function hex2bin(hx:ansistring):ansistring;
15. var ans:ansistring='';
16. var t,r:longint;
17. err:integer;
18. begin
19. for t:=1 to length(hx) do
20. begin
21. val('$'+midstr(hx,t,1),r,err);
22. ans:=ans+inttobin(r,4);
23. end;
24. exit( TrimLeadingZeros(ans));
25. end;
26.  
27. function bin2hex(bn:ansistring):ansistring;
28. var ans:ansistring='';
29. var t,r:longint;
30. err:integer;
31. begin
32. while (length(bn) mod 4 <>0) do
33. begin
34. bn:='0'+bn;
35. end;
36. t:=-3;
37. repeat
38. t:=t+4;
39. if t>length(bn) then break;
40. val('%'+midstr(bn,t,4),r,err);
41. ans:=ans+inttohex(r,1);
42. until false;
43. exit(TrimLeadingZeros(ans));
44. end;
45. var
46. hx,bn:ansistring;
47.  
48.  
49. begin
50. hx:='123456789ABCDEF';
51. writeln('hex ',hx);
52. bn:=hex2bin(hx);
53. writeln('bin ',bn);
54. writeln('hex ',bin2hex(bn));
55.  
56. hx:=hx+hx;
57. writeln;
58. writeln('hex ',hx);
59. bn:=hex2bin(hx);
60. writeln('bin ',bn);
61. writeln('hex ',bin2hex(bn));
62. writeln('Press return to end . . .');
63. readln;
64. end.
65.  

[Frogfather]

Hi!

This can be done with only the system functions:

Code: Pascal  [Select][+][-]

1. var err : integer;
2.      i,k : Int64;
3.      s : string;
4.    ...
5.    val ('%1',i,err);
6.    val ('$123456789ABCDEF',k,err);
7.    s :=  BinStr(i*k,64);          
8.  

There are so many powerful functions in the system unit - often the stuff in the multiple units is redundand.

Winni

Hi Winni,

That looks like a really nice solution - I'll definitely give that a go - thanks!

[jollytall]

I agree for this simple case there are solutions available in the system. My unit:
- can do it in any base 2-16
- can do any length number
- and can do arithmetic with it
So, as said above, for a simple conversion it is a bit of overkill.

@BobDog: Nice to do repeated conversion for long numbers if both are bases 2^n. (What would you like me to check?).
@winnin: In your example, you do not need to multiply by 1. I only used it to indicate that the result I want to see in base 2. However that is not necessary in my unit either, if one is willing to use a function:

Code: Pascal  [Select][+][-]

1. s := ConvertNumberToBinString('$123456789ABCDEF0123456789ABCDEF') also works.