Lazarus

Announcements => Third party => Topic started by: avra on June 25, 2018, 10:20:17 am

Title: BitHelpers - bit manipulation for standard pascal types
Post by: avra on June 25, 2018, 10:20:17 am
BitHelpers

BitHelpers enable additional bit manipulation for qword, longword, word, byte and boolean types which will make your life much easier if you need such a feature.

History

FreePascal type helpers TBooleanHelper (https://www.freepascal.org/docs-html/rtl/sysutils/tbooleanhelper.html), TByteHelper (https://www.freepascal.org/docs-html/rtl/sysutils/tbytehelper.html), TWordHelper (https://www.freepascal.org/docs-html/rtl/sysutils/twordhelper.html), TCardinalHelper (https://www.freepascal.org/docs-html/rtl/sysutils/twordhelper.html) and TQWordHelper (https://www.freepascal.org/docs-html/rtl/sysutils/tqwordhelper.html) do not offer much when bit manipulation and presentation are needed. That's where BitHelpers package jumps in, nicely extending mentioned type helpers.

Installation

While you can simply copy bithelpers unit to your project directory and start using it, the recommended way would be to open bithelpers_pkg.lpk package and compile it. That would add BitHelpers source directory to Lazarus and make it available to all your projects.

Usage
Code: [Select]
  uses
    bithelpers;
  ...
  procedure TForm1.BooleanBitTestBtnClick(Sender: TObject);
  var
    MyBool: boolean;
  begin
    MyBool := true;
    Memo1.Append(MyBool.ToOneZeroString);
    Memo1.Append(MyBool.ToOnOffString); // default is scfUnchangedCase and can be ommited
    Memo1.Append(MyBool.ToOnOffString(scfLowerCase));
    Memo1.Append(MyBool.ToTrueFalseString(scfUpperCase));
    Memo1.Append(MyBool.ToString('OnState', 'OffState')); // true/false custom strings
    Memo1.Append(MyBool.ToString('Укључено', 'Искључено', scfUpperCase)); // when case and unicode matter
  end;
Code: [Select]
  1
  On
  on
  TRUE
  OnState
  УКЉУЧЕНО

Code: [Select]
  procedure TForm1.ByteBitTestBtnClick(Sender: TObject);
  var
    MyByte: byte;
  begin
    MyByte.Clear;                                  // %00000000 MyByte equals 0
    MyByte.Bit[0] := true;                         // %00000001 MyByte equals 1
    MyByte.Bit[2] := true;                         // %00000101 MyByte equals 5
    Memo1.Append(MyByte.ToString);
    Memo1.Append('$' + MyByte.ToHexString);
    Memo1.Append(MyByte.ToBooleanString(lzHideLeadingZeros));   // hide leading zeros
    Memo1.Append(MyByte.ToBooleanString);                       // show leading zeros   
  end;
Code: [Select]
  5
  $05
  101
  00000101

Code: [Select]
  procedure TForm1.WordBitTestBtnClick(Sender: TObject);
  var
    MyWord: word;
  begin
    MyWord.Clear;                  // %0000000000000000 MyWord equals 0
    MyWord.Byte[0] := 2;           // %0000000000000010 MyWord equals 2
    MyWord.Byte[1] := 1;           // %0000000100000010 MyWord equals 258 (2 + 256)
    MyWord.Byte[1].Bit[7] := true; // %0000000100000010 MyWord equals 258 (Beware!!! This DOES NOT set a bit in MyWord !!!)
    MyWord.Bit[10] := true;        // %0000010100000010 MyWord equals 1282 (258 + 2^10)
    Memo1.Append(MyWord.ToString);
    Memo1.Append('$' + MyWord.ToHexString);
    Memo1.Append(MyWord.ToBooleanString(lzHideLeadingZeros)); // hide leading zeros
    Memo1.Append(MyWord.ToBooleanString);                     // show leading zeros
  end;
Code: [Select]
  1282
  $0502
  10100000010
  0000010100000010

Code: [Select]
  procedure TForm1.LongwordBitTestBtnClick(Sender: TObject);
  var
    MyLongword: longword;
  begin
    MyLongword.Clear;                  // %00000000000000000000000000000000 MyLongword equals 0
    MyLongword.Word[0] := 250;         // %00000000000000000000000011111010 MyLongword equals 250
    MyLongword.Word[1].Byte[0] := 100; // %00000000000000000000000011111010 MyLongword equals 250 (Beware!!! This DOES NOT set a byte in MyLongword !!!)
    MyLongword.Byte[1] := 4;           // %00000000000000000000010011111010 MyLongword equals 1274 (250 + 2^(8 + 2), 2^2 = 4)
    MyLongword.Bit[26] := true;        // %00000100000000000000010011111010 MyLongword equals 67110138 (1274 + 2^26)
    Memo1.Append(MyLongword.ToString);
    Memo1.Append('$' + MyLongword.ToHexString);
    Memo1.Append(MyLongword.ToBooleanString(lzHideLeadingZeros)); // hide leading zeros
    Memo1.Append(MyLongword.ToBooleanString);                     // show leading zeros
    Memo1.Append('');
  end;
Code: [Select]
  67110138
  $040004FA
  100000000000000010011111010
  00000100000000000000010011111010

Code: [Select]
  procedure TForm1.QuadwordBitTestBtnClick(Sender: TObject);
  var
    MyQuadword: qword;
  begin
    MyQuadword.Clear;                      // %0000000000000000000000000000000000000000000000000000000000000000 MyQuadword equals 0
    MyQuadword.Longword[0] := 12345;       // %0000000000000000000000000000000000000000000000000011000000111001 MyQuadword equals 12345
    MyQuadword.Longword[1].Word[0] := 100; // %0000000000000000000000000000000000000000000000000011000000111001 MyQuadword equals 12345 (Beware!!! This DOES NOT set a word in MyQuadword !!!)
    MyQuadword.Byte[3] := 2;               // %0000000000000000000000000000000000000010000000000011000000111001 MyQuadword equals 33566777 (12345 + 2^(8 + 8 + 8 + 2), 2^1 = 2)
    MyQuadword.Bit[50] := true;            // %0000000000000100000000000000000000000010000000000011000000111001  MyQuadword equals 1125899940409401 (33566777 + 2^50)
    Memo1.Append(MyQuadword.ToString);
    Memo1.Append('$' + MyQuadword.ToHexString);
    Memo1.Append(MyQuadword.ToBooleanString(lzHideLeadingZeros)); // hide leading zeros
    Memo1.Append(MyQuadword.ToBooleanString);                     // show leading zeros
  end;
Code: [Select]
  1125899940409401
  $0004000002003039
  100000000000000000000000010000000000011000000111001
  0000000000000100000000000000000000000010000000000011000000111001

Code: [Select]
  procedure TForm1.OverlaysTestBtnClick(Sender: TObject);
  var
    MyQuadOverlay: TQuadwordOverlay;
  begin
    MyQuadOverlay.AsQuadword.Clear;
    MyQuadOverlay.AsByte[0] := 100;
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
    MyQuadOverlay.AsLongword[1] := 1;
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
    MyQuadOverlay.AsQuadword.Bit[32] := false;
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
    MyQuadOverlay.AsWordOverlay[3].AsByte[1] := $FF; // recursive overlays are allowed
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
    MyQuadOverlay.AsWord[3].Byte[1].Bit[5] := false; // NO CHANGE !!! Bit is set in a result byte, not in a byte that belongs to MyQuadOverlay
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
    MyQuadOverlay.AsBit[63] := false;
    Memo1.Append(MyQuadOverlay.AsQuadword.ToBooleanString);
  end;
Code: [Select]
  0000000000000000000000000000000000000000000000000000000001100100
  0000000000000000000000000000000100000000000000000000000001100100
  0000000000000000000000000000000000000000000000000000000001100100
  1111111100000000000000000000000000000000000000000000000001100100
  1111111100000000000000000000000000000000000000000000000001100100
  0111111100000000000000000000000000000000000000000000000001100100


Download

https://bitbucket.org/avra/bithelpers


License

BitHelpers package is released under triple license:
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: garlar27 on September 27, 2018, 05:02:33 pm
Hi Avra!!
        Good job!!!

It's a shame not knowing about it 5-7 years ago, it would saved me a lot of time and I would be using it for sure!!!

The only cons I see from the examples (I haven't downloaded the package and didn't checked the full capacity either) is the lack of configurable separators for nibbles and bytes in the ToBooleanString function; and configurable separators for each byte and "n" bytes in the ToHexString function. I need those to ease the logs readability. For instance:
Code: Pascal  [Select][+][-]
  1. 11.24.03.019 - Rx-<STX><$97><NUL><NUL><FS><$C0><$81><FS>
  2. 11.24.03.028 - Rx-<FS><NUL><NUL><FS><ETX>024
  3. 11.24.03.037 - Rx-D
  4. 11.24.03.046 - Tx-<ACK>
  5. 11.24.03.055 -  //   ###   Received:  02 97 00 00 1C C0 81 1C 1C 00 00 1C 03 30 32 34 44
  6. 11.24.03.083 -  //   ###   Status 1: hex( 00 00); bin( 00000000 00000000)
  7. 11.24.03.091 -  //   ###   Status 2: hex( C0 81); bin( 11000000 10000001)
  8. 11.24.03.268 -  //   ###   command PrnTK_TKNC_Item() // Extension [ bin: 00000000 00110000;  Hex: 00 30]:
  9. 11.24.03.287 -  //   ###   string to send:  02 98 0A 1B 02 1C 00 30 1C 1C 1C 1C 1C 53 61 6C 61 6D 65 20 4D 69 6C 61 6E 20 70 69 65 7A 61 1C 31 30 30 30 30 1C 39 30 30 30 30 30 1C 32 31 30 30 1C 1C 1C 1C 1C 37 39 30 31 1C 30 30 1C 37 03 30 44 39 36
  10. 11.24.03.300 - Tx-<STX><$98><LF><ESC><STX><FS><NUL>0<FS><FS><FS><FS><FS>Salame Milan pieza<FS>10000<FS>900000<FS>2100<FS><FS><FS><FS><FS>7901<FS>00<FS>7<ETX>0D96
  11.  

Other thing I needed back then was encoding numbers in a BCD string with a predetermined "endianness" (which could be the same or different from the host PC) and send them to a server. What I did was not optimal but it worked  :-[ .
I also had problem to get the raw value of a number as i t is in memory to a string (for instance: a Byte = 65 converted to string should be "A" if I need it in LE).

It is not a formal feature request since I think I'm not going to use such features in the near future. And I don't know if such things are already implemented.
I used such things to communicate with hardware such like VeriFone PINPads and fiscal printers like Hasar and Epson. Also to communicate with VISA servers (that project was halted or in stand by just before try to comply with IPC standard).-

Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: Thaddy on September 27, 2018, 05:18:30 pm
Hi Avra!!
        Good job!!!

It's a shame not knowing about it 5-7 years ago, it would saved me a lot of time and I would be using it for sure!!!
Five to seven years ago this would not be possible with this syntax..
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: lucamar on September 27, 2018, 05:57:48 pm
Very nice. But ... something of a misname, ToBooleanString, isn't it? Shouldn't have been p.e. ToBinaryString? Or I may be missing something, of course  :D
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: garlar27 on September 27, 2018, 07:58:48 pm
Very nice. But ... something of a misname, ToBooleanString, isn't it? Shouldn't have been p.e. ToBinaryString? Or I may be missing something, of course  :D
I agree, but I think people who knows or usually work with assembler might not think the same.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: AlexTP on September 27, 2018, 08:30:20 pm
New page:
http://wiki.freepascal.org/BitHelpers
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on September 28, 2018, 03:17:07 pm
Good job!!!
Thank you. You are very kind!  :D

The only cons I see from the examples (...) is the lack of configurable separators for nibbles and bytes in the ToBooleanString function...
Nice idea which will improve formatting customization. I don't promise nibbles separator, but I will put bytes separator on my TODO list.

...and configurable separators for each byte and "n" bytes in the ToHexString function.
I like the separator idea for each byte, but I am not sure that "n" bytes separator should end up in BitHelpers. My end goal is that BitHelpers become part of FreePascal. I do not know yet if more preferred way would be to include them as they are - or to embed them deeper with original root helpers for native types (then BitHelpers would not need to be included in unit's uses list at all). If BitHelpers get embedded more into FreePascal, then "n" bytes idea would be an overkill for that use. So, I will put only byte separator idea on my TODO list.

Other thing I needed back then was encoding numbers in a BCD string with a predetermined "endianness"
I feel that BCD conversion should not be part of BitHelpers, and that it should be done externally. I have also put quite an effort to keep "endianness" irrelevant for BitHelpers, and I would like to keep it that way: bits order is always with bit0 being least significant (look at Getbit method), and bytes/words/longwords are the same with bit0/word0/longword0 being least significant (look at type TByteOverlay, TWordOverlay, TLongwordOverlay and TQuadwordOverlay used by GetByte, GetWord and GetLongWord methods). Therefore I believe that BitHelpers should stay clean and endianess needs should be solved externally.

I also had problem to get the raw value of a number as i t is in memory to a string (for instance: a Byte = 65 converted to string should be "A" if I need it in LE)
Use type casting to character. I do not feel that something like ByteToChar belongs to BitHelpers so I will not add it.

Very nice. But ... something of a misname, ToBooleanString, isn't it? Shouldn't have been p.e. ToBinaryString?
You are probably right. Only booleans should have ToBooleanString, while other types should have ToBinaryString. If others agree, I will correct this.

New page:
http://wiki.freepascal.org/BitHelpers
Thank you for the effort!  :)
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: 440bx on September 28, 2018, 04:16:30 pm
You are probably right. Only booleans should have ToBooleanString, while other types should have ToBinaryString. If others agree, I will correct this.
Avra, I concur with everyone else, very nice job.   Thank you for your efforts and hard work.

I am also of the opinion that ToBinaryString would be more appropriate since it makes it clear there is no "logical" characteristic associated with the resulting string.

Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: garlar27 on September 28, 2018, 05:40:03 pm
@avra, I agree on your observations. Specially with keeping a clean code since a clean code is more maintainable.

I want to share some of my code. Hope someone find it useful. If some find a way to make it better or more flexible or fix something wrong, I'll be glad to know.  :D

The SniffBuffer (hex editor style) I use it log during development, mainly when I receive a long formatted string with fixed sized fields. I can count positions sizes and where the encoding is wrong and where de decoding didn't work as expected.
Code: Pascal  [Select][+][-]
  1.    function StrToHex(const S, Prefijo, Sufijo: string): string;
  2.    function StrToHex(S: string): string; {$IFnDEF D_BUG}inline;{$ENDIF}
  3.    function HexToStr(H: string): string;
  4.    function HexToBinStr(H: string): string; {$IFnDEF D_BUG}inline;{$ENDIF}
  5.    function ReplaceSpecialChars(const Buffer: string; const AStart, Prefix, Sufix, AnEnd: string; const ChangeEOL, ChangeTAB: Boolean;
  6.       const CharChangeOption: TCharChangeOption): string;
  7.    function ReplaceSpecialChars(const Buffer: string; const Prefix, Sufix: string; const ChangeEOLandTAB, ToHexa: Boolean): string;
  8.  
  9.    function BufferToHexa_ASCII(const Buffer: string; const HexPrefix, HexSufix: string; const SpaceAsPoint: Boolean = FALSE; const LinesOffset: Integer = 0): string;
  10.    function BuffToH_A(const ABuff: string; const LinesOffset: Integer = 0): string; {$IFnDEF D_BUG}inline;{$ENDIF}
  11.    function SniffBuffer(const ABuffer: string; const HexPrefix, HexSufix: string; const SpaceAsPoint: Boolean = FALSE; const WithTitle: Boolean = TRUE; const LineOffset: Integer = 0): string;
  12.    function SniffBuffer(const ABuffer: string): string;
  13.  
  14. implementation
  15.  
  16. {________________________________________________________________________________________________________________________________ StrToHex }
  17. function StrToHex(const S, Prefijo, Sufijo: string): string;
  18. var
  19.    i: Integer;
  20. begin
  21.    Result := '';
  22.    for i := 1 to Length(S) do begin
  23.       Result := Result + Prefijo + hexStr(Ord(S[i]), 2) + Sufijo;
  24.    end; {<--- del for i }
  25. end; {<--- StrToHex }
  26.  
  27.  
  28. {________________________________________________________________________________________________________________________________ StrToHex }
  29. function StrToHex(S: string): string;
  30. begin
  31.    Result := StrToHex(S, '', '');
  32. end; {<--- StrToHex }
  33.  
  34.  
  35. {________________________________________________________________________________________________________________________________ HexToStr }
  36. function HexToStr(H: string): string;
  37. var
  38.    i: Integer;
  39. begin
  40.    Result := '';
  41.    if odd(Length(H)) then begin
  42.       Exit;
  43.    end;
  44.    for i := 1 to Length(H) div 2 do begin
  45.       Result := Result + Chr(Hex2Dec(H[(i * 2) -1] + H[i * 2]));
  46.    end; {<--- del for i }
  47. end; {<--- HexToStr }
  48.  
  49.  
  50.  
  51. {_____________________________________________________________________________________________________________________________ HexToBinStr }
  52. function HexToBinStr(H: string): string;
  53. begin
  54.    Result := StrToBin(HexToStr(H));
  55. end; {<--- HexToBinStr }
  56.  
  57.  
  58.  
  59. (* http://www.aivosto.com/vbtips/control-characters.html
  60. Character list
  61. *)
  62. {_____________________________________________________________________________________________________________________ ReplaceSpecialChars }
  63. function ReplaceSpecialChars(const Buffer: string; const AStart, Prefix, Sufix, AnEnd: string; const ChangeEOL, ChangeTAB: Boolean;
  64.    const CharChangeOption: TCharChangeOption): string;
  65. var
  66.    i: Integer;
  67.    CambiarLF, CambiarCR: Boolean;
  68.  
  69.    function ToH(const s: Char): string;
  70.    begin
  71.       case s of
  72.           #9 : if ChangeTAB then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  73.          #10 : if CambiarLF  then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  74.          #13 : if CambiarCR  then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  75.          //#32..#126: Result := s;
  76.          #127: Result := StrToHex(s, AStart + Prefix, Sufix + AnEnd);
  77.       else
  78.          Result := s;
  79.          //Result := StrToHex(s, AStart + Prefix, Sufix + AnEnd);
  80.       end;
  81.    end;
  82.    function ToD(const s: Char): string;
  83.    begin
  84.       case s of
  85.           #9 : if ChangeTAB then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  86.          #10 : if CambiarLF  then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  87.          #13 : if CambiarCR  then Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd else Result:= s;
  88.          //#32..#126: Result := s;
  89.          #127: Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd;
  90.       else
  91.          Result := s;
  92.          //Result := AStart + Prefix + IntToStr(Byte(s)) + Sufix + AnEnd;
  93.       end;
  94.    end;
  95.    function ToRS232(const s: Char): string;
  96.    begin
  97.       case s of
  98.           #0 : Result := AStart + '<NUL>'  + AnEnd;
  99.           #1 : Result := AStart + '<SOH>'  + AnEnd;
  100.           #2 : Result := AStart + '<STX>'  + AnEnd;
  101.           #3 : Result := AStart + '<ETX>'  + AnEnd;
  102.           #4 : Result := AStart + '<EOT>'  + AnEnd;
  103.           #5 : Result := AStart + '<ENQ>'  + AnEnd;
  104.           #6 : Result := AStart + '<ACK>'  + AnEnd;
  105.           #7 : Result := AStart + '<BEL>'  + AnEnd;
  106.           #8 : Result := AStart + '<BS>'   + AnEnd;
  107.           #9 : if ChangeTAB then Result := AStart + '<HT>' + AnEnd else Result:= s;
  108.          #10 : if CambiarLF  then Result := AStart + '<LF>' + AnEnd else Result:= s;
  109.          #11 : Result := AStart + '<VT>'   + AnEnd;
  110.          #12 : Result := AStart + '<FF>'   + AnEnd;
  111.          #13 : if CambiarCR  then Result := AStart + '<CR>' + AnEnd else Result:= s;
  112.          #14 : Result := AStart + '<SO>'   + AnEnd;
  113.          #15 : Result := AStart + '<SI>'   + AnEnd;
  114.          #16 : Result := AStart + '<DLE>'  + AnEnd;
  115.          #17 : Result := AStart + '<DC1>'  + AnEnd;
  116.          #18 : Result := AStart + '<DC2>'  + AnEnd;
  117.          #19 : Result := AStart + '<CD3>'  + AnEnd;
  118.          #20 : Result := AStart + '<DC4>'  + AnEnd;
  119.          #21 : Result := AStart + '<NAK>'  + AnEnd;
  120.          #22 : Result := AStart + '<SYN>'  + AnEnd;
  121.          #23 : Result := AStart + '<ETB>'  + AnEnd;
  122.          #24 : Result := AStart + '<CAN>'  + AnEnd;
  123.          #25 : Result := AStart + '<EM>'   + AnEnd;
  124.          #26 : Result := AStart + '<SUB>'  + AnEnd;
  125.          #27 : Result := AStart + '<ESC>'  + AnEnd;
  126.          #28 : Result := AStart + '<FS>'   + AnEnd;
  127.          #29 : Result := AStart + '<GS>'   + AnEnd;
  128.          #30 : Result := AStart + '<RS>'   + AnEnd;
  129.          #31 : Result := AStart + '<US>'   + AnEnd;
  130.          //#32..#126: Result := s;
  131.          #127: Result := AStart + '<DEL>'  + AnEnd;
  132.          #128..
  133.          #255: Result := AStart + '<$'+StrToHex(s)+'>'  + AnEnd;
  134.          //#128: Result := AStart + '<PAD>'  + AnEnd;
  135.          //#129: Result := AStart + '<HOP>'  + AnEnd;
  136.          //#130: Result := AStart + '<BPH>'  + AnEnd;
  137.          //#131: Result := AStart + '<NBH>'  + AnEnd;
  138.       else
  139.          //Result := AStart + Prefix + IntToStr(Byte(s){, 2}) + Sufix + AnEnd;
  140.          Result := s;
  141.       end;
  142.    end;
  143. begin
  144.    Result := '';
  145.    {$IFDEF UNIX}
  146.    CambiarLF := CambiarEOL and TRUE;
  147.    CambiarCR := CambiarEOL and TRUE;
  148.    {$ELSE}
  149.    CambiarLF := ChangeEOL and TRUE;
  150.    CambiarCR := ChangeEOL and TRUE;
  151.    {$ENDIF}
  152.    case CharChangeOption of
  153.       ccoHex      : begin for i := 1 to Length(Buffer) do begin Result := Result + ToH(Buffer[i])    ; end; end;
  154.       ccoDec      : begin for i := 1 to Length(Buffer) do begin Result := Result + ToD(Buffer[i])    ; end; end;
  155.       ccoRS232Name: begin for i := 1 to Length(Buffer) do begin Result := Result + ToRS232(Buffer[i]); end; end;
  156.       ccoASCIIName: begin for i := 1 to Length(Buffer) do begin Result := Result + ToRS232(Buffer[i]); end; end;
  157.    else
  158.       raise Exception.Create('Error inesperado al cambiar cadena en ReplaceSpecialChars.');
  159.    end;
  160. end; {<--- ReplaceSpecialChars }
  161.  
  162.  
  163. {_____________________________________________________________________________________________________________________ ReplaceSpecialChars }
  164. function ReplaceSpecialChars(const Buffer: string; const Prefix, Sufix: string; const ChangeEOLandTAB, ToHexa: Boolean): string;
  165.    function ToH(const s: Char): string;
  166.    begin
  167.       Result := StrToHex(s, Prefix, Sufix);
  168.    end;
  169.    function ToD(const s: Char): string;
  170.    begin
  171.       Result := Prefix + IntToStr(Byte(s)) + Sufix;
  172.    end;
  173. begin
  174.    if ToHexa then begin
  175.       if ChangeEOLandTAB then begin
  176.          Result := StringsReplace(Buffer,
  177.             [#0,#1,#2,#3,#4,#5,#6,#7,#8,#9,#10,#11,#12,#13,#14,#15,#16,#17,#18,#19,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#30,#31,#127],
  178.             [ToH(#0),ToH(#1),ToH(#2),ToH(#3),ToH(#4),ToH(#5),ToH(#6),ToH(#7),ToH(#8),ToH(#9),ToH(#10),ToH(#11),ToH(#12),ToH(#13),ToH(#14),
  179.              ToH(#15),ToH(#16),ToH(#17),ToH(#18),ToH(#19),ToH(#20),ToH(#21),ToH(#22),ToH(#23),ToH(#24),ToH(#25),ToH(#26),ToH(#27),ToH(#28),ToH(#29),ToH(#30),ToH(#31),ToH(#127)],
  180.             [rfReplaceAll]);
  181.          end
  182.       else begin
  183.          Result := StringsReplace(Buffer,
  184.             [#0,#1,#2,#3,#4,#5,#6,#7,#8,#11,#12,#14,#15,#16,#17,#18,#19,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#30,#31,#127],
  185.             [ToH(#0),ToH(#1),ToH(#2),ToH(#3),ToH(#4),ToH(#5),ToH(#6),ToH(#7),ToH(#8),ToH(#11),ToH(#12),ToH(#14),ToH(#15),ToH(#16),
  186.              ToH(#17),ToH(#18),ToH(#19),ToH(#20),ToH(#21),ToH(#22),ToH(#23),ToH(#24),ToH(#25),ToH(#26),ToH(#27),ToH(#28),ToH(#29),ToH(#30),ToH(#31),ToH(#127)],
  187.             [rfReplaceAll]);
  188.       end;
  189.       end
  190.    else begin
  191.       if ChangeEOLandTAB then begin
  192.          Result := StringsReplace(Buffer,
  193.             [#0,#1,#2,#3,#4,#5,#6,#7,#8,#9,#10,#11,#12,#13,#14,#15,#16,#17,#18,#19,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#30,#31,#127],
  194.             [ToD(#0),ToD(#1),ToD(#2),ToD(#3),ToD(#4),ToD(#5),ToD(#6),ToD(#7),ToD(#8),ToD(#9),ToD(#10),ToD(#11),ToD(#12),ToD(#13),ToD(#14),
  195.              ToD(#15),ToD(#16),ToD(#17),ToD(#18),ToD(#19),ToD(#20),ToD(#21),ToD(#22),ToD(#23),ToD(#24),ToD(#25),ToD(#26),ToD(#27),ToD(#28),ToD(#29),ToD(#30),ToD(#31),ToD(#127)],
  196.             [rfReplaceAll]);
  197.          end
  198.       else begin
  199.          Result := StringsReplace(Buffer,
  200.             [#0,#1,#2,#3,#4,#5,#6,#7,#8,#11,#12,#14,#15,#16,#17,#18,#19,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#30,#31,#127],
  201.             [ToD(#0),ToD(#1),ToD(#2),ToD(#3),ToD(#4),ToD(#5),ToD(#6),ToD(#7),ToD(#8),ToD(#11),ToD(#12),ToD(#14),ToD(#15),ToD(#16),
  202.              ToD(#17),ToD(#18),ToD(#19),ToD(#20),ToD(#21),ToD(#22),ToD(#23),ToD(#24),ToD(#25),ToD(#26),ToD(#27),ToD(#28),ToD(#29),ToD(#30),ToD(#31),ToD(#127)],
  203.             [rfReplaceAll]);
  204.       end;
  205.    end;
  206. end; {<--- ReplaceSpecialChars }
  207.  
  208.  
  209. {______________________________________________________________________________________________________________________ BufferToHexa_ASCII }
  210. function BufferToHexa_ASCII(const Buffer: string; const HexPrefix, HexSufix: string; const SpaceAsPoint: Boolean; const LinesOffset: Integer): string;
  211. var
  212.    AsccStr, HexaStr, ln: string;
  213.    a: Char;
  214.    b: Byte;
  215.    i, os_i: Integer;
  216.    NonPrintable: set of Byte;
  217. begin
  218.    Result  := '';
  219.    AsccStr := '';
  220.    HexaStr := '';
  221.    ln      := '';
  222.    a       := #0;
  223.    os_i    := 0;
  224.    NonPrintable      := [0..31,127..255];
  225.    if SpaceAsPoint then begin
  226.       NonPrintable := NonPrintable + [32];
  227.    end;
  228.    for i := 1 to Length(Buffer) do begin
  229.       a := Buffer[i];
  230.       b := Byte(a);
  231.       if b in NonPrintable then begin
  232.          AsccStr := AsccStr + '.';
  233.          end
  234.       else begin
  235.          AsccStr := AsccStr + a;
  236.       end;
  237.       HexaStr := HexaStr + HexPrefix + IntToHex(b, 2) + HexSufix;//' ';
  238.       case Length(AsccStr) of
  239.          8: begin
  240.             HexaStr := HexaStr + ' ';
  241.             AsccStr := AsccStr + ' ';
  242.          end;
  243.          17: begin
  244.             Result := Result + ln + PadLeft(IntToHex(os_i, 8), 8) + '| |' + HexaStr + '| |' + AsccStr + '|';
  245.             ln := LineEnding + PadLeft('|', LinesOffset);
  246.             os_i := os_i +16;
  247.             HexaStr := '';
  248.             AsccStr := '';
  249.          end;
  250.       end;
  251.    end;//} {<--- del for i }
  252.    if HexaStr <> ''  then begin
  253.       Result := Result + ln + PadLeft(IntToHex(os_i, 8), 8) + '| |' + PadRight(HexaStr, 49) + '| |' + PadRight(AsccStr, 17) + '|';
  254.    end;
  255. end; {<--- BufferToHexa_ASCII }
  256.  
  257.  
  258. {_______________________________________________________________________________________________________________________________ BuffToH_A }
  259. function BuffToH_A(const ABuff: string; const LinesOffset: Integer): string;
  260. begin
  261.    Result := BufferToHexa_ASCII(ABuff, '', ' ', TRUE, LinesOffset);
  262. end; {<--- BuffToH_A }
  263.  
  264.  
  265. {_____________________________________________________________________________________________________________________________ SniffBuffer }
  266. function SniffBuffer(const ABuffer: string; const HexPrefix, HexSufix: string; const SpaceAsPoint: Boolean; const WithTitle: Boolean;
  267.    const LineOffset: Integer): string;
  268. var
  269.    t : Integer;
  270. begin
  271.    t := (2 + Length(HexPrefix) + Length(HexSufix)) * 8;
  272.    Result := BufferToHexa_ASCII(ABuffer, HexPrefix, HexSufix);
  273.    if (Result <> '') and WithTitle then begin
  274.  
  275.       Result := PadLeft('' , LineOffset) + PadRight(Format('| Buffer''s Length = %D', [Length(ABuffer)]), 81) + '|'+LineEnding
  276.               + PadLeft('' , LineOffset) + '| Offset | |0  .  .  .  4  .  .  .   8  .  .  .  C  .  .  F  | |0...4... 8...C..F|'+LineEnding
  277.               +                              '|--------| |'+ StringOfChar('-', t -1) + '||' + StringOfChar('-', t) + '| |-------- --------|'+LineEnding
  278.               + PadLeft('|', LineOffset) + Result + LineEnding
  279.               + '|=================================' + StringOfChar('=', t * 2 -1) + '|'+LineEnding;
  280.    end;
  281. end; {<--- SniffBuffer }
  282.  
  283.  
  284. {_____________________________________________________________________________________________________________________________ SniffBuffer }
  285. function SniffBuffer(const ABuffer: string): string;
  286. begin
  287.    if Pos('<?xml', ABuffer) = 1 then begin
  288.       Result := ABuffer;
  289.       end
  290.    else begin
  291.       Result := SniffBuffer(ABuffer, '', ' ', TRUE, TRUE, 0);
  292.    end;
  293. end; {<--- SniffBuffer }
  294.  


Here some routines I use to read/write BCD strings (IMPORTANT: This might not work well on every case, it just worked well with what I needed back then. It is not prepared to work with floats nor negative values and who knows what else is missing)

Code: Pascal  [Select][+][-]
  1.    function IntToBCDStr(const TheValue, TheDigits: QWord; const NonSignifficantZeros: Boolean = TRUE): string;
  2.    function BCDStrToBCD(const AValue: string): tBCD; {$IFnDEF D_BUG}inline;{$ENDIF}
  3.    function BCDStrToInt64(const AValue: string): Int64; {$IFnDEF D_BUG}inline;{$ENDIF}
  4.    function BCDStrToQWord(const AValue: string): QWord; {$IFnDEF D_BUG}inline;{$ENDIF}
  5.  
  6.    function StrToBCDStr(const AValue: string): string;
  7.    function BCDDigitLengthToByteLength(const Ln: Integer): Integer; {$IFnDEF D_BUG}inline;{$ENDIF}
  8.  
  9. implementation
  10.  
  11. {_____________________________________________________________________________________________________________________________ IntToBCDStr }
  12. function IntToBCDStr(const TheValue, TheDigits: QWord; const NonSignifficantZeros: Boolean = TRUE): string;
  13. var
  14.    ABcd: tBCD;
  15.    a{scii}: Byte;
  16.    i, d{igits}, b{ytes}, p{recision}, o{ffset}: Integer;
  17.    s: shortstring;
  18.    e: Int64;
  19. begin
  20.    Result := '';
  21.    ABcd := TheValue;
  22.    s := '';
  23.    o := 0;
  24.    b := 0;
  25.  
  26.    d := BCDDigitLengthToByteLength(TheDigits);
  27.    p := BCDDigitLengthToByteLength(ABcd.Precision);
  28.  
  29.    if odd(ABcd.Precision) then begin
  30.       o := 2;
  31.       b := 1;
  32.       e := trunc(power(10, ABcd.Precision + o));
  33.       ABcd := e + TheValue;
  34.       end
  35.    else begin
  36.       e := 0;
  37.    end;// del if odd(ABcd.Precision)
  38.  
  39.    s := '';
  40.  
  41.    for i := Low(ABcd.Fraction) +b{} to p -1 +b do begin
  42.       a := ABcd.Fraction[i];
  43.       s := s + Char(a);
  44.    end; {<--- del for i }
  45.  
  46.  
  47.    case CompareValue(d, p) of
  48.       -1: begin
  49.          Result := RightStr(s, d);
  50.       end;
  51.       00: begin
  52.          Result := s;
  53.       end;
  54.       01: begin
  55.          if NonSignifficantZeros then begin
  56.             Result := StringOfChar(#0, d - p) + s;
  57.             end
  58.          else begin
  59.             Result := s;
  60.          end;
  61.       end;
  62.    end;
  63. end; {<--- IntToBCDStr }
  64.  
  65.  
  66. {_____________________________________________________________________________________________________________________________ BCDStrToBCD }
  67. function BCDStrToBCD(const AValue: string): tBCD;
  68. //var i: Integer;
  69. begin
  70.    Result := BCDStrToInt64(AValue);//0;//
  71.    {for i := 1 to Length(AValue) do begin
  72.       Result.Fraction[i] := Byte(AValue[i]);
  73.    end;//}
  74. end; {<--- BCDStrToBCD }
  75.  
  76.  
  77. {___________________________________________________________________________________________________________________________ BCDStrToInt64 }
  78. function BCDStrToInt64(const AValue: string): Int64;
  79. begin
  80.    Result := 0;
  81.    Result := StrToInt64(StrToHex(AValue));
  82. end; {<--- BCDStrToInt64 }
  83.  
  84.  
  85. {___________________________________________________________________________________________________________________________ BCDStrToQWord }
  86. function BCDStrToQWord(const AValue: string): QWord;
  87. begin
  88.    Result := 0;
  89.    Result := StrToQWord(StrToHex(AValue));
  90. end; {<--- BCDStrToQWord }
  91.  
  92.  
  93. {_____________________________________________________________________________________________________________________________ StrToBCDStr }
  94. function StrToBCDStr(const AValue: string): string;
  95. var
  96.    i: Integer;
  97. begin
  98.    Result := '';
  99.    for i := 1 to Length(AValue) do begin
  100.       Result := Result + IntToBCDStr(Byte(AValue[i]), 2);
  101.    end; {<--- del for i }
  102. end; {<--- StrToBCDStr }
  103.  
  104.  
  105. {_________________________________________________________________________________________________________________ BCDDigitLengthToByteLength }
  106. function BCDDigitLengthToByteLength(const Ln: Integer): Integer;
  107. begin
  108.    if odd(Ln) then begin
  109.       Result := (Ln + 1) div 2;
  110.       end
  111.    else begin
  112.       Result := Ln div 2;
  113.    end;
  114. end; {<--- BCDDigitLengthToByteLength }
  115.  

Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: VTwin on September 29, 2018, 02:34:27 am
Very nice, thanks! I had not considered such options for basic type helpers, good to know they are available.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: AlexTP on September 30, 2018, 09:16:44 am
@author
feel free to edit Wiki text with new examples.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on October 01, 2018, 09:41:36 am
Very nice, thanks! I had not considered such options for basic type helpers, good to know they are available.
I am glad you like BitHelpers. It all started with a need to present the state of a boolean in my language, and to manipulate some bits inside of a word.

feel free to edit Wiki text with new examples.
After BitHelpers update I will. Thanks again!
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on November 19, 2018, 12:15:47 pm
BitHelpers updated to 1.1.0.7

New feature:
Binary string format capability as promissed.

History:
1.0.0.0 First public version.
1.1.0.7 Renamed ToBooleanString() method into ToBinString(). It is more correct and more consistent naming in line with ToHexString() in SysUtils. Added DefaultHelperFormatSettings record, used by ToBinString() to put prefix, suffix, and separator strings for nibbles, bytes, words and longwords in result binary string (idea was borrowed from DefaultFormatSettings in SysUtils date and time format settings). Added optional THelperFormatSettings record parameter for ToBinString(), when temporary sufix/prefix/separators are preferred over changing global DefaultHelperFormatSettings. Expanded demo project with demonstration of using both default and temporary format settings in ToBinString().

Example for temporary format:
Code: Pascal  [Select][+][-]
  1.   procedure TForm1.TmpFormatBtnClick(Sender: TObject);
  2.   var
  3.     MyQuadWord: qword;
  4.     MyTmpFormat: THelperFormatSettings;
  5.   begin
  6.     MyQuadWord := 1125899940409401;
  7.  
  8.     MyTmpFormat.PrefixString      := '< ';
  9.     MyTmpFormat.SufixString       := ' >';
  10.     MyTmpFormat.ByteSeparator     := ' - ';
  11.     MyTmpFormat.WordSeparator     := ' = ';
  12.     MyTmpFormat.LongwordSeparator := '   ';
  13.     MyTmpFormat.NibbleSeparator   := '.';
  14.  
  15.     Memo1.Append('MyQuadword.ToBinString(MyTmpFormat, true);');
  16.     Memo1.Append(LineEnding + MyQuadword.ToBinString(MyTmpFormat, true)); // true is default so it can be ommited and leading zeros will still be shown
  17.     Memo1.Append('MyQuadword.Longword[1].ToBinString(MyTmpFormat);');
  18.     Memo1.Append(LineEnding + MyQuadword.Longword[1].ToBinString(MyTmpFormat)); // default true is ommited, show higher longword with leading zeros
  19.     Memo1.Append('MyQuadword.Word[3].ToBinString(MyTmpFormat, false);');
  20.     Memo1.Append(LineEnding + MyQuadword.Word[3].ToBinString(MyTmpFormat, false)); // show highest word without leading zeros
  21. end;
gives this output:
Quote
  MyQuadword.ToBinString(MyTmpFormat, true);
  < 0000.0000 - 0000.0100 = 0000.0000 - 0000.0000   0000.0010 - 0000.0000 = 0011.0000 - 0011.1001 >

  MyQuadword.Longword[1].ToBinString(MyTmpFormat);
  < 0000.0000 - 0000.0100 = 0000.0000 - 0000.0000 >

  MyQuadword.Word[3].ToBinString(MyTmpFormat, false);
  < 100 >

and example for permanent format:
Code: Pascal  [Select][+][-]
  1.   procedure TForm1.ChangeFormatBtnClick(Sender: TObject);
  2.   var
  3.     MyQuadWord: qword;
  4.   begin
  5.     MyQuadWord := 1125899940409401;
  6.  
  7.     // all DefaultHelperFormatSettings fields are empty strings by default, so we change them here:
  8.     DefaultHelperFormatSettings.PrefixString      := '{';
  9.     DefaultHelperFormatSettings.SufixString       := '}';
  10.     DefaultHelperFormatSettings.NibbleSeparator   := '_';
  11.     DefaultHelperFormatSettings.ByteSeparator     := '  ';
  12.     DefaultHelperFormatSettings.WordSeparator     := '  ' + Chr(39) + '  '; // single quote
  13.     DefaultHelperFormatSettings.LongwordSeparator := '  "  ';
  14.  
  15.     Memo1.Append(LineEnding + MyQuadword.ToBinString(true));
  16.     Memo1.Append(LineEnding + MyQuadword.Longword[1].ToBinString);
  17.     Memo1.Append(LineEnding + MyQuadword.Word[3].ToBinString(false));
  18.   end;
gives this output:
Quote
  {0000_0000  0000_0100  '  0000_0000  0000_0000  "  0000_0010  0000_0000  '  0011_0000  0011_1001}

  {0000_0000  0000_0100  '  0000_0000  0000_0000}

  {100}

Idea for DefaultHelperFormatSettings and THelperFormatSettings was borrowed from DefaultFormatSettings and TFormatSettings in SysUtils, something you should already be familiar with. Also, if BitHelpers get embedded into FPC then the same settings could be used for ToHexString() which would then be expanded to also support custom formatting.

EDIT: Just updated Wiki page.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on June 17, 2021, 08:21:50 am
BitHelpers updated to 1.2.0.11

New feature:
Added HighestBitPos(), LowestBitPos() and BitsCount().

History:
1.2.0.11 Added HighestBitPos(), LowestBitPos() and BitsCount().
1.1.0.7 Renamed ToBooleanString() method into ToBinString(). It is more correct and more consistent naming in line with ToHexString() in SysUtils. Added DefaultHelperFormatSettings record, used by ToBinString() to put prefix, suffix, and separator strings for nibbles, bytes, words and longwords in result binary string (idea was borrowed from DefaultFormatSettings in SysUtils date and time format settings). Added optional THelperFormatSettings record parameter for ToBinString(), when temporary sufix/prefix/separators are preferred over changing global DefaultHelperFormatSettings. Expanded demo project with demonstration of using both default and temporary format settings in ToBinString().
1.0.0.0 First public version.

Example code:
Code: Pascal  [Select][+][-]
  1.   procedure TForm1.BitsPosAndCountClick(Sender: TObject);
  2.   var
  3.     MyQuadWord: qword;
  4.   begin
  5.     MyQuadWord := $7F00000000000064; // 0111111100000000000000000000000000000000000000000000000001100100
  6.     Memo1.Append(MyQuadword.ToBinString);
  7.     Memo1.Append('MyQuadword.HighestBitPos = ' + MyQuadword.HighestBitPos.ToString);
  8.     Memo1.Append('MyQuadword.LowestBitPos = ' + MyQuadword.LowestBitPos.ToString);
  9.     Memo1.Append('MyQuadword.BitsCount = ' + MyQuadword.BitsCount.ToString);
  10.     Memo1.Append('');
  11.   end;

gives this output:
Quote
  0111111100000000000000000000000000000000000000000000000001100100
  MyQuadword.HighestBitPos = 62
  MyQuadword.LowestBitPos = 2
  MyQuadword.BitsCount = 10
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: schuler on June 24, 2021, 06:43:33 pm
Looks very good.

Wondering if this should become a FPC standard...
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on June 25, 2021, 01:14:48 am
Looks very good.
Thanks. I'm glad you like it.  :D

Wondering if this should become a FPC standard...
Original idea was that once BitHelpers is feature complete and well tested, it becomes a FPC package. Maybe that time has come. We'll see how that goes...
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: Thaddy on June 25, 2021, 09:46:23 am
Note there is overlap with the standard Bithelpers that come with FPC3.2.0 and up, as you know. Maybe we should look at integration.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on June 25, 2021, 12:15:57 pm
Note there is overlap with the standard Bithelpers that come with FPC3.2.0 and up, as you know. Maybe we should look at integration.
Nothing was mentioned about it at https://wiki.freepascal.org/FPC_New_Features_3.2.0 so I thought it was still in trunk only. Therefore I have compared 3.0 and 3.2 sources, and I see that standard type helpers have been extended for bit access with SetBit(), ClearBit(), ToggleBit() and TestBit(). I have a feeling that if we add all methods from my bithelpers to standard types then it would be overcrowded. I don't mind doing it, but although bit manipulation is very important - out of embedded world it is actually not used that often to justify it's place in standard type default helpers, so I would prefer to see it in a separate FPC package and just add bithelpers unit to uses code section when needed. What do you think?

In latest update I have also moved away from standard SHL based bit getting/setting to precalculated BitValueLookupTable[] for speed optimization, but did not make any benchmarks yet to see if there was actual benefit (SHL should be very fast in modern processors). Patches should be made only after benchmarking decision.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: Thaddy on June 25, 2021, 02:35:20 pm
I have a feeling that if we add all methods from my bithelpers to standard types then it would be overcrowded. I don't mind doing it, but although bit manipulation is very important - out of embedded world it is actually not used that often to justify it's place in standard type default helpers, so I would prefer to see it in a separate FPC package and just add bithelpers unit to uses code section when needed. What do you think?
Well, several points:
- I wrote that code with some great feedback and tests by Bart, but I should probably also write a request for documentation (and a proposal documentation to go with it)
- Apart from embedded, it is is useful in several ways, especially Database application, e.g. SQL is based on bit tables and it is highly useful for AI as well. (schuler likes your code not for nothing)
- I wrote it initially because I needed it for electronics on Raspberry Pi and the likes.
- I still think we should avoid overlaps, but lets discuss just the candidates that really warrant inclusion in my four.
Quote
In latest update I have also moved away from standard SHL based bit getting/setting to precalculated BitValueLookupTable[] for speed optimization, but did not make any benchmarks yet to see if there was actual benefit (SHL should be very fast in modern processors). Patches should be made only after benchmarking decision.
I agree. Note I have some updates for implemention of ROL/ROR SAR etc, which is also useful.

About speed: that is indeed highly CPU dependent. Lets come up with some tests.

I really appreciate your contribution! It is a nice library.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on June 25, 2021, 03:41:33 pm
Apart from embedded, it is useful in several ways, especially Database application, e.g. SQL is based on bit tables and it is highly useful for AI as well.
So you prefer adding to syshelp*.inc instead of adding a separate package? If other devs also support that idea then I could do that (if there is a devs resistance to that idea then I would more prefer rejection before then after doing all the work).

- I still think we should avoid overlaps, but lets discuss just the candidates that really warrant inclusion in my four.
What overlaps? I do not have your ToggleBit() and ClearBit() at all, and instead of your TestBit() and SetBit() I use array like access property Bit[] which I like very much and want to keep. Maybe I can change names of that property's getter and setter, since that is the only place where I see an overlap (or to be more precise a method name conflict). If preferred, besides renaming I might also hide those methods...

I really appreciate your contribution! It is a nice library.
You're very kind. Thanks!
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on July 12, 2021, 01:04:55 pm
I have run some simple benchmarks (maybe too simple, but anyway) on x86 Win64, and shift based bit setting was faster 3% if variable was qword, and 10% if variable was word. So I will give up table based bit access and avoid spending 64 qwords for a lookup table. However inlining brings significant speed improvement in loops so I will leave it on.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on January 19, 2022, 11:43:01 am
I am so glad to announce that BitHelpers are now part of trunk FPC RTL. Type helpers for direct access to bit, nibble, byte, word and dword parts in ordinal types are now part of sysutils unit, and customizable boolean, binary and hexadecimal data localized string representation is now part of syshelpers unit. Syntax has been slightly changed to be more in line with FPC (using plurals like SomeOrdType.Bits[3] instead of old SomeOrdType.Bit[3] and similar).

More details here:
https://gitlab.com/freepascal.org/fpc/source/-/issues/39268
https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39267

From now on, all further development will be in FPC trunk. Bitbucket repo will continue to exist only for historical reasons and old code.

This is the output from new console demo:
Code: [Select]
True.ToString                                          = True
True.ToTrueFalseString(scfUpperCase)                   = TRUE
True.ToString('OnState', 'OffState')                   = OnState
True.ToString('Running', 'Stopped', scfUpperCase)      = RUNNING
True.ToString('Ради', 'Не ради', scfUpperCase)         = Ради (console uppers ASCII only, gui uppers unicode as well)
True.ToString(MyBitFormatSettings)                     = MyTrue
MyNativeInt                                            = -10000
MyNativeInt.ToBinString                                = 11111111111111111101100011110000
MyNativeInt.Bits[MyNativeInt.MaxBit].ToTrueFalseString = True
MyDword.ToBinString          = 00000000000111000000001001001010
MyDword.ToBinString(true)    = 00000000000111000000001001001010
MyDword.ToBinString(false)   = 111000000001001001010
MyDword.Bytes[0].ToBinString = 0100_1010   (BinNibbleSeparator = "_")
MyDword.HighestSetBitPos     = 20
MyDword.LowestSetBitPos      = 1
MyDword.SetBitsCount         = 7
MyDword.ToHexString          = 001C024A
MyDword.ToHexString(7)       = 01C024A
MyDword.ToHexString(false)   = 1C024A

DefaultBitFormatSettings:
FALSE : False, False, 0, Off
TRUE  : True, True, 1, On
MyDword.Bits[5].ToTrueFalseString = False (BitTrueString = True)
MyDword.Bits[5].ToString          = False
TryStrToBool('trUE', MyBool)      = TRUE
BoolToStr(true, true)             = True, TrueBoolStrs[0] = True

CustomBitFormatSettings:
FALSE : Неистина, Неистина, O, Искључено
TRUE  : Истина, Истина, X, Укључено
MyDword.Bits[5].ToTrueFalseString = Неистина (BitTrueString = Истина)
MyDword.Bits[5].ToString          = Неистина
TryStrToBool('Истина', MyBool)    = Истина, (MyBool = TRUE)
TryStrToBool('ИСТИна', MyBool)    = Неистина (console uppers ASCII only, gui uppers unicode as well)
BoolToStr(true, true)             = Истина, TrueBoolStrs[0] = Истина
True.ToString                     = Истина

DefaultBinFormatSettings:
MyQword.ToBinString              = 0000000000000000000010101011110011011110111111111111111111111111
MyQword.ToBinString(false)       =                     10101011110011011110111111111111111111111111
MyQword.ToBinString(50)          =               00000010101011110011011110111111111111111111111111

CustomBinFormatSettings:
MyQword.ToBinString       = OOOOOOOO.OOOOOOOO-OOOOXOXO.XOXXXXOO_XXOXXXXO.XXXXXXXX-XXXXXXXX.XXXXXXXX
MyQword.ToBinString(false)       =                XOXO.XOXXXXOO_XXOXXXXO.XXXXXXXX-XXXXXXXX.XXXXXXXX
MyQword.ToBinString(50)          =         OO-OOOOXOXO.XOXXXXOO_XXOXXXXO.XXXXXXXX-XXXXXXXX.XXXXXXXX

DefaultHexFormatSettings:
MyQword.ToHexString                             = 00000ABCDEFFFFFF
MyQword.ToHexString(3)                          =      ABCDEFFFFFF
MyQword.ToHexString(13)                         =    00ABCDEFFFFFF
MyQword.ToHexString(true)                       = 00000ABCDEFFFFFF
MyQword.ToHexString(false)                      =      ABCDEFFFFFF

CustomHexFormatSettings1:
MyQword.ToHexString                             = [$0.0^0.0_0.A^B.C--D.E^F.F_F.F^F.F]
MyQword.ToHexString(true)                       = [$0.0^0.0_0.A^B.C--D.E^F.F_F.F^F.F]
MyQword.ToHexString(false)                      =           [$A^B.C--D.E^F.F_F.F^F.F]

CustomHexFormatSettings2:
MyQword.ToHexString(MyHexFormatSettings)        = $0000:0ABC:DEFF:FFFF
MyQword.ToHexString(MyHexFormatSettings, false) =       $ABC:DEFF:FFFF
MyQword.ToHexString(MyHexFormatSettings, 13)    =    $0:0ABC:DEFF:FFFF

Have fun!  :D 8-) :D
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: mischi on January 19, 2022, 11:48:08 am
Nice. Great work.👍
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: Thaddy on January 19, 2022, 12:49:51 pm
I am so glad to announce that BitHelpers are now part of trunk FPC RTL.
I am also glad that you improved my original code (that was already accepted). Thanks a lot. We had a little communication and everything you added is very useful.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on January 19, 2022, 01:13:55 pm
Nice. Great work.👍
Thanks!  ::)

I am so glad to announce that BitHelpers are now part of trunk FPC RTL.
I am also glad that you improved my original code (that was already accepted). Thanks a lot. We had a little communication and everything you added is very useful.
I'm glad you like it. Thank you for your kind words!  O:-)
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on April 27, 2022, 01:12:25 pm
I am glad to announce that documentation and console example have been added to the latest FPC trunk.

https://gitlab.com/freepascal.org/fpc/source/-/tree/main/packages/rtl-objpas/examples
https://gitlab.com/freepascal.org/fpc/documentation/-/blob/main/xml/syshelpers.xml
https://gitlab.com/freepascal.org/fpc/documentation/-/blob/main/xml/sysutils.xml

GUI example does not belong to FPC so it will remain available at https://gitlab.com/freepascal.org/fpc/source/-/issues/39541 as file attachment 20220120-3-examples.zip.

This concludes migration of bithelpers to FPC trunk. Of course, you may continue to shoot related questions here.

Enjoy!  :D
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: WayneSherman on May 01, 2022, 02:42:41 am
Thank you for your work on these and getting them merged into fpc.

The bit functions such as ClearBit, SetBit, and TestBit take bit index values, but they don't work with bit mask values.  It would be nice to add bit functions that work with masks as well since many APIs and libraries use them.  Something like this:

Code: Pascal  [Select][+][-]
  1. FLAG1 = $01;
  2. FLAG2 = $02;
  3. FLAG3 = $04;
  4. FLAG4 = $08;
  5. ...
  6.  
  7. MyFlags.ClearFlags(FLAG1 or FLAG3);
  8. MyFlags.SetFlags(FLAG2);
  9. if MyFlags.HasAnyFlags(FLAG1 or FLAG4) then...
  10. if MyFlags.HasAllFlags(FLAG1 or FLAG4) then...

File attributes on Windows are defined like this:
Code: Pascal  [Select][+][-]
  1.      FILE_ATTRIBUTE_READONLY             = $0000001;
  2.      FILE_ATTRIBUTE_HIDDEN               = $0000002;
  3.      FILE_ATTRIBUTE_SYSTEM               = $0000004;
  4.      FILE_ATTRIBUTE_DIRECTORY            = $0000010;
  5.      FILE_ATTRIBUTE_ARCHIVE              = $0000020;
  6.      FILE_ATTRIBUTE_DEVICE               = $0000040;
  7.      FILE_ATTRIBUTE_NORMAL               = $0000080;
  8.  
  9. //so you could do this
  10. if MyFileAttr.HasFlags(FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM) then

I recently had to do this:
Code: Pascal  [Select][+][-]
  1.   if GetConsoleMode(TextRec(Input).Handle, ConsoleMode) then begin
  2.     ConsoleMode := ConsoleMode and not ENABLE_QUICK_EDIT_MODE;
  3.     SetConsoleMode(TextRec(Input).Handle, ConsoleMode);
  4.   end;

But would have loved to do this:
Code: Pascal  [Select][+][-]
  1.   if GetConsoleMode(TextRec(Input).Handle, ConsoleMode) then
  2.       SetConsoleMode(TextRec(Input).Handle, ConsoleMode.ClearFlags(ENABLE_QUICK_EDIT_MODE));

Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: WayneSherman on May 01, 2022, 09:52:42 am
I haven't decided if I like "Mask" or "Flag":

Code: Pascal  [Select][+][-]
  1. MyFlag.ClearMask(FLAG1);
  2. MyFlag.SetMask(FLAG3 or FLAG4);
  3. if MyFlags.TestMask(FLAG1 or FLAG3) then...
  4. if MyFlags.TestMaskAll(FLAG1 or FLAG3) then...
  5. if MyFlags.TestMaskAny(FLAG1 or FLAG3) then...

Also, we have a Clear operation, but I did not see a helper to set all bits:
Code: Pascal  [Select][+][-]
  1. MyFlag.Clear;  //not a function, operates on the variable directly
  2. MyFlag.SetAllBits;  //not a function, operates on the variable directly
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: 440bx on May 01, 2022, 09:56:36 am
I haven't decided if I like "Mask" or "Flag":
I "vote" for "Mask"  because it's more descriptive than (not as generic as) "Flag" and also because it's consistent with the helpers' names.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: WayneSherman on May 01, 2022, 05:13:13 pm
or maybe add the word "Bits":

Code: Pascal  [Select][+][-]
  1.     MyFlag.ClearMaskBits(FLAG1);
  2.     MyFlag.SetMaskBits(FLAG3 or FLAG4);
  3.     if MyFlags.TestMaskBits(FLAG1 or FLAG3) then...  //all bits must match for true
  4.  
  5.    //is this even needed?
  6.     if MyFlags.TestMaskAny(FLAG1 or FLAG3) then...  //any bit must match for true

Maybe this was discussed, but the way helpers look, using a dot operator, leads one to believe the variable itself gets modified.  Should a helper like this be written as a function or procedure?

Code: Pascal  [Select][+][-]
  1. MyFlag.ClearMaskBits(FLAG1);  //not a function, operate on the variable directly
  2. //versus
  3. MyFlag := MyFlag.ClearMaskBits(FLAG1);  //a function, MyFlag is not changed by the helper

For example, a record or object generally are expected to operate on their own data:
Code: Pascal  [Select][+][-]
  1. MyStrRec.SetValue('hello world'); //would be expected to change the record variable
  2. MyObject.SetValue(True); //would be expected to modify data in the object.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: 440bx on May 01, 2022, 05:59:30 pm
or maybe add the word "Bits":
A mask is a bit structure, because of that, adding "Bits" feels redundant and increases the amount of typing. I think you had it short and concise to begin with. If I were you, I'd keep the way it was.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: marcov on May 01, 2022, 06:17:56 pm
or maybe add the word "Bits":
A mask is a bit structure, because of that, adding "Bits" feels redundant and increases the amount of typing. I think you had it short and concise to begin with. If I were you, I'd keep the way it was.

He probably is a PICian, like me :-) If you are used to Microchip headers, that feels natural :-)
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: avra on May 02, 2022, 12:02:53 pm
Thank you for your work on these and getting them merged into fpc.
You're most welcome. It was a pleasure, not work :D ::) :D

The bit functions such as ClearBit, SetBit, and TestBit take bit index values, but they don't work with bit mask values.  It would be nice to add bit functions that work with masks as well since many APIs and libraries use them.
Devs are very picky (rightfully) with what goes into sysutils, so you should better discuss with them in the mail list before doing any real work. My wild guess would be that if your patches get accepted, they would need to be in some other unit.
Title: Re: BitHelpers - bit manipulation for standard pascal types
Post by: mas steindorff on May 02, 2022, 09:53:37 pm
I haven't decided if I like "Mask" or "Flag":
also coming from the hardware world, a Flag is a single bit, Mask = several bits or flag(s). mask is what you do to select one to several flags (bits).
TinyPortal © 2005-2018