[SOLVED] Bitpacked record of Boolean flags — problem with reading flag states

[furious programming]

Can someone tell me why the output is the way it is?

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}{$LONGSTRINGS ON}
2.  
3. type
4.   TStatus = bitpacked record
5.     Up: Boolean;
6.     Down: Boolean;
7.     Left: Boolean;
8.     Right: Boolean;
9.     Select: Boolean;
10.     Start: Boolean;
11.     B: Boolean;
12.     A: Boolean;
13.   end;
14.  
15. var
16.   Status: TStatus;
17.   StatusInt: UInt8 absolute Status;
18. begin
19.   Status.Up := True;
20.   Status.Down := True;
21.   Status.Left := True;
22.   Status.Right := True;
23.   Status.Select := True;
24.   Status.Start := True;
25.   Status.B := True;
26.   Status.A := True;
27.  
28.   WriteLn('Status: ', BinStr(StatusInt, 8), LineEnding);
29.  
30.   WriteLn('Up:     ', StatusInt and &00000001 <> 0);
31.   WriteLn('Down:   ', StatusInt and &00000010 <> 0);
32.   WriteLn('Left:   ', StatusInt and &00000100 <> 0);
33.   WriteLn('Right:  ', StatusInt and &00001000 <> 0);
34.   WriteLn('Select: ', StatusInt and &00010000 <> 0);
35.   WriteLn('Start:  ', StatusInt and &00100000 <> 0);
36.   WriteLn('B:      ', StatusInt and &01000000 <> 0);
37.   WriteLn('A:      ', StatusInt and &10000000 <> 0);
38.  
39.   ReadLn();
40. end.

Output:

Code: Pascal  [Select][+][-]

1. Status: 11111111
2.  
3. Up:     TRUE
4. Down:   TRUE
5. Left:   TRUE
6. Right:  FALSE
7. Select: FALSE
8. Start:  FALSE
9. B:      FALSE
10. A:      FALSE

I feel like an idiot.

[Jonas Maebe]

& is octal, % is binary: https://freepascal.org/docs-html/ref/refse6.html

[MathMan]

First I admit that I didn't look up anything prior to this replay, but ...

... if ampersand '&' declares octal numbers this would easily explain it.

What happens if you declare with percent '%'?

Cheers,
MathMan

ah - beaten by Jonas, by a fraction

[Jonas Maebe]

Also keep in mind that the bit order of bitpacked records/arrays is different for big and little endian systems so if you want to be portable, only use the record/array types and not the underlying values directly.

[furious programming]

So I'm an idiot. Thanks for the replies…

Also keep in mind that the bit order of bitpacked records/arrays is different for big and little endian systems so if you want to be portable, only use the record/array types and not the underlying values directly.

AFAIK endianness determines the order of bytes, not bits, so using such a record is safe. I need such a packed type, because I'll have tens of thousands of such records in the stream. By packing eight logical values into one byte, the size of this data in the stream will be eight times smaller, and in my case it matters a lot.

[Blaazen]

AFAIK endianness determines the order of bytes, not bits, so using such a record is safe.

Until you add the ninth boolean to the record.

[Jonas Maebe]

So I'm an idiot. Thanks for the replies…

Also keep in mind that the bit order of bitpacked records/arrays is different for big and little endian systems so if you want to be portable, only use the record/array types and not the underlying values directly.

AFAIK endianness determines the order of bytes, not bits, so using such a record is safe.

In general this is true, but bitpacked records/arrays also use a different bit order on big/little endian systems. The reason is that otherwise accessing bitpacked data changes the bit number depending on the size of the access (1/2/4/8 byte).

From the compiler source:

  (*
    Subsetrefs are used for (bit)packed arrays and (bit)packed records stored
    in memory. They are like a regular reference, but contain an extra bit
    offset (either constant -startbit- or variable -bitindexreg-, always OS_INT)
    and a bit length (always constant).

    Bit packed values are stored differently in memory depending on whether we
    are on a big or a little endian system (compatible with at least GPC). The
    size of the basic working unit is always the smallest power-of-2 byte size
    which can contain the bit value (so 1..8 bits -> 1 byte, 9..16 bits -> 2
    bytes, 17..32 bits -> 4 bytes etc).

    On a big endian, 5-bit: values are stored like this:
      11111222 22333334 44445555 56666677 77788888
    The leftmost bit of each 5-bit value corresponds to the most significant
    bit.

    On little endian, it goes like this:
      22211111 43333322 55554444 77666665 88888777
    In this case, per byte the left-most bit is more significant than those on
    the right, but the bits in the next byte are all more significant than
    those in the previous byte (e.g., the 222 in the first byte are the low
    three bits of that value, while the 22 in the second byte are the upper
    two bits.

    Big endian, 9 bit values:
      11111111 12222222 22333333 33344444 ...

    Little endian, 9 bit values:
      11111111 22222221 33333322 44444333 ...
    This is memory representation and the 16 bit values are byteswapped.
    Similarly as in the previous case, the 2222222 string contains the lower
    bits of value 2 and the 22 string contains the upper bits. Once loaded into
    registers (two 16 bit registers in the current implementation, although a
    single 32 bit register would be possible too, in particular if 32 bit
    alignment can be guaranteed), this becomes:
      22222221 11111111 44444333 33333322 ...
      (l)ow  u     l     l    u     l   u

    The startbit/bitindex in a subsetreference always refers to
    a) on big endian: the most significant bit of the value
       (bits counted from left to right, both memory an registers)
    b) on little endian: the least significant bit when the value
       is loaded in a register (bit counted from right to left)

    Although a) results in more complex code for big endian systems, it's
    needed for compatibility both with GPC and with e.g. bitpacked arrays in
    Apple's universal interfaces which depend on these layout differences).

    Note: when changing the loadsize calculated in get_subsetref_load_info,
    make sure the appropriate alignment is guaranteed, at least in case of
    {$defined cpurequiresproperalignment}.
  *)

[furious programming]

Until you add the ninth boolean to the record.

I will never do this, because NES controller had only 8 buttons.

In general this is true, but bitpacked records/arrays also use a different bit order on big/little endian systems.

Yes, but this only applies to bitpacked types whose size is greater than one byte, or data types whose units cannot be aligned to 8 bits. Therefore, 5-bit and 9-bit values are given in this comment and therefore the indexes of the bits in the units do not matter (therefore, the unit bit indexes are not specified).

In my case, the data packet is always 8 bits long, so endianness does not concern me.

[Jonas Maebe]

On little endian, the first bit of the bitpacked record is at 1 shl 0.

On big endian, the first bit of the bitpacked record is at 1 shl 7.

[winni]

On little endian, the first bit of the bitpacked record is at 1 shl 0.

On big endian, the first bit of the bitpacked record is at 1 shl 7.

Yes. Another step back.

On UCSD-machines it was always the same.
No matter if you worked on an Apple II or a PC.
I am talking about 1980.

Winni

[Jonas Maebe]

On little endian, the first bit of the bitpacked record is at 1 shl 0.

On big endian, the first bit of the bitpacked record is at 1 shl 7.

Yes. Another step back.

On UCSD-machines it was always the same.
No matter if you worked on an Apple II or a PC.
I am talking about 1980.

Winni

The downside of having it the same everywhere is that the bit numbering changes depending on the access size. You then always have to access everything byte by byte on either big or little endian (depending on which bit order you pick) while with the current approach (which is defined by all ABIs I know of) allows operations to be performed using a single 16/32/64 bit access when applicable.

[winni]

[
The downside of having it the same everywhere is that the bit numbering changes depending on the access size. You then always have to access everything byte by byte on either big or little endian (depending on which bit order you pick) while with the current approach (which is defined by all ABIs I know of) allows operations to be performed using a single 16/32/64 bit access when applicable.

Hi!

Already clear.

The reason was the p-machine and the ability to run the p-code-executable on any p-machine: No matter what byte-sex.  So you could run an executable compiled on an Apple II on a PC without any problems. That's the reason why Java has "stolen" the p-code.

Winni

[furious programming]

On little endian, the first bit of the bitpacked record is at 1 shl 0.
On big endian, the first bit of the bitpacked record is at 1 shl 7.

Nevermind. Anyway, this code is supposed to work only on Windows, so I'm not going to mess too much.

This is interesting to me. I am creating a simple file format that allows me to store a recording of Tetris gameplay. The format provides for storing data in binary form, as a set of frames in uncompressed form (each frame is a key frame) and compressed (so that the file on the disk takes up as little space as possible, no key frames).

Each keyframe must contain, inter alia, controller state (eight buttons), which I want to represent as a bitpacked record with a size of 8 bits (1 byte), and the state of the game field in the form of a bitpacked array, in which each cell must contain one of the four types of cubes (empty, color A, B or C) — the playing field is 10×20 in size, which is 400 bits (50 bytes).

From what you write, it appears that if I send someone a binary file containing a gameplay recording (a recording made on my computer), to someone who has a different endianness, the program will not be able to correctly read the data from this file. I understand well?

[PascalDragon]

From what you write, it appears that if I send someone a binary file containing a gameplay recording (a recording made on my computer), to someone who has a different endianness, the program will not be able to correctly read the data from this file. I understand well?

Correct. You'll need to pick one bitness where you then do a bitswap upon reading the corresponding data.

[furious programming]

Is there such a need if the software is to run exclusively on Windows?

Theoretically, only me will use this program, but I would prefer it not to crash in the future when I change my computer to a newer/different one.