Status of FPC 3.4.0 or FPC 4.0.0 [major release]

[440bx]

given FPC 3.2.2 has been in use for the last 4.5 years without major issue, is there any need for a new release?

In addition to what Davo mentioned above, v3.2,2 has some annoying bugs which prevent writing code as it should be and make carrying out some operations quite a bit more complicated than they should be.

An example of that is the inability to define bit fields as they appear in the C definition.  This requires code acrobatics to interpret and use those fields correctly.    That's far from the only case but, since it is one I am dealing with, it is very fresh in my mind.

When FPC is bug free then maybe asking whether or not another release is needed will be appropriate, till then, there is no doubt new releases are needed.

And, even after the compiler is 100% bug free, as Davo mentioned there will be new reasons why another release is needed.

[marcov]

given FPC 3.2.2 has been in use for the last 4.5 years without major issue, is there any need for a new release?

In addition to what Davo mentioned above, v3.2,2 has some annoying bugs which prevent writing code as it should be and make carrying out some operations quite a bit more complicated than they should be.

An example of that is the inability to define bit fields as they appear in the C definition.  This requires code acrobatics to interpret and use those fields correctly.    That's far from the only case but, since it is one I am dealing with, it is very fresh in my mind.

That is a missing feature (and even that can be debated), not a bug. Or did it work in an earlier version?

[Martin_fr]

An example of that is the inability to define bit fields as they appear in the C definition.  This requires code acrobatics to interpret and use those fields correctly.    That's far from the only case but, since it is one I am dealing with, it is very fresh in my mind.

That is a missing feature (and even that can be debated), not a bug. Or did it work in an earlier version?

Actually if I recall that thread correctly: Maybe half/half.

1) The compiler does not offer an e.g. align of word to a 32byte bound. => That definitely is a feature, as the FPC docs afaik do not say it can be done. (And while it may be important for the example case, judgment for the fpc project may differ, if e.g. the judgment is "how many users try to do that" ... anyway that leads way off topic for this thread => feel free to comment on the importance in the original thread of the bitmap declaration)

2) aligning an 0..2^33 sub-range (33bits number) inside a bitpacked record, so it starts after the previous value, i.e. in the middle of a byte.
Afaik, that is a combination of several existing features, that all are documented, and afaik have no exclusions why they should not work together.
IIRC it is reported that this works with the 64 bit compiler, but not the 32 bit compiler => then that is a bug. (importance of that one being fixed compared to others... I guess opinions may vary widely)

[robert rozee]

fpc322 on Linux does not support the current version of SSL

surely that functionality is contained in library (unit), not baked into the compiler ELF file?! anything that is in a unit - that the user makes use of using a USES clause in their code - can be replaced without need for bumping up the version number from 3.2.2. or am i missing something obvious here?

or the user can simply write their own SSL library - i do similar things when using serial ports or sound as the bundled units are (in my opinion) bloated.


Raspberry Pi Pico and ESP32 series

these are both microcontrollers; the number of people who would even attempt to use FPC to build an application for one of these would be minimal (if anyone at all would). the bulk of users are on intel hardware for Windows, Linux, legacy Mac. and just recently ARM for newer Mac.


Macs using ARM chips

see:
https://sourceforge.net/projects/lazarus/files/Lazarus%20macOS%20aarch64/Lazarus%203.8/
the above suggests that there is an FPC 3.2.2 that can target aarch64 (M1+), as well as the below link for RPi:
https://sourceforge.net/projects/lazarus/files/Lazarus%20Linux%20arm64%20DEB/Lazarus%204.0/

admittedly, the RPi seems to be using FPC 3.2.3. i didn't realize there was a 3.2.3 release, or is this a 'special' just for RPi?


the inability to define bit fields as they appear in the C definition

personally, i've never struck this problem. but if i did, it would be pretty easily worked around by just working with an array of bytes. or, again, am i missing something here?


overall, i can't see any particularly strong examples presented. the Mac using ARM example is the most pressing, but that seems to have already been solved in the form of a 3.2.3 version.


cheers,
rob   :-)

[440bx]

That is a missing feature (and even that can be debated), not a bug. Or did it work in an earlier version?

There are several problems with bit fields.

1. there are bugs (outright bugs)
2. design problems (lack of control over bit field alignment and bit count.)
3. the combination of the above 2 causes all kinds of headaches.

By design it could not have possibly worked in an earlier version since there are needed features that are missing.  That in addition to the bugs which further complicate the situation.

I fully understand the aversion to do things the C way since the C way is often rather deficient by design but, there are a few things C got reasonably well designed, the declaration of bit fields is among those.   The point here is that maybe the aversion to the C way has occasionally been taken too far.  IMO, it definitely was as far as bit fields are concerned.

Yes, I would love to have a version of FPC that at least corrects the bugs in that area.  Ideally a future version would improve the design but, I doubt that will happen because it seems that once a feature has been implemented, no matter how poorly, a new implementation is out of the question, e.g, static variables.

[marcov]

That is a missing feature (and even that can be debated), not a bug. Or did it work in an earlier version?

There are several problems with bit fields.

1. there are bugs (outright bugs)

As in specification discrepancies ?

2. design problems (lack of control over bit field alignment and bit count.)

Or maybe differing intentions of use? I can't remember the exact reasons for packed bitfield though. I assumed it was primarily to layout registers in embedded targets.

I fully understand the aversion to do things the C way since the C way is often rather deficient by design but, there are a few things C got reasonably well designed, the declaration of bit fields is among those.   The point here is that maybe the aversion to the C way has occasionally been taken too far.  IMO, it definitely was as far as bit fields are concerned.

The problem is that you refer to your expectation pattern as it being C, while other's do not necessarily do so.

[440bx]

As in specification discrepancies ?

No.  Some records that have bit fields end up having the wrong size and/or incorrect offsets for the fields that follow the bit fields.   That makes the definition unusable.  In the case of Win API definitions, it requires creating definitions that don't match the original specifications thereby creating a bunch of headaches.    I first experienced this problem in v3.0.4 in different ways but, it is present in v3.2.2 (and I believe it's very likely to be present in v3.2.4 - when that is ever released.)

Or maybe differing intentions of use? I can't remember the exact reasons for packed bitfield though. I assumed it was primarily to layout registers in embedded targets.

The programming world has to accept the fact that C (or maybe C++) is king.  Any language, not just FPC, that in any way targets low level programming has to implement features present in C.  Actually, this isn't even about C, it's about how hardware behaves and how to describe its behavior with data structures.  This is why C has bit fields and likely why they are reasonably well designed instead of the absurd and close to useless FPC implementation.  Sorry for being so harsh but, that's the reality.

The problem is that you refer to your expectation pattern as it being C, while other's do not necessarily do so.

See above paragraph.  It's about implementing features that are necessary to model hardware behavior with a minimum of hassle and maximum of convenience.

That's why I support the creation of language standards, because those are usually created by people who understand these things and come up with practical, workable, sensible designs instead of stuff that someone hallucinated that, to boot, doesn't even work as originally hallucinated.

[LeP]

Or maybe differing intentions of use? I can't remember the exact reasons for packed bitfield though. I assumed it was primarily to layout registers in embedded targets.

The programming world has to accept the fact that C (or maybe C++) is king.  Any language, not just FPC, that in any way targets low level programming has to implement features present in C.  Actually, this isn't even about C, it's about how hardware behaves and how to describe its behavior with data structures.  This is why C has bit fields and likely why they are reasonably well designed instead of the absurd and close to useless FPC implementation.  Sorry for being so harsh but, that's the reality.

The problem is that you refer to your expectation pattern as it being C, while other's do not necessarily do so.

See above paragraph.  It's about implementing features that are necessary to model hardware behavior with a minimum of hassle and maximum of convenience.

I do not agree (for what my opinion is worth) with that. C is C, Pascal is Pascal. Is like one said (in the story line) ... huaoo implementing this from java ... this from C# ... this from python ... and this from rust.
Every one can have their needs but one thing is move the compiler every time there is a "HUAOOO" and one thing is to adapt (like create a specialized asm o function) this needs.

Whenever you modify the compiler with new features, you generate potential new bugs and should have more time to validate the functionality.

[runewalsh]

I confirm that bitfields in C and equivalent bitpacked records in FPC give different bitpatterns, try e.g. on Godbolt:

Code: C  [Select][+][-]

1. #include <stdio.h>
2.  
3. struct Bitfield
4. {
5.     union
6.     {
7.         struct
8.         {
9.             int a:5;
10.             int b:10;
11.             int c:15;
12.             int d:18;
13.         };
14.         unsigned char bytes[6];
15.     };
16. };
17.  
18. int main() {
19.     struct Bitfield b;
20.     b.a = 1;
21.     b.b = 2;
22.     b.c = 3;
23.     b.d = 4;
24.     for (int i = 0; i < 6; i++)
25.         printf("%d ", b.bytes[i]); // 65 128 1 0 4 0
26.     return 0;
27. }

Code: Pascal  [Select][+][-]

1. type
2.         Bitfield = record
3.         case int32 of
4.                 0: (
5.                         b: bitpacked record
6.                                 a: 0 .. 1 shl 5 - 1;
7.                                 b: 0 .. 1 shl 10 - 1;
8.                                 c: 0 .. 1 shl 15 - 1;
9.                                 d: 0 .. 1 shl 18 - 1;
10.                         end
11.                 );
12.                 1: (bytes: array[0 .. 5] of byte);
13.         end;
14.  
15. var
16.         b: Bitfield;
17.         i: SizeInt;
18.  
19. begin
20.         b.b.a := 1;
21.         b.b.b := 2;
22.         b.b.c := 3;
23.         b.b.d := 4;
24.         for i := 0 to High(b.bytes) do
25.                 write(b.bytes[i], ' '); // 65 128 1 0 1 0
26. end.

BUT: man, what are you talking about, there are NO GUARANTEES to begin with, even in C: https://stackoverflow.com/a/12118582.

[440bx]

BUT: man, what are you talking about, there are NO GUARANTEES to begin with, even in C: https://stackoverflow.com/a/12118582.

I didn't ask for guarantees.  I asked for a design that is bug free.

In addition to that but not what I have an issue with: bit fields can be made portable across platforms simply by specifying what their arrangement should be regardless of endianness.  Unfortunately, the standard, so far, has chosen to stay away from that.

[creaothceann]

I don't know what C is doing (maybe undefined behavior), but afaik Free Pascal is right here.

Code: Pascal  [Select][+][-]

1.         Bitfield = record
2.                 case int32 of
3.                         0: (
4.                                 b : bitpacked record
5.                                         a : 0..((1 SHL  5) - 1);  // u5     .... .... | .... .... | .... .... | .... .... | .... .... | ...0 0001 = 1
6.                                         b : 0..((1 SHL 10) - 1);  // u10    .... .... | .... .... | .... .... | .... .... | .000 0000 | 010. .... = 2
7.                                         c : 0..((1 SHL 15) - 1);  // u15    .... .... | .... .... | ..00 0000 | 0000 0001 | 1... .... | .... .... = 3
8.                                         d : 0..((1 SHL 18) - 1);  // u18    0000 0000 | 0000 0001 | 00.. .... | .... .... | .... .... | .... .... = 4
9.                                         end);                     //                0           1           0           1         128          65
10.                         1: (bytes : array[0 .. 5] of byte);
11.                 end;
12.  

2) aligning an 0..2^33 sub-range (33bits number) inside a bitpacked record, so it starts after the previous value, i.e. in the middle of a byte.
Afaik, that is a combination of several existing features, that all are documented, and afaik have no exclusions why they should not work together.
IIRC it is reported that this works with the 64 bit compiler, but not the 32 bit compiler => then that is a bug. (importance of that one being fixed compared to others... I guess opinions may vary widely)

Speaking of bit fields, I experienced a bug yesterday - or rather, an internal error. So it doesn't work...

SNES sprite info ("OAM") is 34 bits per sprite, so the 128 sprite entries are stored in 544 bytes: a table of 512 bytes (4 bytes per sprite) followed by a table of 32 bytes (2 bits per sprite). Therefore my data structure definition starts like this:

Code: Pascal  [Select][+][-]

1. unit U_Ricoh_5C7x;  {$ModeSwitch AdvancedRecords}
2.  
3.  
4. // SNES PPU = picture processing unit (Ricoh 5C77 + 5C78)
5.  
6.  
7. interface  /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
8. // uses
9. //      U_Tools;
10.  
11.  
12. // --- from U_Tools ---
13. const Bit0  = 1 SHL  0;  Bits0  = Bit0  - 1;
14. const Bit1  = 1 SHL  1;  Bits1  = Bit1  - 1;    type u1  = 0..Bits1;
15. const Bit2  = 1 SHL  2;  Bits2  = Bit2  - 1;    type u2  = 0..Bits2;
16. const Bit3  = 1 SHL  3;  Bits3  = Bit3  - 1;    type u3  = 0..Bits3;
17. const Bit4  = 1 SHL  4;  Bits4  = Bit4  - 1;    type u4  = 0..Bits4;
18. const Bit5  = 1 SHL  5;  Bits5  = Bit5  - 1;    type u5  = 0..Bits5;
19. const Bit6  = 1 SHL  6;  Bits6  = Bit6  - 1;    type u6  = 0..Bits6;
20. const Bit7  = 1 SHL  7;  Bits7  = Bit7  - 1;    type u7  = 0..Bits7;
21. const Bit8  = 1 SHL  8;  Bits8  = Bit8  - 1;    type u8  = 0..Bits8;
22. const Bit9  = 1 SHL  9;  Bits9  = Bit9  - 1;    type u9  = 0..Bits9;
23. const Bit16 = 1 SHL 16;  Bits16 = Bit16 - 1;    type u16 = 0..Bits16;
24. const Bit24 = 1 SHL 24;  Bits24 = Bit24 - 1;    type u24 = 0..Bits24;
25. const Bit32 = 1 SHL 32;  Bits32 = Bit32 - 1;    type u32 = 0..Bits32;    uint = u32;
26. const                    Bits64 = high(QWord);  type u64 = QWord;
27.  
28.  
29. type
30.         i8 = System.ShortInt;  i16 = System.SmallInt;   i32 = System.LongInt;    i64 = System.Int64;       int = i32;
31.         b8 = System.Boolean;   b16 = System.Boolean16;  b32 = System.Boolean32;  b64 = System.Boolean64;  bool = b32;
32. // ---
33.  
34.  
35. type Ricoh_5C7x = packed record
36.         procedure PowerOn(const PAL : bool = False);
37.         procedure Step;
38.  
39.         type
40.                 OAM_Info1 = bitpacked record
41.                         X, Y, Tile   : u8;    // x position (8 of 9 bits), y position, first 8x8 pixel tile index
42.                         NameTable    : u1;    // where to grab the tiles from - https://wiki.superfamicom.org/sprites
43.                         Palette      : u3;    // a slice of the global palette
44.                         Priority     : u2;    // relationship to the 4 backgrounds
45.                         FlipX, FlipY : bool;  // is sprite mirrored horizontally and/or vertically
46.                         end;  {$if SizeOf(OAM_Info1) <> 4}  {$fatal}  {$endif}
47.  
48.  
49.                 OAM_Info2 = bitpacked record  // using this on ObjectAttributeMemory.Table2[i] causes internal error 20060521 in FPC 3.2.2
50.                         X_Bit8 : u1;  // 9th bit of the sprite's X position
51.                         Size   : u1;  // one of the two sizes possible per screen
52.                         end;
53.  
54.  
55.                 ObjectAttributeMemory = bitpacked record
56.                         case uint of
57.                                 0: (Bytes : array[0..(512 + 32) - 1] of u8);
58.                                 1: (
59.                                         Table1 :           array[u7] of OAM_Info1;  // 512
60.                                         Table2 : bitpacked array[u7] of u2;         // 32
61.                                 );
62.                         end;  {$if SizeOf(ObjectAttributeMemory) <> 544}  {$fatal}  {$endif}
63.  

As the comment says, trying s := OAM_Info2(SNES.PPU.OAM.Table2).Size; causes the error.

(Maybe someone who has a beta version of the compiler could check the error is still there.)

[dbannon]

(SSL) surely that functionality is contained in library (unit)
or the user can simply write their own SSL library
Raspberry Pi Pico and ESP32 series, .... the number of people who would even attempt to use FPC to build an application for one of these would be minimal
Macs using ARM chips ...  see: URL ...
i didn't realize there was a 3.2.3 release,
the inability to define bit fields as they appear in the C definition  ... pretty easily worked around

Summary : I am afraid I have to (respectfully) disagree with every one of those points.  I don't personally run in to all those issues but I know people who do. Yes, the FPC devs could just cut those people free. Over time, as those cuts cut deeper, the user base would dwindle to zero. Thats not how a project works.

Individually : fpc322 has, hard wired, the version number of OpenSSL it knows it can work with. Workaround involves installing the "non-versioned" -dev library (and saying "trust me"). I'd advise against the average programmer (such as me) designing their own SSL libraries. Pico and ESP32 (eg) - remember "write once run anywhere" ? Mac issue : please see the MacOS sub-forum for details, fixed in 323, not in 324rc1 and therefore preventing release of 324. FPC323 is the development stream. If we don't do new releases, the development stream gets further and further away from the release. 440bx, someone I have a lot of respect for, sure seems to want/need that bit/field issue solved, it comes up every few weeks.

Davo

[robert rozee]

(SSL) [...] Raspberry Pi Pico and ESP32 series [...] Macs using ARM chips [...] the inability to define bit fields as they appear in the C definition [...]

Summary : I am afraid I have to (respectfully) disagree with every one of those points [...]

i'm probably approaching this from a slightly different perspective - let me explain. i've worked on a few different (quite large) commercial hardware projects where the final product does not do 100% of what was in the original design brief, and does not meet the requirements of 100% of customers. this is a surprisingly common occurrence - sometimes features are disabled in software, sometimes chunks of circuitry will be removed from the BOM and the associated components not loaded at manufacture time. sometimes the unused components will even be left in place as this is cheaper than amending a BOM. user manuals and advertising material will be edited to remove reference to those things missing or disabled.

why? because sometimes it is better to release a product that satisfies 95% of customers rather than delay the release until everything is working for 100% of customers. this, i am afraid, is the place where i see FPC 3.2.2 as sitting today, imperfect, but mostly usable. sure: one could wait until bit fields are fixed up, Pico and ESP32 supported, but that could take another 2 or 3 years. the SSL situation is unfortunate, but not insurmountable by other means. and the one i see as most important - Macs using ARM - has a path (if not the best path) going forward in 3.2.3.

no doubt there are other roadblocks - but no one has mentioned them here yet.


cheers,
rob   :-)

[creaothceann]

(Maybe someone who has a beta version of the compiler could check the error is still there.)

Tried it with FPC 3.3.1 via fpcupdeluxe, same error.

[Martin_fr]

this, i am afraid, is the place where i see FPC 3.2.2 as sitting today, imperfect, but mostly usable. sure: one could wait until bit fields are fixed up, Pico and ESP32 supported, but that could take another 2 or 3 years. the SSL situation is unfortunate, but not insurmountable by other means. and the one i see as most important - Macs using ARM - has a path (if not the best path) going forward in 3.2.3.

Well, from the statements made by FPC team members: The Pico/ESP/... aren't the hold up.

Mac/ARM is (within the list of hold ups), but the fixing commit is known and was listed in this thread. So the hold up for this is "doing whatever needs to be done with the existing fix"

Otherwise, the problem isn't releasing now, but prioritizing the release work (i.e. start/continue the release process as a priority).
According to statements made on this thread, this work has been taken over by FPK. But, unless I missed it FPK has not commented on what steps are currently next and why they are waiting. And according to the git log of main, its not just a "no time at all" issue, since FPK had time for other commits. So according to what was said on this thread earlier, its his decision to progress the release, or rather currently not to progress it.

If I have misread any of those prior statements, then I apologize for any wrong in that summary.