Lazarus for Windows on aarch64 (ARM64) - Native Compiler

[Wallaby]

Updated set of patches. This resolves a few edge cases that the previous patches did not address:

This crashed:

Code: Pascal  [Select][+][-]

1. program texcept;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   SysUtils;
7.  
8. procedure Test(aArg: array of String);
9. begin
10.   raise Exception.Create('Blubb');
11. end;
12.  
13. begin
14.  
15. end.
16.  

This crashed as well:

Code: Pascal  [Select][+][-]

1. program texcept;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   SysUtils;
7.  
8. type
9.   TTest = class
10.     fName: String;
11.     property Name: String read fName;
12.   end;
13.  
14. procedure Bar(const aArg: array of const);
15. begin
16. end;
17.  
18. procedure Test(aArg: TTest);
19. begin
20.   try
21.     raise Exception.Create('Test');
22.   finally
23.     Bar([aArg.Name]);
24.   end;
25. end;
26.  
27. begin
28.  
29. end.
30.  

In standard generics test this would fail due to a mismatch in how parameters are passed:

Code: Pascal  [Select][+][-]

1. TTestStdCollections.Test_MethodsList

Building is same as the above with win32 FpcupDeluxe, use trunk/main adding two diff files 1268.diff and 1289.diff then installing the cross-compiler and Lazarus.

[msintle]

Updated set of patches. This resolves a few edge cases that the previous patches did not address:

This crashed:

Code: Pascal  [Select][+][-]

1. program texcept;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   SysUtils;
7.  
8. procedure Test(aArg: array of String);
9. begin
10.   raise Exception.Create('Blubb');
11. end;
12.  
13. begin
14.  
15. end.
16.  

This crashed as well:

Code: Pascal  [Select][+][-]

1. program texcept;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   SysUtils;
7.  
8. type
9.   TTest = class
10.     fName: String;
11.     property Name: String read fName;
12.   end;
13.  
14. procedure Bar(const aArg: array of const);
15. begin
16. end;
17.  
18. procedure Test(aArg: TTest);
19. begin
20.   try
21.     raise Exception.Create('Test');
22.   finally
23.     Bar([aArg.Name]);
24.   end;
25. end;
26.  
27. begin
28.  
29. end.
30.  

In standard generics test this would fail due to a mismatch in how parameters are passed:

Code: Pascal  [Select][+][-]

1. TTestStdCollections.Test_MethodsList

Building is same as the above with win32 FpcupDeluxe, use trunk/main adding two diff files 1268.diff and 1289.diff then installing the cross-compiler and Lazarus.

Thanks for the update!

So do we need the previous 7 patches as well as in the earlier instructions?

This is for trunk only again, right?

[AlexTP]

Wallaby,
is your works 'synced' with this merge request?
https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/1268
(updated 1 day ago)

[Wallaby]

These patches are for the trunk/main only and yes, they are actually the latest ones from the MR.

They supersede the old set, so only two files needed. One is from MR 1268 patches SEH on ARM64 (affects multiple files), the other patches code generation related to parameter passing MR 1289 (affects one file).

They are logically separate, so two patches. Currently I have a set of 2,507 tests, primarily taken from the compiler testing framework and other open-source projects - they all pass with the above patches.

[msintle]

Well, our grand test has succeeded. And this is no small feat. We've successfully ported our shell namespace extension from Delphi (which STILL doesn't support ARM64 after a decade!) to Lazarus. And, it works flawlessly - in some respects, better than the Delphi extension itself! You can see it for yourself in action here: www.installaware.com/arm64/diskzip.exe

Don't be misled by the arm64 moniker, the install properly installs 32 bit components on a 32 bit OS, 64 bit components on a 64 bit OS, and ARM64 components on a 64 bit ARM OS (Snapdragon native PCs, or Parallels Windows VMs on apple silicon). The only reason this isn't live on the main site yet is because we're still porting other code from Delphi to Lazarus, but getting back to the point: the shell namespace extension is the hardest one, by and far. And we were able to get it done in under a week! Which says a LOT.

Why is a shell namespace extension so hard, what does it do, and why are we so confident that this validates the Lazarus ARM64 efforts at large?

It's basically a COM object that integrates with File Explorer, using (still) mostly undocumented interfaces (which was part of a lawsuit against Microsoft); in our case, it helps you browse inside archives inside File Explorer just like they are ordinary folders (when they obviously aren't). This is called a non-rooted shell namespace extension (whereas, having a fixed "root" off the Desktop, say for something like a cloud drive, makes it a somewhat simpler rooted extension).

Again, this is the most complex and challenging type of programming undertaking that may actually exist on Windows. You make the slightest mistake, Explorer freezes - or crashes. Your compiler makes the slightest mistake - Explorer freezes, or crashes. The way we've done it, we're showing our own Lazarus form INSIDE File Explorer - so when you browse inside an archive, File Explorer handles the Tree view - and we internally handle and render the List view. Getting this working properly - to say nothing of in a highly performant manner - is truly nothing short of a monumental undertaking. 7-Zip's own author Igor Pavlov - the outright genius who invented 7-Zip - gave up on this decades ago, because he had better ways to spend his time.

Gals and guys, Lazarus handled this beautifully. Again, see for yourself - just double-click any archive, and watch it load inside File Explorer as any ordinary folder (of course, please try this only on an ARM64 Windows instance, or you won't be seeing the Lazarus compiled code in action; but legacy Delphi code instead, which we haven't recompiled for downlevel platforms [as of this writing, the shell namespace extension code we did port to Lazarus has been evolving for the past twenty four {24} years]).

It is worth mentioning that while Windows now does show archives other than ZIP as folders, its own implementation is still extremely slow (try browsing inside archives with [tens of] thousands of files in them and see how long it takes), and can't handle things like copy/paste (copy any number of files from the source archive and extract them transparently using the Paste command on a real file system folder, for example). Copy/paste is actually a great example because guess what, all that copy/paste stuff (especially when the data to paste doesn't even exist yet - it is compressed inside your archive!) relies on even more complex and extremely difficult Windows interfaces which, you guessed it - Lazarus runs flawlessly on ARM64 Windows!!!

We also ported far simpler shell extensions you can check out, of course. Hover your mouse over an archive - you get a tooltip showing what's actually inside. Explore the new Archives tab inside the File Properties window to get a static listing of all the files inside that archive (and view/extract/update items using the relevant buttons). All of it - coming from Lazarus.

A really fun one is - add an archive extension (ex: .7zip) to an existing folder. Instead of Windows renaming that folder, our Lazarus shell extension kicks in - and you get an archive of the specified type, instantly built; out of the intact contents of that folder.

And of course, the usual right-click context menu extensions (including Windows 11 top-level menu extensions, but InstallAware's Context Menu Bridge ports them over to the top-level, we can't take credit for that - although we did natively implement the right-click menus too in Lazarus ARM64 [these are really the least interesting, as ARM64 Windows does successfully emulate x86 and x64 context menu extensions for the most part {I say mostly, because its emulation in that regard has been very buggy and fickle, still unfixed over the years}]).

But none of these other shell extensions - for which, except where noted (just the context menus), you still DO need a pure ARM64 compiler like we now have thanks to the efforts of you amazing folks - are nearly a hundredth as complex as that crown jewel, the shell namespace extension. So as far as we can tell - Lazarus has totally  N A I L E D   I T  here!!!  C O N G R A T U L A T I O N S ! ! !

[msintle]

Here's our porting case study, with tips and tricks we learned along the process ("best practices" in industry jargon):

www.installaware.com/blog/?p=986

And here's our point-and-click installer for Lazarus on ARM64/Windows:

www.installaware.com/lazarus-arm64.msi

Once again, thank you everyone for making this happen!

[Wallaby]

Embarcadero has just released a new version of Delphi with ARM64 support.

The catch is, it's ARM64EC, so it won't allow shell extensions and things like that (I believe ARM64 processes can't load ARM64EC DLLs).

[msintle]

Embarcadero has just released a new version of Delphi with ARM64 support.

The catch is, it's ARM64EC, so it won't allow shell extensions and things like that (I believe ARM64 processes can't load ARM64EC DLLs).

That is absolutely correct, and is the greatest irony of Delphi Florence 13.1!

ARM64EC DLLs may only be loaded by x64 EXE's, and ARM64EC EXE's may only load x64 DLL's - entirely defeating the purpose

That's what Lazarus's TRUE ARM64 is for - and hoping the case study helps people stuck with Delphi port on over.

Once again, congratulations to the whole Lazarus team - we've all done this, together.

I'll refrain from commenting on how this may reflect upon Delphi though

[440bx]

@retrofoxed/Wallaby,

Off topic and again, strictly out of curiosity, did you use an A.I model to help you implement ARM64 Windows SEH correctly ?

As in a previous thread where you answered a similar question, I will _not_ elaborate on the topic no matter what your answer is.  I just want to know if A.I was involved and if the answer is "yes", if possible, how helpful do you consider its input/help was.

Thank you.

[Thaddy]

@440bx

-equally off-topic -
Maybe it is a good idea to open a new topic for that, or even a new forum category.
I tend to support the latter. It is such an important topic and many of us use it on a daily basis. I have - kind of - a log on where AI definitely was a great help, either as time-saver or to correct logic, finding corner cases, or writing tests and its use can differ between amazing, jaw dropping, to horrible and time wasting, equally jaw dropping. And the experiences greatly differ between models with all of them having pro's and con's.

For example this week I wrote the /= bug fix and you can immediately see that the int := int /= float part was helped by AI, the rest being my work, which you can also immediately see, and the test was even pure AI and documented as such. (My coding style and documentation where AI was used - with a different coding style for example)
I think that for the time being at least such new code writing experiences can be enlightening in how AI is a help or a nuisance if we share our experiences.

[440bx]

or even a new forum category.

I would  be in favor of that.

Since there isn't currently such a forum and, the question is directly related to the topic, I figured that I'd ask here instead of creating a new thread just to find out if A.I was helpful for this _specific_ task.  IOW, the question is only 50% off topic

[Thaddy]

Well, then I open a new topic to see if the users support it...

[440bx]

Well, then I open a new topic to see if the users support it...

I have to admit being a little reluctant to do that... I suggested the creation of a forum section for A.I in the following post...
https://forum.lazarus.freepascal.org/index.php/topic,73804.msg579974.html#msg579974
and the thread got locked soon after that.  I don't know if that post contributed to it but, for some reason, I feel A.I isn't generally speaking welcome around here (admittedly, I could be wrong but, that's the impression I have.)

[Wallaby]

@retrofoxed/Wallaby,
Off topic and again, strictly out of curiosity, did you use an A.I model to help you implement ARM64 Windows SEH correctly ?
As in a previous thread where you answered a similar question, I will _not_ elaborate on the topic no matter what your answer is.  I just want to know if A.I was involved and if the answer is "yes", if possible, how helpful do you consider its input/help was.
Thank you.

The same AI (Claude) did most of the work on that issue, with my steering and debugging expertise. It (the AI) had a funny tendency of suggesting a quick and dirty fix like so:

AI: "Hey look, I've fixed everything"
Me: "Nope, it's a shitty workaround masking the bug"
AI: "Yes, you are right!"
Me: "Do this, that and this and get to the bottom of it"
...

Many iterations later we actually arrive at a proper solution.

TBH, while I have no idea how compilers really work and no understanding of assembly, I feel with proper guidance one could easily fork FPC and turn it into a better product. Trim it a little of old baggage (really, does anyone program for Z80 or AMIGA running M68000?) keep the modern platforms only Linux/Windows/macOS/BSD on x64/arm64, maybe riscv64 (Android/iOS would be nice too, but need a widgetset).

The AI is amazing so that it can read specs way faster, design and prototype at 10-100x human speed under proper guidance.

I dare to say the FPC team are either pursing their own areas of interest, or simply not care, as we can see from zero releases for 5 years; my ready-to-merge ARM64 patches were simply ignored, so it's only a matter of time until someone takes matters into their own hands.

[440bx]

retrofoxed, thank you again for the extensive and detailed reply.

I asked the question for two reasons, the first one was, I was genuinely curious to know if you had used A.I to solve the problem.  The second one was, exception handling is not something easy to master and, I've read replies about it given by some A.I models that definitely showed the A.I thing had no clue.

That said, as a result of your comments and those of other users, such as schuler, I've been studying how to use A.I and, Claude Code in particular.  I was curious about how well versed Claude Code was about x64 exception handling and asked a few questions about it, I have to admit that, I was very surprised about the accuracy and high quality of the answers.  Unlike you, I didn't ask it to solve a particular problem but, its theoretical knowledge is solid, it explained the exception handling process in detail and accurately.  I didn't expect that, I thought it would make conceptual mistakes and, to my surprise, it didn't make any.  Granted, I didn't ask it to solve a problem, just to show its knowledge of the subject in the form of explanations but, since it can do that correctly, I figured there is a good chance it could create the correct mechanisms for a compiler to use the facility.

The one thing I did experience in the process is its ability to flatter the "conductor" which I found very amusing <chuckle>  Apparently, it knows something about human nature too