Apropos some of the legal pondering here, I wonder if the AI could be used to write a version of Windows11 (without some of the features) and what would happen if it did?
(Or Wine++.)
Now that is a really _tall_ order.
Compared to writing an operating system like Windows 11, writing a C compiler is a walk in the park (no exaggeration), the scanner and parser parts can be precisely and accurately defined using EBNF. The IR, is also definable and can be interpreted (can be a VM's instruction set.) The CPU's instruction set is reasonably well defined (though not always as extensively as desired.) Again and, very important, I want to emphasize the word _comparatively_.
There is no such thing as EBNF to describe an O/S and that's just one thing. Is it going to be a real time O/S, a pre-emptive scheduling O/S, a cooperative scheduling O/S, is the memory demand-paged ?... is it multitasking ?... is it multi-user ?... does the CPU offer instructions to manage multiple tasks ?... Client/Server architecture ? ... modular or monolithic ? ... do you expect to design the interfaces (API calls) user programs will use to interface with the O/S ? ... will it have a built-in graphic subsystem ? ... the list of considerations goes on and on and on... and unlike a compiler, there is no language (EBNF) to describe any of these features.
OTH, one possibility comes to mind but, I doubt it would be legal and it wouldn't be easy either... an A.I thing can disassemble the entire O/S (that's fairly easy), study the disassembled code and produce equivalent high level language code (I'd love to see the Windows source code in Ada instead of C/C++ junk.) the fact that MS no longer distributes debug versions of Windows adds complexity to the task (e.g, creating reasonable variable names based on usage... that requires a lot of analysis.)
Lastly, without some very serious massaging of the generated source code, it would be patently obvious that the code is a translated copy of copyrighted material. It would be shut down in court before its lawyers have time to recite a hail Mary.
OTH, maybe an A.I model could be trained to study and eventually re-write the Linux kernel. I've read plenty of times that there are parts of the kernel that are only understood by one or two people, it would be nice to have those sections rewritten to make them understandable to a larger number of developers. That said, I don't know if A.I has gotten to be good enough to accomplish that but, odds are reasonably good that it eventually will.