Forum > Other

Interesting work on Universal Executables (Cosmopolitan)

(1/2) > >>


--- Quote ---Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 with the best possible performance and the tiniest footprint imaginable.

--- End quote ---

Pulling this off required...
A new libc implementation

A new linker that lets you build fat binaries

--- Quote ---It's called apelink.c and it's a fine piece of poetry that weaves together the Portable Executable, ELF, Mach-O, and PKZIP file formats into shell scripts that run on most PCs and servers without needing to be installed.

--- End quote ---

A POSIX change

--- Quote ---POSIX even changed their rules about binary in shell scripts specifically to let us do it.
--- End quote ---

Modifications to the C language and GCC:

What about performance and size?

--- Quote ---Cosmo Libc will make your software faster and use less memory too. For example, when I build Emacs using the cosmocc toolchain, Emacs thinks it's building for Linux. Then, when I run it on Windows...It actually goes 2x faster than the native WIN32 port that the Emacs authors wrote on their own. Cosmo Emacs loads my dotfiles in 1.2 seconds whereas GNU Emacs on Windows loads them in 2.3 seconds. Many years ago when I started this project, I had this unproven belief that portability toil could be abstracted by having a better C library. Now I think this is all the proof we need that it's not only possible to make software instantly portable, but actually better too.

For example, one of the things you may be wondering is, "these fat binary files are huge, wouldn't that waste memory?"  The answer is no, because Cosmo only pages into memory the parts of the executable you need. Take for example one of Linux's greatest hits: the Debian Almquist shell.

$ ls -hal /usr/bin/dash
-rwxr-xr-x 1 root root 107K Nov 21  2022 /usr/bin/dash
$ ls -hal /opt/cosmos/bin/dash
-rwxr-xr-x 1 jart jart 983K Oct 15 19:14 /opt/cosmos/bin/dash

Here we see Cosmo's six OS + two architecture fat binary dash is 30% bigger than the one that comes with Alpine Linux (which only supports x86-Linux and dynamically links a separate 600kb Musl library). But if I run them:

$ rusage /usr/bin/dash -c true
took 231µs wall time
ballooned to 688kb in size
needed 183us cpu (0% kernel)
caused 34 page faults (100% memcpy)

$ rusage /opt/cosmos/bin/dash -c true
took 217µs wall time
ballooned to 544kb in size
needed 172us cpu (0% kernel)
caused 36 page faults (100% memcpy)

Here we see Cosmo's fat binary version of dash went faster and used less memory than an x86-Linux-only binary built for Musl Libc. This is due to (1) the magic of modern memory management, where CPU MMUs lazily load 4096 byte blocks at a time; and (2) how carefully apelink plans your executable layout.

--- End quote ---

For another performance comparison, see Production Web Servers

I was skeptical but it does work. At least under Windows. Nice!

What are the hidden costs of this solution?

Because it is obvious that nothing is free. For users of software that runs only on Windows or only on Linux, this solution does not matter. In such cases, there is no point in bearing the costs of compromises included in this solution.

In my opinion, it's just a technical curiosity, showing that such a solution can be implemented. But that doesn't mean that it should be done or that it is worth doing. Either way, it is more ambitious than the primitive, makeshift and naive script-based solutions: JavaScript (Electron, etc.), Python, etc.

The issues with deployment was always the need for a mac to generate binaries for a Mac and that the Linux target is very fragmented. 

So it would be interesting how they deal with that.

At this time, it compiles executables that run on a command line (console mode only).  But that includes web servers which can generate a gui in a (separate) browser.

The "Hermit" project uses it for WASM to bundle a self-contained command line executable that runs on multiple platforms (i.e. it makes the WASM interpreter and runtime universal):

--- Quote ---Hermit bundles a WebAssembly runtime, WASI configuration, and a WebAssembly module into one executable (called a hermit). The same hermit executable runs on multiple platforms, bringing the portability of Wasm to a new level!
--- End quote ---


[0] Message Index

[#] Next page

Go to full version