Single executable file

[kaedusoft]

Hi all. I have a doubt that I could not solve anywhere.

When you write a program in Lazarus, using several units and libraries, compiling generates a unique binary file, whose size depends on the size of the project itself, taking into account whether it contains debug information and so on.

My question is: how can it be compiled in such a way that the executable (binary) is only to start the program, and the rest of the components are compiled separately in libraries?

For example, when using any large system (video game, programs such as CorelDraw, or any application), the executable file is small, and then the program is divided into many different components, usually thousands of files. I do not know how to do that in Lazarus without generating a single executable.

Thank you.

[lucamar]

Build your units into libraries and make your main project load and use it.

[Mr.Madguy]

Yeah, use DLLs. Delphi also has runtime packages. In Delphi I do it, not only because well structured program is good thing. I do it, because I have to do it. Delphi IDE is still 32bit and has some strict project/unit size limitations.

[furious programming]

My question is: how can it be compiled in such a way that the executable (binary) is only to start the program, and the rest of the components are compiled separately in libraries?

If you want to divide the program into small pieces only so that the executable file has a small size, then this solution does not make sense.

For example, when using any large system (video game, programs such as CorelDraw, or any application), the executable file is small, and then the program is divided into many different components, usually thousands of files.

This type of software has a lot of additional files for a reason.

The games have a set of binary files to store data about content, that is mainly maps, textures, sounds, scripts and so on. It allows developers to create and modify game content without having to constantly recompile the code.

Regular applications also have a set of external files for a similar reason. In these files are stored graphics for building the interface, scripts, plugins and any other data on which the program operates but which are not its integral part.

However, DLL libraries are used to separate certain universal functions and resources so that they can be used in any other application. They are not used for weight loss of the executable file.

[Mr.Madguy]

If you want to divide the program into small pieces only so that the executable file has a small size, then this solution does not make sense.

Why not? Sometimes it's reasonable. Whole logic behind splitting program into many modules - is "runtime binding". I.e. when you want to swap things without recompiling anything. Via config files for example. And sometimes main module falls into this category too, so it wouldn't be bad, if it would be stored in separate module. But, yeah. Sometimes it's done purely for feng shui reasons.

[Handoko]

When use it properly, it can be useful. For example, I remember Turbo Pascal 7 has "overlay" feature, which allows the binary to be split into multiple files. And as far as I know that was for limited memory computers to be able to run huge binaries. My first computer has only 640 KB memory.

But if there is no good reason to do it, don't do it. Nowadays, memory isn't expensive, 4 GB RAM (used) is very affordable. And I don't think any of you is going to write program that has the result binary bigger than 1 GB.

[JernejL]

You also have to take into account that large software packages are sometimes split into library for licensing and compilation speed reasons, compilation speed is usually a non-issue in pascal world, but sometimes libraries are made by external vendors, and are easier to work with, and update in form of a dll.
 

[Mr.Madguy]

When use it properly, it can be useful. For example, I remember Turbo Pascal 7 has "overlay" feature, which allows the binary to be split into multiple files. And as far as I know that was for limited memory computers to be able to run huge binaries. My first computer has only 640 KB memory.

But if there is no good reason to do it, don't do it. Nowadays, memory isn't expensive, 4 GB RAM (used) is very affordable. And I don't think any of you is going to write program that has the result binary bigger than 1 GB.

May be 64bit compilers don't have such problem, but 32bit Delphi compiler can't compile unit, larger than around 2Mb (32bit), and can't produce executable module, that is also larger, than around 2Mb (32bit). Delphi barely can compile my main library without throwing "out of memory" errors. That's why it's separated into DLL. That is done, because I don't want my main program to have such problems. And overall. My project is so massive, that Delphi would never be able to compile it as solid executable. It has 33 modules, 18Mb in 32bit mode and 28Mb in 64bit mode. Pure code. No data. So, it's ok to have single executable for small homebrew project. But large professional project just can't have single executable.

[marcov]

Why not? Sometimes it's reasonable. Whole logic behind splitting program into many modules - is "runtime binding". I.e. when you want to swap things without recompiling anything. Via config files for example. And sometimes main module falls into this category too, so it wouldn't be bad, if it would be stored in separate module. But, yeah. Sometimes it's done purely for feng shui reasons.

So first: it rarely pays to do it out of space considerations, because the exe+dll's get bigger, and only with repeated use it gets less, and usually the dlls also need to update regularly.

Recompilation avoidance for static modules can also be a reason, but the best reason is about sharing the more static parts between clients, and plugging only the differences in.

The problem is going back to plain shared lib level limits you in freely using language features over such interface borders.

This can be deliberate, since that also means that the parts can be in other languages or versions, but that also decreases productivity.

That's where library packages (like Delphi bpl) come in, you gain a lot of language freedoms (can use full language), but have to keep packages and core programs synced with eachother.

There are also some hybrids, that only join memory allocators (sharemem)

[Mr.Madguy]

So first: it rarely pays to do it out of space considerations, because the exe+dll's get bigger, and only with repeated use it gets less, and usually the dlls also need to update regularly.

Recompilation avoidance for static modules can also be a reason, but the best reason is about sharing the more static parts between clients, and plugging only the differences in.

The problem is going back to plain shared lib level limits you in freely using language features over such interface borders.

This can be deliberate, since that also means that the parts can be in other languages or versions, but that also decreases productivity.

That's where library packages (like Delphi bpl) come in, you gain a lot of language freedoms (can use full language), but have to keep packages and core programs synced with eachother.

There are also some hybrids, that only join memory allocators (sharemem)

Dunno, how to explain it better. I'll give you an example. I had old DOSEx program, that supported VGA, SVGA VBE 1.0, SVGA VBE 2.0 and Windows GDI displays. Supported via drivers. And then I decided to add VGA Mode X support to it. So, I just made new driver without even touching initial program. It's called scalability. I don't have time for it, but I have even better idea. To separate renderers from drivers. For example VGA, SVGA and Windows GDI could share Chain renderer. This way I wouldn't need to make separate drivers for, lets say, Chain VGA and Plane VGA. Same for SVGA. It would be one single driver, but using two different renderers. Cool stuff, you know. It's needed to allow user to add/remove features by himself - without asking you to recompile anything. It's needed to be able to add features from 3rd party developers. It's needed to allow user to customize your program. It's needed to be able to update things separately.

Just imagine project like 3DSMax with hundreds of plugins, compiled into one single executable. Dunno, it's just bad style.

[PascalDragon]

When use it properly, it can be useful. For example, I remember Turbo Pascal 7 has "overlay" feature, which allows the binary to be split into multiple files. And as far as I know that was for limited memory computers to be able to run huge binaries. My first computer has only 640 KB memory.

But if there is no good reason to do it, don't do it. Nowadays, memory isn't expensive, 4 GB RAM (used) is very affordable. And I don't think any of you is going to write program that has the result binary bigger than 1 GB.

May be 64bit compilers don't have such problem, but 32bit Delphi compiler can't compile unit, larger than around 2Mb (32bit), and can't produce executable module, that is also larger, than around 2Mb (32bit). Delphi barely can compile my main library without throwing "out of memory" errors. That's why it's separated into DLL. That is done, because I don't want my main program to have such problems. And overall. My project is so massive, that Delphi would never be able to compile it as solid executable. It has 33 modules, 18Mb in 32bit mode and 28Mb in 64bit mode. Pure code. No data. So, it's ok to have single executable for small homebrew project. But large professional project just can't have single executable.

Considering that the FPC's compiler executables are already around 2 MB in size, I don't see a real problem with FPC. 
As another example two main units of mORMot have a size greater than 2 MB as well (as source files) and they compiled with 32-bit Delphi from the get go. So clearly it's not the compiler itself that's at fault there...  (though I personally consider such units essentially crying for some refactoring  )

[marcov]

The problem with modularizing apps is not to get "something working", but to really control and benefit from it.

Making small testing programs that very carefully skirt what is possible is easy. Taking a large program (like some of the commercial suites you name, or even something one or two magnitudes smaller), and divide it in multiple parts, handle versioning, and keep some productivity and flexibility, THAT is the challenge.

[Mr.Madguy]

Considering that the FPC's compiler executables are already around 2 MB in size, I don't see a real problem with FPC. 
As another example two main units of mORMot have a size greater than 2 MB as well (as source files) and they compiled with 32-bit Delphi from the get go. So clearly it's not the compiler itself that's at fault there...  (though I personally consider such units essentially crying for some refactoring  )

Dunno. As I understand, problem is with debug info. I use generics in my project and they produce very large debug info. And problem is with address space, not with available memory. 32bit application has only around 2GB address space. Delphi uses built-in DLL compiler in order to perform on a fly compiling. I.e. that compiler shares address space with IDE itself. Yeah, this problem is fixed in latest Delphi version. It's large address aware and allows using of external compiler. But I don't have one.

[kaedusoft]

Thanks for all. I will tray to build units into libraries.