* * *

Author Topic: Making Lazarus IDE even better  (Read 2358 times)

soerensen3

  • Full Member
  • ***
  • Posts: 152
Making Lazarus IDE even better
« on: April 04, 2018, 03:34:30 pm »
First I have to say I really love the Lazarus IDE and it is still unmatched in many places (I especially love the code editor) compared to other IDE's. However I think there is still room for more in some other places. Or to put it in another way, I see a lot of unused potential...
The good thing about Lazarus is that it is very extendable. That everything in theory could be changed if necessary. But some parts are kind of "hard coded" into the application which in my opinion should be or at least don't have to be.

When I tried to make a few changes to the Project Inspector and Package Windows to merge them into one window I found that it is rather complicated to simply change just one behaviour. You could of course change the original code but it is maybe not desired to do so, because some people like it the way it is now. So it is unlikely that this change is accepted for trunk. So the better approach is probably to make it a separate installable extension. This is however rather complicated because first you'd have to reimplement all the behaviour you want to keep by copying the code and make it work from inside your package and second you have to find a way to surpress showing the old windows which is at least not documented. It would be more intuitive if you had a package to begin with, where you can just play and get a feel for the functionality. I have been working first with Delphi and later with Lazarus for a long time but still changing the IDE feels like something with a steep learning curve. I presume this keeps a lot of developers from even trying it. For example the package inspector has had this stupid dialog to tell you that reordering the files via drag and drop is not supported, which is probably more annoying than the fact that it is not supported. I don't see it getting implemented soon anyway. But no one ever tried to change it, maybe because it is very complicated to dig through all the code of the IDE.
My idea is that it would be a lot easier if all the "Components" (as in parts of the IDE, not LCL components) would consequently be in separate packages which in theory could be removed or replaced if desired. Only the core functionality should stay in the main application. The only problem would be the communication between the windows but I think the solution for this is (at least for most of the cases) already there. The *Intf units provide a way to access the project and packages, as well as opened code buffers and forms, independent of the windows displaying them by access structures for reading/writing and by using hooks for events. So the only problem, as far as I see is moving some code portions to packages.

With the Installed Packages Dialog and Online Package Manager in mind it would be really simple to install other IDE extensions and configure the IDE the way you want it and make it as distraction free as possible. For example if you do not want the assembler window to show up because you have no clue about assembler anyway you can just remove the package (eg. http://forum.lazarus.freepascal.org/index.php/topic,40371.msg278868.html#msg278868). Or iff you only write non GUI applications you could simply remove all the GUI plugins and have a clutter free IDE (I think there were several request and approaches for a minimal IDE (eg. https://forum.lazarus.freepascal.org/index.php?topic=37464.0 or http://www.pascal-central.com/lwp/lightweight.html and I think there was an abandoned repo for a lightweight IDE somewhere but I can't find it) where people gave up after some time. Following the suggested approach would open the way for custom Lazarus Distribution (Which CodeTyphon is kind of) but make it more simple to maintain them. If design time packages would have some sort of tags what they provide it would even be possible to show conflicting packages (I'm thinking of something similar as with Linux packages). This would not be minor changes but I think a lot of the functionality is already there and I think it wouldn't break any compatibility with projects made in Lazarus and only a little with IDE plugins because the funtionality would still be there only moved to packages. As I said in the beginning I see a lot of unused potential. It would hopefully make it easier to develop IDE plugins and so encourage more developers to try.

I also understand if you say I don't want the IDE to change, I like it the way it is but if we move everything to packages it would not change visually. It would only be better structured and make way for extending it more easily.

Roughly the steps would be:
  • Strip the IDE to only the core functionality (Plugin functionality, Help, Open/Close projects,..)
  • (Optional) Create mockup packages to test the IDE
  • Create/Replace the packages in form of IDE plugins to get back all of the functionality (Here a lot of people could contribute at the same time without getting in each others way)

Of course it would throw up a lot of questions, which parts should remain core. Also describing this sounds probably more simple than it is in fact so I'm really interested in your opinions. What do you think?

Also with plugins in mind: Right now every design time package is compiled into the application. What exactly keeps us from using libraries (.dll, .so, .dynlib) for plugins? This way if a plugin crashes it would not necissarily break the whole application and plugins could be loaded on runtime.
Lazarus 1.9 with FPC 3.0.4
Target: Manjaro Linux 64 Bit (4.9.68-1-MANJARO)

Pascal

  • Hero Member
  • *****
  • Posts: 699
Re: Making Lazarus IDE even better
« Reply #1 on: April 04, 2018, 04:19:53 pm »
Sounds good to me  :)

We probably have to extend the *Intf units. I had to make several extentions for my One-Click-Profiler-Addon.
laz trunk - fpc trunk 32bit - Windows 10 Pro x64 (1803)

Blaazen

  • Hero Member
  • *****
  • Posts: 2607
  • POKE 54296,15
    • Eye-Candy Controls
Re: Making Lazarus IDE even better
« Reply #2 on: April 04, 2018, 05:07:51 pm »
You should present your ideas on Lazarus mailing list. Not all developers follow the forum and only they can tell you whether these changes will be accepted and, if yes, how to do them at most feasible way.
Lazarus 2.1.0 r59015M FPC 3.3.1 r39660 x86_64-linux-qt Chakra, Qt 4.8.7/5.11.1, Plasma 5.13.4
Lazarus 1.8.2 r57369 FPC 3.0.4 i386-win32-win32/win64 Wine 3.14

Try Eye-Candy Controls: https://sourceforge.net/projects/eccontrols/files/

engkin

  • Hero Member
  • *****
  • Posts: 2101
Re: Making Lazarus IDE even better
« Reply #3 on: April 04, 2018, 05:15:19 pm »
You will need dynamic packages which, AFAIK, is not implemented yet. And some people will object using this concept because it increases the time it takes the IDE to start.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4781
    • wiki
Re: Making Lazarus IDE even better
« Reply #4 on: April 04, 2018, 07:30:10 pm »
Generally, there could be some benefits from modularization. Especially if parts that go into packages can then get test cases.

On the other hand: "Never touch a running system". Well at least not  without need, that is not unless there is actual new work that requires the change. (Or hugely benefits)

The mere promise "this change will bring new developers to the project", well that promise has been made plenty of times. And some of the requests were fulfilled, yet there is not even one case were the promise was fulfilled. So that promise has no longer any value at all.

As for practical considerations...

There is the question of available men hours. The best chance of any such thing to happen is to find a developer (mailing list) that is willing to apply your patches (small steps that can be checked easily). For a developer to be willing to do this, there probably has to be a concrete benefit. E.g. a package you work on, that requires the changes, and that the developer thinks worth supporting.

About "small steps". It is also likely (actually it is sure) that not everything will be pulled out into package, but only small parts, for which a need can be shown.


---
my 2 cents.

soerensen3

  • Full Member
  • ***
  • Posts: 152
Re: Making Lazarus IDE even better
« Reply #5 on: April 05, 2018, 12:56:10 am »
Thank you for your constructive responses.

@engkin: Not sure if I understand why it is not possible, but it seems you know what you are talking about. Anyway it's is not important for my main question.

I think I will present the idea on the mailing list when I find the time to summarize it and maybe someone is interested.

About the small steps: It would also be possible the other way around. Take one part and remove it from the main code and put it into a package and then reintegrate it.

Of course it is likely that new bugs are introduced and that therefore nobody wants to touch the running system but it also would make it easier to debug in the future if not all the components were mingled together. For example take the delphi converter. It is spread across a lot of files and which makes it very complicated for someone who didn't create it to even find out in which files it is used. It seems to be used by CodeTools which by itself is a package but is also part of the Lazarus project (I only had a quick look). Also there is the dialogs for the converter which are spread out over different units. And the functionality is really not an integral part of the IDE (It could as well be a separate program). This complexity might be the reason why only a core developer can estimate the effort of it. But then again if only the core developers can contribute there is also less man hours. This again supports the idea of making the IDE better structured.

The reason why I asked on this forum is to get an opinion. If no one is interested in the following this plan everything stays the same and no men hours are wasted (Except for the hours reading this posts :P ). If however enough people were interested in this and maybe more developers could be found to help this would already contribute to an active community, which is very important for any open source project. So the benefit would be there rightaway. A core developer could assign tasks depending on their skills. One component is removed from core and put into a package and then a pull request can be done for testing. If it loses drive nothing is lost.
I know from other projects that they have road maps. While Lazarus has one too (http://wiki.lazarus.freepascal.org/Roadmap) it is really outdated. I don't know if there is another kind of road map I didn't find. I only saw there is a change history for each version which is up to date but it only reflects changes already done. It at least looks like it is lacking direction. Maybe it would help to define a road map.
Lazarus 1.9 with FPC 3.0.4
Target: Manjaro Linux 64 Bit (4.9.68-1-MANJARO)

taazz

  • Hero Member
  • *****
  • Posts: 5336
Re: Making Lazarus IDE even better
« Reply #6 on: April 05, 2018, 03:12:47 am »
You will need dynamic packages which, AFAIK, is not implemented yet. And some people will object using this concept because it increases the time it takes the IDE to start.
you are half right. As things are now if you want access to pascal objects you have to be compiled in the IDE (or become unstable), if you use interfaces (as in com interfaces) you can have a dll based plugin system that will not only free you from compile in but also compile with the same lazarus version, which is way cooler than dynamic packages.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4781
    • wiki
Re: Making Lazarus IDE even better
« Reply #7 on: April 05, 2018, 01:09:54 pm »
Yes the road map is really outdated. There really isn't one. Since all developers work in their spare time, the available time keeps changing. Also developers pick the tasks they want to do. There is no management to assign tasks.

Also to get more responses from developers you may want to join the mailinglist. Not all of us (I am one of them) hang out in the forum. But we are all on the mailing list.

And since every developer specializes in different parts, to get more than a "nice idea" or "no my cup of tea", you need to pick an actual example.

If the developer who maintains the code you are interested in changing, ... if he generally agrees, then you need to show that you are willing to put the work in, willing to do it according to guidelines (which may well mean, that when your first patch is reviewed, you need to change big parts of it again). And that you are willing to stick around, until the whole thing has been in production a while and the first few waves of bugs are dealt with.

-----------------------
I am mainly looking at SynEdit/SourceEditor and the debugger.

When I first worked on the debugger as a none core developer, I remember I had to wait for feedback (the maintainer was busy so sometimes it took a couple of weeks, but often it also was just days). I had to learn his ideas how it should be designed, and explain what I indented with my proposed changes.
Now (almost 10 years on) most of the debugger code moved to packages. (That wasn't the plan (not even in my mind) 10 years ago, but eventually became necessary a few years back. So it only was done in order to be able to add new features)


taazz

  • Hero Member
  • *****
  • Posts: 5336
Re: Making Lazarus IDE even better
« Reply #8 on: April 05, 2018, 01:37:56 pm »
Personally I would start small (really small), creating a small add in that will load any dll in a specified folder (add it to the options dialog) that would register the add in in the dll with the IDE. After that I would convert existing functionality to a dll addin start with something even smaller like a component editor that could be extended to allow multi selection. See if the interfaces can support it and how if there is anything it can be done to make them easier to use and why. Then talk with the developers for those changes. Mailing list, bug tracker makes little difference as long as you get feedback. See how that goes and if it can be used in a step by step process to create the modularized IDE you want.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4781
    • wiki
Re: Making Lazarus IDE even better
« Reply #9 on: April 05, 2018, 02:13:45 pm »
For example if you do not want the assembler window to show up because you have no clue about assembler anyway you can just remove the package

This would be to fine grained. The assembler is part of the debugger. The debugger (should) consist out of 2 packages.

1) The backend. This is already a package (in fact there are several to choose from)

2) The front end. This is still integrated directly into the IDE.

The Asm window is part of 2.
Even if all of (2) would go into a package, it would still be part of this overall package.

Of course (package or not) maybe hooks could be created to allow to register substitute windows for each debugger window.
I am making this statement purely "theoretical". I have not yet looked at the implications that may have.... And it may not be desireable in the end.
Most of the debug windows are an integral part of the IDE (both in user-experience/expectation and in implementation detail/needs)

The debugger (frontend) needs to very closely work with at the very least:
- the source editor (not just the gutter, but also editing, inserting new lines....)
- hints
- the IDE (menus / toolbars)
  e.g. compile and close-IDE need to be aware of the debugger running
- codetools

All those aren't blocker. But if you try to connect all of this over to many packages, then it may make the code much harder to read. Which would be the opposite of what you wanted to archive.

Anyway it is not impossible that over time some of the debugger frontend code moves to one package. But probably only if there is a concrete benefit for the IDE. For now all the windows are in a separate folder already. So easy to find.

Apropos: Opening the asm window is probably in the DebugManager (or something to do with that). And that probably has to stay in the IDE.

Anyway: Blocking the asm window via a package is probably much harder than adding an option to the existing code.

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3489
  • I like bugs.
Re: Making Lazarus IDE even better
« Reply #10 on: April 06, 2018, 10:41:44 pm »
Of course it is likely that new bugs are introduced and that therefore nobody wants to touch the running system but it also would make it easier to debug in the future if not all the components were mingled together.
Yes, modular code with less dependencies is always good.
In fact I have refactored the code related to projects and packages in my local Git branch during the past few months. The main goal is to remove LCL dependency from LazBuild but it improves the modularity for other purposes, too. It requires lots of refactoring. Many units used by LazBuild either directly or indirectly depend on LCL but they logically should not.
I hope I get something ready to be shown during next month (May).
In the future it would allow separating the package system code into its own package for example.
FYI: This work is pure refactoring. It will not add any new interfaces. However you can extend the interfaces with your patches if needed. It is another issue.

Quote
For example take the delphi converter. It is spread across a lot of files and which makes it very complicated for someone who didn't create it to even find out in which files it is used.
No. It is very easy to find out because they are all in one "converter" directory. Having many unit files instead of few very big ones is also a good thing.

Quote
It seems to be used by CodeTools which by itself is a package but is also part of the Lazarus project (I only had a quick look).
No. The converter uses CodeTools, not the other way around. (Take a better look).

Quote
Also there is the dialogs for the converter which are spread out over different units. And the functionality is really not an integral part of the IDE (It could as well be a separate program).
A separate cmd line program could be useful in some situations. When a GUI is used then Lazarus IDE has a big advantage: the converted project is opened in the IDE ready to be run. See, the functionality is an integral part of the IDE, yes!
Moving the converter code into a package and creating a cmd line program would be accepted/applied if somebody implements it.

Quote
This complexity might be the reason why only a core developer can estimate the effort of it. But then again if only the core developers can contribute there is also less man hours. This again supports the idea of making the IDE better structured.
I don't follow you now. Anybody can contribute, not only core developers. Besides, a "core developer" is not a person who magically understands and remembers everything after he gets commit rights. No, a core developer is somebody who "bit the bullet", studied and debugged code for hours, days, weeks ... then created patches to improve and fix it. Finally he got commit rights after providing many valid patches.
You can do the same if you want.

Quote
The reason why I asked on this forum is to get an opinion. If no one is interested in the following this plan everything stays the same and no men hours are wasted (Except for the hours reading this posts :P ). If however enough people were interested in this and maybe more developers could be found to help this would already contribute to an active community, which is very important for any open source project. So the benefit would be there rightaway. A core developer could assign tasks depending on their skills.
Ok. I guess I am counted as a "core developer". I now assign you the tasks you suggested. :-)
Otherwise your suggestion is wishful thinking. In a volunteer FOSS project tasks cannot be assigned to anybody really. Just "getting opinions" of such a big refactoring effort is quite useless.

Quote
I know from other projects that they have road maps. While Lazarus has one too (http://wiki.lazarus.freepascal.org/Roadmap) it is really outdated. ... Maybe it would help to define a road map.
Yes, it shoud be updated. There is no way to ensure that any roadmap schedule will hold but at least it could work as an inspiration for some people.

@taaz and others: A dynamic package (.dll, .so) is another issue. It is kind of an implementation detail of how a package is implemented. The modularity of code can be improved regardless of it.
« Last Edit: April 07, 2018, 07:35:00 am by JuhaManninen »

soerensen3

  • Full Member
  • ***
  • Posts: 152
Re: Making Lazarus IDE even better
« Reply #11 on: April 08, 2018, 12:30:55 am »
Quote
The debugger (frontend) needs to very closely work with at the very least:
[...]
All those aren't blocker. But if you try to connect all of this over to many packages, then it may make the code much harder to read. Which would be the opposite of what you wanted to archive.

If I understood the Interface units correctly the/one purpose is to connect different parts of the IDE. The good about is that you do not all the dependencies in one unit and that GUI and functionality are separated. So (without checking the code) it would be possible to provide all the functionality in an editor interface and a debugger interface (Set breakpoint), which could be core. How everything is displayed (I mean the GUI part) would be part of another IDE package which does not have to be core and thus can be replaced if required. The editor and debugger GUI part would then only have to reference the interfaces instead of each other. Or did I get you wrong?


Quote
Yes, modular code with less dependencies is always good.
In fact I have refactored the code related to projects and packages in my local Git branch during the past few months. The main goal is to remove LCL dependency from LazBuild but it improves the modularity for other purposes, too. It requires lots of refactoring. Many units used by LazBuild either directly or indirectly depend on LCL but they logically should not.
I hope I get something ready to be shown during next month (May).
In the future it would allow separating the package system code into its own package for example.
FYI: This work is pure refactoring. It will not add any new interfaces. However you can extend the interfaces with your patches if needed. It is another issue.
This sounds interesting!

Quote
No. It is very easy to find out because they are all in one "converter" directory. Having many unit files instead of few very big ones is also a good thing.
You're right I didn't notice that not all units were added to the project file and assumed, looking at the treeview, that the other units were in a different folder then.
Quote
No. The converter uses CodeTools, not the other way around. (Take a better look).
You're right here as well! was refering to the part of CodeTools where it provides the functionality for converting delphi units but it doesn't have a dependency on this units (This would not even be possible I think because these are project units).

Quote
I don't follow you now. Anybody can contribute, not only core developers. Besides, a "core developer" is not a person who magically understands and remembers everything after he gets commit rights. No, a core developer is somebody who "bit the bullet", studied and debugged code for hours, days, weeks ... then created patches to improve and fix it. Finally he got commit rights after providing many valid patches.
You can do the same if you want.

Okay everyone can in theory contribute but if you have to look to all the ide's source code to understand how everything is structured instead of one package where everything is together and clearly isolated from the other parts it kind of scares me of to try it in the first place. Maybe I should explain how I came to write this post. I read the discussion about the Free Pascal logo on the mailing list. (For all not the mailing list) Someone proposed a new FP logo and the idea was that it would motivate more young developers when the logo and the website look more modern. I also read parts of this guide how to build an Open Source community (https://opensource.guide/building-community/). Further down it says:
Quote
Resist fixing easy (non-critical) bugs. Instead, use them as opportunities to recruit new contributors, or mentor someone who’d like to contribute. It may seem unnatural at first, but your investment will pay off over time. For example, @michaeljoseph asked a contributor to submit a pull request on a Cookiecutter issue below, rather than fix it himself.
If we had an up to date roadmap with small achievable task it could (hopefully) animate new developers to contribute. Developers could then pick a task from the list which is probably better than assigning a task. Maybe an explanation of the task would help there.
As probably most of the developers here I have little spare time so its just not realistic I do all the proposed changes myself. I would still like to contribute with one part (Maybe I try the Package/Project Inspector because I always wanted to adapt that to my needs) but it will probably take some time. I was hoping that some people might want to join in.
Lazarus 1.9 with FPC 3.0.4
Target: Manjaro Linux 64 Bit (4.9.68-1-MANJARO)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4781
    • wiki
Re: Making Lazarus IDE even better
« Reply #12 on: April 08, 2018, 10:31:45 am »
Quote
The debugger (frontend) needs to very closely work with at the very least:

If I understood the Interface units correctly the/one purpose is to connect different parts of the IDE. The good about is that you do not all the dependencies in one unit and that GUI and functionality are separated. So (without checking the code) it would be possible to provide all the functionality in an editor interface and a debugger interface (Set breakpoint), which could be core. How everything is displayed (I mean the GUI part) would be part of another IDE package which does not have to be core and thus can be replaced if required. The editor and debugger GUI part would then only have to reference the interfaces instead of each other.

The debugger interface currently indeed acts as an API.  (But not to the IDE)

The debugger interface does not set a breakpoint to the source editor (or IDE). It does the opposite: If a breakpoint was set in the IDE, then it tells gdb to set this breakpoint too. (or instead of gdb, whatever debugger backend is running).

The debugger interface was created to allow work on fpdebug. (gdb replacemnt)
----

There already is a package IDEIntf, which has various ...Intf units. (Like SourceEditorIntf)
Those units allow 3rd party code to interact with the IDE. (But they do not (yet) include the IDE parts of the debugger).

Also IDEIntf is LGPL, while IDE is GPL. This means you can write close sourced packages for the IDE, as long as you only need IDEIntf.

Maybe I should explain how I came to write this post. I read the discussion about the Free Pascal logo on the mailing list. (For all not the mailing list) Someone proposed a new FP logo and the idea was that it would motivate more young developers when the logo and the website look more modern.


As I wrote on a previous post of this very thread:

That promise - any promise of the kind "Do this, and it will bring new users/contributors" - has been made before. Plenty of times. And when some of the demanded things had been done, the promise never was fulfilled.

Sure there are places in the sourced that could be improved. And the docs have even more potential for improvements.
But it is a vicious circle. Once a person worked through the sources, they start doing other work on the sources....

Also splitting the sources will not make it that much easier. You then need to add hooks, extra calls, .... And overall you still have the same amount of code. (Documentation could help so. If it is kept up to date, so probably needs to be in the source, and really short / comments on the function declaration)

Example:
A separate package for the IDE frontend of the debugger, would still be full of interaction with the rest of the IDE (because the debugger affects so many parts). And it would still have interaction (calls and events) with the debugger backend.

In fact, all the code that you need for the asm window, is in the folder "debugger", and one unit in the ide folder: "ide/debugmanager.pas". That is as good as a package of its own, don't you think?
(Of course, even though the backend is already a package, it probably needs design changes, to support a really good solution. But for a quick fix, you may be able to ignore this.)
And yes context menus for the source editor are defined in the source editor. That can be moved. And while it will be a better design, it will mean to split the menu creation into many packages. This may make it harder to understand.

You still need to understand those interactions, or changes you make may have really bad side effects.
You can not suppress all uses of the asm window (maybe you can on your local copy). Because if the user opens it from the menu, or pop up menu (could be in the IDE or the debugger), or dedicated short cut, in all those cases it must still work.

Still, yes. If it can be made easier, then cool. I am all for it.

====
Edit:

Don't get me wrong. I know I sound very critical of the idea, but I am not against it.

I simple think that the outcome to be expected from such work, may be less than you advertise.

On the other hand there are other good reasons that can be added:
- re-usability
- ability to add tests

« Last Edit: April 08, 2018, 10:47:53 am by Martin_fr »

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus