Example projects window

[dbannon]

OK, this is pretty strange, my Lazarus Src is in ~/bin/Lazarus/lazarus-drb

Code: Bash  [Select][+][-]

1. dbannon@dell:~/bin/Lazarus/lazarus-drb$ ls -l ~/bin/Lazarus/lazarus-drb
2. total 306696
3. drwxrwxr-x 87 dbannon dbannon      4096 Mar  5 10:23 components
4. drwxrwxr-x  2 dbannon dbannon      4096 Feb 24 09:24 converter
5. -rw-rw-r--  1 dbannon dbannon     18088 Feb 24 09:24 COPYING.GPL.txt
6. .....
7. dbannon@dell:~/bin/Lazarus/lazarus-drb$ pwd
8. /home/dbannon/bin/Lazarus/lazarus-drb
9.  
10. dbannon@dell:~/bin/Lazarus/lazarus-drb$ rm -Rf ~/.lazarus
11.  

And thats where I am and I removed the default lazarus config files. So, I start lazarus, let it create new config, exit and have a look at LazrusDir in ~/.lazarus/environmentoptions.xml 

Code: Bash  [Select][+][-]

1. dbannon@dell:~/bin/Lazarus/lazarus-drb$ ./lazarus
2. Hint: (lazarus) [TMainIDE.ParseCmdLineOptions] PrimaryConfigPath="/home/dbannon/.lazarus"
3. Hint: (lazarus) [TMainIDE.ParseCmdLineOptions] SecondaryConfigPath="/etc/lazarus"
4. ....
5.  
6. dbannon@dell:~/bin/Lazarus/lazarus-drb$ grep LazarusDir ~/.lazarus/environmentoptions.xml
7.     <LazarusDirectory Value="../bin/Lazarus/lazarus-drb">
8.     </LazarusDirectory>
9. dbannon@dell:~/bin/Lazarus/lazarus-drb$ ls ../bin/Lazarus/lazarus-drb
10. ls: cannot access '../bin/Lazarus/lazarus-drb': No such file or directory
11.  
12. dbannon@dell:~/bin/Lazarus/lazarus-drb$ ls -l ../lazarus-drb
13. total 306696
14. drwxrwxr-x 87 dbannon dbannon      4096 Mar  5 10:23 components
15. drwxrwxr-x  2 dbannon dbannon      4096 Feb 24 09:24 converter
16.  

Its actually, in this case, two directories out !

But if we force a more complicated LazConfig dir and Lazarus decides not to try to save a relative path, all is well -

Code: Bash  [Select][+][-]

1. dbannon@dell:~/bin/Lazarus/lazarus-drb$ ./lazarus --pcp=~/bin/LazConfig/lazarus-test
2. ....
3. dbannon@dell:~/bin/Lazarus/lazarus-drb$ grep LazarusDir ~/bin/LazConfig/lazarus-test/environmentoptions.xml
4.     <LazarusDirectory Value="/home/dbannon/bin/Lazarus/lazarus-drb">
5.     </LazarusDirectory>

So, the problem definitely relates to the conversion to a relative path, a conversion that is, IMHO, quite unnecessary. I am seeing the same thing with Lazarus 2.2.0

Anyone have another view ?

Davo

[JuhaManninen]

Yes, it was a bug in Lazarus code. I fix it in 9f142eef14.
I had added the relative path myself but the IDE accepted only a false one because of the bug. Now it works.
I guess everybody has used absolute paths so far.

Some notions about the examples GUI:
- Projects also from my writable Lazarus source directory are copied to the config dir. They could be compiled in place.
- The Key Words column for some projects is so wide that the next Path column shifts far away. You cannot even adjust the width manually.
- The Category CheckBoxes could all be ON initially. Now they are OFF but all items in the list show anyways.
- You could consider using TListViewFilterEdit as a filter control.
- The source files don't have any license headers. (L)GPL requires such a header.

Is there a way to download online examples? I guess not yet.

[dbannon]

Yes, it was a bug in Lazarus code. I fix it in 9f142eef14.

OK, so I will definitely need to do some rebasing now 

- Projects also from my writable Lazarus source directory are copied to the config dir. They could be compiled in place.

Thats intentional. It means that you can have a play with the code, maybe mess it up completely and refresh from the original and start again. Is one of my KPIs.

- The Key Words column for some projects is so wide that the next Path column shifts far away. You cannot even adjust the width manually.

Yes, I am not that happy about that, should be able to make it adjustable but not much more can be done as far as I can see. Truncating the keywords seems worse than hiding the path, in practice, no one really needs to see the path anyway. Maybe better to not display it at all ?

- The Category CheckBoxes could all be ON initially. Now they are OFF but all items in the list show anyways.

Ah, thats an historical accident, one I should have fixed long ago.

- You could consider using TListViewFilterEdit as a filter control.

Hmm, never heard of it before now. Almost no docs. The filtering happens in two separate ways (Category and Keyword) and is performed by the list I maintain in the data unit. I am not sure it can be done in the UI space.

- The source files don't have any license headers. (L)GPL requires such a header.

Easily fixed.

Is there a way to download online examples? I guess not yet.

Not easily. When you suggested sticking to keeping the examples in the src tree, I concentrated on that approach. The on-line code is in there hidden with some $ifdefs and can be brought back and updated in light of what I know now.  But we cannot use httpclient on MacOS to download https files so until that situation improves, I don't see it as a good approach. https://forum.lazarus.freepascal.org/index.php/topic,58392.0.html

I will address the things you mention, rebase and see what it all looks like. I have a src tree here now with all the old Examples Window and its infrastructure stripped out. But will need to do that rebase I suspect.

Davo

[JuhaManninen]

OK, so I will definitely need to do some rebasing now 

Yes, run "git rebase main". Then force push to GitLab.
Anybody who has cloned your fork repo must then delete their local "newexamples" branch and create it again. No big deal. I think I am the only one who has done that.

[dbannon]

OK, rebase worked like a charm. I have used it in the past and regretted it (indeed, I tared up my working dir before I started). The changes to remove the old Examples Window is not really extensive but it does touch a good number of files. I will rebase frequently from now on ....

So, its usable now.  I'll address at least some of the matters you mention if I have time tomorrow.

Davo

[dbannon]

OK, I have simplified the ListView, it now shows only Example name and keywords. If user clicks it, the full path to the example is printed in the memo along with metadata.

The category checkboxes work more sensibly now and I have added a license statement, the usual Lazarus one I hope, to each source file.

Davo

[JuhaManninen]

OK, rebase worked like a charm. I have used it in the past and regretted it (indeed, I tared up my working dir before I started).

Regretted because of conflicting commits? Now there is only a small chance for them.
In some situations when rebasing over many commits causing conflicts, you must solve too many of them. "git rebase" iterates over the commit history one by one. If many commits change the same file causing a conflict, every such conflict must be solved separately.
In that case "git merge" is better. It just joins the branches together leaving the intermediate steps out.

The changes to remove the old Examples Window is not really extensive but it does touch a good number of files. I will rebase frequently from now on ....

Your change to lazarus.lpi was not right. You added more lines instead of removing ExampleManager.
Otherwise your changes look good.
I confess I don't always understand the changes that "gitk --all" shows. I attach here a screenshot of how it shows your latest "newexamples" branch. Why are the same commits shown twice? It is not a problem in your commits, I just don't fully understand Git's logic yet.

Other notions: You should remove code straight away instead of commenting it out. The removed code can always be found in revision control history.
For example:

Code: Pascal  [Select][+][-]

1. -  ecManageExamples          = ecFirstLazarus + 826;
2. +  // ecManageExamples          = ecFirstLazarus + 826;

The first line removing ecManageExamples is enough.

[dbannon]

Your change to lazarus.lpi was not right. You added more lines instead of removing ExampleManager.

Interesting. While I have removed ExampleManger, running a diff against a month old lazarus.lpi indicates that I somehow added ProjectWizardDialog to the lpi file, I did not do so consciously  (the local lazarus.lpi is the now current one in my repo, the ~/Downloads/lazarus.lpi  is a month old).

Code: Bash  [Select][+][-]

1. dbannon@dell:~/bin/Lazarus/lazarus-drb/ide$ sdiff -bs lazarus.lpi ~/Downloads/lazarus.lpi
2.         <ComponentName Value="ProjectWizardDialog"/>          <
3.         <ResourceBaseClass Value="Form"/>                     <
4.         <UnitName Value="ProjectWizardDlg"/>                  <
5.         <UnitName Value="KeyMapping"/>                        <
6.                                                               >       </Unit>
7.                                                               >       <Unit>
8.                                                               >         <Filename Value="examplemanager.pas"/>
9.                                                               >         <IsPartOfProject Value="True"/>
10.                                                               >         <HasResources Value="True"/>
11. dbannon@dell:~/bin/Lazarus/lazarus-drb/ide$ grep ProjectWizardDialog lazarus.lpi
12.         <ComponentName Value="ProjectWizardDialog"/>
13. dbannon@dell:~/bin/Lazarus/lazarus-drb/ide$ grep ProjectWizardDialog ~/Downloads/lazarus.lpi

I believe that ProjectWizardDialog should be mentioned in the lpi file, while I can remove it I think it would be a bad mistake. I suspect it appeared there in a manner related to the rebasing and is someone else's commit (in official Lazarus repo) as discussed below. I think ....

I confess I don't always understand the changes that "gitk --all" shows. I attach here a screenshot of how it shows your latest "newexamples" branch. Why are the same commits shown twice? It is not a problem in your commits, I just don't fully understand Git's logic yet.

No, neither do I. I think it relates to the rebasing.  Looking at some affected files, I see content changes that do not relate to what I did in the first of the each pair of commits. That has to be someone else's commit off in the official branch that appears here during the refresh of the code base. 

( a comment : Gihub has a lot better GUI than Gitlab when it comes to inspecting past commits. Using gitlab, such "what changed when ?" questions need to be answered using git back here at home, on Github, I can browse to the commit and view it directly.)

Other notions: You should remove code straight away instead of commenting it out.

Yes, noted. My usual model with a fairly complicated edit is that I comment out code and stamp it with a 'DRB', then I can compile, repeat. Leaving the lines there helps me find related things that might be otherwise left dangling.  Then, when I am happy, I grep for the "DRB"s and remove them. In the case you mention, appears I forgot to stamp those lines. 

Davo

[dbannon]

Hmm, bad new. Seems I cannot rebase any more.

This is, I must say, not particularly surprising to me, I have seen it before. I am getting a lot of conflicts like this -

Code: [Select]

CONFLICT (rename/delete): examples/testall.pp deleted in Examples tested, categorised, metadata added, restructured and renamed to components/codetools/codetools.inc in HEAD. Version HEAD of components/codetools/codetools.inc left in tree.

CONFLICT (rename/delete): examples/helloform.pp deleted in Examples tested, categorised, metadata added, restructured and renamed to doceditor/frmabout.pp in HEAD. Version HEAD of doceditor/frmabout.pp left in tree.
("Examples tested, categorised, metadata added, restructured" is the name of the commit. Git has spotted a file being deleted (or moved) and has searched around for its "replacement", in this case, there is none).

Most are non plain text files, harder to auto merge I expect. But harder to manually edit out the merge messages too, try it with an icon or .res file !

The issue is that there are simply too many edits in there, its too much for one rebase on top of another to handle. I am not sure if its a stack type problem, seems that git is guessing and realizes its guessing wrongly. Notice how it seems to have searched the whole source tree for what might be a replacement for the file its deleting ?  While I admire its persistence, its all wrong !

 There is absolutely no way we can risk sending official lazarus a merge request based on this current code base.

I believe that we must revert to my first suggestion, break it up into several, smaller merges -

1. Contents of ~/examples.   This by far the biggest change and the one that is triggering the rebase problem. And, its totally non controversial. What I have done there is still usable with existing system and if my plans are rejected, all that needs happen is removal of a hundred or so metadata files. The restructure, fix ups and removal of irrelevant examples is a good thing in any case.

2. Add the new package into ~/components - again, does no harm, easily removed if someone decides its a bad thing. It does not stop the existing one working, it allows someone to choose to install the package and see it all work along side the existing Examples Manager.

3. Finally, the potentially destructive bit. Changes to a number of core files in ~/ide, mostly the removal of the existing Examples Manager. 

4. Adding metadata files to all of the examples in other parts of the source tree. I have not yet started on this stage.

I can very easily recover the files needed to do step one and two.  It may be possible to recover already edited files from step three (or manually reedit may be safer).  No point in starting that if stage one and stage two have not been accepted into the official main. Too much chance of this happening again and my time and effort does need to be assigned some value.

I will have to take it out of the git environment, create a lazarus fork, make a new tree, move my stage one and two file in.

Davo

David

[JuhaManninen]

Hmm, bad new. Seems I cannot rebase any more.

No problem. Moving and modifying files may be a challenge for rebase indeed. I will merge your changes to the official repo through patches.
Merge requests are not needed. They are clumsy anyways. No modification from my side is possible with them.

While I have removed ExampleManger, running a diff against a month old lazarus.lpi indicates that I somehow added ProjectWizardDialog to the lpi file, I did not do so consciously  (the local lazarus.lpi is the now current one in my repo, the ~/Downloads/lazarus.lpi  is a month old).

ProjectWizardDialog was there already. The IDE sometimes adds tags like <ComponentName Value="ProjectWizardDialog"/> etc. for some reason. You can see your own changes easily with gitk.

Gihub has a lot better GUI than Gitlab when it comes to inspecting past commits. Using gitlab, such "what changed when ?" questions need to be answered using git back here at home, on Github, I can browse to the commit and view it directly.)

You don't need Gitlab or Gihub GUI for inspecting past commits! There are much better tools for that, especially gitk. It shows a single branch by default and all branches with "gitk --all".
It has 3 panes with all the relevant information:
- List of commits with their messages, author and date.
- List of files in a selected commit.
- Contents of a selected commit as a diff.

Moving between commits is easy also without mouse clicks. IMO it is a superior tool also compared to TortoiseGit which many people praise.
Gitlab web GUI is needed mostly for bug reports after the initial cloning. All development and pulling / pushing happens with local Git tools.

The earlier issue of identical commits in the history was a result of your rebase / merge exercises somehow. I don't know how.   I was not able to flatten the history there.
No problem, I moved all your changes to my development repo through patches, then joined them to 3 commits and pushed.
Congratulations! Your revamped example projects and your new GUI are there now.
You should delete your existing newexamples branch and create a new one from "main" HEAD.

One more notion about the GUI: Double-click downloads the selected project but does not open it which is counter-intuitive. A user expects it to open.
The whole concept of "downloading" a local example project is weird. You should hide the copying process then from a user.
With online examples the word "download" makes sense.

[dbannon]

No problem. Moving and modifying files may be a challenge for rebase indeed. I will merge your changes to the official repo through patches.
Merge requests are not needed. They are clumsy anyways. No modification from my side is possible with them.

Right, I will remember that. Patches don't take any history with them, much better for this way of working !

....There are much better tools for that, especially gitk.

I saw you mention gitk, thought it was a typo !  TCL/TK ? really ?
OK, installed, yes it works. But still looks like a TCL/TK app.

Congratulations! Your revamped example projects and your new GUI are there now.

And thanks for your patience, I have learnt quite a lot. Will now start, time permitting, on the remainder of the examples in the tree. All they will need is an appropriate metadata file added, that should not worry git at all.

One more notion about the GUI: Double-click downloads the selected project but does not open it which is counter-intuitive. A user expects it to open.

The whole concept of "downloading" a local example project is weird. You should hide the copying process then from a user.
With online examples the word "download" makes sense.

Agree. I am sure we'll find more problems before this project is finished.
I'll batch them up as I work through the 'other' examples, will get you another big patch at some stage.

Thanks for your help and patience here Juha !

Davo

EDIT- Also still need to have the new package auto installed. I guess thats a makefile thing ?
EDIT - worse than that, my code to grey out the Examples button on the ProjectWizard is not working, so, fresh install, no Examples until you manually install the package and no errro msg. Sigh ....

[JuhaManninen]

I saw you mention gitk, thought it was a typo !  TCL/TK ? really ?
OK, installed, yes it works. But still looks like a TCL/TK app.

There are many new GUI apps with bells and whistles and fancy graphics but their usability sucks.
Gitk is the opposite. Its GUI looks outdated but the usability is super. It is even quick in its actions despite TCL.

TCL is not a nice language, I know. I had to use it in a objet DB system where it was a macro language.

I guess you already know "git gui". I use it always to commit code. It allows you to select files and even lines inside a file to include in the commit. Also allows to easily join your new changes with the last commit.
Super good usability again.

[dbannon]

I tend to work from the command line where I can and have not looked for a (desktop) git GUI.

Github is pretty good for those situation where a GUI is indicated, examining commits and so on, being on line must be a slower but I have not found that a problem.  I'll try gitk for gitlab related things as it seem my {best|only} choice. Thats all good.

OK, now that I know its preferred to generate a patch rather than git merge, I think I will create two branches of my existing gitlab fork, one will show changes to the Package (as mentioned above) and the other will slowly accrue the metadata files in all the examples that exist in directories other than ~/examples

As neither will involve mass renaming of files, rebasing should be safe.

I do not, at present, understand how I would make the new package part of a standard install. I suspect its Makefile and fpcmake stuff ?  Not my favorite place to play.

Davo



 

[wp]

I tend to work from the command line where I can and have not looked for a (desktop) git GUI.

Github is pretty good for those situation where a GUI is indicated, examining commits and so on, being on line must be a slower but I have not found that a problem.  I'll try gitk for gitlab related things as it seem my {best|only} choice. Thats all good.

OK, now that I know its preferred to generate a patch rather than git merge, I think I will create two branches of my existing gitlab fork, one will show changes to the Package (as mentioned above) and the other will slowly accrue the metadata files in all the examples that exist in directories other than ~/examples

As neither will involve mass renaming of files, rebasing should be safe.

I do not, at present, understand how I would make the new package part of a standard install. I suspect its Makefile and fpcmake stuff ?  Not my favorite place to play.

Davo

I don't know whether I like the new structure of the examples folder in the IDE. There is a "top-level" folder "Beginner" which contains all the single mini sample projects that where all in the examples folder of the older version, but now each in its own folder - that's perfect, I like it. But there is also a folder "Components", it contains subfolders for "cell_overflow", "columneditors", "embedded_images", "merged_cells" - all these were contained in a "grid_examples" folder in the old version, and it was clear that all these sample project somehow dealt with the Lazarus grid. This information is lost now. I know that the meta-data of these projects contain the information that they are about grids, but this forces me to use the new example projects windows. So far, I found everything needed from the file system alone. Giving up this feature clearly would be a weakness of the new example system.

Can't you move the mentioned project folders into a subfolder "grid_examples" of "Components"? (BTW: I would prefer all folder names to be in lower-case)

I did not read everything about this topic, and I am not sure if the next topic is planned anyway: There are a lot of examples in the (Lazarus)/Components folder. They should be accessible from the new examples window, too. But without moving them to the examples folder: When I work with TAChart I expect the examples to be in the TAChart installation folder, not somewhere else.  The meta-data structure should be flexible enough to accept a project at any location of the Lazarus source tree.

[JuhaManninen]

Github is pretty good for those situation where a GUI is indicated, examining commits and so on, being on line must be a slower but I have not found that a problem.  I'll try gitk for gitlab related things as it seem my {best|only} choice. Thats all good.

Github and Gitlab are servers. You don't need their web GUI for your development process. I personally use only the bug tracker.
A big benefit of Git is its local commits and revision history. Why would you examine commits with a clumsy online browser interface when you can do it locally?
"gitk" + "git gui" are not your only choice. There is for example SmartGit, a Java app, free for FOSS development. It has a modern GUI and nice graphics etc., you may like it. I don't see any usability benefit over "gitk" and "git gui" though.

OK, now that I know its preferred to generate a patch rather than git merge, I think I will create two branches of my existing gitlab fork, one will show changes to the Package (as mentioned above) and the other will slowly accrue the metadata files in all the examples that exist in directories other than ~/examples

You may not need your fork at all soon. I have asked commit rights for you into the official repository. I have duties during the spring and summer and don't want to work as a middle man.
You can have many local branches and experiment with them. It is possible to push a branch to the Gitlab server for others to test, too, although most people just test the "main" branch.

I do not, at present, understand how I would make the new package part of a standard install. I suspect its Makefile and fpcmake stuff ?  Not my favorite place to play.

We need help with that. Mattias knows the Make systems best.

BTW, there is a reported problem with MacOS :
 https://forum.lazarus.freepascal.org/index.php/topic,58631
On Linux things work.