Example projects window

[dbannon]

OK, thanks folks for some valuable (and encouraging) feedback.

Juha,
Sorry, I come from a background where a lot of emphasis was placed on planning, I need to see and "socialize" (an IBM term) my plan before proceeding. So -
1. Move my examples gitlab tree to somewhere under FPC/Lazarus. Might save everyone a lot of time if I am granted access to (just) that new tree. This is easily reversed if we decide against this model.
2. Replace Lazarus Examples Window in Lazarus with the new one - it will see only ~/examples/* (about 100). As you suggest, I'll bring that up in a fork. Or Getmem's package idea ?
3. Remove  examples from ~/examples. Leave five there, they will provide the initial "recent projects" list in a new install.
4. Start processing remaining examples under eg ~/components/*/examples|demos but leave 'tests' in place. Some tests are also usable as examples as well and these will be duplicated, many tests are not good examples and tests belong with their components.

Step 2 and 3 can happen after a lot of testing and will require developer merging of course. People who know their way around the src tree will still find the ~/components/*/examples but no so easily. But, as you say, its trunk !

Step 2 could be made a lot easier with GetMem's package suggestion, maybe thats a good approach, then, the transition would be considerably softened ?  Your thoughts please ?

Yep, agree 100% that most non-ObjectInspector examples  should go. In most cases, they are really only demonstrating how to to declare and create visual components anyway so rewriting is not useful.

CCR - my current scanner looks for the OPM metafiles in Lazarus Config. If a metafile is added to a CCR project and a user installs that project using OPM, then the scanner will find and list it. In all cases, IMHO, examples found in installed OPM projects should be copied to the defined space (also under Lazconfig dir) so the original code remains untouched.

Standalone App - no, it has no future beyond the transition. Good testbed, code will be reused and maybe a small tool to write good JSON metadata will emerge but whats there now is just for now.

GetMem
Wow, thats a big help and a very sensible approach, removes quite a lot of the transition problems. I will have a play.

ASBzone - Thanks !

Davo

[dbannon]

...also master.ex-meta does not pass an online json validator.

Thats very, very messy. JSON does not allow newline characters in a JSON string, but, as luck would have it, our JSON parser (and JSONTools and, apparently many others) is quite happy with it. I'd be quite unhappy if this had gone on much longer before being spotted, thanks !

I'll have to convert to \\n and do some cross platform testing. Sadly, it negates my reason for choosing JSON, readable content. Tempting as it is, I cannot rely on a non-compliant feature of our parser. Sigh....

each and every caption/text should be a resourcestring

I have a few string literals in there that will need converting, most of the GUI content will be done automatically.  In the event that someone wants to translate the metadata, my plan is to have a separate files in each directory with a po file like 'es', 'ru' etc. The actual translation would be quite a big job and my guess is it will be quite selective, so the master metadata for each language will probably be mostly English unless someone gets really keen. Code changes to support it is minimal.

TreeView v. ListView.

Now, I am not sure about that. A treeview approach will require a user to click to expand each item to view it, the listview requires just a scroll. But I will definitly consider and maybe code up an example ....

Again, thanks for the feedback, especially picking up that bad json - its not a good situation at all.

I have tried out your "Package Template", that is really neat, makes it all look so easy ! Very grateful.

 Davo

[balazsszekely]

@Davo

It's not yet clear what will happen with the example folder. Personally I would move it to an external server, but I'm not the one who decides such things. As always things are not black and white,  there are pros and cons no matter what solution is chosen. Anyways if the example folder remains part of Lazarus, you should reuse the existing Tools->Example projects dialog. If the folder is moved to a server, then the package approach is a far better one. Code moved from IDE to a package, will make Lazarus faster and slimmer, because a large part of the packages are only loaded in the bigide.

PS: Other implementation details can be discussed later.

[wp]

Personally I would move it to an external server, but I'm not the one who decides such things. As always things are not black and white,  there are pros and cons no matter what solution is chosen.

One con of the external server is that examples will change with the Lazarus version (new examples added, deprecated ones removed, some edited due to changes in properties or FPC calling parameters etc). Keeping the examples within the Lazarus distribution makes sure that they always are opened by the correct version.

[trev]

One con of the external server is that examples will change with the Lazarus version (new examples added, deprecated ones removed, some edited due to changes in properties or FPC calling parameters etc). Keeping the examples within the Lazarus distribution makes sure that they always are opened by the correct version.

Why not version the external server examples tree eg using version branches or tags or whatever. Lazarus should know its own version.

[JuhaManninen]

It's not yet clear what will happen with the example folder. Personally I would move it to an external server, but I'm not the one who decides such things. As always things are not black and white,  there are pros and cons no matter what solution is chosen. Anyways if the example folder remains part of Lazarus, you should reuse the existing Tools->Example projects dialog. If the folder is moved to a server, then the package approach is a far better one. Code moved from IDE to a package, will make Lazarus faster and slimmer, because a large part of the packages are only loaded in the bigide.

Don't reuse the existing Tools->Example Projects dialog. It needed a big revamp anyways. Now the revamp will be thorough. Please remove the old GUI when adding the new one.
Yes, the new example projects GUI should be a package which registers itself under Tools menu and to global options window. We should make Lazarus more modular anyways. For example the Delphi converter should be a package, too.

Why not version the external server examples tree eg using version branches or tags or whatever. Lazarus should know its own version.

The same discussion now happens in Lazarus dev list and here, but no problem.
The examples must be tested with few recent versions. Typically examples are simple and libraries are more or less backwards compatible. I believe the examples can be made to work with older versions quite easily. No big problems there.
Yes, the examples should be under revision control where dbannon and current Lazarus devs have write access. I am not sure what is the right place.

[dbannon]

Personally, I think just noting, in the metadata, any constraint is more user friendly. If we enforce it, hiding or refusing to download for example, the user feels dis-empowered. Better to tell them about the issue and let them experiment with it if they like.

We would need to document just what the issue is anyway. And some constraints can be flexible.

"This example requires Lazarus 2.0.12 or later because it uses BlarBlar"

"This example works with Lazarus 2.0.8 and before if you uncomment the {.$DEFINE OLDLAZARUS}"

Davo

[trev]

"This example requires Lazarus 2.0.12 or later because it uses BlarBlar"
"This example works with Lazarus 2.0.8 and before if you uncomment the {.$DEFINE OLDLAZARUS}"

Why not simply omit the example from the versioned tree? It may be "fixed" later and re-included or it may simply disappear into the bit bucket of history.

Remember, this all began with a discussion about beginners and the included examples. You may already have restraints based on OS and in some cases OS version. No need to make it more complicated than necessary or add to the amount of labour required for continued maintenance which is something I often find to be overlooked (in a previous life in loose-leaf publishing, but the same issue applies here).

[JuhaManninen]

It seems the example directory must stay there in Lazarus sources. It is OK, part of the projects can be moved from there gradually.
@dbannon, can you please create a GitLab fork with a GUI replacing the current Example Projects dialog, and options page registered to the global Options.
More people will pay attention then and maybe propose further improvements.
The final location for online examples can be discussed later. I have no opinion or preference for it.

[dbannon]

OK, I guess a decision made elsewhere ?

Thats OK, no problem. I have a working package already but honestly, thats all down to the package template Getmem sent me, made it trivial. And we could have that working in parallel to existing one, so easy and safe transition. But no problems Juha, I'll fix it.

A more serious problem might be the fact that I cannot, totally dependably, get my https downloader working on MacOS. We cannot depend on the user having installed openssl and the Mac provided one is dodgy (IMHO) in earlier releases of MacOS.

Unless this can be sorted out, Examples on line is looking risky too I am afraid. Sigh...

Davo

[JuhaManninen]

OK, I guess a decision made elsewhere ?

No decision really. The discussion stalled after few comments. The best way to get things forward is to provide code to test. Let's try to get it into Lazarus trunk which is tested by many people.

Thats OK, no problem. I have a working package already but honestly, thats all down to the package template Getmem sent me, made it trivial. And we could have that working in parallel to existing one, so easy and safe transition. But no problems Juha, I'll fix it.

Package is good. I didn't study GetMem's package but the IDE integration at least should use a package. The existing Example Projects dialog is not a package. It can stay there in parallel if you want, but can also be removed.
Remember, this is not a critical feature. It can be under construction in trunk for a while. Remember also that the open source development model is basically meritocracy. Voting power goes up with accomplishments. You don't only need to take commands from others.

A more serious problem might be the fact that I cannot, totally dependably, get my https downloader working on MacOS. We cannot depend on the user having installed openssl and the Mac provided one is dodgy (IMHO) in earlier releases of MacOS.
Unless this can be sorted out, Examples on line is looking risky too I am afraid. Sigh...

I believe it can be sorted out. OPM works in MacOS, right? It can be temporarily broken for MacOS, no problem.

[dbannon]

OK, now I am afraid I don't understand what you are saying Juha.

Maybe if I read your message a bit more carefully.   

> It seems the example directory must stay there in Lazarus sources. It is OK, part of the projects can be moved from there gradually.

OK, so you mean the dir (and its contents) must stay in the Laz SRC tree. But not necessarily be used by the new Examples Window ? Fine by me. Maybe a good idea to update it with the work I have done on the Examples but no rush.

> @dbannon, can you please create a GitLab fork with a GUI replacing the current Example Projects dialog, and options page registered to the global Options.
> Package is good.

I really don't understand that one. I do not need to fork Lazarus to make a package, Lazarus already has all the support I need built in, nothing need change in Lazarus at this stage. Except, perhaps, to intercept a user click on the Project Wizard -> View Examples Project.

In https://gitlab.com/dbannon/laz_examples/-/tree/main/Utility/pkg you will find a working Package that adds a menu item to the Tools menu, it opens an Examples Window that does most of what we need. Copy the files somewhere and open the lpk file.

Unless, of course, if you are using a Mac.

I have no idea how to connect that Package to the Project Wizard but as a try out now, its up and running via the Tools Menu.

> The existing Example Projects dialog is not a package.

No, it was a test bed for some code and some ideas. Please make out I never mentioned it.

Davo

[JuhaManninen]

OK, so you mean the dir (and its contents) must stay in the Laz SRC tree. But not necessarily be used by the new Examples Window ?

As long as the directory stays there, it should be used by the new Examples Window.

I really don't understand that one. I do not need to fork Lazarus to make a package, Lazarus already has all the support I need built in, nothing need change in Lazarus at this stage. Except, perhaps, to intercept a user click on the Project Wizard -> View Examples Project.

In https://gitlab.com/dbannon/laz_examples/-/tree/main/Utility/pkg you will find a working Package that adds a menu item to the Tools menu, it opens an Examples Window that does most of what we need. Copy the files somewhere and open the lpk file.

They must be copied under the Components directory at some point. It makes things easier if you already have them in the correct place in your GitLab fork. There will be Makefiles which need help from others etc.
Then merging to Lazarus trunk can be done easily.
I can look at your latest package version next week and add it to trunk. Do you have the options page registration there? Once it is in trunk, you must continue with a GitLab Lazarus fork anyways.
(Sorry, I am not at my development computer now.)

I have no idea how to connect that Package to the Project Wizard but as a try out now, its up and running via the Tools Menu.

Which Project Wizard you mean? The Project -> New Project dialog?

[dbannon]

Juha, I'll make some more readable notes on https://wiki.freepascal.org/Lazarus_Examples_Window but for now, this is how I see where we are.

As long as the directory stays there, it should be used by the new Examples Window.

OK, some issues. My version of the Examples Dir has a number of changes, if they are going to be used, we need to decide which of these changes need be pushed up to the fork -
1. The metadata files - thats a must.
2. Directory Structure, I have built a dir structure based on Category just to make it more manageable. Its not necessary, just tidier.
3. Changes made to examples. Strongly Recommended. See note A.

So, easiest approach for me and probably best outcome for Lazarus is for me to replace all the Examples in ~/examples with my set but leave out my new directory structure. So, implement 1 and 3 above. I'll reverse the directory restructure (because it will confuse the existing Examples Window). This will be first merge request. It should not affect anything except removing misleading examples 

Then I will need to make some changes to new Examples Window so that it looks to the examples in the local directory rather than on-line. Easy, I'll use some {$ifdef so we don't loose (yet) the online capability. This solves my MacOS problem too. 

If, at some later stage, we do decide to go online, the underlying structure will still be there.

I'll then push my new Examples Window package up, it will appear directly under ~/components and, if installed, will only change the tools menu but will provide improved access to ~/examples and will solve the read only issue (because when a user chooses to first open an example, we'll make a working copy in user space). This will be the second merge.

Then, all the other examples (eg under ~/components) will need to be tested and have metadata added (they will remain where they are in the Laz src tree but will also be copied to user space when needed). This will be quite long series of merge requests.
 

They must be copied under the Components directory at some point...... options page registration ?

OK, right now, its a Package, its gets everything it needs from LazarusIDE variable, it does not need any options set at present (see note B). So, no, no "options page registration". I have no idea what that means by the way.

Which Project Wizard you mean? The Project -> New Project dialog?

Yep, thats what I mean.  its caption is "Project Wizard" and it currently has a Button that triggers the current Examples Window. Maybe that can be done with the LazarusIDE variable, it seems to be amazingly flexible.

Note A - A number of Examples have been edited, updated or skipped because they are no longer useful. For example some Examples needed extra units "used" as things have been moved. Many Examples have been skipped because they pre-dated the Object Inspector. Some Examples are still marked as needing some more work. That was substantial amount of effort and was needed for any of the outcomes. While it does improve the overall quality, more needs to be done. So far, the Examples have only been tested on Linux, they will require testing on Windows and MacOS too.

Note B - The new Example Window reads things like LazConfigDir from the LazarusIDE variable. Will need to read LazSrcDir. At present it has no options of its own but at least one person has requested the ability to direct downloaded (or copied) Examples to a directory of their choice rather than automatically under LazConfigDir. I assumed this would a small config screen associated with the new Examples Window, to be done later.  Is this what you are referring to as "options page registration" ?

[JuhaManninen]

I'll reverse the directory restructure (because it will confuse the existing Examples Window).

Just remove the existing one as I initially suggested. I don't know why you want to keep it there.

Then I will need to make some changes to new Examples Window so that it looks to the examples in the local directory rather than on-line. Easy, I'll use some {$ifdef so we don't loose (yet) the online capability. This solves my MacOS problem too. 
If, at some later stage, we do decide to go online, the underlying structure will still be there.

I think online is the way to go. Supporting both local and online examples would be best.

Note B - The new Example Window reads things like LazConfigDir from the LazarusIDE variable. Will need to read LazSrcDir. At present it has no options of its own but at least one person has requested the ability to direct downloaded (or copied) Examples to a directory of their choice rather than automatically under LazConfigDir. I assumed this would a small config screen associated with the new Examples Window, to be done later.  Is this what you are referring to as "options page registration" ?

Yes, a package that integrates in the IDE and needs settings should add a configuration page (or pane) to the global options window, Tools->Options.
IdeIntf has an API for it. I can find an example package that already does it later. ToDoList maybe...