Example projects window

[dbannon]

I never advocated a ZIP file - I advocated links to SourceForge et al code repos.

Example projects need to be bundled into something and zip is both the Lazarus preferred model and is reasonably cross platform. if not zip, what else would we use ? Contents could just sit as files in a directory on, eg Gitlab, one directory per project but maybe a little harder to manage.

Again, I never advocated either ZIP files or uploading them to the Wiki.

No, you did not. And I clearly misread that. But you equally clearly missed my words that "I am definitely not against the idea".

You seem to have setup a straw man just so you could demolish it

I think that is unfair and uncalled for. It really does not matter where we keep gz/xz/zip/dir contents, if it has access similar to the wiki, its not safe enough. If it has gatekeepers, that can work as long as we can manage the extra load. Its the usual security v. convenience argument, neither SourceForge nor Github/lab would allow unauthenticated uploads anyway.

This idea is far from a bad one, it just needs some discussion, please encourage it.

Davo

 

[JuhaManninen]

Because, from what I understand, one of the issues in the bug report is that end users who install Lazarus using deb/rpm/pakman etc end up with it all in Read Only space.  Its not a packaging thing, its a ReadOnly issue. So, if they try a build an example project, it cannot write. I, personally always build from source but lots and lots of people install distro packages.

A user must be able to compile the code while it is open in the IDE. He may want to modify and debug it, just to learn and experiment.
All those Linux packages would require lots of maintenance work. It is not realistic. They are not even very useful because the ReadOnly issue remains.
An online example project system indeed would solve the ReadOnly issue.

Category : beginner
Keywords : buttons, memo

Yes, both Category and Keywords sound good.

Returning to the original issue, improving the current Example Projects GUI...
It can and should be done anyway, regardless of any future online systems.
Is somebody willing to do it? What do you think of my mockup GUI?
If nobody else volunteers, I will look at it during this winter.

[af0815]

A example system like OPM - i said like !! The goal are, it can be maintained and you can download only the examples you want. Not needed to load all examples at once. If you use gitlab, how can you saperate all the examples ? And you must make it possible to test the examples for the actual stable (and comming) version of Lazarus and FPC. Nothing more boring than a not working sample.

But this is definitly not a job for the actual OPM. It can use the techniques, but must have a team to handle this.

ANd what is about fppkg, as nmed in the first post - is it useable/working for Lazarus too ? I have never heard in Lazarus (except of problem at installtime).

[Martin_fr]

Sorry, Martin. This is not a good idea: It will be an unacceptable burdon to GetMem who handles all the additions to OPM manually. And unless "examples" are separated from "components" in some kind of GUI, components will be buried underneath loads of examples.

Well, yes: But the burden can be shared. With the correct infrastructure to maintain the OPM database, other could help.

If the database is moved to a gitlab repo (or the database is (auto-)generated from a git repo), then all team members (or selected team members) could help.

And, in any case

Doing it via opm, and the "example window" searching packages, also offers the advance that existing packages (e.g. packages that add new components), can include examples for those components.

needs to be taken care off.

Already today "ATSynEdit" has a folder "app" which contains demos.
It would be great, if the "Example Window" could show them, wouldn't it?

But if, that is the case, then we have an entirely working system for "online examples". Should we then really duplicate it?
Or should we think about how to distribute the maintenance work?

[Martin_fr]

Because, from what I understand, one of the issues in the bug report is that end users who install Lazarus using deb/rpm/pakman etc end up with it all in Read Only space.  Its not a packaging thing, its a ReadOnly issue. So, if they try a build an example project, it cannot write. I, personally always build from source but lots and lots of people install distro packages.

When rebuilding the IDE, ppu files (and the new exe) are written to a folder inside the primary config path.

A similar concept is needed here.
Plus, if the user wants to change the files, then the project should be saved to a new location at this point. (the IDE could offer that, whenever there is a write permission issue).
Or the "example window" could offer to copy the files before opening the example.

Such functionality could also go along with "files saved for compilation" should not be saved in the project folder, because you may not want to change your files. Compilation should be done by saving to a temp folder (all that as an optional feature). But that part is off topic.

[Martin_fr]

1) Remove all the examples from the distributions

* Reduces distribution size (I didn't even know there were examples)

It's 60 MB, but text only. Compressed maybe 10 MB.
I think that is the least of the issues.

3) Link to a to-be-created Wiki page which lists example category pages
4) Add examples to appropriate yet-to-be-created category pages
5) Example entry in Wiki links to SourceForge/Github/Gitlab for the source for each example

IMHO, some examples should be included in the main distro.
Maybe that could be more selective.

If the examples are included. And if they actually work, then that removes one (big-ish) source of problem for users.
Examples would probably more often be used by less experienced users. So that actually matters.

If some examples would be removed from the default-distro, then there should be a download via OPM (or if needs must, something similar). IMHO Download links to "SourceForge/Github/Gitlab"  are not a good replacement.

If Examples are stored online (for opm), or even included in the distro, then nothing stops us from having the description in a readme, and displayed in the IDE example window.
An additional wiki page is still ok (some examples already have this)

[wp]

One thing to be considered when Examples are removed from the main distribution: Sample code is coupled to the distribution. When a Laz version introduces a new feature the corresponding sample project will adopt to the new feature. As a consequence the sample will not be usable by older Laz versions. This means that OPM (or whatever makes the samples available) will have to keep sample projects at a variety of versions!

Isn't it possible to have the installer copy the Examples folder after installation to user space?

[Martin_fr]

Isn't it possible to have the installer copy the Examples folder after installation to user space?

Well this is mainly relevant for installations to the default/global install folder.

Such an install can then be used by any user on this system. Even by users (accounts) that are created at a later time.
So those users do not have a home folder yet. Where should the installer move the files too?

The IDE, could copy the example folder to the primary-conf-path, on its very first run. It does the same with the default "environmentoptions.xml" that is in the main folder of the install. That is, if the pcp does not yet have an "environmentoptions.xml".

But that creates a new problem. The very problem that we already have for the "lazarus.dci" file.

The "example folder" and the "lazarus.dci" can have updates in newer versions. But the copy in the pcp can contain user changes.
So with the "lazarus.dci" currently you do not get new templates when you update your IDE.
And that would happen to examples too.

If we copy, examples on request, we have the same problem. But at least on a smaller scale. Also in that case, because we still need to implement this copy-code, we can add a dialog and ask if a new copy from the install folder should be created.
I.e. we can list the copy, and the original as two entries. We can check, if there is a diff between them (and it does not matter if the user made changes, or the install was updated). If the user made changes, they may want the option to continue with them, revert their copy, or make a new copy in a different folder.

[dbannon]

OK, so we have two broad models available. Both models require a review of and (indexing, meta data) additions to existing set of examples.

Built In
An improved version of what we now have. It requires -

• A rewritten GUI
• (Linux Package Only) A means to move the example projects into user space. This is not an install time operation because root installs don't know about users. Nor should it move all Examples into user space, many user may never look at more than one example. So, it sounds like its a case of when the user selects a particular example, its moved into user space (in the ~/.lazarus area) and opened in te IDE. Thats easy.   

Online Model
I think overloading the OPM would be a bad stratagic mistake. The OPM is an incredibly valuable resource and one we should not complicate or enlarge without due consideration. Too many eggs in one basket argument applies.

But a new stand alone, online service may well be a good idea. I have been working recently with an app that hits the Github API, its very rich and easy to use. I expect Gitlab has the same approach.  Issues include -

• The need for Gatekeepers. No, not getmem again!
• Its a far bigger change than the alternative model.
• It would scale better, making the distribution a bit smaller is a good idea.
• Its good PR, those little project names will come up in the search engines.

In answer to Juha's question, I am certainly willing to be involved but we both know Juha would do a far better job (but good use of his time ?). But we must agree on an approach first. We seem to be a long way from there now.

OK, some wild and uninformed speculation of how an online model would work -

Examples and Demos are defined as small projects demonstrating only a small number of features. This is not a show case and a projects with more than X lines are not accepted. Given the small size of projects, pull requests may not be the best way to handle additions/updates ??

A github/lab site, maybe under the existing FPC one. Several gatekeepers are given commit access. Top level is category, below that are all the projects in that category, pairs of files, a metadata file and a archive (a zip file).

Gatekeepers are sent a metadata file and a zip file of a standalone project, maybe a lazarus project or just a single code file. Only the gatekeepers must have git installed.

They need to examine code for safety (and correctness ??) If happy,  its pushed.

A script reads all the metadata (or just the new one) and category based pages are generated. Either on the Lazarus/FPC wiki or the gitlab wiki ?? A master metadata file is generated.

Potential end users can download those zip files themselves.

Lazarus has a GUI window that reads and displays the master metadata file, a click will download the selected zip (into ~/.lazarus config area) and open it in the IDE.

Too easy !

EDIT : WP's issue about changes due to version of FPC/Lazarus. Most examples would be FPC/Lazarus independent. Those that demonstrate something that exists in one version but not another would be marked accordingly.

Davo



 

[dbannon]

OK, here is a first cut at a gitlab examples repository

https://gitlab.com/dbannon/fpexamples

I have put two categories up, Beginner and NoDesigner, just as eg.  But I have put two versions of each up, Beginner and BeginnerAlt, difference being how files are stored in there. The BeginnerAlt is more traditional, unarchived and is more readable. Thoughts ?

And, yep, dead easy to use their API to read the metadata and to download files from there.

Davo

[Martin_fr]

OK, here is a first cut at a gitlab examples repository

https://gitlab.com/dbannon/fpexamples

Not sure why the name is "FP..."?

1) Why zip?
- If we have IDE integration, we can download files/folders as required
- If there is more than one example, GitLab offers "download folder as zix" => so all that is needed is "each example has a folder of its own"
- Without zip, examples can be browsed online.

2) "Hello World"
I try to imagine a user to whom this is of value....

IMHO, this really would need to be a tutorial.
=> That is, an IDE plugin that would open a "guide window" leading the user step-by-step through the process of creating the "hello word" project.
Starting the tutorial, the user would get instructions to
- New Project: Application
- Select TMemo on the component bar
- Click the form, and drag the size, or resize afterwards
- but the buttons on the form
- add the events
- run
(actually, one button to change the form.caption would be a fine start...)

Just seeing a form, with two buttons, even with some text explaining how the got there....
Yes, on the wiki, with images, that would be different. But that would defeat the idea of "like OPM" (download, and explore locally).

3) Examples for components
Like IIRC the AtSynEdit demo. A form with the component, and the buttons, checkboxes, ... to modify different properties of the component.
So the user can choose different "line-wrap" options, and immediately see the effect.

Only problem(s), as pointed out before:
- If you use OPM to get AtSynEdit, then you need a 2nd download for the examples (if indeed the decision is to have examples in a diff repo).
- If you explore the OnlineExamples, and download AtSynEdit-examples, then there must be a dependency to trigger OPM.

As far as I can see, there is a given dependency between OnlineExamples and OPM.
Attempting to divide them, will create extra complexity.

We do already have OPM packages, that include examples (component + examples). I doubt we can prevent this.
Whatever "tool" (example window , online-example-mgr) we will have in the IDE:
=> It must display
- examples from the installer
- examples from OPM packages (at least the "component + examples")
- maybe OPM packages, not yet downloaded, but know to have examples included? (they would need to have a flag)
- and (if indeed created): the "online example only repo"

[JuhaManninen]

In answer to Juha's question, I am certainly willing to be involved but we both know Juha would do a far better job (but good use of his time ?). But we must agree on an approach first. We seem to be a long way from there now.

Actually I only promised to work on the window for local example projects, yet I am happy if somebody else does it. Whoever does it, let other people know to prevent duplicate work.

For an online project system it makes no sense to store ZIP packages in a revision control system like GitLab. Revision control is meant for source code.
ZIP files can be stored in the same server as the OPM packages. Remember, the sources of OPM packages are also maintained in some external revision control system.
A competing concept would be to load source code directly from GitLab or GitHub. It was already discussed when OPM was designed. For OPM it didn't fit well because quality control is difficult to enforce. For example projects it would work better as quality control is not as important.
There is such a plugin for Delphi's packages and projects IIRC.

[dbannon]

Not sure why the name is "FP..."?

It stands for Fine Programing.

- If there is more than one example, GitLab offers "download folder as zix" => so all that is needed is "each example has a folder of its own"

Indeed ! I did not know that (Github, my usual platform does not allow that). I have restructured it. Thats heaps better !

2) "Hello World, I try to imagine a user to whom this is of value...."

Its only an example of an example. But new user of a programming language, build system etc looks for Hello World. Its of no earthly use expect allowing to new user to confirm compiler works as expected and binary runs. It takes up no room but there is no commitment to it being there at the end if you find it a problem.

IMHO, this really would need to be a tutorial.

Sure, this model would support a tutorial series, but it would be silly discussing content before the framework is agreed on.

Only problem(s), as pointed out before:
- If you use OPM to get AtSynEdit, then you need a 2nd download for the examples (if indeed the decision is to have examples in a diff repo).
- If you explore the OnlineExamples, and download AtSynEdit-examples, then there must be a dependency to trigger OPM.
.....

The need to resolve dependencies is in just about any development environment. People who are going to stay with us and develop real apps must learn that process. The example project we are addressing here is just where they start out, we cannot undertake to solve all problems beyond that stage. That said, its sure no worse than where we are now, if AtSynEdit decides to move its examples to this project, the meta data will say, "but first install AtSynEdit" and docs for AtSynEdit will say look for the examples in Lazarus Examples. I guess it will depend on just how "3rd Party" a package is.

Thanks for your input Martin

Davo

[dbannon]

Actually I only promised to work on the window for local example projects, yet I am happy if somebody else does it. Whoever does it, let other people know to prevent duplicate work.

I already have code that can upload and download from Github, superficial experiments yesterday indicates Gitlab is much the same. So, maybe I get the transport side of things working and you make it look presentable ? I'd just use my temp gitlab repo, later we'd need to redirect it to somewhere more official. And then integrate that into Lazarus.

The task of reviewing existing projects, adding necessary meta data, pushing up to a repo ......

My experience, from a past life, is managing projects like this can be done from a wiki page, does that suit ? 

For an online project system it makes no sense to store ZIP packages ...

Yes, Martin pointed out that Gitlab (unlike Github) allows an interactive user to pull down a directory, converting to zip on the fly, thats excellent.

The question that needs to be resolved is, IMHO, is this the way we want to go ?

(Please note, I am NOT trying to grab ownership of all this, far from it, but I am just trying to push it along ....)

Davo

[Martin_fr]

2) "Hello World, I try to imagine a user to whom this is of value...."

Its only an example of an example. But new user of a programming language, build system etc looks for Hello World. Its of no earthly use expect allowing to new user to confirm compiler works as expected and binary runs. It takes up no room but there is no commitment to it being there at the end if you find it a problem.

Sorry, I did not meant to make this about "finding a problem". But rather "pointing out more potential".

I am aware, this is an "example of an example". And for that, it could have been an empty "program Foo; begin end;".
- It shows the infra structure (e.g. meta data, repo structure, ...).
- It gives some ideas of what the "example system" would have to comprise.

So, indeed my introduction about the lack of "value" was wrong. My apologies for that.

I thoughtlessly used this as an opener, to bring up the idea of tutorials (see meta-data below) as something to comprise. Your example just happens to the kind, that IMHO could be made into a tutorial.

IMHO, this really would need to be a tutorial.

Sure, this model would support a tutorial series, but it would be silly discussing content before the framework is agreed on.

framework and content are somewhat bound together....

The more is known about the content, the better the framework can be planed.
Sure "KISS" is a could principle. And we can add more as we get there.
But
- meta-data files will have to be extended too, and once the first version of the "example mgr" is out, meta-data requires backward compatibility.
  Planing ahead, can help with that. E.g. planing for different example kinds: component-behaviour, tutorial, comparing-related-features, ...
- The form and place of storage (folder vs zip ...) are also long term decisions.
- Compatibility (and/or integration) with OPM, is yet another "hard to change" decision.
  As pointed out: OPM packages already contain examples. So our meta data, and storage requirements should allow such OPM packages to be recognized.

So the below is not about what to implement now.
But about, what to consider when making decisions.

The OPM integration has still a lot of unconsidered points.

Well, I guess - ideally - the "example window" (in its new form) will show everything that is (or has) example(s).
- installed
- online
- and if our online DB is separated from OPM, then also OPM packages with examples

But in reverse, if a user browses OPM packages, and if they (or some of them) have meta-data about their examples:
- Should then the local OPM browser window, not also allow to filter by "has example" ?
- And if the local OPM browser window, can show "only with example", should then the local OPM browser window not also be able to show the entries from the new example repo?
  (Especially, since the new example repo could also contain examples for OPM packages. Just examples that have not been bundled with the OPM package itself.)

Note that:
- "local OPM browser window" and "OPM online database" are separate entities.
   Changes to the "local OPM browser window" do not affect the maintenance work of the "OPM online database".
- Adding "example meta data" to OPM packages may affect that maintenance.
   But this is independent of changing the "local OPM browser window". This would be desirable, even just for the new "example window".

So, unless we want to exclude OPM packages from having examples (within the example window), it seems to me that the two need to be very closely coupled.

We can have separate databases for the two indeed. IMHO that works well with any point I mentioned above.
It does mean however:
- the example system must access both databases.
- Contributors must deal with two teams of maintainers to submit their work.

The latter is, if they submit work in both categories. OPM packages, that just happen to include examples would be dealt with by OPM maintenance, and thus the OPM maintainer would also decide about some additions to the examples list.

Anyway: Just my two pennies.

If it is thought better to limit the extend, then that's that.
Also if the idea to add those later, and the hope is that the code will be easy to change, even if it was designed without taking those possibilities into account, well then hopefully this will work out.

Otherwise, consider taking them into account now.