Online Package Manager

[balazsszekely]

Hi,

 I decided to implement the online package manager(opkman). The first alpha version is ready and it works like this: the packages are zipped and stored in a webserver(repository) along with a json file. I chose zip to minimize server load. The json file contains all the necessary info about the packages. Once installed into the IDE, the opkman downloads, then serialize the json file into a package list. The list is  displayed in a tree. On request the packages can be downloaded or/and installed. For now the repository is read-only, it contains only eight, well known(I guess?) packages. The nature/type of the eight package is irrelevant for now, since we are only testing the functionality of the opkman. Few notes:
 
 1. Supported versions: Laz 1.6+(FPC 3.0.0+). I did the implementation/testing with Lazarus trunk(FPC 3.0.0)
 2. Tested under the following widgetsets: Win32/64, GTK2, Carbon
 3. After install, you can find opkman in the "Package" menu
 4. Link: http://wiki.freepascal.org/Online_Package_Manager
 
 Please test! Suggestions are welcome.
 
 PS: Big thanks to @Leledumbo for hosting the repository.
 
regards,
GetMem


Edit: stable != alpha(thanks Graeme) + screenshot added.

[Graeme]

stable(alpha) version

stable and alpha are two opposites of the scale. I had to laugh when I read that.

Thanks for sharing, and good luck with the package manager though.

[Bostjan]

Is this to be something like "NuGet" for MS visual studio ? (https://en.wikipedia.org/wiki/NuGet, https://www.nuget.org/)

If so, I think it is really good idea. Worst part of Delphi/Lazarus for me is that you must hunt various packages across the internet and central repository would be so much better.

[balazsszekely]

@Graeme
stable and alpha are two opposites of the scale. I had to laugh when I read that.

What I meant to say is the first alpha that's stable enough for test, but you're right, it's kinda confusing.  Point taken, I edited my post.

@Bostjan
Is this to be something like "NuGet" for MS visual studio ? (https://en.wikipedia.org/wiki/NuGet, https://www.nuget.org/)

Yes, something similar.

Worst part of Delphi/Lazarus for me is that you must hunt various packages across the internet and central repository would be so much better.

I agree 100%.

[rvk]

Nice work

Error in trunk:

Code: [Select]

opkman_mainfrm.pas(129,48) Error: Incompatible types: got "<procedure variable type of procedure(TObject;AnsiString;TErrorType;const AnsiString) of object;Register>" expected "<procedure variable type of procedure(TObject;UTF8String;TErrorType;const AnsiString) of object;Register>"
on

Code: Pascal  [Select][+][-]

1.   PackageDownloader.OnJSONDownloadCompleted := @DoOnJSONDownloadCompleted;

Making it this works:

Code: Pascal  [Select][+][-]

1.     procedure DoOnJSONDownloadCompleted(Sender: TObject; AJSON: UTF8String; AErrTyp: TErrorType; const AErrMsg: String = '');

TJSONStringType is an UTF8String in trunk.

Suggestion #1: Making the packages-list more compact. Only expanding when choosing the +

Suggestion #2: Putting the "Online Package Manager" above the "Install/Uninstall packages". I'm very used to be "Install" being the last in that menu Although, if this becomes the standard the position at the bottom might be preferred.

Note: The GetIt Package Manager of Delphi resides in the Tools menu. But Delphi doesn't have a dedicated "Packages" menu, only a Component-menu where "Install packages" sits.

Question: What packages will be provided in the end. Will the lazarus-ccr packages/components be added? Now you have the standard packages in the Lazarus distribution, the lazarus-ccr and everything else with separate download via Wiki-links. Maybe all these should be combined.

If these are not combined maybe it should be possible to add multiple "Remote repositories". (That would also allow you to make your own repository in addition to the default ones provided)

[Phil]

(That would also allow you to make your own repository in addition to the default ones provided)

I would assume this would be the case. That way, a package author only needs to publicize the URL of the package zip. The user types/pastes this URL into the package manager, which does the rest.

I would use a separate .zip for each package. And each .zip would contain its own JSON file for the package. Put a master list of packages elsewhere.

Take a look at how this is implemented in other IDE/apps. For example, QGIS has supported Python plugins for years and has hundreds of plugins to manage. Look at the QGIS External (official) and Experimental plugins dialogs here:

http://www.qgistutorials.com/en/docs/using_plugins.html

Note that official plugins need to be approved. Here's their list of items to pay attention to for fast approval:

http://plugins.qgis.org

[lainz]

I agree with Phil, so for example I can upload a release on GitHub that in fact is a package, and then anyone can just copy and paste the URL to this program and get the package installed.

[Phil]

Exactly. The only question in my mind is whether there should be any sort of "official" stamp, or perhaps just a requirement that the package have an appropriate license (modified LGPL, for example), documentation, etc. as indicated by the contents of the JSON file.

[howardpc]

Apart from questions of improving the functionality, reliability, security etc. of the package manager, careful thought needs to be given to non-software issues such as long term funding of webserver costs, who (or what grouping) has control of the 'official' repositories, what are the criteria for inclusion/rejection of submitted packages, whether some PPA-type system should be in place for addition of private repositories, who will police it in terms of enforcing adherence to the stated policies, and so on.
This is a bold start, but there could be a minefield ahead in terms of future maintenance and oversight, not so much of the software itself, but of the data it manages and the ongoing quality of that data.

[lainz]

If money is an issue I recommend to host it on GitHub/Sourceforge that has free website that is managed with a repository/ftp for project or organization.

Use standard releases for each package and only mantain a list of json that points to another url where the real zip is available (the zip can be in any host service).

If you choose GitHub/Sourceforge you can add as many mantainers you want, simple give them access to put files in the website.

[rvk]

Use standard releases for each package and only mantain a list of json that points to another url where the real zip is available (the zip can be in any host service).
If you choose GitHub/Sourceforge you can add as many mantainers you want, simple give them access to put files in the website.

Yes, I would also advise using existing download locations. And because GitHub and Sourceforge are the most commonly used it would be best to implement downloading directly from there. It would also have the benefit of having the latest version and always be up to date.

The json-collection could even be hosted on Github or Sourceforge and downloaded from there on start of the manager. The json-collection would point to the actual download of given project. (Although there should be a solution found for the individual component/packages from lazarus-ccr because they can't be individually downloaded as zip)

Or it should be from here but I'm not sure how old those are. The RichMemo in the current manager is more up to date.

[Phil]

who (or what grouping) has control of the 'official' repositories, what are the criteria for inclusion/rejection of submitted packages, whether some PPA-type system should be in place for addition of private repositories, who will police it in terms of enforcing adherence to the stated policies, and so on.

I'm not necessarily suggesting an "official" repository. That might not work for Lazarus. For example, the QGIS repository has rules that are probably far too rigid for Lazarus (http://plugins.qgis.org). It needs to fit the "culture" of Lazarus. For example, QGIS is a very Euro-centric effort, with many core members being employees of Euro government agencies or their contractors, so a rigid submission policy might naturally fit their bureaucratic modus operandi.

However, that doesn't mean the project manager couldn't have technical requirements. For example (and only as examples):

(1) Package must have a proper license. If it's the same as LCL or FPC RTL, just state so or point to the FPC modified LGPL doc files. If it's something else, anything, just have a link to it (Eclipse, BSD, etc.). The only thing unacceptable might be if nothing is specified in the JSON file. By clearly stating what license is used, the user of the package manager can decide beforehand whether that's even a package they're interested in.

(2) If the package depends on LCL (ie, has LCL under RequiredPgks in .lpk file), then it must specify what LCL interfaces it has been tested against. I typically test against Carbon, Win32, GTK2 at a minimum so that I can legitimately say that the package is cross-platform.

That sort of thing.

[molly]

Just wanted to mention Delphinus in case it wasn't already (even though i seem to remember getting the name from this board).

Being able to at least store packages inside your own github account would help tremendously as right now things are scattered all over the internets. At least it would also allow for individuals to create their own packages so these things can be shared.

Decisions about a official or even semi-official package server would perhaps be something for the foundation as well ?

[Phil]

Decisions about a official or even semi-official package server would perhaps be something for the foundation as well ?

Lazarus already has official packages, which are the ones deemed worthy or essential enough to include with the IDE.

What a package manager can do is establish an unofficial list somewhere between the official IDE packages and the CCR slushpile. At the very least it indicates that a package is still alive and that the author is available to put in the minimal effort of "regularizing" it to meet the package manager's requirements.

[Graeme]

I agree with Phil, so for example I can upload a release on GitHub that in fact is a package,...

As I already mentioned in the mailing list, and somebody else mentioned it here... The best approach seems to be whan  Delphinus does on Github. Everything is hosted there.  Delphinus use the Github API to detect new packages and automatically present those in the IDE. Anybody can add there own packages via Github plus the required JSON file. No central repository or hosting location is needed. This is the best of all worlds. Now simply port it to Lazarus IDE and enjoy.