Online Package Manager

[minesadorada]

Re:Voting system
Could we have a TPopupNotifier or similar show over each star graphic showing what it means?  That way, the voting would be equitable.

..something like:

Popup text over 1 : "Unuseable. Buggy and/or obselete"
Popup text over 2 : "Compiles/Installs, but has many unfixed bugs"
Popup text over 3 : "OK, but poor support for fixes and/or updates"
Popup text over 4 : "OK, but unsuitable for production"
Popup text over 5 : "Perfection!"

[balazsszekely]

@Juha

I didn't mean the external JSON thing only. I meant the authors and maintainers should also deliver new versions for the master repository. That is the "official" place after all.

That was my original idea, a well maintained central repository, but then everyone wanted updates  . Anyway, once the database server is up and running(I already discussed with Marc about the details), we can give rights to package maintainers, so in the end a direct upload to the central repository will be possible in the future.

The external JSON update still looks like a little confusing design oddity to me, but no worries, I believe it is needed.

Every major package system works in a similar way. Maybe you didn't follow the latest post, but almost every limitation is lifted now. The package maintainer doesn't have to follow any naming convention, zip structure, etc. Just generate a JSON with OPM, fill in two items, and that's all there is to it. If someone can came up with more simple design I'm open to suggestions.

Hey, let's document things first for a change. What will the server SW do? Could you at least make a bullet point list please. I can try to make a sequence diagram after it.

I do not understand the question, I mean what list should I create? The server SW is only needed as a gateway between OPM and the database(it will be PostgreSQL), since a direct connection is not safe.
The database will contain the:
  - table with users(package maintainers) and hashed passwords
  - table with user rights
  - table with packages
  - table with votes
  etc.
With time I plan to replace the main JSON(from the central repository) with a table. It's much easier to maintain then a JSON file.

@minesadorada

Re:Voting system
Could we have a TPopupNotifier or similar show over each star graphic showing what it means?  That way, the voting would be equitable.

..something like:

Popup text over 1 : "Unuseable. Buggy and/or obselete"
Popup text over 2 : "Compiles/Installs, but has many unfixed bugs"
Popup text over 3 : "OK, but poor support for fixes and/or updates"
Popup text over 4 : "OK, but unsuitable for production"
Popup text over 5 : "Perfection!"

Ok, but we all must agree on the list, because it can be very subjective. For example in your list after 4(which is still a crapy level) comes 5 Pefection

[JuhaManninen]

That was my original idea, a well maintained central repository, but then everyone wanted updates  . Anyway, once the database server is up and running(I already discussed with Marc about the details), we can give rights to package maintainers, so in the end a direct upload to the central repository will be possible in the future.

Sometimes it makes sense to follow your vision and not listen to others (much).
Anyway, let's see how it will work out.

Every major package system works in a similar way.

Not really. They usually have only a central repository. Then you can see if your local package is up-to-date or not. Easy, logical and intuitive.
With our hybrid system a local package can be up-to-date with the central repository but not with an external update source, at the same time. Not logical nor intuitive.

I do not understand the question, I mean what list should I create? The server SW is only needed as a gateway between OPM and the database(it will be PostgreSQL), since a direct connection is not safe.

List the technical details of what the gateway does. Protocols used over the net, data formats, authentication methods (is the forum user account still the plan).
How do you ensure the secure connection? Protocols and libraries used for it?
Is asynchronous communication needed, in addition to what TCP/IP already does?
Etc...
Think you are delegating the job to me and you must explain what must be done.

[JuhaManninen]

Popup text over 1 : "Unuseable. Buggy and/or obselete"
Popup text over 2 : "Compiles/Installs, but has many unfixed bugs"
Popup text over 3 : "OK, but poor support for fixes and/or updates"
Popup text over 4 : "OK, but unsuitable for production"
Popup text over 5 : "Perfection!"

Such explanations are not needed. The quality is always a subjective matter and it must be so. There is an infinite number of criteria by which people can judge a package. Trying to fit them all into those short explanations is a swamp.
If you look at other sites that allow voting, they also don't try to explain the scale.
For example SourceForge allows voting for different properties:

• Ease
• Features
• Design
• Support

each having from 1 to 5 starts. The star-scale however is not explained because (I guess) it is so obvious.
1 star means bad and 5 stars mean good.
That is a sufficient explanation for our case, too. A scale from bad to good.

BTW, your texts already raise questions. No package can reach "Perfection!", thus 5 stars cannot be used. Even 4 stars is still unsuitable for production, thus no Lazarus package can be used in production.

[minesadorada]

Popup text over 1 : "Unuseable. Buggy and/or obselete"
Popup text over 2 : "Compiles/Installs, but has many unfixed bugs"
Popup text over 3 : "OK, but poor support for fixes and/or updates"
Popup text over 4 : "OK, but unsuitable for production"
Popup text over 5 : "Perfection!"

Such explanations are not needed. The quality is always a subjective matter and it must be so. There is an infinite number of criteria by which people can judge a package. Trying to fit them all into those short explanations is a swamp.
If you look at other sites that allow voting, they also don't try to explain the scale.
For example SourceForge allows voting for different properties:

• Ease
• Features
• Design
• Support

each having from 1 to 5 starts. The star-scale however is not explained because (I guess) it is so obvious.
1 star means bad and 5 stars mean good.
That is a sufficient explanation for our case, too. A scale from bad to good.

BTW, your texts already raise questions. No package can reach "Perfection!", thus 5 stars cannot be used. Even 4 stars is still unsuitable for production, thus no Lazarus package can be used in production.

I'm willing to be convinced. Let's see how the voting system works out without explanations...
My original list was just a flag to raise - I expected it to be changed by consensus.

[balazsszekely]

Not really. They usually have a central repository. Then you can see if your local package is up-to-date or not. Easy, logical and intuitive.
With our hybrid system a local package can be up-to-date with the central repository but not with an external update source. Not logical nor intuitive.

The thing is we have an update system now, we can't put the toothpaste back to the tube.  Seriously it would be a shame to drop it.

List the technical details of what the gateway does. Protocols used over the net, data formats, authentication methods (is the forum user account still the plan). How do you ensure the secure connection? Protocols and libraries used for it? Etc...
Think you are delegating the job to me and you must explain what must be done.

OPM with TFPHtmlClient sends Post/Get requests to the web server like any web browser would do. The protocol is https. It's logical to use it, since the traffic is already encrypted + the database can be also accessed with a browser. The web server can be Apache or a full fpc based server. A third possibility is an Apache with a FastCGI coded in fpc. FastCGi basically it's a interface between a program and a web server. The data format is simple, just basic command for the database: "Ïnsert into TableName(...) values(...)". I cannot tell you more details about the actual implementation until it's not decided what to use: php vs. fpc fastCGI vs full fpc server(it's an overkill for a small project like this). If we choose php I can do all the coding alone, if not I need help because I have little experience in server side coding with fpc. It would be fun to learn, but I'm busy implementing other stuff in OPM. The database server is PostgreSQL, according to Marc is very efficient and secure. I use firebird in my applications. Regarding the forum user account, it can be used, but since we create a new db. with users and passwors for voting/uploding packages to central repository it make sense to use the new username and password.

[Rayvenhaus]

So, the best way to update OPM at this time is to use SVN?  It's not showing any updates from inside OPM for OPM.

[balazsszekely]

@Rayvenhaus
So, the best way to update OPM at this time is to use SVN?  It's not showing any updates from inside OPM for OPM.

Yes, please update from SVN. I almost every day commit something new, it would be an overkill to update the central repository each day. More over, I think I will remove OPM from the repository. It make no sense to keep it there since it's part of lazarus.

[JuhaManninen]

The thing is we have an update system now, we can't put the toothpaste back to the tube.  Seriously it would be a shame to drop it.

Yes. I am not even suggesting that.

Regarding the forum user account, it can be used, but since we create a new db. with users and passwors for voting/uploding packages to central repository it make sense to use the new username and password.

So your client GUI will have an option to create a new user account? That is OK.

I don't think the CGI / FastCGI makes much difference because voting does not stress the server heavily.
CGI would be easy to implement.
Anyway, someone with experience of FPC on server side should make skeleton code which can then be improved.
Or maybe an existing example can be used as a skeleton.

[lainz]

I have a question, once the system is ready, for example I create an account it counts for voting and uploading packages, what's the limit? For example I can upload multiple packages, upload duplicate packages, upload some malware. Or it will be like is now moderated?

Edit: A suggestion -> when there's a new version available in the repository show it in bold, the same as when there's a new external version.

[JuhaManninen]

I have a question, once the system is ready, for example I create an account it counts for voting and uploading packages, what's the limit? For example I can upload multiple packages, upload duplicate packages, upload some malware. Or it will be like is now moderated?

Direct upload must be allowed only for few and selected people. There will be some admin work involved when the write access is given to them.
Rating will be allowed for anybody who logs in.

Let's see how the rating works.
Typically the results are biased to the positive side because only people who use and like a certain package rate it.
People who don't use it, don't rate it either.
I just noticed that Tcl is the "Projects Of The Month" in SourceForge.
 https://sourceforge.net/projects/tcl/
It got 60 ratings and all of them full 5 stars. Uhhh! In reality it is a horrible language. I guess 60 people in the world like it and they all went to rate it.

[balazsszekely]

@lainz
I have a question, once the system is ready, for example I create an account it counts for voting and uploading packages, what's the limit? For example I can upload multiple packages, upload duplicate packages, upload some malware. Or it will be like is now moderated?

Yes, it must be moderated to prevent spam, malware etc.

A suggestion -> when there's a new version available in the repository show it in bold, the same as when there's a new external version.

Done r. 53704. Please test.

@Juha
It got 60 ratings and all of them full 5 stars. Uhhh! In reality it is a horrible language. I guess 60 people in the world like it and they all went to rate it.

This is funny!

[minesadorada]

Initial Wiki page for the External JSON Editor:
http://wiki.lazarus.freepascal.org/opmjsonupdateeditor

It got 60 ratings and all of them full 5 stars. Uhhh! In reality it is a horrible language. I guess 60 people in the world like it and they all went to rate it.

This can be the problem.  In a 5-star system, voting stars 2,3,and 4 are pretty meaningless without explanation, so people usually vote nothing or 1 (bad) or 5 (good) - a bit like the FaceBook 'like' button.
If the displayed stars are an average of all votes this could be sort-of okay.  The best-of-all system would be a reviews list webpage with individual ratings and comments (like imdb) but I do understand this would be shooting too high at this stage.  Let's see how it goes.

Re Direct upload: Perhaps to a 'sandbox' repository; then the moderator vets before moving to the 'real' repository?  This has the advantage that the author can test it him/herself (changing OPM/Options/General/Remote Repository to the sandbox URL) before notifying the moderator.  Only the moderator would have write access to the 'real' repository, which is the secure and accountable option.

Re Direct 'Upload': IMO the easiest way is to manage it is via svn access to the sandbox repository.  It could even be hosted for free on SF, so logins/account management/malware scanning/version control etc  are already done for you. (just add the prospective author's SF accountname to the OPMSandbox project as a 'Developer')  It already works fine for ccr and svn is available for all platforms AFAIK.  In SF there are separate permissions available for file management and svn access.

[minesadorada]

OPM Rev 53704:
Screenshot 1 - after a ForceNotify
Screenshot 2 - after a version change

Question: Is there a process planned for merging updates into the main repository after a suitable period?

[balazsszekely]

@minesadorada

Initial Wiki page for the External JSON Editor:
http://wiki.lazarus.freepascal.org/opmjsonupdateeditor

You should add a link to your wikipage: http://wiki.freepascal.org/Online_Package_Manager#Create_JSON_for_upates

OPM Rev 53704:
Screenshot 1 - after a ForceNotify
Screenshot 2 - after a version change

The screenshots are OK. There should be no difference visually between ForceNotify and a Version change(except the version number).

Question: Is there a process planned for merging updates into the main repository after a suitable period?

Yes. Most likely will be an option to directly upload a package to main repository. Authentication needed.