Online Package Manager

[lainz]

I agree with minesadorada, GitHub or SourceForge is the place to share code.

Anyone can create a new repository in their own accounts and then send you the mail (like I did for all my packages).

The thing that you can automate is the sending of the mail inside the OPM.

[minesadorada]

Anyone can create a new repository in their own accounts and then send you the mail (like I did for all my packages).

My point is: they don't need to create a repository outside of the OPM_CCR one.  All they need is a SF username and OPM_CCR access for uploads and svn commit.  Their 'stable_release' code is held in OPM_CCR.  If they want to copy it from some other repository, that's not part of the OPM equation.

All the OPM 'action' is in the OPM_CCR SF repository under svn version control.  It is however a 'sandbox' and no-one but the OPM moderator can alter the official online repository.  Direct uploads would be a disaster-in-waiting.

I stress 'version control' because I feel this essential to successful OPM deployment in Laz 1.8, and @GetMem has rightly handled it well for installation and updates.

[balazsszekely]

We discussed this before, if a login system must be used then better implement our own and host it in the Lazarus server. Using a third party hosting doesn't look to good in my opinion, even if sourceforge and github are more then professional services. It's like microsoft servers running on linux not IIS.
Until somebody implements the login system we need a temporal solution, even if it's not 100% secure.

[minesadorada]

We discussed this before, if a login system must be used then better implement our own and host it in the Lazarus server. Using a third party hosting doesn't look to good in my opinion, even if sourceforge and github are more then professional services. It's like microsoft servers running on linux not IIS.
Until somebody implements the login system we need a temporal solution, even if it's not 100% secure.

Fair enough - I sort-of agree.  But a SF-based system can easily be migrated when resources are available.  I can't see your objection to a 3rd-party supplier like SF or GitHub as important enough to delay setting up an initial system.  As I have said before, CCR has been hosted by SF for ages without complaint and it formed the basis for the initial OPM packages.

The assumption I would question is:  Why is a Lazarus server more 'professional' than a SourceForge server?  Both are open-source providers.  Which would you trust more if you were new to the Lazarus project, and which could you be more sure to continue to host your component code in 10 year's time?
'More professional' is an assumption that doesn't apply to everyone.

All this only applies to a 'sandbox'.  The 'real' OPM server should be in-house, of course.

[lainz]

What do you need?

A place to upload all zip + json packages and then be reviewed?

A place to upload all zip + json and automatically added to OPM?

Also if is the first, you will be the only one reviewing?

[balazsszekely]

@lainz
The first one, personally I would prefer a database, it's also useful for the voting system + other stuff. After a login everyone can upload packages(zip + json + updatejson), but the packages are only visible/downloadable after moderation. A few, trusted users will have reviewing rights. I prefer not to reviewing at all, since I'm busy with other things.

minesadorada

The assumption I would question is:  Why is a Lazarus server more 'professional' than a SourceForge server?  Both are open-source providers.  Which would you trust more if you had never heard of the Lazarus project, and which could you be more sure to continue to host your code in 10 year's time?

Probably the servers are less professional since it's an open source project with limited founding, but that's the whole point, if we manage to make a pascal based, safe login/package managment system, lazarus/fpc would look more professional to other people

[lainz]

Ok. We know what you need.

Sorry for my lack of knowledge, a database can contain zip files and json files?

The language you want to do the server side is FPC or PHP for example? ok you edited your message, it should be FPC

[balazsszekely]

@lainz

Sorry for my lack of knowledge, a database can contain zip files and json files?

Blob fields can contain any binary format.

The language you want to do the server side is FPC or PHP for example?

If I had the possibility to choose php, the loging system would be long implemented. However some core developer insist to be fpc as a showcase. I have little experience coding server side stuff with fpc,  I hopping somebody with more experience will help me. Another solution is to learn it myself, but since my time is limited the progress will be slow.

[minesadorada]

Probably the servers are less professional since it's an open source project with limited founding, but that's the whole point, if we manage to make a pascal based, safe login/package managment system, lazarus/fpc would look more professional to other people

Well, I guess it's what I argued for many posts ago on this thread, so I do agree.   Good luck with implementing as secure and user-friendly a system in-house as SourceForge before Laz1.8 comes out.  You know that SF gives you a free online DB - not that it would be needed for the sandbox system as I proposed.

I keep stressing it's a 'dual' system.  The sandbox in Sourceforge, but the 'real' server written in pascal and hosted in-house (this would include the voting system and DB)  The best of both worlds, with all the security risks and version control faffle borne by SourceForge and administered by a trusted person who is the only one with access to the 'real' server.

[balazsszekely]

@minesadorada
As a temporary solution we can use sourceforge, but without a login system. We need a public account, with limited storage place(100 Mb), where everyone can upload files, but only 1-2 person can delete. The upload must be done through OPMs "Create repository package" dialog? Forget the secure part, I know is not secure as a temporary solution is ok.

I keep stressing it's a 'dual' system.  The sandbox in Sourceforge, but the 'real' server written in pascal and hosted in-house (this would include the voting system and DB)  The best of both worlds, with all the security risks and version control faffle borne by SourceForge and administered by a trusted person who is the only one with access to the 'real' server.

Yes, this can be done. Move all the possible crap to sourceforge, and keep just the links in a db located in lazarus server + voting system + users. 

[minesadorada]

@minesadorada
As a temporary solution we can use sourceforge, but without a login system. We need a public account, with limited storage place(100 Mb), where everyone can upload files, but only 1-2 person can delete. The upload must be done through OPMs "Create repository package" dialog? Forget the secure part, I know is not secure as a temporary solution is ok.

I think you are distracted by automation.  Why should OPM manage the upload?
The only automation OPM could usefully do is send an email with the SF username to add to the OPM_CCR project.

There is a HUGE disadvantage to the SF temporary solution which I have not mentioned.  If you are have 'developer' access, you could potentially alter other OPM_CCR menber's code.  But this has been true for the SF CCR repository for ages, and it hasn't resulted in chaos.

I could potentially mess up the entire CCR with a careless commit, and could all the others with a CCR SF account - but it has never happened!

Investigate the SourceForge permissions system for file upload and svn access - it's better than you think.
If you can make a component in Laz/fpc you can surely manage version control and uploading to a SF 'sandbox' project!  No need for OPM to hold your hand..

[balazsszekely]

The only automation OPM could usefully do is send an email with the SF username to add to the OPM_CCR project.

There is no native component to send mails in fpc/lazarus. I cannot use third party components since OPM is part of lazarus.

[minesadorada]

The only automation OPM could usefully do is send an email with the SF username to add to the OPM_CCR project.

There is no native component to send mails in fpc/lazarus. I cannot use third party components since OPM is part of lazarus.

@GetMem
Understood.  A dialog can do the job though, so not a showstopper.  We have to assume a component developer is not an idiot.  Too much automation is not always desirable

As a non cutting-edge Laz developer, I can handle getting a SF account, Installing SVN/TortoiseSVN and uploading my source with an OPM json for approval - I can even handle updates (thanks to my groovy GUI).

What I'm saying is - if I can do it - anyone can and if they can't, maybe they are not the kind of developer to reliably maintain an OPM component in Laz1.8.

There are dozens of components in SF ccr and some are in active development even today.  It works; so current active component developers seem to cope with SF svn.  The proof is in the pudding (as they say).  The change would be simple - instead of the SF CCR repository, set up an OPM_CCR repository and clone the current contents.  Then advertise via OPM - access to component developers as their definitive repository for stable versions to include in OPM.  A risk-free strategy.

I realise that the current 40+ components in OPM have been chosen for stability rather than maintainability, but the next 40 OPM components need to be judged by different criteriia.  I wrote CryptIni specifically to test the OPM system, so I have some experience to offer.

I would suggest that your main worry is being too busy to un-vet a crap component that doesn't compile and is full of bugs that will never be fixed.  This could happen with an over-automated system, and would reflect badly on Lazarus 1.8.  I have no idea what a solution for that would look like but my 'SF sandbox' proposal minimises the chance - proposed components 'sit there' until the OPM moderator has time and energy to vet them.

@Juha's ideas are sound if applied to the in-house server.  I am suggesting an extra 'sandbox' layer for security and maintainability and above all - to maintain the Lazarus reputation for solid bug-free built-in components.

[lainz]

@lainz

Sorry for my lack of knowledge, a database can contain zip files and json files?

Blob fields can contain any binary format.

The language you want to do the server side is FPC or PHP for example?

If I had the possibility to choose php, the loging system would be long implemented. However some core developer insist to be fpc as a showcase. I have little experience coding server side stuff with fpc,  I hopping somebody with more experience will help me. Another solution is to learn it myself, but since my time is limited the progress will be slow.

Thanks for the clarification, I will learn databases maybe the next year in university.

Of course PHP is designed specifically for that kind of websites.

[JuhaManninen]

All the OPM 'action' is in the OPM_CCR SF repository under svn version control.  It is however a 'sandbox' and no-one but the OPM moderator can alter the official online repository.  Direct uploads would be a disaster-in-waiting.
I stress 'version control' because I feel this essential to successful OPM deployment in Laz 1.8, and @GetMem has rightly handled it well for installation and updates.

Now there is a misunderstanding somewhere. The only task now is to transfer a package generated by the Opkman to the server repository, through the admin's approval.
This transfer does not require version control. The original sources of those packages are typically under version control but that is a different topic.
Thus I don't see any reason why SourceForge or similar should be used as a "sandbox", a transfer area. DropBox or Google Docs or whatever would suit better if it must be a public commercial provider.
BTW, nobody has suggested direct upload to the final repository. The discussion is about a transfer area where Opkman could automatically move packages, to be inspected later by admin.

I would suggest that your main worry is being too busy to un-vet a crap component that doesn't compile and is full of bugs that will never be fixed.  This could happen with an over-automated system, and would reflect badly on Lazarus 1.8.  I have no idea what a solution for that would look like but my 'SF sandbox' proposal minimises the chance - proposed components 'sit there' until the OPM moderator has time and energy to vet them.

@Juha's ideas are sound if applied to the in-house server.  I am suggesting an extra 'sandbox' layer for security and maintainability and above all - to maintain the Lazarus reputation for solid bug-free built-in components.

No, we don't need another layer for security and maintainability. We already have such a layer, namely a human inspection.
In the beginning many people wanted to have a fully automatic Delphinus-style system because only it would be flexible enough. Now you think even a controlled delivery through admin moderation is not enough?

The packages delivered are not part of Lazarus, they are 3rd party packages. The admin's duty is not to rate them. That's why there will be a rating system. Having some lower quality packages is part of reality.

Whatever server is used for the temporary transfer, a single public account / password should be enough initially. Later it can be improved.
I didn't know there is no FTP protocol in FPC libs, but indeed HTTP works equally well.

If I had the possibility to choose php, the loging system would be long implemented. However some core developer insist to be fpc as a showcase. I have little experience coding server side stuff with fpc,  I hopping somebody with more experience will help me. Another solution is to learn it myself, but since my time is limited the progress will be slow.

No, I don't really insist it. I only said I would like to see an FPC solution. I was hoping somebody else will join and do an initial version. Maybe not.
If nobody does it, please feel free to use PHP.