Recent

Author Topic: Online Package Manager  (Read 510518 times)

lainz

  • Hero Member
  • *****
  • Posts: 3699
  • Leandro Diaz
Re: Online Package Manager
« Reply #1905 on: July 25, 2020, 05:37:57 am »
>>Personaly I would like to see a solution where getmem is able to deligate most if his current work, preferably in a safe manner. At least to a degree where he is able to feel comfortable

I think that's is the point.

Yes a site that's restricted is a good idea. Not so open like anyone can post but a safe place.

GetMem

  • Hero Member
  • *****
  • Posts: 3757
Re: Online Package Manager
« Reply #1906 on: July 27, 2020, 07:25:01 am »
Sorry for the late reply, I was away for the weekend. 

@lainz
Quote
Ok, I'm using postgres, so I know how to use it. About the console application, I've coded some endpoints in pascal as well, I can't remember wich tools used but is just like do the query to the database and return as json.

About the main json, maybe an endpoint called main.json will do the trick  :) Like it generates the json file with all the packages. Of couse is not optimal because it needs to download a huge file, but maybe adding some pagination will do.

And web interfaces is what I do most of the time, so I can help, but we need to organize in some way
Thank you! I can also handle the database part, but web development isn't my strong point. The fact that you are willing to help, it means a lot to me.

Quote
of course is a lot of work and maybe you as me want to take weekends for fun not to keep coding like in the whole week
Exactly! To make things worse, I have two small children. Coding in my free time, instead of playing with my kids is a sin in my opinion. :)



@TRon
Quote
However, and I express it gladly again, my main concern is/was about packages that are indeed abandoned, then somehow ended up in OPM _and_ doing so by using different source-code.

It seems perfectly fine for blissfully unaware reactors to point me to how FOSS works (which, for these kind of things doesn't btw) but the above situation, for me personally, is a complete no-no.

Either you use the original sources that are abandoned (and perhaps not work) or you take over the project so that you are able to make changes to the original project and allow for others to take note of that.

Simply making changes to an abandoned project to make it work, adding it to OPM and then pointing towards the original repo/author, crossing fingers no-one will ever notice is just plain rude to everybody except for the one making use of these kind of tactics
I agree, but the vast majority of the packages are submitted by forum users like you. Perhaps I should do a background check for each package, but this would require a tremendous amount of time, so I only remove binaries to prevent a possible infection. Please note that some well known package can contain binaries(dll, so), but it was explicitly requested by the package developer, without them the package wouldn't work properly. By the way I removed Internettools from OPM, the package is actively developed and the author regularly visits this forum, as you correctly mentioned in one of your previous posts. 

Quote
Personaly I would like to see a solution where getmem is able to deligate most if his current work, preferably in a safe manner. At least to a degree where he is able to feel comfortable.
That would be great! This project is no longer something I enjoy.

TRon

  • Hero Member
  • *****
  • Posts: 535
Re: Online Package Manager
« Reply #1907 on: August 01, 2020, 03:17:19 am »
I agree, but the vast majority of the packages are submitted by forum users like you. Perhaps I should do a background check for each package, but this would require a tremendous amount of time, so I only remove binaries to prevent a possible infection.
Please note that some well known package can contain binaries(dll, so), but it was explicitly requested by the package developer, without them the package wouldn't work properly.
A good thing that you at least uphold some standards that you are able to manage. However, it raises another concern, namely the integrity of the code itself.

I fully understand that you do not have the time to check each and every offered package yourself.

How about not accepting any packages that do not have an origin for their sources ? That is a thing that could be automated ? Each package contain sources, and they should match with those from their origin. In case it doesn't then such package will not be accepted or indicated as a package without (official/matching) origin.

That will put some of the burden back to the submitter. I fully realise that will probably also mean less packages being submitted. However, since i've given some of these things some though, I don't think you are able to relieve yourself without making some of such (perhaps harsh) decisions.

afaik people who genuinely maintain a project that support OPM would have no issue whatsoever with such a rule as they already have this kind of infrastructure set up.

At least that way the end-user would be able to either not being offered untrusted sources or at least ones that are indicated as such (in which case the end-user can decide him/herself if it's worth the trouble, e.g. you would probably not have heard from me in the first place instead of being a pita now  ;) )

Quote
By the way I removed Internettools from OPM, the package is actively developed and the author regularly visits this forum, as you correctly mentioned in one of your previous posts. 
Thank you for doing so. Although that would perhaps be a bummer for the person who created and submitted the package, it is imho better this way.

Quote
That would be great! This project is no longer something I enjoy.
I feel your burden there. In that regards I really wish I could be of more help. Note that i'm often thinking about how things could be improved but, it seems there are so many obstacles in the way that things become stuck when working out solutions that have a too naive approach.

One of the things I still miss a little is your input. Perhaps you already expressed yourself in the past (that I am unaware off) but it would be nice to know what you yourself think could be done to relieve your stress. And by that I meant without just simply turning things over to someone else. You must have some idea's about that, or not ?

I can suggest a multitude of changes that could perhaps improve things but for some of them I have no idea if they are actually feasible to realise (without too much work).

For example, I've seen a mention of some sorts of quality system so that users could submit their experience or offer a grade for the package. Bad experience, bad grade. Good experience, good grade. Too much bad grades call for (manual) inspection and/or removal. At the same time you mentioned that being a shiteload of work for you to be able to incorporate such a thing into OPM (not to mention adding/maintaining an infrastructure for keeping track of that information).

In theory it is a good idea, as it would also be able to gather some information with regards to popularity of a package (although perhaps you are able to see some of that already because of the number of downloads/installations of a particular package). The more a particular package is valued the more attention it deserves, in case people report issues. But in the end we don't know if self-regulating in such a way is going to work for OPM.

wp

  • Hero Member
  • *****
  • Posts: 7521
Re: Online Package Manager
« Reply #1908 on: August 02, 2020, 12:32:06 pm »
GetMem, I need to work on the JVCL contribution to OPM because the JvPascalInterpreter package does not compile on 64 bit and this prevents usage of the OPM installation even if the user does not need it at all.

Is there a way that I can keep this package within the OPM-zip (JvPascalInterpreter does work on 32 bit) but exclude it from the default OPM installation?
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

Soner

  • Full Member
  • ***
  • Posts: 191
Re: Online Package Manager
« Reply #1909 on: August 02, 2020, 03:55:48 pm »
GetMem, I need to work on the JVCL contribution to OPM because the JvPascalInterpreter package does not compile on 64 bit and this prevents usage of the OPM installation even if the user does not need it at all.

Is there a way that I can keep this package within the OPM-zip (JvPascalInterpreter does work on 32 bit) but exclude it from the default OPM installation?
wp I use that for such cases:
Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2. {$mode objfpc}{$H+}
  3. interface
  4.  
  5. {$IfDef CPU64}
  6.   //here code for 64bit or warn
  7.   {$warning you can't use this unit  for cpu64}
  8. implementation
  9. {$else}
  10. //code 32bit cpu
  11. implementation
  12. {$EndIf}
  13. end.
  14.  

lainz

  • Hero Member
  • *****
  • Posts: 3699
  • Leandro Diaz
Re: Online Package Manager
« Reply #1910 on: August 02, 2020, 05:03:41 pm »
Hi, there is a new version of BGRABitmap that fixes compilation for Linux (of BGRAControls)
https://forum.lazarus.freepascal.org/index.php/topic,24239.msg372105.html#msg372105

As well the directory of BGLControls has changed to solve the compilation problem with FPC 3.2

GetMem

  • Hero Member
  • *****
  • Posts: 3757
Re: Online Package Manager
« Reply #1911 on: August 03, 2020, 11:53:50 am »
@TRon
Quote
How about not accepting any packages that do not have an origin for their sources ? That is a thing that could be automated ? Each package contain sources, and they should match with those from their origin. In case it doesn't then such package will not be accepted or indicated as a package without (official/matching) origin.

That will put some of the burden back to the submitter. I fully realise that will probably also mean less packages being submitted. However, since i've given some of these things some though, I don't think you are able to relieve yourself without making some of such (perhaps harsh) decisions.

afaik people who genuinely maintain a project that support OPM would have no issue whatsoever with such a rule as they already have this kind of infrastructure set up.

At least that way the end-user would be able to either not being offered untrusted sources or at least ones that are indicated as such (in which case the end-user can decide him/herself if it's worth the trouble, e.g. you would probably not have heard from me in the first place instead of being a pita now  ;) )
OK. From now on, I will pay more attention to submitted packages and their sources. Unfortunately the damage is already done, nowadays I rarely add new packages to the repository, only update old ones. If you notice some issues, please let me know and we(the community) can decide what to do next: create a fork, delete it from OPM, etc...

Quote
One of the things I still miss a little is your input. Perhaps you already expressed yourself in the past (that I am unaware off) but it would be nice to know what you yourself think could be done to relieve your stress. And by that I meant without just simply turning things over to someone else. You must have some idea's about that, or not ?
As I mentioned in one of my previous post, we should create a database(https://packages.lazarus-ide.org/) and implement a login system, where package maintainers can update their packages without my intervention, via OPM or web interface. Also forum user can rate packages. @Lainz offered to help, but still there is too much work, even for two persons.   

Quote
For example, I've seen a mention of some sorts of quality system so that users could submit their experience or offer a grade for the package. Bad experience, bad grade. Good experience, good grade. Too much bad grades call for (manual) inspection and/or removal. At the same time you mentioned that being a shiteload of work for you to be able to incorporate such a thing into OPM (not to mention adding/maintaining an infrastructure for keeping track of that information).

In theory it is a good idea, as it would also be able to gather some information with regards to popularity of a package (although perhaps you are able to see some of that already because of the number of downloads/installations of a particular package). The more a particular package is valued the more attention it deserves, in case people report issues. But in the end we don't know if self-regulating in such a way is going to work for OPM.
This is already implemented locally, you can click and rate various packages(see rating column in OPM), but is kinda useless, until the ratings are not centralized.



@wp
Quote
Is there a way that I can keep this package within the OPM-zip (JvPascalInterpreter does work on 32 bit) but exclude it from the default OPM installation?
Yes it is. I updated the package. JvPascalInterpreter is no longer installed by default.


@lainz
Quote
Hi, there is a new version of BGRABitmap that fixes compilation for Linux (of BGRAControls)
https://forum.lazarus.freepascal.org/index.php/topic,24239.msg372105.html#msg372105

As well the directory of BGLControls has changed to solve the compilation problem with FPC 3.2
Done. I updated BGRABitmap.

wp

  • Hero Member
  • *****
  • Posts: 7521
Re: Online Package Manager
« Reply #1912 on: August 03, 2020, 12:47:19 pm »
Quote
Is there a way that I can keep this package within the OPM-zip (JvPascalInterpreter does work on 32 bit) but exclude it from the default OPM installation?
Yes it is. I updated the package. JvPascalInterpreter is no longer installed by default.
Thank you.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

ASBzone

  • Sr. Member
  • ****
  • Posts: 460
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Online Package Manager
« Reply #1913 on: August 03, 2020, 07:17:35 pm »
Exactly! To make things worse, I have two small children. Coding in my free time, instead of playing with my kids is a sin in my opinion.

Amen!  Don't fall into the trap of deprioritizing the family...


This project is no longer something I enjoy.

Even though I do not use OPM myself, I would like to thank you for your efforts on this project.

I also think you should consider establishing an SLA for how long it will take from the time you are aware of a request to update a package in OPM, until it has been completed.   Whether than a 2 days, 3 days, a week or whatever is reasonable for you.

Consider doing this now, while the larger discussions about how to delegate this work start up, so you will have some sanity, and everyone can get some expectations set.

Thanks again for your tireless efforts here.
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.388)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

lainz

  • Hero Member
  • *****
  • Posts: 3699
  • Leandro Diaz
Re: Online Package Manager
« Reply #1914 on: August 04, 2020, 01:30:20 am »
@lainz
Quote
Hi, there is a new version of BGRABitmap that fixes compilation for Linux (of BGRAControls)
https://forum.lazarus.freepascal.org/index.php/topic,24239.msg372105.html#msg372105

As well the directory of BGLControls has changed to solve the compilation problem with FPC 3.2
Done. I updated BGRABitmap.

Thanks  :)

GetMem

  • Hero Member
  • *****
  • Posts: 3757
Re: Online Package Manager
« Reply #1915 on: August 04, 2020, 06:07:38 pm »
@ASBzone
Thank you for your kind words and suggestions.

Cyrax

  • Hero Member
  • *****
  • Posts: 832
Re: Online Package Manager
« Reply #1916 on: August 12, 2020, 07:31:19 pm »
Why OPM doesn't check if there exists already downloaded/installed package in the user system so it won't need to be redownloaded again during (re) installation process?

ASBzone

  • Sr. Member
  • ****
  • Posts: 460
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Online Package Manager
« Reply #1917 on: August 12, 2020, 07:56:26 pm »
@ASBzone
Thank you for your kind words and suggestions.

You are very welcome.  :D
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.388)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

GetMem

  • Hero Member
  • *****
  • Posts: 3757
Re: Online Package Manager
« Reply #1918 on: August 12, 2020, 08:57:08 pm »
@Cyrax
Quote
Why OPM doesn't check if there exists already downloaded/installed package in the user system so it won't need to be redownloaded again during (re) installation process?
Uncheck OPM->Options->General->Always force download and extract.

lainz

  • Hero Member
  • *****
  • Posts: 3699
  • Leandro Diaz
Re: Online Package Manager
« Reply #1919 on: August 23, 2020, 11:44:51 pm »
Hi GetMem, I've modified a bit the website to look fine on mobile, and as well I've put a material design theme for the site
https://forum.lazarus.freepascal.org/index.php/topic,51174.msg375262.html#msg375262

 

TinyPortal © 2005-2018