Recent

Author Topic: Online Package Manager  (Read 436343 times)

avra

  • Hero Member
  • *****
  • Posts: 1712
    • Additional info
Re: Online Package Manager
« Reply #1635 on: July 10, 2019, 08:46:44 am »
I ran a few test with Lazarus Trunk/FPC 3.0.4, unfortunately I cannot reproduce the issues you mentioned. I will also try FPC 3.2.0 soon, most likely the problem only occur with a newer version of freepascal. Can you please test with Lazarus 2.0.2 or/and Trunk, combined with FPC 3.0.4? For now, 2.0.2/3.0.4 is the officially supported version.
I will. Thanks for trying.

I would like to support FPC trunk too if possible, but I had issues in the past when I tried to install packages. The issues were unrelated to OPM, the internal package system could not install the packages("incompatible ppu files" if I remember correctly).
I have found a work around for "incompatible ppu" which works each time I face this problem:
https://forum.lazarus.freepascal.org/index.php/topic,43646.msg326093.html#msg326093
I have no desire to support trunk which is a moving target, but official FPC 3.2 should be around the corner, and I have to use it for several reasons so it has my full attention. Unfortunately as you could see I could not make pl_dwscript work with it.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

lainz

  • Hero Member
  • *****
  • Posts: 3276
    • Lainz
Re: Online Package Manager
« Reply #1636 on: July 10, 2019, 11:35:41 pm »
@Lainz
By CGI I meant https://en.wikipedia.org/wiki/Common_Gateway_Interface and yes the console application will run on a linux server, the same server where the package files are located. OPM will communicate with that console application, which will update the database internally. I don't know any specific details about the database, if I understood correctly the site admin prefers PostgreSQL. The console application can be written in any language, of course freepascal is the prefered one. :)

Nice. Well so is not that hard I think, but of course it takes a lot of time to make something good.

If I understand CGI correctly, it must connect to the database each time, since is a console application that:
- runs, connects to the db, get / put data, makes the response and then is closed.

Is not a service that keeps running.

And yes, I think FPC will do.

And I have a lot of questions since I never coded a CGI, for example, if you need to connect to the DB to debug the application, must use a local database and a local apache server? Or must have a directory to test in the real server, and always copy and paste that binary to the server to test?
Sounds somewhat tedious if the last one is the workflow.  :)

lucamar

  • Hero Member
  • *****
  • Posts: 2076
Re: Online Package Manager
« Reply #1637 on: July 11, 2019, 01:25:31 am »
And I have a lot of questions since I never coded a CGI, for example, if you need to connect to the DB to debug the application, must use a local database and a local apache server? Or must have a directory to test in the real server, and always copy and paste that binary to the server to test?
Sounds somewhat tedious if the last one is the workflow.  :)

The "normal" workflow for building CGIs is to have a local server (doesn't need to be Apache, almost any will do) and, if a DB is needed, a local DB. Only the last testing steps should be done iin a "real" server--although never in the production one!--so that you can test whether the network layer interferes unduly with your program (it shouldn't but ... Murphy!).

Note also that, normally, from the CGI's p.o.v. the web server is almost always local. That is, the server calls the CGI as you would, for example, in a program using TProcess.

If the DB connection is slow to stablish and may impair your CGI's performance you should investigate FastCGI. With it your program is run once and kept "alive" so it in turn can keep alive the connection to the DB. Of course, this posses a small security risk but in normal scenarios it should be almost negligible.

HTH
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.2/2.0.4  - FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Online Package Manager
« Reply #1638 on: July 11, 2019, 08:44:54 am »
@avra
1. I fixed pl_DWScript, now it should work with FPC 3.2.0 too. I also updated in OPM.
2. I did try to install/reinstall as many packages as possible with Lazarus Trunk/FPC 3.2.0(pl_X and others), but the error you mentioned eludes me. Everything works out of the box.
3. Unfortunately the "Clean all" suggestions wont work with FPC trunk. A typicall example is BGRABitmap combined with BGRAControls. After you install BGRABitmap manually, then try to install BGRAControls, the package system will complain about "icompatible ppu" files, although you just compiled BGRABitmap so everything should be compatible.  :)  I had to delete the lib folder inside BGRABitmap, then I was able to installl both package. Everything works fine with FPC 3.0.4, and since FPC trunk is not yet officially suppoted, I did not investigate the issue more thoroughly.

PS: (2) Here is a somewhat boring video about the install process: https://www.youtube.com/watch?v=IRJkyiOn3AA

@Lainz
Quote
And I have a lot of questions since I never coded a CGI
Neither do I. This is why I never had the motivation to implement that feature(plus the lack of time). If I understood correctly the CGI is needed mainly for security purposes, if OPM directly updates the database, nasty things can happen.

@lucamar
Thanks for the suggestions.
« Last Edit: July 11, 2019, 10:25:10 am by GetMem »

lucamar

  • Hero Member
  • *****
  • Posts: 2076
Re: Online Package Manager
« Reply #1639 on: July 11, 2019, 02:55:40 pm »
A CGI is not intrinsicallly different from any other console app: you read your input from the "terminal" (or the environment), fiddle with it, do your thing and output the response.

Only, in a CGI the inputs/outputs must be somewhat strictly formatted and the domain is quite "closed", which is why it's possible (and desirable!) to have a "CGI Program" category in the development environment with some goodies added to help you get easily the request fields and buid the response ones.

Otherwise? Just a standard program. That is, after all, the beauty of the CGI compared to server-specific modules, etc.
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.2/2.0.4  - FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

avra

  • Hero Member
  • *****
  • Posts: 1712
    • Additional info
Re: Online Package Manager
« Reply #1640 on: July 11, 2019, 02:59:07 pm »
1. I fixed pl_DWScript, now it should work with FPC 3.2.0 too. I also updated in OPM.
Very, very, very nice. Thank you! :D I will update ct4laz repo.

2. I did try to install/reinstall as many packages as possible with Lazarus Trunk/FPC 3.2.0(pl_X and others), but the error you mentioned eludes me. Everything works out of the box.
It would be very good news if Lazarus trunk does not suffer the "icompatible ppu" problem, and even better if patch makes it's way into fixes.

3. Unfortunately the "Clean all" suggestions wont work with FPC trunk. A typicall example is BGRABitmap combined with BGRAControls. After you install BGRABitmap manually, then try to install BGRAControls, the package system will complain about "icompatible ppu" files, although you just compiled BGRABitmap so everything should be compatible.  :)  I had to delete the lib folder inside BGRABitmap, then I was able to installl both package. Everything works fine with FPC 3.0.4, and since FPC trunk is not yet officially suppoted, I did not investigate the issue more thoroughly.
In similar cases "Clean All" didn't work for me either, but "Clean common files" did work (with mentioned compilation switches). Strange but true in my case, and tested up to unofficial FPC 3.2 fixes. I also do not bother much with FPC trunk since it's a moving target.

PS: (2) Here is a somewhat boring video about the install process: https://www.youtube.com/watch?v=IRJkyiOn3AA
Hopefully during the weekend I will catch time to look and do promised tests.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

lainz

  • Hero Member
  • *****
  • Posts: 3276
    • Lainz
Re: Online Package Manager
« Reply #1641 on: July 14, 2019, 12:18:20 am »
Hi GetMem, can you please update in OPM BGRAControls and BGRABitmap?

Thanks.

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Online Package Manager
« Reply #1642 on: July 14, 2019, 06:14:20 am »
@Lainz
Did you go undercover?  :D

Quote
Can you please update in OPM BGRAControls and BGRABitmap?
Done.

lainz

  • Hero Member
  • *****
  • Posts: 3276
    • Lainz
Re: Online Package Manager
« Reply #1643 on: July 14, 2019, 06:15:33 pm »
@Lainz
Did you go undercover?  :D

 :o That didn't work...

Thanks for updating the packages.

mdbs99

  • Full Member
  • ***
  • Posts: 143
    • Github
Re: Online Package Manager
« Reply #1644 on: July 17, 2019, 03:09:08 pm »
GetMem,

There is an situation on mORMot Packages (since we published, but I haven't time until now) that is inconvenient.

If I add two packages (or more) like `momort_base`, `mormot_rest`, and `mormot_sqlite3` I got messages like this:

Code: Pascal  [Select]
  1. Warning: other unit files search path (aka unit path) of "mormot_sqlite3 1.18" contains "<PATH>\lib", which belongs to package "mormot_base"
  2. Warning: other unit files search path (aka unit path) of "mormot_rest 1.18" contains "<PATH>\lib\SQLite3", which belongs to package "mormot_sqlite3"
  3. Warning: other unit files search path (aka unit path) of "mormot_base 1.18" contains "<PATH>\lib", which belongs to package "mormot_sqlite3"
  4.  
  5. Note: Duplicate unit "mORMotWrappers" in "mormot_base 1.18", orphaned ppu "<PATH>\lib\Packages\lib\i386-win32\mORMotWrappers.ppu"
  6. Note: Duplicate unit "mORMotWrappers" in "mormot_rest 1.18", ppu="<PATH>\lib\Packages\lib\i386-win32\mORMotWrappers.ppu", source="<PATH>\lib\SQLite3\mORMotWrappers.pas"
  7.  

The simpler way to fix that is having different source directories for each package — like Zeos does, for example.
Another option is having just one or two packages, as all sources stay (almost) together, however this issue could happen, anyway... but less.
Finally, I can think in one package with just basic units added, because some units is used just with another packages — again Zeos + SynDBZeos — and if the user doesn't have such libs, he/she couldn't compile. But, without those units added, it is possible to compile *and* having others (SynDBZeos) in PATH, just in case.

I will ask Arnaud about these options. But what do you think is the best?

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Online Package Manager
« Reply #1645 on: July 17, 2019, 04:08:57 pm »
Hi mdbs99,

Unfortunately I'm not familiar with mORMot, so I don't have any preference. You can add as many subfolders and/or dependencies you like. If the package compiles with at least one combination of FPC/Lazarus, I can add it to OPM.

mdbs99

  • Full Member
  • ***
  • Posts: 143
    • Github
Re: Online Package Manager
« Reply #1646 on: July 17, 2019, 04:18:22 pm »
GetMem,

I've already talked to Arnaud: we will create just one package.
I'm working on it. After I send a PR (and this be approved, I hope), I'll warn you to update.

Thank you.

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Online Package Manager
« Reply #1647 on: July 17, 2019, 04:43:14 pm »
I've already talked to Arnaud: we will create just one package.
I'm working on it. After I send a PR (and this be approved, I hope), I'll warn you to update.
Thank you. I will add the package after it's ready.

mdbs99

  • Full Member
  • ***
  • Posts: 143
    • Github
Re: Online Package Manager
« Reply #1648 on: July 18, 2019, 07:13:55 pm »
Thank you. I will add the package after it's ready.

Hello GetMem,

It's done!

Please, could you update the packages?
At the end we stay with two packages: The `mormot_base` and `mormot_cross` (this last was renamed from mormot_crossplatform to mormot_cross).

Just in case, you can see the PR here to understand the changes.

thank you.

--

EDIT: there is something wrong in compilation on i386, not related to Packages. Please, wait until everything is done. Sorry.
« Last Edit: July 19, 2019, 02:26:11 am by mdbs99 »

mdbs99

  • Full Member
  • ***
  • Posts: 143
    • Github
Re: Online Package Manager
« Reply #1649 on: July 19, 2019, 02:48:40 pm »
GetMem,

The fix for compilation in i386 was applied.

Packages are working. However, we have just one inconvenient issue, which I've posted on Lazarus' list but, as you (all) know much about Packages, maybe can help me:

https://lists.lazarus-ide.org/pipermail/lazarus/2019-July/236725.html

You can update mORMot Packages now, if you want. But would be better to fix this inconvenient before. Thanks.

best regards,
Marcos Douglas