Lazarus

Programming => Networking and Web Programming => Topic started by: BosseB on December 15, 2019, 06:12:16 pm

Title: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 06:12:16 pm
I have built a Raspberry Pi based data collection system and I have used FreePascal and Lazarus to program its control software running on the RPi.

This consists of a few console programs intended to be run by cron.
The data are managed inside an SQLite database (for which I have received good help on this forum!)

Now I want to build a configuration utility that will be web based (Apache2 on the RPi) and originally I thought that I would use PHP7 and straight HTML coding.
But that would mean to somehow re-create the database and other functions I have already developed in Pascal so it would work with PHP...
Not so fun.

Why web based?
Well, it makes it possible to use any computer with a web browser to configure the system when connected to the same network rather than logging on to the RPi itself, which is headless (no monitor or keyboard attached).


Question:
Is there some simple way to call processing functions written in Pascal from within PHP such that it can retrieve data to display to the user or execute commands and feed back results?

All I really want is read and write to the SQLite database and also scan a few directories for certain files, list and down/upload these.

Has someone here been there and done that and is willing to share the experience?
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: PaulRowntree on December 15, 2019, 06:22:43 pm
Ignorant question coming ...Why do you need the PHP intermediate layer? 
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 15, 2019, 06:23:29 pm
I'd suggest https://wiki.freepascal.org/CGI_Web_Programming (https://wiki.freepascal.org/CGI_Web_Programming) and related pages.

I've just set up a very simple webpage on a server run by our ISP which will eventually be a portal to various backend stuff- probably in VMs. They don't support PHP etc but do support CGI, and I wrote a simple script in Perl which expands SSIs in a page passed as the parameter and emits the result.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 15, 2019, 06:30:42 pm
You don't need to use PHP to make web apps. Just use Pascal directly, either as CGI, FastCGI, or Apache module. Free Pascal provides fpWeb framework for everything you need to make web apps. I've made lots of web apps using FPC since like 15 years ago. I've never need PHP or nodeJS or any non-Pascal language for that. Now it's even better, Free Pascal has a sister project named pas2js (https://wiki.freepascal.org/pas2js) that convert pascal program into javascript. So, you can even write the front-end side using pascal as well. Isn't that awesome? 😊
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 06:58:52 pm
Ignorant question coming ...Why do you need the PHP intermediate layer?
Because that is the only way I (kind of) know how to use for programming stuff on webpages...
There is a way to call an external executable on the webserver from PHP and somehow retrieve the result, I guess. Since I have written a number of related utilities in pascal and compiled for Raspbian they are available to be called. Only I don't really know how to extract the result from them...
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 07:00:19 pm
I'd suggest https://wiki.freepascal.org/CGI_Web_Programming (https://wiki.freepascal.org/CGI_Web_Programming) and related pages.

I've just set up a very simple webpage on a server run by our ISP which will eventually be a portal to various backend stuff- probably in VMs. They don't support PHP etc but do support CGI, and I wrote a simple script in Perl which expands SSIs in a page passed as the parameter and emits the result.

MarkMLl
Hi Mark,
I will have a look at that documentation.
Thanks.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 07:03:49 pm
You don't need to use PHP to make web apps. Just use Pascal directly, either as CGI, FastCGI, or Apache module. Free Pascal provides fpWeb framework for everything you need to make web apps. I've made lots of web apps using FPC since like 15 years ago. I've never need PHP or nodeJS or any non-Pascal language for that. Now it's even better, Free Pascal has a sister project named pas2js (https://wiki.freepascal.org/pas2js) that convert pascal program into javascript. So, you can even write the front-end side using pascal as well. Isn't that awesome? 😊
First off, I am an old hand so I do not know much about "webapps", what are these?

Do you mean one can use the Lazarus desktop programming and somehow show the programs through a web server?
Regarding Javascript I am totally unfamiliar with Java so I'd rather stay away from that. Started using Pascal back in the 1980-es, though.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 15, 2019, 07:06:13 pm
Bo, first step: find out what scripting etc. facilities are supported by your webserver. Second step: do not under any circumstances be tempted to write your own custom webserver.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: lainz on December 15, 2019, 08:14:32 pm
Regarding Javascript I am totally unfamiliar with Java so I'd rather stay away from that. Started using Pascal back in the 1980-es, though.

Javascript is not Java.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 09:20:20 pm
I now looked (briefly) at the links provided and it seems like a webapp is more or less an invisible processing system. The examples I looked at looked mostly like "Hello world" type programs operating in a console.

But I need a GUI configuration form where I can enter or select values and then submit them to a web server handler.
For me this has always been a php file receiving the data and then doing something with them.

In the current case I want to check the validity of the data and store in the database or maybe trigger some server side action.
Also on request from the user retrieve result data as files or display. This would be PHP code calling pascal programs for retrieval and then rendering the result into a GUI form sent back to the user.

Rather than programming these database and file system operations in PHP my thought was to use PHP to extract the input data from the user (form submission) and then repackage these into a call to my pascal program that does the real work towards the database and hardware....
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 15, 2019, 09:31:09 pm
As I said, you start off by finding out what facilities your web server (Apache?) is configured to provide. It might even have PHP.

Any HTML page- without Javascript and potentially without CSS- has a basic set of interactive data entry components. Even without PHP etc., CSS allows you to make components visible/invisible depending on what else is going on. Javascript is vastly overused in my opinion, mainly by addons which benefit their writer more than the user (e.g. targetted advertising under the control of Google et al.). But detailed discussion is way off topic.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: lucamar on December 15, 2019, 09:36:18 pm
Rather than programming these database and file system operations in PHP my thought was to use PHP to extract the input data from the user (form submission) and then repackage these into a call to my pascal program that does the real work towards the database and hardware....

It depends heavily on what you're currently doing (if anything) and what your final goal is but if I were you I would forget about PHP and let my Pascal program manage everything. That way you can avoid most (if not all) problems of interoperabilrity between the two programs.

Whether the backend is CGI/FastCGI, an Apache module or whatever I'm fairly sure it can be done in Pascal without more than the normal problems.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 15, 2019, 10:30:34 pm
As I said, you start off by finding out what facilities your web server (Apache?) is configured to provide. It might even have PHP.
As I think I have mentioned but possibly in another thread the target for my system is a Raspberry Pi running Raspbian Buster and with Apache2 + PHP etc installed. Not MySql though, I decided against its complications and am using an SQLite database instead.
The web interface is not a user interface as such, it is just a front end to set up (configure) the system and define the automatic tasks to execute. In order to use it one has to be physically connected to the RPi network.
Quote
Any HTML page- without Javascript and potentially without CSS- has a basic set of interactive data entry components. Even without PHP etc., CSS allows you to make components visible/invisible depending on what else is going on.
Yes, I have designed a prototype webpage in php lingo that shows the boxes, lists and selectors I need. It works ok right now but it does not do anything of course.

I think I will just use the PHP exec() function with my own pascal programs and get it done....
Of course I will have to create the php webpages also, but it should not be a big deal.
And I already know the way to extract the posted data from the form so I can stuff it onto a call to my pascal program.
Will do some experimentation shortly.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 16, 2019, 02:50:28 am
It seems Bosse doesn't have proper knowledge about web apps nor web pages. I wrote a tutorial about that, but unfortunately it's written in Bahasa Indonesia not English, because it's for my fellow Pascal programmers in Indonesia. The tutorial explains from the very basic of HTTP, HTML, CSS, JavaScript, scripting, web server features (based on Apache), communication between web server and our program (in Pascal), etc.

PHP is just ONE of MANY ways to make a web apps. There are many other better ways than using PHP for web apps. Web apps are like desktop app but using web interface (instead of GUI or CLI). So, instead using OS GUI APIs to display things to user, it's using HTML and CSS and using Javascript to make to do the logic (on the browser side). Oh, Javascript is NOT Java.

Basically, a web app starts from a browser request to your web server. Your web server should know the URL requested is handled by other application. Your web server forward the request to the application handler which is your pascal program. Your program then reads the input given by your web server (using either CGI, FastCGI, or Apache module). The easiest way is using CGI since all you need to do is read input from standard input and send output to standard output. Your program response to your web server request by sending output in HTML document along with the required CSS and Javascript code (either internal or external to the HTML document). Your web server reads your response and forward it to the browser that requested at the beginning. That's what happened in every single request and response of HTTP communication.

Some people think that the application part (the program that you wrote to handle web server request) can only be written in PHP. It's very wrong. You can even write it in bash script or batch file. There's nothing that PHP can do and other language can't. My first web app was written in Turbo Pasal 5.5.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 16, 2019, 02:55:00 am
Along with the tutorial, I also made a WebCRT (https://github.com/pakLebah/webCRT) unit to demonstrate how easy it's to write web apps using Pascal and the unit. It's almost as easy as using the standard CRT unit.

For more control but with more "manual" way, you may use my webUtils (https://github.com/pakLebah/dasarweb) unit, like to handle sessions and cookies. The repo contains the codes of my web app tutorial. I think you can learn something from them although you don't read the tutorial.

Hope it will help.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: valdir.marcos on December 16, 2019, 06:03:43 am
Along with the tutorial, I also made a WebCRT (https://github.com/pakLebah/webCRT) unit to demonstrate how easy it's to write web apps using Pascal and the unit. It's almost as easy as using the standard CRT unit.

For more control but with more "manual" way, you may use my webUtils (https://github.com/pakLebah/dasarweb) unit, like to handle sessions and cookies. The repo contains the codes of my web app tutorial. I think you can learn something from them although you don't read the tutorial.

Hope it will help.
Impressive. Thanks for sharing.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: avra on December 16, 2019, 08:59:23 am
You can simply add JsonRPC to your application and expose some of it's functionality to 3rd parties. If you browse fpc sources you will find some examples. There is also CeosMW JSONRPC which can be downloaded via OPM or through https://github.com/jbsolucoes/ceosmw. CeosMW components make this task very easy.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: PascalDragon on December 16, 2019, 09:05:29 am
But I need a GUI configuration form where I can enter or select values and then submit them to a web server handler.
For me this has always been a php file receiving the data and then doing something with them.
PHP essentially boils down to reacting to a HTTP request (e.g. GET index.php) by sending a HTTP response to the client which (in most cases) will simply be HTML text. That is nothing special of PHP and can just as easily be down with FPC (or any other language in fact).
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 16, 2019, 09:19:25 am
Some people think that the application part (the program that you wrote to handle web server request) can only be written in PHP. It's very wrong. You can even write it in bash script or batch file. There's nothing that PHP can do and other language can't. My first web app was written in Turbo Pasal 5.5.

The crucially important thing is that the web server should have a mechanism to pass a request for certain content to custom code. That custom code could be PHP, SSI (in the original page), CGI (a script in a separate directory), or a server-specific mechanism to improve the efficiency of any of those (e.g. Apache's ModPerl, which eliminates the overhead of a load/compile every time a script is run).

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 16, 2019, 11:36:03 am
It seems Bosse doesn't have proper knowledge about web apps nor web pages.
You are quite right about that concerning "Web apps"! It is the main reason I am asking here...
Web pages in general I do know, I have written sites since the 1990:es using just a text editor...
Quote

I wrote a tutorial about that, but unfortunately it's written in Bahasa Indonesia not English, because it's for my fellow Pascal programmers in Indonesia. The tutorial explains from the very basic of HTTP, HTML, CSS, JavaScript, scripting, web server features (based on Apache), communication between web server and our program (in Pascal), etc.
Maybe I can try sending it through Google Translate?

Quote
PHP is just ONE of MANY ways to make a web apps. There are many other better ways than using PHP for web apps. Web apps are like desktop app but using web interface (instead of GUI or CLI). So, instead using OS GUI APIs to display things to user, it's using HTML and CSS and using Javascript to make to do the logic (on the browser side). Oh, Javascript is NOT Java.
I looked at your example and realized that:
1) One can use a straight pascal cli program to handle the POST or GET call.
2) One just has to place the program into the cgi-bin dir of the webserver and set its name as the form handler
3) One builds the response page as html in pascal code and put return variable values into that then writes it out as a writeln

So I should be able to do this in my command line programs written with Lazarus!

Quote
The easiest way is using CGI since all you need to do is read input from standard input and send output to standard output. Your program response to your web server request by sending output in HTML document along with the required CSS and Javascript code (either internal or external to the HTML document). Your web server reads your response and forward it to the browser that requested at the beginning. That's what happened in every single request and response of HTTP communication.

Some people think that the application part (the program that you wrote to handle web server request) can only be written in PHP. It's very wrong. You can even write it in bash script or batch file. There's nothing that PHP can do and other language can't. My first web app was written in Turbo Pasal 5.5.

Very helpful response! Thank you very much!

I have a few clarification questions:

1) Displaying the main form using a pascal program?
Since I am using web forms for my configuration and status pages it would be better if I can synchronize the web form and processing inside the pascal program by having the actual page display also controlled by the pascal code. One less thing to get out of sync....
So can I display my config using an url such as this:
Code: [Select]
http://rpimonitor/cgi-bin/mainpageWhere mainpage is my pascal program for showing the main page...
Or maybe cgi-bin is not directly available on the browser (I think it is typically located above the document root) so I may need an index.php file on the server that calls the cgi-bin script? If so how would that be done?

2) Form handler type GET and POST?
The two types I know of are POST and GET where the submit button will in the GET case put the input data into the URL itself whereas in the POST case the data is sent in another way, which I don't know how...
When dealing with form handlers there are different ways to deal with that in PHP, how does it work in cgi pascal programs?
Say I use the POST method in the form, how do the data arrive into the pascal cmd line program?

Grateful for your input!"
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 16, 2019, 11:54:04 am
I looked at your example and realized that:
1) One can use a straight pascal cli program to handle the POST or GET call.
2) One just has to place the program into the cgi-bin dir of the webserver and set its name as the form handler
3) One builds the response page as html in pascal code and put return variable values into that then writes it out as a writeln

So I should be able to do this in my command line programs written with Lazarus!

That's right. But you'll need to look at the CGI environment variables that are made available to your script, and different servers make different one available.

> Or maybe cgi-bin is not directly available on the browser (I think it is
> typically located above the document root) so I may need an index.php
> file on the server that calls the cgi-bin script? If so how would that be done?

Note that cgi-bin is entirely a server-side thing. On the browser you've got Javascript (disabled to varying extents by security addons), and very rarely you might come across Java, ActiveX and so on.

> 2) Form handler type GET and POST?

One pipes the submission, the other encodes it in the URI. There have also been various server-specific methods in the past, e.g. PATCH, but those are rarely implemented these days.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 16, 2019, 12:32:03 pm
I found this Apache tutorial (https://httpd.apache.org/docs/2.4/howto/cgi.html) concerning CGI and its various aspects...

So I think I am on the right way now.  :D
I believe that my compiled pascal program will work if it is placed in a dir, which has been marked in the apache conf file using the directive ScriptAlias. I will check that as soon as I have an example program to test with.


Seems like the input data are available like this:
POST:
on STDIN as a string like name=value&name2=value&name3=value

GET:
in an environment variable QUERY_STRINGS in the same format as above
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 16, 2019, 12:46:27 pm
On a standard Debian system I found that Apache was configured to expect CGIs in /usr/lib/cgi-bin and that provided I set up a symlink it worked fine; I'd be surprised if Raspbian were different. I ended up with /var/www/html/cgi-bin which isn't quite the classic layout but if it works...

BE TOLD: treat QUERY_STRINGS and the rest with caution. I think that one's fairly universally implemented, but there are others relating to referring document etc. which you'll need and are somewhat variable.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 16, 2019, 05:16:19 pm
Maybe I can try sending it through Google Translate?
If you're so interested, you may get my free mini ebook about web programming using (Free)Pascal here (https://pak.lebah.web.id/ebook/pascal.id_kulgram2.pdf) (it's a 2.2 MB PDF file). And the code of the examples of the book are here (https://github.com/pakLebah/dasarweb).

I looked at your example and realized that:
1) One can use a straight pascal cli program to handle the POST or GET call.
2) One just has to place the program into the cgi-bin dir of the webserver and set its name as the form handler
3) One builds the response page as html in pascal code and put return variable values into that then writes it out as a writeln

So I should be able to do this in my command line programs written with Lazarus!
Indeed! :)

Very helpful response! Thank you very much!
You're welcome. I'm glad to be able to help you.

I have a few clarification questions:

1) Displaying the main form using a pascal program?
Since I am using web forms for my configuration and status pages it would be better if I can synchronize the web form and processing inside the pascal program by having the actual page display also controlled by the pascal code. One less thing to get out of sync....
So can I display my config using an url such as this:
http://rpimonitor/cgi-bin/mainpage
Where mainpage is my pascal program for showing the main page…
The main page of your program is the output of the program itself. If your mainpage[.cgi] produce correct HTML document along with the required CSS and JS files, then the client browser will render it to the user like a normal web page.

Or maybe cgi-bin is not directly available on the browser (I think it is typically located above the document root) so I may need an index.php file on the server that calls the cgi-bin script? If so how would that be done?
The cgi-bin folder is configurable. It's the folder that specially set for F/CGI binary files, by default. Of course you may change it to any folder. Just study the Apache setting for executable binary files.

2) Form handler type GET and POST?
The two types I know of are POST and GET where the submit button will in the GET case put the input data into the URL itself whereas in the POST case the data is sent in another way, which I don't know how…

When dealing with form handlers there are different ways to deal with that in PHP, how does it work in cgi pascal programs?
Say I use the POST method in the form, how do the data arrive into the pascal cmd line program?
Very simple actually. Every GET variables and its value will be stored in an environment variable named QUERY_STRING, you can use the GetEnvironmentVariable() function to read the content. And POST variables and its values streamed into standard input, you simply use the standard readln procedure to get the content.

Or, you better study my tutorial and read the examples. Good luck! :)
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 16, 2019, 06:30:55 pm
The main page of your program is the output of the program itself. If your mainpage[.cgi] produce correct HTML document along with the required CSS and JS files, then the client browser will render it to the user like a normal web page.

Or the main page can be a frameset, with at least some of the constituent frames being generated by CGI. That has the advantage that the scripts can look at the referrer and have some degree of confidence that they're not being fed malicious parameters, rather than letting a potentially-untrusted user drive the scripts directly.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: zamronypj on December 17, 2019, 06:57:32 am

Question:
Is there some simple way to call processing functions written in Pascal from within PHP such that it can retrieve data to display to the user or execute commands and feed back results?

All I really want is read and write to the SQLite database and also scan a few directories for certain files, list and down/upload these.

Has someone here been there and done that and is willing to share the experience?

mixing Pascal with PHP is actually hard.


Writing PHP extension requires you to use C language or Zephir language. I do not know if Free Pascal can be used to write PHP extension. (maybe Free Pascal can but, AFAIK, no PHP headers are converted yet)

So the simplest way is just to use Pascal or PHP. PHP can access SQLite database as well as Free Pascal.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: egsuh on December 17, 2019, 07:47:48 am
Once you have web-server program, then you may use Lazarus program (I mean Windows visual program if the terminals are running on Windows --- I do not know about Linux) using Synapse... You don't have to bother write HTML (and CSS, JS, etc.).
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: avra on December 17, 2019, 08:31:41 am
mixing Pascal with PHP is actually hard.
Not if you use JsonRPC on Pascal side (like CeosMW), and JsonRPC on PHP side (like https://github.com/datto/php-json-rpc).
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: PascalDragon on December 17, 2019, 09:23:18 am
I looked at your example and realized that:
1) One can use a straight pascal cli program to handle the POST or GET call.
2) One just has to place the program into the cgi-bin dir of the webserver and set its name as the form handler
3) One builds the response page as html in pascal code and put return variable values into that then writes it out as a writeln

So I should be able to do this in my command line programs written with Lazarus!
You can also take a look at fpWeb which is part of FPC and Lazarus and would deal with all the background stuff of request variables, headers, etc. You can find an example for CGI here (https://wiki.freepascal.org/fcl-web). Please note that you do not need to use Lazarus for this. You can just as well use fpWeb inside a "command line" application.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 17, 2019, 11:34:20 am
I looked at your example and realized that:
1) One can use a straight pascal cli program to handle the POST or GET call.
2) One just has to place the program into the cgi-bin dir of the webserver and set its name as the form handler
3) One builds the response page as html in pascal code and put return variable values into that then writes it out as a writeln

So I should be able to do this in my command line programs written with Lazarus!
You can also take a look at fpWeb which is part of FPC and Lazarus and would deal with all the background stuff of request variables, headers, etc. You can find an example for CGI here (https://wiki.freepascal.org/fcl-web). Please note that you do not need to use Lazarus for this. You can just as well use fpWeb inside a "command line" application.
When I am referring to Lazarus I am just indicating my use of it as a development IDE.
The actual programs are just non-visual command line programs communicationg via start arguments, environment variables and STDIO interchanges. Meant to run on an RPi without anyone logged on and no display or keyboard attached.
Also the RPi will be interfaced this way only for configuration by a single user and once that is done it will be on its own...
And I have scrapped the idea of using PHP in favour of directly calling the pascal programs from Apache.
So the subject of this therad is now a bit misleading....
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: mr-highball on December 17, 2019, 03:59:43 pm
I wrote some microservices that handled requests to/from a sqlite db a little while back, and it used the standalone web framework that comes with fpc (which makes it incredibly easy to deploy).
Here's the repos,
https://github.com/mr-highball/dcl-hackathon-2019 (initial repo with some implemented services)
https://github.com/mr-highball/dcl-microservices (trimmed up repo with only core stuff)

This is an example web service which was successfully built/deployed on a linux droplet running with digital ocean (so cross compiling to pi should work no prob). It takes image url's and dithers them down to a reduced color space, and offers endpoints to fetch the result from a sqlite db.
https://github.com/mr-highball/dcl-hackathon-2019/tree/master/services/image-service

here's some examples of an endpoint being used to return some content to the user,
https://github.com/mr-highball/dcl-hackathon-2019/blob/master/services/image-service/src/controller.status.pas#L146
https://github.com/mr-highball/dcl-hackathon-2019/blob/master/services/image-service/src/controller.registration.pas#L549

some niceties that you get if you use the stuff I wrote would be the automatic execution/serialization of db calls that contain result sets,
https://github.com/mr-highball/dcl-hackathon-2019/blob/master/services/image-service/src/controller.status.pas#L253

While the use case is not probably close to what you're doing, the code may help point you in the right direction.
By no means is this the only way to do things, just hoping it may help  :)


Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 17, 2019, 07:55:05 pm
I am sorry to have to come back here after I thought I had it all fixed....

I have created a getwebpage program in FPC and what it does is that it will output a webpage built internally by writing it out STDOUT when executed.
The content depends on how it is called:
- if the env variable QUERY_STRING is present it will read that and parse it (case GET)
- if there is a command line argument it will read this and parse it (case POST)
- if there is no data of any of the two kinds it will output an error page.

In the two first cases the output is a web page source with preamble, head with style section, body and the end tags. Preamble is the Mime header "Content-type: text/html"
Body is the decoded data from the input.
I have verified the creation of the webpage by setting up the env variable with the type of data from a GET operation and when I run this from the command line and send the output into a file it contains the exact expected content.

But when I call this from a web browser via Apache CGI handling I am always getting error 500 and the Apache error log says this:

Code: [Select]
[Tue Dec 17 19:18:04.837217 2019] [cgi:error] [pid 24911] [client 192.168.119.166:58361] malformed header from script 'getwebpage': Bad header: <!DOCTYPE html public >
The data sent out from the program looks like this:
Code: [Select]
Content-type: text/html

<!DOCTYPE html public >

<html>
<head>
<title>Monitoring Config</title>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<style>
... long style section, then:
table, td, th {
  border: 1px solid black;
}
</style>
</head>

<body>
<p>Got Data:<br>namn=Olle<br>trafik=hemsk<br>Test=7</p>
</body></html>

I have googled extensively for the error and what I have seen and included without positive result (the error persists) is:
- I must include the Mime header as the first output followed by a blank line (done)
- I must include a blank line after the </head> line (done)

I have also looked at web pages that work normally around the web and checked their sources only to find the same kind of data I have put in the start of the file...

Why am I not able to get Apache to accept the page I am sending from my FPC program?
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: lucamar on December 17, 2019, 08:20:02 pm
You should be sending out quite a lot more headers than just Content-Type (for example, Content-Lenght, IIRC). Read carefully the HTTP RFCs to see which headers are mandatory andhow to build a well-structured response.

Or better yet: use a CGI/HTML library which will do most of that for you automagically, even if just fcl-web, creating a "CGI application" in Lazarus.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 17, 2019, 08:21:58 pm
Stop making life difficult for yourself, and start off with a /minimal/ page: cut out the metas, style and so on.

You need

Code: [Select]
Content-type: text/html; charset=ISO-8859-1

followed by two newlines to give yourself a blank line. Then something like

Code: [Select]
<html>
  <head>
  </head>
  <body>
  </body>
</html>

I'm not entirely sure about this, but assume for the moment that Apache is a unix program that it wants the unix line-end convention. The pages I created a couple of weeks ago, which I'm using with Apache and Perl, certainly use \n rather than \r\n

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 17, 2019, 08:29:45 pm
You should be sending out quite a lot more headers than just Content-Type (for example, Content-Lenght, IIRC).

Most of which will be generated by Apache. The only one that /has/ to be generated by a CGI script is Content-Type, and I suspect that many browers will make an interim assumption of HTML at least until they've seen whether there's a !DOCTYPE and whether the document markup language (some derivative of SGML) has anything useful in the headers.

So the only thing that /really/ has to be output is a blank line, and any extraneous characters can foul that up.

MarkMLl.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: dsiders on December 17, 2019, 08:55:13 pm
But when I call this from a web browser via Apache CGI handling I am always getting error 500 and the Apache error log says this:

Code: [Select]
[Tue Dec 17 19:18:04.837217 2019] [cgi:error] [pid 24911] [client 192.168.119.166:58361] malformed header from script 'getwebpage': Bad header: <!DOCTYPE html public >
The data sent out from the program looks like this:
Code: [Select]
Content-type: text/html

<!DOCTYPE html public >

<html>
<head>
<title>Monitoring Config</title>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<style>
... long style section, then:
table, td, th {
  border: 1px solid black;
}
</style>
</head>

<body>
<p>Got Data:<br>namn=Olle<br>trafik=hemsk<br>Test=7</p>
</body></html>


Using a valid DOCTYPE would probably help:

HTML 5
Code: XML  [Select][+][-]
  1. <!DOCTYPE html>

HTML 4.01 Strict
Code: XML  [Select][+][-]
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

HTML 4.01 Transitional
Code: XML  [Select][+][-]
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

HTML 4.01 Frameset
Code: XML  [Select][+][-]
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">

XHTML 1.0 Strict
Code: XML  [Select][+][-]
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

XHTML 1.0 Transitional
Code: XML  [Select][+][-]
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  2.  

XHTML 1.0 Frameset
Code: XML  [Select][+][-]
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Frameset//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd">

XHTML 1.1
Code: XML  [Select][+][-]
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

You should also make sure the content-type in the Apache HEAD matches the UTF-8 content-type used in your HTML. You can modify the Apache config using:

Code: Text  [Select][+][-]
  1. AddDefaultCharset utf-8
or
Code: Text  [Select][+][-]
  1. AddCharset utf-8 .htm .html .js .css

Hope that helps...
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 18, 2019, 03:26:01 am
The most minimalist code for CGI is as simple as this:

Code: Pascal  [Select][+][-]
  1. program test;
  2.  
  3. begin
  4.   writeln('content-type: text/html;');
  5.   writeln;
  6.   writeln('Hello world!');
  7. end.

Compiled with FPC 3.0.4. It works perfectly fine on my Ubuntu + Apache server here:
https://pak.lebah.web.id/test.cgi
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 18, 2019, 08:27:39 am
I have got it working now but I sure do not know why it does (or why it didn't before)....

What I did is that I stripped down the generated html to bare minimums with just a few lines, no head etc. Then it worked.
Then I added back part by part and it still worked so finally I had the exact same page as before minus the line
<!DOCTYPE html public >
Still working so I added that too in and it still works!
So now I am back to square one with the same output but now it is accepted by Apache and sent back to the client....

Concerning the various doctype lines I have tried several of them and the last was cut from the source of a working webpage on the net. All earlier gave Apache hiccups and errors, but now not. I don't get it...
Now using:
Code: Pascal  [Select][+][-]
  1. <!DOCTYPE html>

There is only one item I did between non-working and working and this is trying to add fpWeb to Lazarus on suggestion from one contributor. Since Lazarus refused to rebuild the IDE whatever I did I went into the source dir and did:
Code: Pascal  [Select][+][-]
  1. make clean
  2. make bigide OPT=-dFPC_ARMHF FPC=/home/pi/bin/ppcarm
Then I had a fresh Lazarus and could rebuild the IDE with my own selection of extra packages plus fpWeb.
Next thing I did was to test a super-simple example, which worked. And finally went back as described above and it all worked.
Go figure.....
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: MarkMLl on December 18, 2019, 09:12:29 am
You probably screwed a line ending around that special blank line. Remember that on Linux if you pipe a program's output through xxd you see the hex.

MarkMLl
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 18, 2019, 02:06:01 pm
I have got it working now but I sure do not know why it does (or why it didn't before)....
Glad to read you made it worked.

Then I had a fresh Lazarus and could rebuild the IDE with my own selection of extra packages plus fpWeb.
This is my personal opinion, take it with extra grain of salt. I think fpWeb is great and all. But for simple web apps, it's overkill. I've made lots of simple web apps with 1-5 forms to serve less than 100 users. For that purpose, a simple and straight-forward unit like webCRT or webUtils is enough. The executable file size is a lot smaller too which is pretty important for CGI app (faster loading).

Unless you need to make a huge and complex web apps, fpWeb is more appropriate. It provides you with everything you need. It also scales very well. It's even better if you made your fpWeb app as FastCGI that responses instantly to Apache requests so it could handle lots more loads. It's like the desktop-class framework for web app, while unit like webCRT or webUtils is like a simple library for console or command line program in the DOS era.

Well, your mile may vary of course. Just share a thought of mind. :)
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: BosseB on December 18, 2019, 05:05:47 pm
This is my personal opinion, take it with extra grain of salt. I think fpWeb is great and all. But for simple web apps, it's overkill. I've made lots of simple web apps with 1-5 forms to serve less than 100 users. For that purpose, a simple and straight-forward unit like webCRT or webUtils is enough. The executable file size is a lot smaller too which is pretty important for CGI app (faster loading).
My current usage is for exactly 1 user...  ;D
To configure the RaspberryPi system before deployment and for navigating recorded data. All inside an SQLite database.
Speed is not a real issue.

Where do I find the webCRT and webUtils units?
EDIT:
Sorry, I found them both as links to GitHub in reply #14 on this thread by you....

EDIT2:
Now I have downloaded the two units from GitHub and also translated webUtils comments to English from Malayan
Using Google Translate, of course. It is pretty good at this...

Quote
Unless you need to make a huge and complex web apps, fpWeb is more appropriate. It provides you with everything you need. It also scales very well. It's even better if you made your fpWeb app as FastCGI that responses instantly to Apache requests so it could handle lots more loads. It's like the desktop-class framework for web app, while unit like webCRT or webUtils is like a simple library for console or command line program in the DOS era.
I installed the fpWeb package but then my original design started working so I have not looked further into it.
What does it give me when the simple CL program now works OK?
From the wiki I found I can create a CGI application (which is what this is all about) but when I tested it all I see is a very sparse main application with an initialization section for a web oriented object...
Quote
Well, your mile may vary of course. Just share a thought of mind. :)
I have a very simple use case but I want to avoid all hardware like displays, keyboards and the like. That is why I zoomed in on using a web based system.
The RPi will be set up as a WiFi Access Point to which you can connect your phone or tablet and then use the web browser to contact the system for config and results browsing. The RPi will have no Internet connection so you are the only user on the system.
I will make sure there is an index file that redirects to the web app so the proper pages will be shown.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: bee on December 19, 2019, 02:11:54 am
My current usage is for exactly 1 user...  ;D
To configure the RaspberryPi system before deployment and for navigating recorded data. All inside an SQLite database.
Speed is not a real issue.
Well… that's a perfect case for my webCRT or webUtils unit. I also have an SQLite tutorial. But again, it's in Bahasa Indonesia.

Now I have downloaded the two units from GitHub and also translated webUtils comments to English from Malayan
Using Google Translate, of course. It is pretty good at this…
It's actually Bahasa Indonesia, not Malayan. But it doesn't matter, for english speaker, both are pretty much the same anyway.

From the wiki I found I can create a CGI application (which is what this is all about) but when I tested it all I see is a very sparse main application with an initialization section for a web oriented object…
Yes, fpWeb has lots of boilerplate. It's too complex for a simple web app. That's why I made webCRT unit. If you want more nifty control over the protocol, you could use the webUtils unit.

I have a very simple use case but I want to avoid all hardware like displays, keyboards and the like. That is why I zoomed in on using a web based system.
Yes, that's the advantage of web application.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: PaulRowntree on December 19, 2019, 04:19:35 am
Along with the tutorial, I also made a WebCRT (https://github.com/pakLebah/webCRT) unit to demonstrate how easy it's to write web apps using Pascal and the unit. It's almost as easy as using the standard CRT unit.

For more control but with more "manual" way, you may use my webUtils (https://github.com/pakLebah/dasarweb) unit, like to handle sessions and cookies. The repo contains the codes of my web app tutorial. I think you can learn something from them although you don't read the tutorial.

Hope it will help.
Thanks Bee!
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: ThomasK on September 17, 2021, 05:51:54 pm
This is my personal opinion, take it with extra grain of salt. I think fpWeb is great and all. But for simple web apps, it's overkill. I've made lots of simple web apps with 1-5 forms to serve less than 100 users. For that purpose, a simple and straight-forward unit like webCRT or webUtils is enough. The executable file size is a lot smaller too which is pretty important for CGI app (faster loading).
My current usage is for exactly 1 user...  ;D
To configure the RaspberryPi system before deployment and for navigating recorded data. All inside an SQLite database.
Speed is not a real issue.

Where do I find the webCRT and webUtils units?
EDIT:
Sorry, I found them both as links to GitHub in reply #14 on this thread by you....

EDIT2:
Now I have downloaded the two units from GitHub and also translated webUtils comments to English from Malayan
Using Google Translate, of course. It is pretty good at this...

Quote
Unless you need to make a huge and complex web apps, fpWeb is more appropriate. It provides you with everything you need. It also scales very well. It's even better if you made your fpWeb app as FastCGI that responses instantly to Apache requests so it could handle lots more loads. It's like the desktop-class framework for web app, while unit like webCRT or webUtils is like a simple library for console or command line program in the DOS era.
I installed the fpWeb package but then my original design started working so I have not looked further into it.
What does it give me when the simple CL program now works OK?
From the wiki I found I can create a CGI application (which is what this is all about) but when I tested it all I see is a very sparse main application with an initialization section for a web oriented object...
Quote
Well, your mile may vary of course. Just share a thought of mind. :)
I have a very simple use case but I want to avoid all hardware like displays, keyboards and the like. That is why I zoomed in on using a web based system.
The RPi will be set up as a WiFi Access Point to which you can connect your phone or tablet and then use the web browser to contact the system for config and results browsing. The RPi will have no Internet connection so you are the only user on the system.
I will make sure there is an index file that redirects to the web app so the proper pages will be shown.

Hi Bo,

can you provide me with the translations? - I want to do more or less the same thing, a single page with data from my solar system.

Thanks and Best Regards,

Thomas
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: ojz0r on September 17, 2021, 09:04:31 pm
ThomasK: I've got something similar running right now but presenting temperature sensor data on a webserver and also controlling my house district heating system setpoint. Everything running with only pascal on my raspberry pi zero w.
The page itself is html with values in html <span> id's changed with an exported javascript file which is exported on http request and served.
Title: Re: Using pascal programs as backend for PHP based webpages?
Post by: devEric69 on September 18, 2021, 10:28:13 am
AMHO, use this https://wiki.freepascal.org/fpWeb_Tutorial (https://wiki.freepascal.org/fpWeb_Tutorial) or use this https://www.php.net/manual/fr/index.php (https://www.php.net/manual/fr/index.php).

Mixing the two isn't recommended, if its possible: be aware that mixing the 2 languages means that the CGI-Pascal-fpWeb named session - said PASCAL_SESSID, or whatever you want - knows how to retrieve the Php named session - said PHP_SESSID, or whatever you want - in order to be assigned with its\some web session variables (using embedded COOKIE or URL's GET or URL's POST shuttle), or vice versa. Absolutely possible. But this complicates things.

nb1: Php was created as a *.htm template handling laguage. For information, fpWeb has a *.htm template engine named fpTemplates.
nb2: CGI-Pascal-fpWeb is just a console program.
TinyPortal © 2005-2018