FCGI (FastCGI) step by step, using fpweb/fcl-web for Windows/Linux

[bobo]

I've created this readme in the past few days as I was running some tests, hope it will help people set up and use FCGI with Lazarus/FPC, or at least help to understand it better
Note, you will need the latest SVN version of /packages/fcl-web/ of FPC for everything to work as explained here (version 2.5.1 or later).
There are also example programs under /packages/fcl-web/examples/  . Many of them has all three of CGI/FCGI/Apache module versions.

===============================================================================
FCGI-README.txt
Last modified: Attila Borka 05/31/2011
-------------------------------------------------------------------------------
This readme will try to explain and help developers how to be able to
use the FastCGI protocol in Lazarus (package fpweb/lazweb) and FPC
(/packages/fcl-web) applications with the Apache web server.

There are two FCGI implementations we are able to use at the moment (FPC
rev17597 /2.5.1/) with FPC/Lazarus:

1. mod_fcgid maintained by Apache (ASF) http://httpd.apache.org/mod_fcgid/
Easy to set up and use, Apache develops and maintains it. Just needs a
download/install and it is ready to go, even on Linux.
There are a lot of options and fine tuning that can be used, please refer to
their documentation at http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html .

2. mod_fastcgi from http://www.fastcgi.com/ the original FCGI "inventors"
The latest downloads are available from http://www.fastcgi.com/dist/ . Do not
be alarmed if the latest one is years old. FCGI is a very stable protocol
since many years (started in 1996).
On Linux, you will need to compile the mod_fastcgi module from the source
codes, as binaries are only available for Windows as a DLL.
Note, that mod_fastcgi supports the use of External FCGI Servers, which makes
it possible to run the FCGI application from within Lazarus for example, and
be able to debug it with the GUI on the fly while it is generating the
response pages. This can be a good choice for a development environment.
The documentation is at http://www.fastcgi.com/drupal/node/6 .


There are benefits to use both, everyone needs to decide which one fits their
needs the best for a development environment or a live web server after some
research on these modules.

Do not forget, that unless you use the External FCGI Server way, you need to
restart your web server every time you recompile your FCGI program.
In case you use the External FCGI Server, you need to restart the application
itself after every recompile for the changes to take effect in the web server
responses. The Apache server does not need to be restarted every time in this
latter case.

===============================================================================
1. mod_fcgid from Apache
1.1 on Windows
1.2 on Linux
2. mod_fastcgi from fastcgi.com
2.1 on Windows
2.1 on Linux
===============================================================================

1. mod_fcgid from Apache:
=========================

1.1 mod_fcgid from Apache on Windows:
-------------------------------------
Setup Steps:
1.1.1 Download from http://httpd.apache.org/download.cgi#mod_fcgid the latest
Windows zip file and unpack as it instructs you to, over the installed Apache
directory. This puts the mod_fcgid.* files under the current /modules
directory of Apache.

1.1.2 Now, mod_fcgid is ready to be used as soon as we load it with Apache.

1.1.3 If you have not done so yet, compile your FCGI application.

1.1.4 Then, edit the Apache /conf/httpd.conf file and add to the end:
############
LoadModule fcgid_module "modules/mod_fcgid.so"
<IfModule mod_fcgid.c>
  <Directory "<Path_To_Your_FCGI_application>">
    SetHandler fcgid-script
#    Options +ExecCGI  <- not needed if ScriptAlias is used below
    Order allow,deny
    Allow from all
  </Directory>
#optionally, to shorten the URL and to not display the executable file name
#(if ScriptAlias is used, no +ExecCGI is needed above)
  ScriptAlias /myfcgid "<Path_To_Your_FCGI_application>/<Your_FCGI_application>"
</IfModule>
############
Example:
LoadModule fcgid_module "modules/mod_fcgid.so"
<IfModule mod_fcgid.c>
  <Directory "C:/My Programs/LazarusFCGITest">
    SetHandler fcgid-script
    Order allow,deny
    Allow from all
  </Directory>
  ScriptAlias /myfcgid "C:/My Programs/LazarusFCGITest/helloworld.exe"
</IfModule>

Note, there are many ways to configure the FCGI applications, this is just but
one example. You can check the Apache and mod_fcgid documentation for
alternatives.

1.1.5 Start/Restart your Apache server so it will load your FCGI application.
If everything went according to plan, your FCGI application should be listed
in the Windows task manager as running.

1.1.6 Open your web browser and try to call your new FCGI application.
Example:
http://127.0.0.1:8080/myfcgid/func1call
or
http://127.0.0.1/myfcgid/func1call
depending on your Apache installation and configuration. "myfcgid" is the
ScriptAlias name specified for the FCGI application, func1call is the action
name we want to call within our default web module. If you have multiple web
modules, you can enter the desired web module name before the action name, for
example:
http://127.0.0.1:8080/myfcgid/webmodule1/func1call
or
http://127.0.0.1/myfcgid/webmodule1/func1call

If there is any problem, you can try and check the Apache error.log for clues.


1.2 mod_fcgid from Apache on Linux:
-----------------------------------
Setup Steps:
1.2.1 Install the mod_fcgi module for Apache with your distro's package
manager.
Example on Ubuntu:
sudo apt-get install libapache2-mod-fcgid

This downloads, installs and configures mod_fcgid without the need to do
anything else.

1.2.2 Now, mod_fcgid is ready to be used as soon as we load it with Apache.
mod_fcgid.so should be sitting in the /usr/lib/apache2/modules/ directory (on
Ubuntu)

1.2.3 If you have not done so yet, compile your FCGI application.

1.2.4 Edit the Apache configuration file (/etc/apache2/apache2.conf on Ubuntu)
and add to the end:
############
LoadModule fcgid_module "<Path_To_Mod>/mod_fcgid.so"
<IfModule mod_fcgid.c>
  <Directory "<Path_To_Your_FCGI_application>">
    SetHandler fcgid-script
#    Options +ExecCGI  <- not needed if ScriptAlias is used below
    Order allow,deny
    Allow from all
  </Directory>
#optionally, to shorten the URL and to not display the executable file name
#(if ScriptAlias is used, no +ExecCGI is needed above)
  ScriptAlias /myfcgid "<Path_To_Your_FCGI_application>/<Your_FCGI_application>"
</IfModule>
############
Example:
LoadModule fcgid_module "/usr/lib/apache2/modules/mod_fcgid.so"
<IfModule mod_fcgid.c>
  <Directory "/home/johndoe/LazarusFCGITest">
    SetHandler fcgid-script
    Order allow,deny
    Allow from all
  </Directory>
  ScriptAlias /myfcgid "/home/johndoe/LazarusFCGITest/helloworld"
</IfModule>

(the project was compiled into directory /home/johndoe/LazarusFCGITest/ , and
the FCGI application is called helloworld with no file extension)

Note, there are many ways to configure the FCGI applications, this is just but
one example. You can check the Apache and mod_fcgid documentation for
alternatives.

1.2.5 Start/Restart your Apache server so it will load your FCGI application.
If everything went according to plan, and you do a "sudo netstat -l"
(on Ubuntu) from a terminal window, there should be a new line in the result
list looking something like this:
... LISTENING   XXXX   /var/lib/apache2/fcgid/sock/...
indicating, that the Apache mod_fcgid module has loaded your FCGI application.

1.2.6 Open your web browser and try to call your new FCGI application.
Example:
http://127.0.0.1/myfcgid/func1call

"myfcgid" is the ScriptAlias name specified for the FCGI application, and
func1call is the action name we want to call within our default web module.
If you have multiple web modules, you can enter the desired web module name
before the action name, for example:
http://127.0.0.1/myfcgid/webmodule1/func1call .

If there is any problem, you can try and check the Apache error.log for clues.


2. mod_fastcgi from fastcgi.com:
================================
Unlike the Apache developed mod_fcgid, mod_fastcgi has two main operating
modes for FCGI applications. One is very similar to mod_fcgid (where Apache
itself loads the FCGI application at startup), and one called External FCGI
Server mode. With this latter, Apache will not load the FCGI application at
startup, but it has to be running on its own when a web request arrives
(either as a system service/daemon, or a simple running application) and
listening. This makes it possible to run the FCGI application from within a
debugger interactively, to see/track the request handling like any normal GUI
application debugging.

2.1 mod_fastcgi from fastcgi.com on Windows:
--------------------------------------------
Setup Steps:
2.1.1 Download the latest SNAP or stable DLL from http://www.fastcgi.com/dist/
and put it into the Apache /modules/ directory.

2.1.2 If you have not done so yet, compile your FCGI application.
In case you want to set up an External FCGI Server, then you must specify a
port number in your main project file (.lpr) before the Application.Run
instruction (this is the only change needed).
Example:
<...snip...>
  Application.Initialize;
  Application.Port:=9999;//Port the FCGI application is listening on
  Application.Run;
<...snip...>

2.1.3 Edit the Apache /conf/httpd.conf file and add to the end:

 2.1.3.a External FCGI Server
############
LoadModule fastcgi_module "modules/<mod_fastcgi_DLL_name>"
<IfModule mod_fastcgi.c>
  <Directory "<Path_To_Your_FCGI_application>">
#    Options +ExecCGI  <- not needed if ScriptAlias is used below
    Order allow,deny
    Allow from all
  </Directory>
#External FCGI app, has to be started and running when a request comes in
  FastCgiExternalServer "<Path_To_Your_FCGI_application>/<Your_FCGI_application>" -host 127.0.0.1:<Port> -idle-timeout 30 -flush
#optionally, to shorten the URL and to not display the executable file name (if used, no +ExecCGI is needed above):
  ScriptAlias /myfcgi "<Path_To_Your_FCGI_application>/<Your_FCGI_application>"
</IfModule>
############
Example:
LoadModule fastcgi_module "modules/mod_fastcgi-2.4.6-AP22.dll"
<IfModule mod_fastcgi.c>
  <Directory "C:/My Programs/LazarusFCGITest">
    Order allow,deny
    Allow from all
  </Directory>
  FastCgiExternalServer "C:/My Programs/LazarusFCGITest/helloworld.exe" -host 127.0.0.1:9999 -idle-timeout 30 -flush
  ScriptAlias /myfcgi "C:/My Programs/LazarusFCGITest/helloworld.exe"
</IfModule>

 2.1.3.b Regular FCGI Server
Replace the FastCgiExternalServer line above with
  FastCgiServer "<Path_To_Your_FCGI_application>/<Your_FCGI_application>" -idle-timeout 30

Example:
LoadModule fastcgi_module "modules/mod_fastcgi-2.4.6-AP22.dll"
<IfModule mod_fastcgi.c>
  <Directory "C:/My Programs/LazarusFCGITest">
    Order allow,deny
    Allow from all
  </Directory>
  FastCgiServer "C:/My Programs/LazarusFCGITest/helloworld.exe" -idle-timeout 30
  ScriptAlias /myfcgi "C:/My Programs/LazarusFCGITest/helloworld.exe"
</IfModule>

2.1.4 Start/Restart your Apache server.
If you will use the FastCgiExternalServer, then start your application
manually, so it can start accepting incoming requests from the web server.
If you use the FastCgiServer, then your FCGI application should be launched
by the Apache server when it starts and should be listed in the Windows task
manager as running.

2.1.5 Open your web browser and try to call your new FCGI application.
Example:
http://127.0.0.1:8080/myfcgi/func1call
or
http://127.0.0.1/myfcgi/func1call
depending on your Apache installation and configuration.

"myfcgi" is the ScriptAlias name specified for the FCGI application, and
func1call is the action name we want to call within our default web module.
If you have multiple web modules, you can enter the desired web module name
before the action name, for example:
http://127.0.0.1:8080/myfcgi/webmodule1/func1call
or
http://127.0.0.1/myfcgi/webmodule1/func1call

If there is any problem, you can try and check the Apache error.log for clues.


2.2 mod_fastcgi from fastcgi.com on Linux:
------------------------------------------
Setup Steps:
2.2.1 There are no binaries offered for download for Linux, so we need to get
the mod_fastcgi source codes from http://www.fastcgi.com/dist/ and compile it.

 2.2.1.1. If the Apache development package headers are not installed, then
we need to get them first:
(on Ubuntu)
>sudo apt-get install apache2-dev

 2.2.1.2. Get the fastcgi module sources (get the latest SNAP or highest
version stable release from http://www.fastcgi.com/dist/ )

>cd /home/johndoe
>wget http://www.fastcgi.com/dist/mod_fastcgi-xxxxxxxxxxx.tar.gz

Example: wget http://www.fastcgi.com/dist/mod_fastcgi-SNAP-0910052141.tar.gz

 2.2.1.3. Unpack the source code files.
>tar -xzf mod_fastcgi-*.tar.gz

 2.2.1.4. Configure the makefile (we use Apache 2.2, so need the Makefile.AP2).
>cd mod_fastcgi-*
>cp Makefile.AP2 Makefile

 2.2.1.5. Edit the copied Makefile and set top_dir to the proper apache source
directory created by the Apache development package install. For example, on
Ubuntu it is /usr/share/apache2 (containing the .mk file).

 2.2.1.6. Compile and install (on Ubuntu)
>make
>sudo make install

2.2.2. Now, we should have a mod_fastcgi.so in /usr/lib/apache2/modules (on
Ubuntu)

2.2.3. If you have not done so yet, compile your FCGI application.
In case you want to set up an External FCGI Server, then you must specify a
port number in your main project file (.lpr) before the Application.Run
instruction (this is the only change needed).
Example:
<...snip...>
  Application.Initialize;
  Application.Port:=1234;//Port the FCGI application is listening on
  Application.Run;
<...snip...>

2.2.4 Edit the Apache configuration file (/etc/apache2/apache2.conf on Ubuntu)
and add to the end:
 2.2.4.a External FCGI Server
############
LoadModule fastcgi_module "<Path_To_Mod>/mod_fastcgi.so"
<IfModule mod_fastcgi.c>
  <Directory "<Path_To_Your_FCGI_application>">
#    Options +ExecCGI  <- not needed if ScriptAlias is used below
    Order allow,deny
    Allow from all
  </Directory>
#External FCGI app, has to be manually started and running when a request comes in
  FastCgiExternalServer "<Path_To_Your_FCGI_application>/<Your_FCGI_application>" -host 127.0.0.1:<Port> -idle-timeout 30 -flush
#optionally, to shorten the URL and to not display the executable file name (if used, no +ExecCGI is needed above):
  ScriptAlias /myfcgi "<Path_To_Your_FCGI_application>/<Your_FCGI_application>"
</IfModule>
############
Example:
LoadModule fastcgi_module "/usr/lib/apache2/modules/mod_fastcgi.so"
<IfModule mod_fastcgi.c>
  <Directory "/home/johndoe/LazarusFCGITest">
    Order allow,deny
    Allow from all
  </Directory>
  FastCgiExternalServer "/home/johndoe/LazarusFCGITest/helloworld" -host 127.0.0.1:1234 -idle-timeout 30 -flush
  ScriptAlias /myfcgi "/home/johndoe/LazarusFCGITest/helloworld"
</IfModule>

 2.2.4.b Regular FCGI Server
Replace the FastCgiExternalServer line above with
  FastCgiServer "<Path_To_Your_FCGI_application>/<Your_FCGI_application>" -idle-timeout 30

Example:
LoadModule fastcgi_module "/usr/lib/apache2/modules/mod_fastcgi.so"
<IfModule mod_fastcgi.c>
  <Directory "/home/johndoe/LazarusFCGITest">
    Order allow,deny
    Allow from all
  </Directory>
  FastCgiServer "/home/johndoe/LazarusFCGITest/helloworld" -idle-timeout 30
  ScriptAlias /myfcgi "/home/johndoe/LazarusFCGITest/helloworld"
</IfModule>

(the project is compiled into directory /home/johndoe/LazarusFCGITest/ , and
the FCGI application is called helloworld with no file extension)

Note, there are many ways to configure the FCGI applications, this is just but
one example. You can check the Apache and mod_fastcgi documentation for
alternatives. For example, some more information available at
http://www.fastcgi.com/docs/faq.html#typical_httpd.conf

2.2.5 Start/Restart your Apache server.
If FastCgiExternalServer is used, then start your application manually, so it
can start accepting incoming requests from the web server.
On the other hand, if FastCgiServer is used and everything went according to
plan, then the Apache error.log should contain a warning message about your
FCGI application, saying that it was started. Something like:
[warn] FastCGI: server "/home/johndoe/LazarusFCGITest/helloworld" started...

2.2.6 Open your web browser and try to call your new FCGI application.
Example:
http://127.0.0.1/myfcgi/func1call
"myfcgi" is the ScriptAlias name specified for the FCGI application, and
func1call is the action name we want to call within our default web module.
If you have multiple web modules, you can enter the desired web module name
before the action name, for example:
http://127.0.0.1/myfcgi/webmodule1/func1call

If there is any problem, you can try and check the Apache error.log for clues.

===============================================================================

[nicke85]

Thank you for your time bobo..tutorial is very nice..do you plan to make pdf maybe!?

[bobo]

I did not plan to make it in other formats, anyone is free to do so.  This is now included in the latest FPC SVN under /packages/fcl-web/src/base/FCGI-README.txt

If something is not clear, please let me know so the file can be updated.

We have also moved (as well as updated/fixed for the latest fcl-web changes) the fpweb examples from Lazarus to FPC under /packages/fcl-web/examples/ because LCL is no longer required for fpweb applications.

[xenblaise]

hi
Just want to ask first before applying this on the near future,
Thus this make my web app rans faster?

thanks

[bobo]

The short answer: Yes.

The long answer and the comparison between CGI and FCGI can be found here:
http://www.fastcgi.com/drupal/node/6?q=node/15

However, for simple, low traffic websites the difference might not be noticeable.

[jixian.yang]

FastCGI can not be found in SVN18020 of Freepascal.

[Silvio Clécio]

The long thread about FCGI:

http://www.mail-archive.com/lazarus@lists.lazarus.freepascal.org/msg20061.html

[kodok_buncit]

hi bobo and all..
is this fcgi app multhithread?

[bobo]

kodok_buncit:
The CGI modules are handling one request at a time - there is no need for multi-thread theoretically - because the web server starts up a new instance of the module for each request. If there are multiple simultaneous requests are arriving, then there will be multiple instances of the CGI application running at the same time.

For FCGI modules, since they are sitting in the memory, it would be possible to do multi-threading, but the standard is to have a queue for multiple request handling and the protocol has an identifier for every request so the server does not mixes up the responses from the same FCGI application instance.
A "pseudo" multi-threading is done kind of the same way as with CGI modules (where every request spawns a new copy of the CGI application) because the web server (Apache in these examples) can spawn multiple instances of the FCGI module to sit in the memory and to wait for incoming requests.
There are many many configuration settings for the web server on how to do the FCGI request/response handling, the best if you read up on them in the documentation.
For example, how many instances of the FCGI application to spawn, after how many requests to restart an FCGI instance, after how much idle time to restart an FCGI instance, etc. (these vary between the mod_fcgid and mod_fastcgi, each has their own settings)

[kodok_buncit]

thanks bobo, i'm curious about how implemented connection pooling(database) on fcgi environment and share object instance accross all client/request so the app can be more efficient.
I will read more about fastcgi

[GiovanniF]

" using fpweb/fcl-web for Windows/Linux"

How can one "use fpweb/fcl-web" with LINUX?

Does ANYONE know a way to install the components ( fpweb ) under LINUX
since the package ( lpk ) file is missing?

Thanks for ANY helpö.

[Leledumbo]

How can one "use fpweb/fcl-web" with LINUX?

Does ANYONE know a way to install the components ( fpweb ) under LINUX
since the package ( lpk ) file is missing?

Thanks for ANY helpö.

Missing? What do you have under components/fpweb then?

[GiovanniF]

Well......

Read the message:  I am talking about LINUX version of fpweb.

IF you go to the components DIR and look inside the fpwep component DIR
you will see that the file weblaz.lpk is MISSING.

Have a nice day.

[BigChimp]

On Debian x64 with FPC fixes_2.6, Lazarus trunk (today) I have:

Code: [Select]

ls -al /home/pascaldev/lazarus/components/fpweb/weblaz.lpk
-rw-r--r-- 1 pascaldev pascaldev 3867 Feb 18 15:16 /home/pascaldev/lazarus/components/fpweb/weblaz.lpk
It compiles as well...

Perhaps you have a faulty download/buggy snapshot/svn version?

[TheBlackSheep]

Hi all
 
I struggled with this for a while but think I'm getting somewhere now;

Using the httpd-2.4.1-win32-ssl_0.9.8t.zip  & mod_fcgid-2.3.7-win32.zip from ApacheLounge - installed apache as a service and put the fcgid module in the modules folder in the apache folder structure.

Created a folder "C:\FastCGI" and set the following in the apache http.conf

Code: [Select]

<Directory "C:/FastCGI">
Options All
AllowOverride All
Require all granted
</Directory>

LoadModule fcgid_module modules/mod_fcgid.so
<IfModule mod_fcgid.c>
  <Directory "C:/FastCGI">
   SetHandler fcgid-script
    Order allow,deny
    Allow from all
  </Directory>
  ScriptAlias /myfcgid "C:/FastCGI/helloworld.exe"
  ScriptAlias /fcgilist "C:/FastCGI/listrecords.exe"
</IfModule>
The bit I struggled with was that the demos have the listening "port" set to 2015 in the application source (the lpr file) - once you comment this out it seems to work fine with this setup - both applications are launched the first time they are called from the browser and remain in memory for each subsequent request.

Using the other fastcgi dll (i.e. not the apache fcgid module but the fastcgi module) with the dll renamed and the following configuration (the above fcgi version commented out)

LoadModule fastcgi_module modules/mod_fastcgi-2.4.6-AP22.so

I get an error trying to start apache;

"cannot load C:/Apache24/modules/mod_fastcgi-2.4.6-AP22.so into server: The specified procedure could not be found."

so this version might not work with Apache 2.4.1 - does anyone know of a verison that does or have I missed something?

I'm not really bothered about this latter module unless it's the only way to do realtime debugging so the question is using apache's own fcgid module (the first setup) is there an easy way to setup the configuration to do realtime debugging (i.e. set breakpoints etc)?

TheBlackSheep