Multi-threaded web server example - performance tuning?

[Akira1364]

I just did another little test of my own, with a modified version of an example from Brook 5.0 (which I linked to in an earlier comment.) The program looks like this:

Code: Pascal  [Select][+][-]

1. program HelloHTTPServer;
2.  
3. {$mode ObjFPC}
4. {$ImplicitExceptions Off}
5.  
6. uses
7.   CMem,
8.   {$ifdef Unix}
9.   CThreads,
10.   {$endif}
11.   SysUtils,
12.   BrookLibraryLoader,
13.   BrookHTTPRequest,
14.   BrookHTTPResponse,
15.   BrookHTTPServer;
16.  
17. type
18.   THTTPServer = class(TBrookHTTPServer)
19.   protected
20.     procedure DoRequest(ASender: TObject;
21.                         ARequest: TBrookHTTPRequest;
22.                         AResponse: TBrookHTTPResponse); override;
23.   end;
24.  
25.   procedure THTTPServer.DoRequest(ASender: TObject;
26.                                   ARequest: TBrookHTTPRequest;
27.                                   AResponse: TBrookHTTPResponse);
28.   begin
29.     AResponse.Send('Hello,world!', 'text/plain', 200);
30.   end;
31.  
32. begin
33.   if not TBrookLibraryLoader.Load(TBrookLibraryLoader.LIB_NAME) then begin
34.     WriteLn(ErrOutput, 'Library not loaded.');
35.     Halt(1);
36.   end;
37.   with THTTPServer.Create(nil) do begin
38.     Port := 8080;
39.     ConnectionLimit := 100;
40.     Threaded := True;
41.     NoFavicon := True;
42.     Open();
43.     if not Active then Exit();
44.     ReadLn;
45.     Free();
46.   end;
47. end.

The AB output was, as I expected, quite good IMO:

D:\fpcupdeluxe\lazarus\components\brookframework\Examples\Console\FPC>ab -c 500 -n 500 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      500
Time taken for tests:   0.062 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      57000 bytes
HTML transferred:       6000 bytes
Requests per second:    8001.79 [#/sec] (mean)
Time per request:       62.486 [ms] (mean)
Time per request:       0.125 [ms] (mean, across all concurrent requests)
Transfer rate:          890.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.0      0      16
Processing:    16   32   9.3     31      47
Waiting:        0   31  10.6     31      47
Total:         16   33   9.3     31      47

Percentage of the requests served within a certain time (ms)
  50%     31
  66%     31
  75%     31
  80%     47
  90%     47
  95%     47
  98%     47
  99%     47
 100%     47 (longest request)

Come to think of it, I do also think Indy has a thread-pool-based scheduler component that can be used with the HTTP server, so that might be another library test worth looking into also.

[botster]

Time taken for tests:   0.062 seconds

That is a darn good number depending, of course, on your hardware. I'm not running anything high-end as you suspected; just a Core 2 Duo E7600 @ 3GHz with 4GB RAM.

I looked at the Brook framework. I'd have to download and compile a C library to use it. So, I can write in Pascal, but I'd have to use a C library to get performance? IMO, that would not be a very good showcase scenario for Free Pascal and could possibly lead to being mocked even of only lightheartedly.

I finally waded through some of the Indy10 documentation and found only a couple of GUI-based examples online. After much trial and error, I was able to get a minimal server operational.

It's performance is slightly worse than fphttpserver (@ low 300's of ms) turning in test times of roughly 300ms-500ms. Granted, that is without thread-pooling. But none of the other minimalistic servers I coded up and tested used pooling either, unless they did so internally by default.

[Leledumbo]

I looked at the Brook framework. I'd have to download and compile a C library to use it. So, I can write in Pascal, but I'd have to use a C library to get performance? IMO, that would not be a very good showcase scenario for Free Pascal and could possibly lead to being mocked even of only lightheartedly.

The author wants to:
- Make Brook available not only for modern Pascal languages (v4 was also available for Python if I'm not mistaken)
- Utilize the already proven libmicrohttpd, PCRE and some other existing libraries (i.e. no wheel reinventing, indeed libmicrohttpd is nicely implemented and quite oriented towards performance)

However, he knew that many of us are die hard Pascal fans that's alergy to non-pure Pascal code, so he created another one: https://github.com/risoflora/brookfreepascal, which is closer to the original Brook up to v3 (I still use this at work). The unfortunate part is that the two are not interchangeable, brookfreepascal is not simply brookframework with libsagui replaced by the same library implemented in pure Pascal.

[botster]

However, he knew that many of us are die hard Pascal fans that's alergy to non-pure Pascal code, so he created another one: https://github.com/risoflora/brookfreepascal, which is closer to the original Brook up to v3 (I still use this at work). The unfortunate part is that the two are not interchangeable, brookfreepascal is not simply brookframework with libsagui replaced by the same library implemented in pure Pascal.

Indeed, they appear to be significantly different. BrookFreePascal (BFP) appears to be mainly a CGI framework. (The README for the Brook framework doesn't say exactly what it is or for what use cases it would be applicable ) A screenshot on the BFP home page (https://risoflora.github.io/brookfreepascal/) suggests that it could be used to create an embedded server. So, I looked through the code a bit, and the only possible HTTP server related unit I found is BrookFCLHttpAppBroker which ... relies on FPHttpServer.

At one point in my research, I must have gotten some search terms right and discovered a program in an article that, I believe, you wrote at https://pascalgeek.blogspot.com/2012/06/encryption-decryption-and-asynchronous.html ("fcl-net, the undocumented treasure"). It's a relatively simple multi-threaded TCP server! I struggled and over time managed to adapt that program to correctly read and respond to a request from a web browser. Aside from a just a few minor changes including adding the 'cmem' unit and defining the "Response" buffer globally, here is what I ended up with for the TClientHandlerThread.Execute procedure:

Code: Pascal  [Select][+][-]

1. procedure TClientHandlerThread.Execute;
2. var
3.   Request: array[0..1023] of char;
4.   Done: Boolean;
5. begin
6.   Done := false;
7.   repeat
8.     try
9.       FClientStream.Read(Request, Length(Request))
10.       FClientStream.Write(Response, Length(Response))
11.       FClientStream.Free;
12.       Exit;
13.     except
14.       on e: EStreamError do begin
15.         Done := true;
16.       end;
17.     end;
18.   until Done;
19.   WriteLn(AddrToString(FClientStream.PeerAddress) + ' disconnected');
20. end;
21.  

That server program yields results similar to the FPHttpServer-based server example, from Rosetta Code, that I started with:

Concurrency Level:      500
Time taken for tests:   0.232 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      32000 bytes
HTML transferred:       12500 bytes
Requests per second:    2157.69 [#/sec] (mean)
Time per request:       231.729 [ms] (mean)
Time per request:       0.463 [ms] (mean, across all concurrent requests)
Transfer rate:          134.86 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   12  12.0     14      28
Processing:     0    8  14.0     10     198
Waiting:        0    6  13.7      5     198
Total:          0   20  21.8     28     218

Percentage of the requests served within a certain time (ms)
  50%     28
  66%     36
  75%     36
  80%     37
  90%     38
  95%     38
  98%     39
  99%     59
 100%    218 (longest request)

The numbers look good except for that last 1% of requests served outlier of 218ms. Occasionally, though, that same server will yield an anomalous result:

Concurrency Level:      500
Time taken for tests:   0.094 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      32000 bytes
HTML transferred:       12500 bytes
Requests per second:    5317.68 [#/sec] (mean)
Time per request:       94.026 [ms] (mean)
Time per request:       0.188 [ms] (mean, across all concurrent requests)
Transfer rate:          332.35 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5   18  10.6     15      34
Processing:     5   14   3.7     14      32
Waiting:        3    9   4.6      9      29
Total:         18   32  11.0     24      65

Percentage of the requests served within a certain time (ms)
  50%     24
  66%     43
  75%     44
  80%     45
  90%     46
  95%     46
  98%     47
  99%     47
 100%     65 (longest request)

ALL good numbers and definitely comparable with the Golang and Rust server implementations. But unfortunately, this type of result is the exception, and I have no clue how to reproduce it consistently.

And, confusingly to me, the FPHttpServer-based server under "Siege" (`siege -b -c500 -r1 http://localhost:8080/`) will be relatively consistent in performance (1.49-3.81 secs) while the TCP server's performance will get progressively worse on each successive run eventually slowing to a crawl. Golang and Rust server implementations are both rather consistent under siege at around 0.22-1.28 secs.

Anyway, I've spent more time than I should have exploring this potential project. I sincerely appreciate everyone's comments and suggestions.

[zamronypj]

Running apache benchmark to test http server on same machine may not yield accurate result.  Higher CPU usage by apache benchmark may result in poor performance of http server due to both competing for CPU usage.