Recent

Author Topic: Multi-threaded web server example - performance tuning?  (Read 11172 times)

botster

  • New Member
  • *
  • Posts: 39
Multi-threaded web server example - performance tuning?
« on: March 25, 2019, 01:49:03 am »
I found the following simple webserver example on Rosetta Code and adapted it to be multi-threaded by adding the 'cthreads' unit and changing "Serv.Threaded" to True. I have no idea if that was the preferred way to do that since I cannot find any documentation for 'fphttpserver'. I compiled it with just the "-O3" option.

Using the Apache HTTP benchmarking tool to test it for concurrent performance, I found the results disappointing:

> ab -c 500 -n 500 http://localhost:8080/
Quote
This is ApacheBench, Version 2.3 <$Revision: 1748469 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:       
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      500
Time taken for tests:   0.302 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      46000 bytes
HTML transferred:       6000 bytes
Requests per second:    1657.92 [#/sec] (mean)
Time per request:       301.583 [ms] (mean)
Time per request:       0.603 [ms] (mean, across all concurrent requests)
Transfer rate:          148.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   13  17.4      0      41
Processing:     6   31  25.1     20     223
Waiting:        6   19  17.0     18     217
Total:          6   44  39.0     20     241

Percentage of the requests served within a certain time (ms)
  50%     20
  66%     76
  75%     84
  80%     86
  90%     94
  95%    103
  98%    117
  99%    121
 100%    241 (longest request)

Here is the code:
Code: Pascal  [Select][+][-]
  1. program HelloWorldServer;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.     cthreads, Classes, fphttpserver;
  6.  
  7. Type
  8.     TTestHTTPServer = Class(TFPHTTPServer)
  9.     public
  10.         procedure HandleRequest(
  11.             Var ARequest: TFPHTTPConnectionRequest;
  12.             Var AResponse : TFPHTTPConnectionResponse); override;
  13.     end;
  14.  
  15. Var
  16.     Serv : TTestHTTPServer;
  17.  
  18. procedure TTestHTTPServer.HandleRequest(
  19.     var ARequest: TFPHTTPConnectionRequest;
  20.     var AResponse: TFPHTTPConnectionResponse);
  21.  
  22.     Var
  23.         F : TStringStream;
  24.  
  25.     begin
  26.     F:=TStringStream.Create('Hello,World!');
  27.     try
  28.         AResponse.ContentLength:=F.Size;
  29.         AResponse.ContentStream:=F;
  30.         AResponse.SendContent;
  31.         AResponse.ContentStream:=Nil;
  32.     finally
  33.         F.Free;
  34.     end;
  35. end;
  36.  
  37. begin
  38.     Serv:=TTestHTTPServer.Create(Nil);
  39.     try
  40.         Serv.Threaded:=True;
  41.         Serv.Port:=8080;
  42.         Serv.AcceptIdleTimeout:=1000;
  43.         Serv.Active:=True;
  44.     finally
  45.         Serv.Free;
  46.     end;
  47. end.
  48.  

I have searched the forum (& elsewhere) and read several posts, but found only two that seem even somewhat relevant.

The first is:
http://forum.lazarus.freepascal.org/index.php/topic,21524.msg126020.html#msg126020
But, I don't understand why one would give a threaded server its own thread. Why not just let the server handle threads?

The second is:
http://forum.lazarus.freepascal.org/index.php/topic,31340.msg200664.html#msg200664
The last post in that thread (by Leledumbo) says:
Quote
Paste server uses thread pool, fphttpserver doesn't. I think that's the biggest factor if you compare multithreaded feature of a web server. Michael as the author of fpWeb and the underlying fcl-web package once said he actually never meant the embedded http server to be used in production

Okay then.

Is there a package that is better suited to this simple example?

Or, is there a way to optimize this example for better performance?
Lee

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Multi-threaded web server example - performance tuning?
« Reply #1 on: March 25, 2019, 06:11:23 am »
Is there a package that is better suited to this simple example?
No ATM.
Or, is there a way to optimize this example for better performance?
Use what's meant to be used for production level performance: apache/nginx as reverse proxy listening to fpweb's fastcgi service. That's what I use at work.

vangli

  • New Member
  • *
  • Posts: 44
Re: Multi-threaded web server example - performance tuning?
« Reply #2 on: March 25, 2019, 07:50:40 am »
For my own interrest, did you try also using 'cmem' unit, as this in documetation is said to be much faster on some systems?

uses
    cthreads, cmem, Classes, fphttpserver;

It could be intereseting to see if it changes something in your test.

Bent  :)

Regards Bent

botster

  • New Member
  • *
  • Posts: 39
Re: Multi-threaded web server example - performance tuning?
« Reply #3 on: March 26, 2019, 06:33:21 am »
Bent, I just tried it according to your suggestion, ie. with the 'cmem' unit.

It does seem to be just slightly better performance with more consistent numbers over several test runs. But, it is still relatively slow. Here are the numbers for the best run (what gets me is that 228ms outlier):

Quote
Concurrency Level:      500
Time taken for tests:   0.230 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      46000 bytes
HTML transferred:       6000 bytes
Requests per second:    2174.92 [#/sec] (mean)
Time per request:       229.894 [ms] (mean)
Time per request:       0.460 [ms] (mean, across all concurrent requests)
Transfer rate:          195.40 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    6  10.5      0      29
Processing:     0    7  14.0      3     204
Waiting:        0    6  13.7      2     204
Total:          0   13  20.4      4     228

Percentage of the requests served within a certain time (ms)
  50%      4
  66%     11
  75%     22
  80%     37
  90%     38
  95%     39
  98%     41
  99%     41
 100%    228 (longest request)

Thank you for the suggestion.
Lee

vangli

  • New Member
  • *
  • Posts: 44
Re: Multi-threaded web server example - performance tuning?
« Reply #4 on: March 26, 2019, 08:17:32 am »
Lee, thanks for doing the alternative test with cmem. I have never done it myself, just taking for granted that cmem is faster and at least do not hurt. It seems that the claim for faster memory management in multithreaded applications are realy true. Good to know.

Hope you find your true bottleneck, whatever it is.

Regards Bent, Oslo, Norway
« Last Edit: March 26, 2019, 08:19:34 am by vangli »
Regards Bent

asdf121

  • New Member
  • *
  • Posts: 35
Re: Multi-threaded web server example - performance tuning?
« Reply #5 on: March 26, 2019, 12:48:14 pm »
You could try some other memory managers like FastMM4, ScaleMM or SynScaleMM. Maybe they perform much better in multithreading :)
Other idea might be to optimize your code at all. Instead of creating a new TStringStream each time, use one to serve all as it does the same thing for all clients.
Using compression like gzip (if supported) would also help, at least for bigger data.

mr-highball

  • Full Member
  • ***
  • Posts: 233
    • Highball Github
Re: Multi-threaded web server example - performance tuning?
« Reply #6 on: March 26, 2019, 02:37:53 pm »
Have you seen this library?
https://github.com/fundamentalslib/fundamentals5

specifically : https://github.com/fundamentalslib/fundamentals5/blob/master/Source/HTTP/flcHTTPServer.pas#L280

I've been meaning play around with the http components to see how they stack up to some other packages I've used, but I just haven't gotten around to it...  :-[

botster

  • New Member
  • *
  • Posts: 39
Re: Multi-threaded web server example - performance tuning?
« Reply #7 on: March 26, 2019, 07:43:26 pm »
You could try some other memory managers like FastMM4, ScaleMM or SynScaleMM. Maybe they perform much better in multithreading :)
Other idea might be to optimize your code at all. Instead of creating a new TStringStream each time, use one to serve all as it does the same thing for all clients.

SynScaleMM requires FastMM4, and I can get neither FastMM4 nor ScaleMM to compile even after switching the mode to 'delphi'. I tried using only one instance of TStringStream by moving its declaration to the global scope and creating & initializing it once in the main procedure. Performance is about the same, but now I get a significant number of failed requests.


Have you seen this library?
https://github.com/fundamentalslib/fundamentals5

I took a cursory look at that. It does not appear to implement multi-threading. I may have missed it.


Thank you all for the ideas.

It seems that simply plugging in a few units and coding up a comparably performing webserver, even as a trivial example, is not easily doable. To me, that is sadly unfortunate. The first professional work I ever did was with Turbo Pascal.
Lee

mr-highball

  • Full Member
  • ***
  • Posts: 233
    • Highball Github
Re: Multi-threaded web server example - performance tuning?
« Reply #8 on: March 26, 2019, 08:26:07 pm »
You may want to consider taking another look at fundamentals ;)
There's use of a critical section in the http server class for handling events and the underlying server is a TF5TcpServer which even mentions in the comments at the top of the unit "making use of worker thread"

Like I said, I haven't had time to go through everything but I believe it is a good option. Additionally you could install indy with fpcupdeluxe which may also do what you want.

botster

  • New Member
  • *
  • Posts: 39
Re: Multi-threaded web server example - performance tuning?
« Reply #9 on: March 26, 2019, 11:33:36 pm »
You may want to consider taking another look at fundamentals ;)

... Additionally you could install indy with fpcupdeluxe which may also do what you want.

You are right. The TCP Server, a dependency of the HTTP Server, does indeed use threads. However, I can find no documentation whatsoever for Fundamentals5; not even any examples for guidance. Maybe it's just me, but I don't think I should have to analyze the code just to attempt to figure out how to use it.

I downloaded and launched fpcupdeluxe; selected my preferred install directory, went to Modules, and attempted to install 'indy':
Quote
fpcupdeluxe: info: UniversalInstaller (BuildModule: indy): Building module indy...

fpcupdeluxe: info: UniversalInstaller (BuildModule: indy): Building module indy running all InstallExecute commands in:
Name=indy
Description="Indy 10 networking library for FPC / Lazarus"
Installdir=$(basedir)/ccr/$(name)
Enabled=0
SVNURL=https://svn.atozed.com:444/svn/Indy10/trunk/
UserName=Indy-Public-RO
Password=
ArchiveURL=https://indy.fulgan.com/ZIP/Indy10_5495.zip
AddPackage=$(Installdir)/Lib/indylaz.lpk
UnInstall=rm -Rf $(Installdir)
Category=miscellaneous

fpcupdeluxe: info: UniversalInstaller (InstallPackage): Installing indylaz version 10.6.2
Executing: /home/user/Coding/FPC-Lazarus/lazarus/lazbuild --version (working dir: )
fpcupdeluxe: info: UniversalInstaller (InstallPackage): Error trying to add package indylaz
fpcupdeluxe: WARNING: UniversalInstaller (InstallPackage): Error while installing package /home/user/Coding/FPC-Lazarus/ccr/indy/Lib/indylaz.lpk.


SUCCESS: Fpcupdeluxe ended without errors.

Hmmm, wonder what the error was exactly.


Anyway, at this point, I have had enough frustration. This is what invariably happens every time I have taken yet another hopeful look at Free Pascal. Every time, no, inadequate, and/or incongruent documentation or not being able to simply "get there from here" has cost me many lost hours of searching and trying without accomplishing anything productive.


Thank you all for your help.  :)
Lee

mr-highball

  • Full Member
  • ***
  • Posts: 233
    • Highball Github
Re: Multi-threaded web server example - performance tuning?
« Reply #10 on: March 27, 2019, 12:19:07 am »
Hmm, well that's unfortunate. Better luck next time I suppose? fpc takes some getting used to, but once you do it's a wonderful tool.

If you have room for just a little bit more frustration...
--
Out of curiosity I looked back at your original benchmarks and was wondering what you would expect to be "good results"? Your longest request was 241ms, amd unless I'm missing something, that doesn't seem too bad when thinking back to response times I've seen in .net or java.
For example do you have comparison metrics to other web frameworks/languages for similar test runs? If you do I could try and put together a test with fundamentals or another implementation to see how well they stack up.

asdf121

  • New Member
  • *
  • Posts: 35
Re: Multi-threaded web server example - performance tuning?
« Reply #11 on: March 27, 2019, 01:03:23 pm »
Just download Indy from https://indy.fulgan.com/ and add its units to your search path.
When compiling via command line add the needed subdir via
Code: Pascal  [Select][+][-]
  1. -FuIndy10dir/*
Documentation can be found online:
http://ww2.indyproject.org/docsite/html/frames.html?frmname=topic&frmfile=index.html or on Indy's website for download.
I think it's the TIdHTTPServer but you'll find a lot of examples for Indy via Google ;)

fundamentals5 code looks indeed very good and clean but there is absolutely no documentation or examples  >:D %) So might be a good idea to not use it or at least it's nothing you can use without spending much time and having good coding skills... :'(

Akira1364

  • Hero Member
  • *****
  • Posts: 561
Re: Multi-threaded web server example - performance tuning?
« Reply #12 on: March 27, 2019, 09:48:30 pm »
Apart from not using a thread pool, FCL-Web kind of suffers from way-too-much-try-finally syndrome internally.

I'm not aware of any libraries for Object Pascal other than the new version of Brook Framework:

https://github.com/risoflora/brookframework

that would (or physically could) get close to the kind of performance you seem to want.
« Last Edit: March 28, 2019, 01:25:48 am by Akira1364 »

botster

  • New Member
  • *
  • Posts: 39
Re: Multi-threaded web server example - performance tuning?
« Reply #13 on: March 27, 2019, 11:32:58 pm »
If you have room for just a little bit more frustration...

Lol. Okay, I'll bite.  :)

Quote from: mr-highball
Out of curiosity I looked back at your original benchmarks and was wondering what you would expect to be "good results"?

That is a discerning question since the meaning of "good results" is bound to be subjective without defining some criteria.

To me, that term means showing performance, productivity (RAD), or both. In regards to productivity, I was thinking primarily of doing more with less (ie. fewer lines of code); to a lesser extent, ease of library/extension management; and, minorly, a general feeling of compilation time.

Here's a little of my backstory for context. I've recently become interested in ActivityPub (World Wide Web Consortium protocol for decentralized social networking). While looking at the languages used for various implementations, I saw that most were written in scripting, or some form of JIT compiled scripting, languages. Only a few are written in truly compiled languages, including Golang and Rust.

I thought that creating an open-source project for ActivityPub in Free Pascal might be a great way to showcase FP, get some eyes on it, and perhaps garner some interest. The question I needed to answer first, though, is would FP produce comparatively "good (enough) results" to counter the undoubtedly forthcoming question, "Why should Free Pascal be used when we can have A and/or B with X language?"

So, I did a 'net search for, "simple multi-threaded http server example in X" where X = (Free Pascal | Golang | Rust). Simple examples were easily found on page one of the results for Golang and Rust. A simple FP example (from Rosetta Code) wasn't found until page two; which, I suppose, is understandable since the know-it-alls say that Pascal is a "dead" language.

My tests show Golang to be almost three times faster than FP. And, the example I found on Stack Overflow, which I adapted by removing just two lines related to timing, is only fifteen lines of code including whitespace.

It took just a bit more to adapt the multi-threaded (MT) server example in the Rust community's, "The Book". I ended up with 27 lines and performance, surprisingly, slightly worse than that of Golang.

Neither Golang nor Rust required me to exercise package installation contortions to implement a simple MT HTTP server. To be fair, neither did FP when using 'fphttpserver', but it seems that is not an acceptable solution.

Using the Apache HTTP server benchmarking tool to send a total of 500 concurrent requests, FP gives consistent total times in the low 200's of milliseconds. Golang is consistent at around 70-80ms, and Rust, 80-100ms.

So, it appeared to me that, comparatively speaking, both Golang and Rust have both A and B; Free Pascal, not so much. Hence, my disappointment.

I'd be glad to post the relative code and benchmark results if you're interested.


And, just BTW...
Quote from: mr-highball
fpc takes some getting used to, but once you do it's a wonderful tool.

I don't doubt that at all which is why I keep taking yet another hopeful look. It's not getting used to the tool itself with which I have so much difficulty. It's trying to navigate its ecosystem.
Lee

Akira1364

  • Hero Member
  • *****
  • Posts: 561
Re: Multi-threaded web server example - performance tuning?
« Reply #14 on: March 28, 2019, 12:07:41 am »
Using the Apache HTTP server benchmarking tool to send a total of 500 concurrent requests, FP gives consistent total times in the low 200's of milliseconds. Golang is consistent at around 70-80ms, and Rust, 80-100ms.

Those don't actually seem as far off as it sounded like from what you were saying before. It does however seem like you must be running pretty high-end hardware, now that I think about the results relatively.

How does this version of the program perform for you? (Note how I've increased the QueueSize parameter of the server... the default is 5. You can increase it further from there and see how that works, also.)

Code: Pascal  [Select][+][-]
  1. program HelloWorldServer;
  2.  
  3. {$mode ObjFPC}{$H+}
  4. {$ImplicitExceptions Off}
  5.  
  6. uses
  7.   CMem,
  8.   {$ifdef Unix}
  9.   CThreads,
  10.   {$endif}
  11.   FPHTTPServer;
  12.  
  13. type
  14.   TTestHTTPServer = class(TFPHTTPServer)
  15.   public
  16.     procedure HandleRequest(var ARequest: TFPHTTPConnectionRequest;
  17.                             var AResponse: TFPHTTPConnectionResponse); override;
  18.   end;
  19.  
  20.   procedure TTestHTTPServer.HandleRequest(var ARequest: TFPHTTPConnectionRequest;
  21.                                           var AResponse: TFPHTTPConnectionResponse);
  22.   begin
  23.     with AResponse do begin
  24.       Content := 'Hello,World!';
  25.       Code := 200;
  26.       SendContent();
  27.     end;
  28.   end;
  29.  
  30. begin
  31.   with TTestHTTPServer.Create(nil) do try
  32.     Threaded := True;
  33.     Port := 8080;
  34.     QueueSize := 25;
  35.     AcceptIdleTimeout := 1000;
  36.     Active := True;
  37.   finally
  38.     Active := False;
  39.     Free();
  40.   end;
  41. end.

Regarding your Indy "package" download/install attempt from before, do you actually have the Lazarus IDE installed? It might be easier in general for you to just get the source off github, too:

https://github.com/graemeg/indy.git
« Last Edit: March 28, 2019, 01:21:14 am by Akira1364 »

 

TinyPortal © 2005-2018