TFPHTTPServer, client disconnect, "Missing HTTP protocol version in request" err

[torumyax]

I'm trying to implement an embedded HTTP server using TFPHTTPServer from fcl-web library.

First, I compiled the example project which comes from fpc.

simplehttpserver.pas

C:\lazarus\fpc\3.0.4\source\packages\fcl-web\examples\httpserver

https://github.com/alrieckert/freepascal/blob/master/packages/fcl-web/examples/httpserver/simplehttpserver.pas

For testing, I accessed to the server using normal browser. After a short period of time, I get a lots of

[Debugger Exception Notification]

Project *** raised exception class 'EHTTPServer' with message:
Missing HTTP protocol version in request

At first, I had no idea why. I thought this was browser's problem and I was trying to figure out why the browser is not sending HTTP protocol version in the request.
But it turns out ... it's just client browsers disconnected.

http://lists.freepascal.org/pipermail/fpc-pascal/2014-April/041720.html


So, looks like it's NOT about the HTTP protocol version.

unit fphttpserver; 
resourcestring
  SErrMissingProtocol = 'Missing HTTP protocol version in request';

maybe something like

SErrDisconnectedByClient = 'Disconnected by client';


And I looked for "IgnoreLostConnections" property but in vain. Try to "try .. catch" without directly editing the TFPHTTPServer ... failed.


I think TFPHTTPConnection needs OnDisconnectByClient or something.. But am not really familiar with socket networking... I mean I just started lazarus a few month ago...

What should I do?


 
Also, related, the example simplehttpserver.pas gives me

simplehttpserver.pas(10,49) Fatal: Cannot find wmecho used by simplehttpserver of the Project Inspector.

Removing the wmecho from using fixed problem.



I'm using
Lazarus 1.8.2 r57369 FPC 3.0.4 x86_64-win64-win32/win64

[torumyax]

OK, I think I manage to add "OnDisconnectedByClient" to TFPHttpServer class.

I just Copied fphttpserver.pp from
C:\lazarus\fpc\3.0.4\source\packages\fcl-web\src\base
and rename it to fphttpserver2.pas and included to a project and made some changes.

It seems to be working.. at least  'Missing HTTP protocol version in request' exception won't show up every time browser disconnect itself with timeout.

Also I made some change to 'Error reading data from the socket' exception, so that OnError will be called instead of exception. This will be normally called when you close browser right after you send request to the server.

I attached the source. I haven't tested this much because I haven't even started my project yet.

DISCLAIMER: I haven't tested on other platforms, and I don't really know what I am doing.
So please let me know if I'm doing it wrong.

[Thaddy]

I have absolutely no clue (well, I have...) why you did not use inheritance.
There's no need to copy the unit. Simply include it and derive from the class you want to extend.

[torumyax]

Code: Pascal  [Select][+][-]

1.  why you did not use inheritance.

Well, (1) because I 'm not trying to extend it. (Sorry my English is bad.) I'm trying to fix it.
(2) because this is a test to see what caused the problem and if the fix works or not and it was easier for me to just copy in this case just like git clone.
(3) I'm hoping if this fix is indeed correct, some one who are much more experienced than I am could implement it.


The problem is when

Code: Pascal  [Select][+][-]

1. r:=FSocket.Read(FBuffer[1],ReadBufLen);

returns 0, (I believe this is a Windows specific behavior.)

Code: Pascal  [Select][+][-]

1.   If (Pos('HTTP/',S)<>1) then
2.     Raise EHTTPServer.CreateHelp(SErrMissingProtocol,400);

this is called which is wrong and it gives us a wrong error message.

So, what I did was adding (and some more..)

Code: Pascal  [Select][+][-]

1.     if ((r=0) and (FSocket.LastError = 0)) then
2.       HandleDisconnectedByClient();

Code: Pascal  [Select][+][-]

1. { TFPHTTPConnection }
2.  
3. function TFPHTTPConnection.ReadString : String;
4.  
5.   Procedure FillBuffer;
6.  
7.   Var
8.     R : Integer;
9.  
10.   begin
11.     SetLength(FBuffer,ReadBufLen);
12.     r:=FSocket.Read(FBuffer[1],ReadBufLen);
13.  
14.     If r<0 then
15.       // -Deleted
16.       //Raise EHTTPServer.Create(SErrReadingSocket);
17.       //
18.  
19.       // +Added
20.       HandleRequestError(EHTTPServer.Create(SErrReadingSocket));
21.       //
22.  
23.     if (r<ReadBuflen) then
24.       SetLength(FBuffer,r);
25.  
26.     // +Added
27.     if ((r=0) and (FSocket.LastError = 0)) then
28.       HandleDisconnectedByClient();
29.     //
30.   end;
31.  

I have no idea what side effects it might have. And I'm sure there are some other cases when this condition ((r=0) and (FSocket.LastError = 0)) might be met other than "client disconnect"..

But I vaguely remember when I use TCPClient in .net, TCPClient socket/Networkstream never returned 0 (callback never called if there is no data to read) as long as it's connected.

I could simply ignore the client disconnect, but

Code: Pascal  [Select][+][-]

1. it IS an error condition (client went away unexpectedly) and you should be aware of it.

So, please some one ...


Please see this thread.

http://lists.freepascal.org/pipermail/fpc-pascal/2014-April/041715.html

http://lists.freepascal.org/pipermail/fpc-pascal/2014-April/041719.html

[torumyax]

I've found a similar code in c++.

Code: C  [Select][+][-]

1.     // Receive until the peer closes the connection
2.     do {
3.  
4.         iResult = recv(ConnectSocket, recvbuf, recvbuflen, 0);
5.         if ( iResult > 0 )
6.             printf("Bytes received: %d\n", iResult);
7.         else if ( iResult == 0 )
8.             printf("Connection closed\n");
9.         else
10.             printf("recv failed: %d\n", WSAGetLastError());
11.  
12.     } while( iResult > 0 );
13.  

https://msdn.microsoft.com/en-us/library/windows/desktop/ms740121%28v=vs.85%29.aspx

and this

If the remote host shuts down the Socket connection with the Shutdown method, and all available data has been received, the EndReceive method will complete immediately and return zero bytes.

https://msdn.microsoft.com/en-us/library/ms145145(v=vs.110).aspx


So, below would be fine?

Code: Pascal  [Select][+][-]

1.  
2.     SetLength(FBuffer,ReadBufLen);
3.     r:=FSocket.Read(FBuffer[1],ReadBufLen);
4.  
5.     If r<0 then
6.       HandleRequestError(EHTTPServer.Create(SErrReadingSocket));
7.     if (r=0) then
8.       HandleDisconnectedByClient();
9.     if (r<ReadBuflen) then
10.       SetLength(FBuffer,r);
11.  

[torumyax]

Not as elegant as original code.. but here is what I've done. (update. I attached my test project file)

It's very difficult to debug. Some times "OnDisconnectedByClient" called twice, some times never called. It depends on the client browser. or I'm missing something.

Code: Pascal  [Select][+][-]

1.   Procedure FillBuffer;
2.   Var
3.     R : Integer;
4.   begin
5.     SetLength(FBuffer,ReadBufLen);
6.     r:=FSocket.Read(FBuffer[1],ReadBufLen);
7.  
8.     If r<0 then
9.       HandleRequestError(EHTTPServer.Create(SErrReadingSocket));
10.     //+Added
11.     ////////////////////////////////////////////////////////
12.     if (r=0) then
13.       HandleDisconnectedByClient();
14.     ////////////////////////////////////////////////////////
15.     if (r<ReadBuflen) then
16.       SetLength(FBuffer,r);
17.   end;

Code: Pascal  [Select][+][-]

1. // +Added
2. procedure TFPHTTPConnection.HandleDisconnectedByClient();
3. begin
4.  
5.   If Assigned(FOnDisconnected) then
6.     try
7.       FOnDisconnected(Self);
8.     except
9.       // ?
10.     end;
11. end;

Code: Pascal  [Select][+][-]

1.   { TFPHTTPConnection }
2.  
3.   TFPHTTPConnection = Class(TObject)
4.   private
5.     FOnError: TRequestErrorHandler;
6.     //+Added
7.     ////////////////////////////////////////////////////////
8.     FOnDisconnected: TDisconnectedByClientHandler;
9.     ////////////////////////////////////////////////////////
10.  

Code: Pascal  [Select][+][-]

1. function TFPHTTPConnection.ReadRequestHeaders: TFPHTTPConnectionRequest;
2.  
3. Var
4.   StartLine,S : String;
5. begin
6.   // +Added
7.   ////////////////////////////////////////////////////////////////////////////
8.   StartLine:=ReadString;
9.   if (Trim(StartLine) = '') then
10.     begin
11.     Result:=nil;
12.     exit;
13.     end;
14.   ////////////////////////////////////////////////////////////////////////////
15.  
16.   Result:=Server.CreateRequest;
17.   try
18.     Server.InitRequest(Result);
19.     Result.FConnection:=Self;
20.  
21.     // -Deleted.
22.     ////////////////////////////////////////////////////////////////////////////
23.     //StartLine:=ReadString;
24.     ////////////////////////////////////////////////////////////////////////////
25.     ParseStartLine(Result,StartLine);
26.     Repeat
27.       S:=ReadString;
28.       if (S<>'') then
29.         InterPretHeader(Result,S);
30.     Until (S='');
31.     Result.RemoteAddress := SocketAddrToString(FSocket.RemoteAddress);
32.     Result.ServerPort := FServer.Port;
33.   except
34.     FreeAndNil(Result);
35.     Raise;
36.   end;
37. end;

Code: Pascal  [Select][+][-]

1. procedure TFPHTTPConnection.HandleRequest;
2.  
3. Var
4.   Req : TFPHTTPConnectionRequest;
5.   Resp : TFPHTTPConnectionResponse;
6.  
7. begin
8.   Try
9.     SetupSocket; // < this is for **ux. don't know
10.     // Read headers.
11.     Req:=ReadRequestHeaders;
12.     ////////////////////////////////////////////////////////////////////////////
13.     // +Added
14.     if not Assigned(Req) then
15.       exit;
16.     ////////////////////////////////////////////////////////////////////////////

Code: Pascal  [Select][+][-]

1. unit fphttpserver
2. Type
3.   TFPHTTPConnection = Class;
4.   // +Added
5.   ////////////////////////////////////////////////////////////////////////////
6.   TDisconnectedByClientHandler = Procedure (Sender : TObject) of object;
7.  ////////////////////////////////////////////////////////////////////////////

Code: Pascal  [Select][+][-]

1.   TFPHTTPConnection = Class(TObject)
2.  
3.   private
4.   // +Added
5.   ////////////////////////////////////////////////////////////////////////////
6.     FOnDisconnected: TDisconnectedByClientHandler;
7.   ////////////////////////////////////////////////////////////////////////////
8.   Protected
9.   // +Added
10.   ////////////////////////////////////////////////////////////////////////////
11.     procedure HandleDisconnectedByClient(); virtual;  
12.   ////////////////////////////////////////////////////////////////////////////
13.   Public
14.   // +Added
15.   ////////////////////////////////////////////////////////////////////////////
16.     Property OnDisconnectedByClient : TDisconnectedByClientHandler Read FOnDisconnected Write FOnDisconnected;  
17.   ////////////////////////////////////////////////////////////////////////////
18.  

Code: Pascal  [Select][+][-]

1. procedure TFPCustomHttpServer.DoConnect(Sender: TObject; Data: TSocketStream);
2.  
3. Var
4.   Con : TFPHTTPConnection;
5.  
6. begin
7.   Con:=CreateConnection(Data);
8.   try
9.     Con.FServer:=Self;
10.     Con.OnRequestError:=@HandleRequestError;
11.     ////////////////////////////////////////////////////////////////////////////
12.     // +Added
13.     Con.OnDisconnectedByClient:=@HandleDisconnectedByClient;
14.     ////////////////////////////////////////////////////////////////////////////
15.     if Threaded then
16.       CreateConnectionThread(Con)
17.     else
18.       begin
19.       Con.HandleRequest;
20.       end;
21.   finally
22.     if not Threaded then
23.       Con.Free;
24.   end;
25. end;