Lazarus

Announcements => Third party => Topic started by: iLya2IK on February 26, 2021, 05:26:41 pm

Title: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on February 26, 2021, 05:26:41 pm
The fpweb library is certainly good, but it clearly lacks modern features - namely, full-fledged http/1.1 with persistent connections, and also http/2. I completed my search for the desired analog by creating a server with all the necessary. I would like to present to your attention my project - WCHTTPServer.

The project builds on the fcl-web library and extends it to increase functionality:

Client management using cookies (saving and maintaining sessions).

Link to project on GitHub : https://github.com/iLya2IK/wchttpserver (https://github.com/iLya2IK/wchttpserver)

At the moment, the server follows the RFC 7540 specification fairly closely, but I continue to test and debug to meet a number of non-obvious protocol requirements. For testing, I use the h2spec and h2load utilities. 

Code: XML  [Select][+][-]
  1. # h2spec -p 8080 http2 -t -k -o 20
  2. 94 tests, 91 passed, 0 skipped, 3 failed
  3. Hypertext Transfer Protocol Version 2 (HTTP/2)
  4.   4. HTTP Frames
  5.     4.2. Frame Size      
  6.       × 1: Sends a DATA frame with 2^14 octets in length
  7.         -> The endpoint MUST be capable of receiving and minimally processing frames up to 2^14 octets in length.
  8.            Expected: HEADERS Frame (stream_id:1)
  9.              Actual: Connection closed
  10.             Comment: Error in h2spec - DATA payload size didn't equal the value of "content-length" header.  
  11.  
  12.   5. Streams and Multiplexing
  13.     5.5. Extending HTTP/2      
  14.       × 2: Sends an unknown extension frame in the middle of a header block
  15.         -> The endpoint MUST treat as a connection error of type PROTOCOL_ERROR.
  16.            Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR)
  17.                      Connection closed
  18.              Actual: Timeout
  19.             Comment: I didn't find any references in RFC 7540 that unknown extension frame in the middle
  20.                      of a header block should cause a connection error
  21.  
  22.   8. HTTP Message Exchanges
  23.     8.1. HTTP Request/Response Exchange
  24.       8.1.2. HTTP Header Fields
  25.         8.1.2.2. Connection-Specific Header Fields          
  26.           × 1: Sends a HEADERS frame that contains the connection-specific header field
  27.             -> The endpoint MUST respond with a stream error of type PROTOCOL_ERROR.
  28.                Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR)
  29.                          RST_STREAM Frame (Error Code: PROTOCOL_ERROR)
  30.                          Connection closed
  31.                  Actual: DATA Frame (length:845, flags:0x01, stream_id:1)
  32.                 Comment: I know that this violates the requirements of RFC 7540, but I sincerely consider them
  33.                          redundant and the type of reaction to the header value should remain on the server side      

And here are the results of testing with h2load
Code: XML  [Select][+][-]
  1. # h2load -n32000 -c32 -m10 --header=connection:keep-alive --header=cookie:cid=11  https://localhost:8080        
  2. starting benchmark...
  3. spawning thread #0: 32 total client(s). 32000 total requests
  4. TLS Protocol: TLSv1.2
  5. Cipher: ECDHE-RSA-AES256-GCM-SHA384
  6. Server Temp Key: X25519 253 bits
  7. Application protocol: h2
  8. progress: 10% done
  9. progress: 20% done
  10. progress: 30% done
  11. progress: 40% done
  12. progress: 50% done
  13. progress: 60% done
  14. progress: 70% done
  15. progress: 80% done
  16. progress: 90% done
  17. progress: 100% done
  18.  
  19. finished in 17.22s, 1858.68 req/s, 1.54MB/s
  20. requests: 32000 total, 32000 started, 32000 done, 32000 succeeded, 0 failed, 0 errored, 0 timeout
  21. status codes: 32000 2xx, 0 3xx, 0 4xx, 0 5xx
  22. traffic: 26.46MB (27746560) total, 126.38KB (129408) headers (space savings 95.01%), 25.79MB (27040000) data
  23.                      min         max         mean         sd        +/- sd
  24. time for request:    16.59ms    327.96ms    150.22ms     35.42ms    77.80%
  25. time for connect:     7.17ms       1.96s       1.15s    618.49ms    62.50%
  26. time to 1st byte:    59.55ms       2.00s       1.20s    629.72ms    62.50%
  27. req/s           :      58.09       72.88       61.87        4.42    81.25%
  28.  
  29. # h2load -n32000 -c32 -t8 -m1 --h1 --header=connection:keep-alive --header=cookie:cid=11  https://localhost:8080
  30. -t: warning: the number of threads is greater than hardware cores.
  31. starting benchmark...
  32. spawning thread #0: 4 total client(s). 4000 total requests
  33. spawning thread #1: 4 total client(s). 4000 total requests
  34. spawning thread #2: 4 total client(s). 4000 total requests
  35. spawning thread #3: 4 total client(s). 4000 total requests
  36. spawning thread #4: 4 total client(s). 4000 total requests
  37. spawning thread #5: 4 total client(s). 4000 total requests
  38. spawning thread #6: 4 total client(s). 4000 total requests
  39. spawning thread #7: 4 total client(s). 4000 total requests
  40. TLS Protocol: TLSv1.2
  41. Cipher: ECDHE-RSA-AES256-GCM-SHA384
  42. Server Temp Key: X25519 253 bits
  43. Application protocol: http/1.1
  44. progress: 10% done
  45. progress: 20% done
  46. progress: 30% done
  47. progress: 40% done
  48. progress: 50% done
  49. progress: 60% done
  50. progress: 70% done
  51. progress: 80% done
  52. progress: 90% done
  53. progress: 100% done
  54.  
  55. finished in 29.06s, 1101.26 req/s, 1.01MB/s
  56. requests: 32000 total, 32000 started, 32000 done, 32000 succeeded, 0 failed, 0 errored, 0 timeout
  57. status codes: 32000 2xx, 0 3xx, 0 4xx, 0 5xx
  58. traffic: 29.39MB (30816000) total, 2.53MB (2656000) headers (space savings 0.00%), 25.79MB (27040000) data
  59.                      min         max         mean         sd        +/- sd
  60. time for request:     7.99ms    208.16ms     26.31ms      5.94ms    93.71%
  61. time for connect:     8.22ms       3.08s       1.25s    943.18ms    62.50%
  62. time to 1st byte:    99.48ms       3.12s       1.31s    913.25ms    62.50%
  63. req/s           :      34.42       38.74       36.31        1.25    62.50%
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on May 11, 2021, 05:18:26 pm
The latest update adds support for the websocket protocol, and therefore the demo site has been changed. As an example of interaction over the websocket protocol, the JSON-RPC application level protocol (well, or some of its similarity  ::) - you can change it to your own) is used. Now I am working on adding the permessage-deflate extension (RFC 7692)
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: Leledumbo on May 14, 2021, 05:23:49 am
While I admire this work, I find it hard to not have a clear separation between which files belong to the library/package, demo/examples and so on. See fcl-web's own directory structure as an example.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on May 14, 2021, 11:21:50 am
While I admire this work, I find it hard to not have a clear separation between which files belong to the library/package, demo/examples and so on. See fcl-web's own directory structure as an example.

Yes, I get it - this is really a problem. I will start reworking the repository in the near future.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on May 15, 2021, 03:37:52 pm
The structure of the project has been put in order. Added lpk file. The "permessage-deflate" extension for websocket has been fully implemented.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: Leledumbo on May 18, 2021, 03:07:28 am
The structure of the project has been put in order. Added lpk file. The "permessage-deflate" extension for websocket has been fully implemented.
Perfect. Now let's see if I can make a hello world demo, surely it's much simpler than the current full fledged demo with database and all.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on May 18, 2021, 09:27:17 am
This is great.If you have any questions, I'm happy to answer them
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: nomorelogic on May 22, 2021, 12:45:05 pm
really a great work!
I’m studying the source to use your wchttpserver as a RestJsonServer and I have some questions.

Question 1: QueryFields

Added in InitializeJobsTree a new endpoint

Code: Pascal  [Select][+][-]
  1. procedure InitializeJobsTree;
  2. begin
  3.   ...
  4.   WCJobsTree.Values['/items.json'] := TWCMyItem;
  5.  
  6. end;
  7.  

Added in unit WCServerTestJobs
Code: Pascal  [Select][+][-]
  1.   { TWCMyItem }
  2.  
  3.   TWCMyItem = class(TWCMainClientJob)
  4.   public
  5.     procedure Execute; override;
  6.   end;
  7.  
  8. ...
  9.  
  10. procedure TWCMyItem.Execute;
  11. var s1, s2: string;
  12. begin
  13.     s1:=Request.QueryFields.Values['id'];
  14.     s2:=Request.QueryFields.Values['name'];  // Edit: here I get an empty string
  15.     Response.Content:=TWCTestWebClient(Client).AnalizeRequest(s1, s2);
  16.  
  17.     inherited Execute;
  18. end;
  19.  

and

Code: Pascal  [Select][+][-]
  1. function TWCTestWebClient.AnalizeRequest(const AId, AName: string): string;
  2. var jo: TJSONObject;
  3. begin
  4.   jo:=TJSONObject.Create;
  5.   try
  6.     jo.Add('Id', AId);
  7.     jo.Add('Name', AName);
  8.     result := jo.AsJSON;
  9.   finally
  10.     FreeAndNil(jo);
  11.   end;
  12. end;
  13.  

it seems that only the first of the queryfields is available: characters following "&" are ignored
using curl as test:
Code: Pascal  [Select][+][-]
  1. $ curl --cacert localhost.crt --request POST https://localhost:4444/items.json?id=1&name=hello
  2. { "Id" : "1", "Name" : "" }
  3.  


Question 2: Headers
How I can read the headers of a request?


Question 3: Post data
How I can read the "post data" of a request?
(I refer to -d parameter of curl command)
Code: Pascal  [Select][+][-]
  1. $ curl --cacert localhost.crt --request POST https://localhost:4444/items.json?id=1&name=hello -d "{ \"ItemName\":\"AItemName\", \"ItemDescriprion\": \"AItemDescription\" }"
  2.  

Thanks you again for your work!

nomorelogic

Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on May 22, 2021, 06:28:51 pm
really a great work!

Thank you, I appreciate it.

https://localhost:4444/items.json?id=1&name=hello -d "{ \"ItemName\":\"AItemName\", \"ItemDescriprion\": \"AItemDescription\" }"

The code you wrote is absolutely correct. I tried to write it by my own - and it works correctly from a browser - both fields are detected. Unfortunately, I was unable to reproduce the error. Please pay attention to the correctness of the command line. I don't often use curl, but maybe this post will help you https://stackoverflow.com/questions/10060093/special-characters-like-and-in-curl-post-data

For information
The project relies heavily on the class structure and fpWeb functionality. In particular, the query string analysis is performed by the fpWeb tools. The QueryFields are filled in the TRequest.ProcessQueryString (fpWeb) method, where the QueryString (this is all to the right of '?') is passed as parameter. This code is in the HTTPDefs module, but it is not part of the wcHTTPServer.

If you want to detect complex queries
All your content objects are in the Request.Content string. Example:
Code: Text  [Select][+][-]
  1. curl -d '{"ItemName":"AItemName", "ItemDescriprion": "AItemDescription"}' -H "Content-Type: application/json" -X POST "http://localhost:4444/items.json?id=1&name=hello"
Then parse content in Job
Code: Pascal  [Select][+][-]
  1. procedure TWCMyItem.Execute;
  2. var jsonObj : TJSONObject;
  3. s1, s2 : String;
  4. begin
  5.   try
  6.     s1:=Request.QueryFields.Values['id'];
  7.     s2:=Request.QueryFields.Values['name'];
  8.     jsonObj:= TJSONObject(GetJSON(Request.Content));
  9.     if assigned(jsonObj) then
  10.     begin
  11.       TWCTestWebClient(Client).DoSomething(s1, s2, jsonObj.Get('ItemName', ''),
  12.                                                jsonObj.Get('ItemDescriprion', ''));
  13.       Response.Content := cSStatusOKJSON;
  14.     end else
  15.       Response.Content := cSStatusBADJSON;
  16.   finally
  17.     if assigned(jsonObj) then FreeAndNil(jsonObj);
  18.   end;
  19.   inherited Execute;
  20. end;      
  21.  

The better way for POST requests is to push all parameters to the content string (for my opinion):
Code: Text  [Select][+][-]
  1. curl -d '{"id":1, "name": "hello", "ItemName":"AItemName", "ItemDescription": "AItemDescription"}' -H "Content-Type: application/json" -X POST http://localhost:4444/items.json

With included wcUtils module you can do this simple way
Code: Pascal  [Select][+][-]
  1. procedure TWCMyItem.Execute;
  2. var Pars : Array [0..3] of Variant;
  3. begin
  4.   // ESWGetHeaderContent replaced by DecodeJsonParams in the latest version
  5.   if ESWGetHeaderContent(Request.Content, ['id', 'name', 'ItemName', 'ItemDescription'], @(Pars[0]), [0,'','','']) then
  6.   begin
  7.     TWCTestWebClient(Client).DoSomething(Pars[0], Pars[1], Pars[2], Pars[3]);
  8.     Response.Content := cSStatusOKJSON;
  9.   end else
  10.     Response.Content := cSStatusBADJSON;
  11.   inherited Execute;
  12. end;      
  13.  

After latest updates you can use the Params/ParPtr/Param properties of the TWCRequest class:
Code: Pascal  [Select][+][-]
  1. procedure TWCMyItem.Execute;
  2. begin
  3.   if DecodeJsonParams(Request.Content, ['id', 'name', 'ItemName', 'ItemDescription'], Params, [0,'','','']) then
  4.   begin
  5.     TWCTestWebClient(Client).DoSomething(Param[0], Param[1], Param[2], Param[3]);
  6.     Response.Content := cSStatusOKJSON;
  7.   end else
  8.     Response.Content := cSStatusBADJSON;
  9.   inherited Execute;
  10. end;      
  11.  
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: nomorelogic on May 24, 2021, 09:14:17 am
Please pay attention to the correctness of the command line. I don't often use curl, but maybe this post will help you https://stackoverflow.com/questions/10060093/special-characters-like-and-in-curl-post-data

Hi
as suggested in link you reported, using double quote for the entire url, it works

thanks for all the explanations
nomorelogic
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: nomorelogic on June 28, 2021, 12:09:34 pm
Hi I'm here again to ask for support.

I need help in the build-response process and a memory lack I get.
I refer to previous posted code like this:
Code: Pascal  [Select][+][-]
  1.   { TWCMyItem }
  2.  
  3.   TWCMyItem = class(TWCMainClientJob)
  4.   public
  5.     procedure Execute; override;
  6.   end;
  7.  
  8. ...
  9.  
  10. procedure TWCMyItem.Execute;
  11. var s1, s2: string;
  12. begin
  13.     s1:=Request.QueryFields.Values['id'];
  14.     s2:=Request.QueryFields.Values['name'];
  15.     Response.Content:=TWCTestWebClient(Client).AnalizeRequest(s1, s2);
  16.  
  17.     inherited Execute;
  18. end;
  19.  

In TWCMyItem.Execute procedure I create a client class to solve the request.
The client create seems to be linked to a session so (I think) this client will be released when session ends.

In my case, cause I'm writing a json rest server, I don't want to use sessions, so I need to free clients after the response is ready.

So I tried following code but this doesn't avoid the memory lack.
Code: Pascal  [Select][+][-]
  1. procedure TWCMyItem.Execute;
  2. var s1, s2: string;
  3. begin
  4.     s1:=Request.QueryFields.Values['id'];
  5.     s2:=Request.QueryFields.Values['name'];
  6.     Response.Content:=TWCTestWebClient(Client).AnalizeRequest(s1, s2);
  7.     WebContainer.RemoveClient(Client);
  8.  
  9.     inherited Execute;
  10. end;
  11.  

What is the best way to destroy the client to avoid this memory lack?

Another question: is there a gracefully way to stop server using endpoint?

Thanks in advance
nomorelogic


Edit:
removed not necessary comment in posted code
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on June 29, 2021, 08:19:19 am
Hi I'm here again to ask for support.
It is wonderful. Each of your questions develops a project. I created it for my narrow needs, but I would like to make it more versatile

The client create seems to be linked to a session so (I think) this client will be released when session ends.
Yes It is. The session created every time you connect unknown client with empty or unrecognized "cid" cookie

In my case, cause I'm writing a json rest server, I don't want to use sessions, so I need to free clients after the response is ready.
...
What is the best way to destroy the client to avoid this memory lack?
This server works according to the principles of targeting a web client and has a built-in mechanism for creating and maintaining sessions. But you can work around it simply by defining an inheritor for the TWCPreAnalizeClientJob class.

Code: Pascal  [Select][+][-]
  1. procedure TWCPACNoSessionJob.Execute;
  2. var ASynThread : TWCMainClientJob;
  3.     aClient : TWebClient;
  4.     aConsumeResult : TWCConsumeResult;
  5. begin
  6.   try
  7.      if not (Assigned(FConn) and TWCHttpServer(FConn.Server).ServerActive) then
  8.        Exit;
  9.      aConsumeResult := FConn.ConsumeSocketData;
  10.      if aConsumeResult = wccrOK then begin
  11.        aClient := WebContainer.AddClient(Request, 'empty');
  12.        if not assigned(aClient) then begin
  13.          Application.SendError(Response, 405);
  14.          Exit;
  15.        end else begin
  16.          aClient.Initialize;
  17.        end;      
  18.        FConn.SetSessionParams(aClient, nil);
  19.        //
  20.        if assigned(aClient) then begin
  21.          ASynThread := GenerateClientJob;
  22.          if Assigned(ASynThread) then
  23.          begin
  24.            FConn := nil; //now fconn is part of ASynThread job
  25.            Application.ESServer.AddToMainPool(ASynThread);
  26.          end;
  27.        end;
  28.      end;
  29.   except
  30.     on E: Exception do ; // catch errors. jail them in thread
  31.   end;
  32. end;
And then set the ServerAnalizeJobClass property while initializing the application.
Code: Pascal  [Select][+][-]
  1. Application.ESServer.ServerAnalizeJobClass := TWCPACNoSessionJob;
I have no way to test it yet, so please use this code with caution for now. In the next update, I will include this code in the release.
Another question: is there a gracefully way to stop server using endpoint?
Yes, sure. You can use the server.cfg file and the "Maintaining"."Shutdown" property.
You can set the property to true value to stop the server while execution.
Code: Text  [Select][+][-]
  1. "Maintaining": {
  2.         "Shutdown":true
  3. }
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: nomorelogic on June 29, 2021, 10:28:38 am
I have no way to test it yet, so please use this code with caution for now. In the next update, I will include this code in the release.

really many thanks :)
These days I’ll do some tests like you suggested
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on June 29, 2021, 08:29:25 pm
really many thanks :)
These days I’ll do some tests like you suggested

I invite you to try the last commit. I have implemented all the suggestions written here earlier in the demo "examples/restjson".

The changes mainly affected two lines in file wcrestjsondemo.lpr.
Code: Pascal  [Select][+][-]
  1. WebContainer.Verbose := false; // reduces disk load as it stops writing
  2. //new client information to the database
  3. Application.WebClientClass:= WCTestClient.TWCTestWebClient; //this line was deleted.
  4. //The parent class TWCWebClient is used.
  5. //There is no need to redefine it as clients are now faceless.
  6. //All business logic can be moved to file wcrestjsonjobs.pas.  

And now the TWCPreThread class is the inheritor of TWCPreAnalizeClientNoSessionJob (the new class in the wcApplication unit)
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on June 30, 2021, 01:50:38 pm
REST JSON simple server

Short description
The server operates in the REST architecture mode. The server can accept POST requests and respond with JSON objects. The server simulates a simple online store capable of creating customers, products, and adding products to a customer cart.

Server testing
Code: Text  [Select][+][-]
  1. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/addClient.json?id=1&name=vasya"
  2.  
  3. {"result":"OK"}
  4.  
  5. > curl -k -d '{"shrtName":"item1", "fullName": "item num 1", "descr":"description for item1", "cost":500}' -H "Content-Type: application/json" -X POST "https://localhost:8080/addItem.json"
  6.  
  7. {"result":"OK"}
  8.  
  9. > curl -k -d '{"shrtName":"item2", "fullName": "item num 2", "descr":"description for item2", "cost":1500}' -H "Content-Type: application/json" -X POST "https://localhost:8080/addItem.json"
  10.  
  11. {"result":"OK"}
  12.  
  13. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/getItem.json?iId=1"
  14.  
  15. {"shrtName":"item1","fullName":"item num 1","descr":"description for item1","cost":500,"result":"OK"}
  16.  
  17. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/getClientByName.json?name=vasya"
  18.  
  19. {"cId":1,"result":"OK"}
  20.  
  21. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/addToBasket.json?cId=1&iId=2&cost=500"
  22.  
  23. {"result":"OK"}
  24.  
  25. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/addToBasket.json?cId=1&iId=1&cost=500"
  26.  
  27. {"result":"OK"}
  28.  
  29. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/addToBasket.json?cId=1&iId=1&cost=500"
  30.  
  31. {"result":"OK"}
  32.  
  33. > curl -k -H "Content-Type: application/json" -X POST "https://localhost:8080/getBasket.json?cId=1"
  34.  
  35. [{"iId":2,"cost":500},{"iId":1,"cost":500}]
  36.  
  37.  
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: senglit on January 13, 2022, 10:05:11 am
hi,

when I tried to install it I got a error:oglregexprwrapper.pas(17,3) Fatal: Cannot find regexpr_t used by OGLRegExprWrapper of package commonutils_ilya2ik.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: noszone on January 13, 2022, 11:25:15 am
Is your component fully supporting all major operaions? I mean is it safe to use it in live production project?
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on January 14, 2022, 01:11:00 pm
hi,

when I tried to install it I got a error:oglregexprwrapper.pas(17,3) Fatal: Cannot find regexpr_t used by OGLRegExprWrapper of package commonutils_ilya2ik.

Hello.
The problem is caused by the fact that the current version of Lazarus uses an outdated RegExpr module. If your version of Lazarus already has REVersionMajor >= 1 in the RegExpr module, then it is correct to change two lines in module OGLRegExprWrapper:

Code: Pascal  [Select][+][-]
  1. {.$define regexpr_new}
  2. {$define regexpr_t}

to

Code: Pascal  [Select][+][-]
  1. {$define regexpr_new}
  2. {.$define regexpr_t}

Otherwise, it is strongly recommended to add all files from the folder "src" from repository https://github.com/andgineer/TRegExpr to the project folder or to the commonutils folder, and then rename regexpr.pas to regexpr_t.pas

If you do not want to download a new version of TRegExpr, you can limit yourself to the following change in OGLRegExprWrapper module
Code: Pascal  [Select][+][-]
  1. {.$define regexpr_new}
  2. {.$define regexpr_t}
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on January 14, 2022, 01:45:39 pm
Is your component fully supporting all major operaions? I mean is it safe to use it in live production project?

This product is just a convenient tool for realizing your specific needs - banking, shopping, streaming, etc. This server by itself cannot guarantee the security of transactions within your or any other external software - this product was not developed for this purpose. The business logic of such a live production project must be programmed by you yourself by, for example, connecting to a database management system with data replication, etc. Whereas WCHTTPServer will provide a platform and interface for your business logic to interact with the web. This product was developed by me for myself - to calculate engineering and scientific problems on a remote server. The server copes with this task regularly. Of course, it does not provide such a volume of features and a set of tools as many (non)commercial products, but in return it provides deep integration with the free pascal/lazarus development environment, reasonable query processing speed and ease of deployment on server hardware.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: senglit on January 21, 2022, 03:36:54 am
It works! thank you!

hi,

when I tried to install it I got a error:oglregexprwrapper.pas(17,3) Fatal: Cannot find regexpr_t used by OGLRegExprWrapper of package commonutils_ilya2ik.

Hello.
The problem is caused by the fact that the current version of Lazarus uses an outdated RegExpr module. If your version of Lazarus already has REVersionMajor >= 1 in the RegExpr module, then it is correct to change two lines in module OGLRegExprWrapper:

Code: Pascal  [Select][+][-]
  1. {.$define regexpr_new}
  2. {$define regexpr_t}

to

Code: Pascal  [Select][+][-]
  1. {$define regexpr_new}
  2. {.$define regexpr_t}

Otherwise, it is strongly recommended to add all files from the folder "src" from repository https://github.com/andgineer/TRegExpr to the project folder or to the commonutils folder, and then rename regexpr.pas to regexpr_t.pas

If you do not want to download a new version of TRegExpr, you can limit yourself to the following change in OGLRegExprWrapper module
Code: Pascal  [Select][+][-]
  1. {.$define regexpr_new}
  2. {.$define regexpr_t}
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on February 08, 2022, 12:51:26 pm
Added a new example for WCHTTPServer.

REST Web Camera server (https://github.com/iLya2IK/wcwebcamserver)

The server operates in the REST architecture mode. The server can accept POST requests and respond with JSON objects. The server is designed to collect images from cameras (devices) and forwards messages between devices to control the periphery via an HTTP 2 connection. A detailed implementation of an external device based on "ESP32-CAM" is given in the example webcamdevice (https://github.com/iLya2IK/webcamdevice). The example of a desktop application for external device controlling and viewing images is webcamclientviewer (https://github.com/iLya2IK/webcamclientviewer).
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: nicknikos on June 16, 2022, 09:33:04 am
iLya2IK,

Congratulations for your excellent code!!
I' m really interested in using your contributed code in an internal application for serving various terminals with database data.
One question though..After i ran the tests (hpec,hload) the cpu load on Windows 10 pro 64 kept high (100%).The same hapens when connections arrive to the server (tested in all forms http,restjson,webcamserver). What should one do release the cpu utilization?

Thanks in advance!!
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on June 24, 2022, 07:01:53 am
Congratulations for your excellent code!!

Thanks for your interest in my project  :)

After i ran the tests (hpec,hload) the cpu load on Windows 10 pro 64 kept high (100%).The same hapens when connections arrive to the server (tested in all forms http,restjson,webcamserver). What should one do release the cpu utilization?

Yes. Thanks to you, I noticed this error in the operation of the server in a Windows environment. The fact is that I program mainly in a Linux environment, and I haven't tested a server under Windows for a long time. Now I have made the necessary adjustments to the code of the wcNetworking.pas module, which, in my opinion, significantly reduced the load on the cpu.

As for the wcNetworking.pas module - in the TThread.Execute method used the Sleep(0) call - in a Windows environment, this caused a serious increase in CPU load. Sleep(0) I changed to Sleep(1).
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: iLya2IK on December 12, 2022, 03:08:00 pm
HTTP2 Web Cam Software Package

As an example of using this HTTP 2 server, a software package was developed and published for collecting and processing streaming data and messages from various devices using a single JSON subprotocol.
The software package includes a server and a package of client applications and libraries.

- wcwebcamserver (https://github.com/iLya2IK/wcwebcamserver) (Lazarus/Free Pascal) is a server designed to collect images and data streams from cameras (devices) and forward messages between devices to control peripherals via an HTTP 2 connection.
- The abstract client for Lazarus (https://github.com/iLya2IK/wccurlclient) is the curl client (Lazarus/Free Pascal).
- A detailed implementation of an external device based on "ESP32-CAM" is given in the example webcamdevice (C).
- An example of a desktop application (https://github.com/iLya2IK/webcamclientviewer) for managing external devices and viewing images is webcamclientviewer (Lazarus).
- An example of an Android application for controlling external devices, chatting and streaming is wcwebcameracontrol (Java).

wcWebCamClient library API User's Guide (https://ilya2ik.github.io/wcwebcamclient_lib/index.html)
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: senglit on February 20, 2024, 04:36:50 pm
When I run the demo project the server rejected the connection from the browser. And I found these infos in the logwebtest.db:
table: log

Code: Text  [Select][+][-]
  1. 2024-02-20 14:07:45     9       1       Server started
  2. 2024-02-20 14:07:45     9       1       Server initialized
  3. 2024-02-20 14:07:45     9       2       Error (ESocketError) : Binding of socket failed: 8000
  4. 2024-02-20 14:07:45     9       2       Error (ESocketError) : Binding of socket failed: 8000
  5. 2024-02-20 14:07:45     9       2       Error (ESocketError) : Binding of socket failed: 8000
  6. ... and another 1,000+ same error message
  7.  

table: sessions
Code: Text  [Select][+][-]
  1. 1       2024-02-20 09:52:55
  2. 2       2024-02-20 09:55:17
  3. 3       2024-02-20 10:26:14
  4. 4       2024-02-20 10:26:42
  5. 5       2024-02-20 10:26:56
  6. 6       2024-02-20 13:48:08
  7. 7       2024-02-20 13:53:52
  8. 8       2024-02-20 13:54:03
  9. 9       2024-02-20 14:07:45
  10. 10      2024-02-20 15:18:33
  11.  

table: sqlite_sequence
Code: Text  [Select][+][-]
  1. sessions        10
  2. log     32670
  3.  

I didn't change anything in your demo project. I just put a zlib1.dll into system32 folder and put localhost.crt/csr into openssl folder to make it runable.

What did I possibly missing?
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: Leledumbo on February 22, 2024, 06:09:54 am
Binding of socket failed: 8000
...
What did I possibly missing?
Something else is already listening on that port. You can use either netstat on cmd or Get-NetTCPConnection on PowerShell to see who. Or maybe just hit localhost:8000 in the browser and see what comes up.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: senglit on February 24, 2024, 01:32:44 pm
Tried by another port 3333(ensured it's not ocupied by using Get-NetTCPConnection). But still can not connect. The request header is normal, but no response.

quite strange. Maybe something wrong with my zlib1.dll or localhost.crt/csr?

Binding of socket failed: 8000
...
What did I possibly missing?
Something else is already listening on that port. You can use either netstat on cmd or Get-NetTCPConnection on PowerShell to see who. Or maybe just hit localhost:8000 in the browser and see what comes up.
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: Thaddy on February 24, 2024, 01:41:33 pm
if i read zlib combined with dll I get worried.  Why will people simply abandon it?
Title: Re: HTTP/2+HTTP/1.1+WebSocket server with Free Pascal
Post by: Leledumbo on February 25, 2024, 09:36:09 am
Tried by another port 3333(ensured it's not ocupied by using Get-NetTCPConnection). But still can not connect. The request header is normal, but no response.

quite strange. Maybe something wrong with my zlib1.dll or localhost.crt/csr?
Is it still the "Binding of socket failed" error?
TinyPortal © 2005-2018