Forum > Networking and Web Programming
HTTP/2+HTTP/1.1+WebSocket server written with Lazarus (Free Pascal)
PierceNg:
--- Quote from: iLya2IK on February 26, 2021, 05:02:41 pm ---I checked the server with the h2spec utility: 41 passed from 94 tests.
I found some bugs in my code caused by a misunderstanding of the http2 protocol (mostly in the part of informing clients about malformed requests and protocol errors). I also found that tests have their own bugs and some ambiguous testing algorithms. For example - Test 5.1/1 waits for the GOAWAY frame with PROTOCOL_ERROR, but section 6.1 in RFC 7540 requires GOAWAY frame with STREAM_CLOSED error. Another example - Test to emulate maximum concurrent streams. On my server, the test program was unable to create so many concurrent streams - no matter how I increase or decrease the MAX_CONCURRENT_STREAMS value - they seem to be quickly snatched up by worker threads.
The author has yet to work out his testing program better, but now it is a powerful tool and I intend to use it further to improve the reliability and adequacy of my server.
--- End quote ---
This is great - both your implementation and h2spec will improve through this interaction.
Natural generative adversarial neural networks at work. :D
iLya2IK:
Testing status now using h2spec: 94 tests, 84 passed, 0 skipped, 10 failed
Failed tests
--- Code: XML [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} --- 4. HTTP Frames 4.2. Frame Size × 1: Sends a DATA frame with 2^14 octets in length -> The endpoint MUST be capable of receiving and minimally processing frames up to 2^14 octets in length. Expected: HEADERS Frame (stream_id:1) Actual: Connection closed Comment: Error in h2spec - DATA payload size didn't equal the value of "content-length" header. 5. Streams and Multiplexing 5.1. Stream States 5.1.2. Stream Concurrency × 1: Sends HEADERS frames that causes their advertised concurrent stream limit to be exceeded -> The endpoint MUST treat this as a stream error of type PROTOCOL_ERROR or REFUSED_STREAM. Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR) RST_STREAM Frame (Error Code: PROTOCOL_ERROR) GOAWAY Frame (Error Code: REFUSED_STREAM) RST_STREAM Frame (Error Code: REFUSED_STREAM) Connection closed Actual: DATA Frame (length:845, flags:0x01, stream_id:201) Comment: Ambiguous testing algorithm in h2spec. Streams quickly snatched up by worker threads 5.5. Extending HTTP/2 × 2: Sends an unknown extension frame in the middle of a header block -> The endpoint MUST treat as a connection error of type PROTOCOL_ERROR. Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR) Connection closed Actual: Timeout Comment: I didn't find any references in RFC 7540 that unknown extension frame in the middle of a header block should cause a connection error 6. Frame Definitions 6.5. SETTINGS 6.5.3. Settings Synchronization × 1: Sends multiple values of SETTINGS_INITIAL_WINDOW_SIZE -> The endpoint MUST process the values in the settings in the order they apper. Expected: DATA Frame (length:1, flags:0x00, stream_id:1) Actual: DATA Frame (length:845, flags:0x01, stream_id:1) Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. 6.9. WINDOW_UPDATE 6.9.1. The Flow-Control Window × 1: Sends SETTINGS frame to set the initial window size to 1 and sends HEADERS frame -> The endpoint MUST NOT send a flow-controlled frame with a length that exceeds the space available. Expected: DATA Frame (length:1, flags:0x00, stream_id:1) Actual: DATA Frame (length:845, flags:0x01, stream_id:1) Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. × 2: Sends multiple WINDOW_UPDATE frames increasing the flow control window to above 2^31-1 -> The endpoint MUST sends a GOAWAY frame with a FLOW_CONTROL_ERROR code. Expected: GOAWAY Frame (Error Code: FLOW_CONTROL_ERROR) Actual: Timeout Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. × 3: Sends multiple WINDOW_UPDATE frames increasing the flow control window to above 2^31-1 on a stream -> The endpoint MUST sends a RST_STREAM frame with a FLOW_CONTROL_ERROR code. Expected: RST_STREAM Frame (Error Code: FLOW_CONTROL_ERROR) Actual: Timeout Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. 6.9.2. Initial Flow-Control Window Size × 1: Changes SETTINGS_INITIAL_WINDOW_SIZE after sending HEADERS frame -> The endpoint MUST adjust the size of all stream flow-control windows. Expected: DATA Frame (length:1, flags:0x00, stream_id:1) Actual: DATA Frame (length:845, flags:0x01, stream_id:1) Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. × 2: Sends a SETTINGS frame for window size to be negative -> The endpoint MUST track the negative flow-control window. Expected: DATA Frame (length:1, flags:0x00, stream_id:1) Actual: Timeout Comment: Agreed. I working to fix this. My problem - i don't clearly understand the WINDOW_UPDATE mechanism. 8. HTTP Message Exchanges 8.1. HTTP Request/Response Exchange 8.1.2. HTTP Header Fields 8.1.2.2. Connection-Specific Header Fields × 1: Sends a HEADERS frame that contains the connection-specific header field -> The endpoint MUST respond with a stream error of type PROTOCOL_ERROR. Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR) RST_STREAM Frame (Error Code: PROTOCOL_ERROR) Connection closed Actual: DATA Frame (length:845, flags:0x01, stream_id:1) Comment: I know that this violates the requirements of RFC 7540, but I sincerely consider them redundant and the type of reaction to the header value should remain on the server side
trev:
@iLya2IK: Great work! I've added a link to the Wiki Web Development Portal.
iLya2IK:
The project has undergone major changes related to debugging of flow control (according to sec.5.2 RFC 7540). Here are the actual results of testing with utilities.
--- Code: XML [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---# h2spec -p 8080 http2 -t -k -o 2094 tests, 91 passed, 0 skipped, 3 failedHypertext Transfer Protocol Version 2 (HTTP/2) 4. HTTP Frames 4.2. Frame Size × 1: Sends a DATA frame with 2^14 octets in length -> The endpoint MUST be capable of receiving and minimally processing frames up to 2^14 octets in length. Expected: HEADERS Frame (stream_id:1) Actual: Connection closed Comment: Error in h2spec - DATA payload size didn't equal the value of "content-length" header. 5. Streams and Multiplexing 5.5. Extending HTTP/2 × 2: Sends an unknown extension frame in the middle of a header block -> The endpoint MUST treat as a connection error of type PROTOCOL_ERROR. Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR) Connection closed Actual: Timeout Comment: I didn't find any references in RFC 7540 that unknown extension frame in the middle of a header block should cause a connection error 8. HTTP Message Exchanges 8.1. HTTP Request/Response Exchange 8.1.2. HTTP Header Fields 8.1.2.2. Connection-Specific Header Fields × 1: Sends a HEADERS frame that contains the connection-specific header field -> The endpoint MUST respond with a stream error of type PROTOCOL_ERROR. Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR) RST_STREAM Frame (Error Code: PROTOCOL_ERROR) Connection closed Actual: DATA Frame (length:845, flags:0x01, stream_id:1) Comment: I know that this violates the requirements of RFC 7540, but I sincerely consider them redundant and the type of reaction to the header value should remain on the server side
And here are the results of testing with h2load
--- Code: XML [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---# h2load -n32000 -c32 -m10 --header=connection:keep-alive --header=cookie:cid=11 https://localhost:8080 starting benchmark...spawning thread #0: 32 total client(s). 32000 total requestsTLS Protocol: TLSv1.2Cipher: ECDHE-RSA-AES256-GCM-SHA384Server Temp Key: X25519 253 bitsApplication protocol: h2progress: 10% doneprogress: 20% doneprogress: 30% doneprogress: 40% doneprogress: 50% doneprogress: 60% doneprogress: 70% doneprogress: 80% doneprogress: 90% doneprogress: 100% done finished in 17.22s, 1858.68 req/s, 1.54MB/srequests: 32000 total, 32000 started, 32000 done, 32000 succeeded, 0 failed, 0 errored, 0 timeoutstatus codes: 32000 2xx, 0 3xx, 0 4xx, 0 5xxtraffic: 26.46MB (27746560) total, 126.38KB (129408) headers (space savings 95.01%), 25.79MB (27040000) data min max mean sd +/- sdtime for request: 16.59ms 327.96ms 150.22ms 35.42ms 77.80%time for connect: 7.17ms 1.96s 1.15s 618.49ms 62.50%time to 1st byte: 59.55ms 2.00s 1.20s 629.72ms 62.50%req/s : 58.09 72.88 61.87 4.42 81.25% # h2load -n32000 -c32 -t8 -m1 --h1 --header=connection:keep-alive --header=cookie:cid=11 https://localhost:8080-t: warning: the number of threads is greater than hardware cores.starting benchmark...spawning thread #0: 4 total client(s). 4000 total requestsspawning thread #1: 4 total client(s). 4000 total requestsspawning thread #2: 4 total client(s). 4000 total requestsspawning thread #3: 4 total client(s). 4000 total requestsspawning thread #4: 4 total client(s). 4000 total requestsspawning thread #5: 4 total client(s). 4000 total requestsspawning thread #6: 4 total client(s). 4000 total requestsspawning thread #7: 4 total client(s). 4000 total requestsTLS Protocol: TLSv1.2Cipher: ECDHE-RSA-AES256-GCM-SHA384Server Temp Key: X25519 253 bitsApplication protocol: http/1.1progress: 10% doneprogress: 20% doneprogress: 30% doneprogress: 40% doneprogress: 50% doneprogress: 60% doneprogress: 70% doneprogress: 80% doneprogress: 90% doneprogress: 100% done finished in 29.06s, 1101.26 req/s, 1.01MB/srequests: 32000 total, 32000 started, 32000 done, 32000 succeeded, 0 failed, 0 errored, 0 timeoutstatus codes: 32000 2xx, 0 3xx, 0 4xx, 0 5xxtraffic: 29.39MB (30816000) total, 2.53MB (2656000) headers (space savings 0.00%), 25.79MB (27040000) data min max mean sd +/- sdtime for request: 7.99ms 208.16ms 26.31ms 5.94ms 93.71%time for connect: 8.22ms 3.08s 1.25s 943.18ms 62.50%time to 1st byte: 99.48ms 3.12s 1.31s 913.25ms 62.50%req/s : 34.42 38.74 36.31 1.25 62.50%
iLya2IK:
Improved support for TLS v1.3, optimized the speed of the network code (read latency increased when the server was idle).
Navigation
[0] Message Index
[#] Next page
[*] Previous page