Missing requests with TPHttprouter after TCP-Window-Resize

[smallfreak]

I'm trying to debug a web service, that is consumed by a Web-Radio device.

The program (not mine) looks fine and the API does all that is expected, when called from a browser or SOAPUI. It uses the fcl-web library and the HTTPRouter component

But when the physical device calls the API, I find many of the requests are just not received at the web service. I do see them in the Wireshark protocol arriving at the network card. They are just not processed. Instead the packet is answered with a Connection-Reset [RST].

The traffic is quite low, payload sizes of < 500Bytes. Essentially it's just another URL request. This all does easily fit in a standard TCP frame and it's not even the reply that gets lost, but the request, even if I can watch it arriving in the Packet trace.

I suspect an intermediate "TCP-Window-Resize" request that is issued from the device, just before calling the API. This in turn may be because of the previous reply that handed over an encryption password, so maybe the device get's a little slow for the next conversation.

If I try the exact URL from a regular browser, the packet gets received and logged. So the Service routing is correct. It just does not get triggered. The service even has a default catch-all routing that is logged as well, in case the URL IS wrong and replies a 404.

If comment out all routngs but the default one, hence reply a 404 on all requests, the "missing" packet gets recieved and logged - but there is no Window-Resize request issued from the client.

The client is connected via WLAN, the server is a local executable on my Ubuntu 24.10.

I tried running the server in a Docker on my NAS, where it exhibits essentially the same behavior. I do not see the "search" request logged, after the login. But due to the nature of the network it is PITA to get these packages traced.

I did try a packet trace on the Internet Router, when the radio communicates with it's native Cloud Service. The Window-Resize-Requests are quite similar, but certainly the service processes the request.

So this might either be some quirx in my (W)LAN or something in the used fcl-web library.

Dos this sound familiar to anyone? Any idea, how I can debug such a problem?

[Thaddy]

Deliberately, manually, resizing a tcp-window during traffic - especially communicating with just one or two end-points - is something that needs recovery and always generates extra traffic to restore packages that are missed.
Are you sure you really wanted that? I don't think so. You are mishandling the protocol, not the code. tcp is about guaranteed delivery, but not if you start mingling with its window size. For that you need UDP. Or are you using udp tunneled over tcp? Then the tcp tunnel would be restored but the udp will NOT be restored and will drop packages.
You must have very slow connections because I did not see this issue for at least 15 years.
Requesting a window change needs to be done on or immediately after handshake, not during the real connection on older hardware.
That is, more modern routers support dynamically adjustment but that is the router software itself and you should not touch it.
Also note the tcp window preferably needs to be a power of two.

You can test that with the iperf tool.

[smallfreak]

Not "I" am fiddling with the Window-Size. It's the device. I'm just watching the traffic happen that way.

A handshake about the next window size is part of EVERY "ACK" package and part of the normal TCP protocol. It's one of the means to keep the traffic in a digestible volume on a fast line. Part of that are "queue full" TCP signals, in case that more data is sent than could be processed since the last ACK. The sender then has to shut up until another "queue empty" and resend the bytes that were not part of the last ACK'ed size. This is how TCP works. Especially with slow devices on a fast lane. This was NOT the case here. Just very small packages, well within any window-size (I think 4K is the smallest possible) and even well within the size of a single TCP frame. Ideally the window-size should be a multiple of the TCP frame payload-size which usually is 1500 minus overhead bytes.

So this is essentially a "You may send up to xxx bytes in a row for your next transmission, then expect another ACK".

If you watch Wireshark protocols, you will notice a "win=xxxxx" part in every TCP header to signal the partner about the capability to receive for the next package.

The problem I suspect is, that the pledge for downsizing of the window-size done by the device might confuse the web service that in turn stumbles over the next request FROM that device.

A pure guess, as TCP handling should be done correctly on the driver level, not in the application. It might be a mere coincidence, that just the request after such a downsizing handshake is not processed.

That's why I'm asking for ideas how to debug such a situation. Can I get a breakpoint low enough in the stack to catch the "GET" package before the HTTProuter tries to match the URL against the learned routes? Just to be certain it DOES get received anywhere in the application? My first assumption was that the routes are not OK and that there is just no target to receive it.

it does not get deployed to any of the routes ad not to the default route. Instead it gets reset [RST] at a lower level of the stack. But where and why?

The route definition is

Code: Pascal  [Select][+][-]

1.   HTTPRouter.RegisterRoute('/setupapp/*/search.asp', @GetRadioBrowserSearchedStations, false);

Essentially the same as the route to the login, that preceeds the call ant which is logged:

Code: Pascal  [Select][+][-]

1.   HTTPRouter.RegisterRoute('/setupapp/*/loginxml.asp', @SetupAppLoginXMLPage, false);

The first line in every routing target is a call to log the incident:

Code: Pascal  [Select][+][-]

1. procedure GetRadioBrowserSearchedStations(AReq: TRequest; ARes: TResponse);
2. var
3.   LRBStations: TRBStations;
4. begin
5.   Logging(ltDebug, AReq.Method+' '+AReq.URI);
6.   try
7.     if AReq.QueryFields.Values[PATH_PARAM_SSEARCH_TYPE]='3' then    // In some cases "3" mean a "Play" option.
8. ...
9.  

Neither is the log written nor does a breakpoint on line 5 fire - when there was this [TCP Window Update] frame before the call. And the same is for the "default" route target. I don't arrive there.

Told that - if I try the same calls in the same sequence from SOAPUI or just a browser, the call gets logged and the breakpoint is hit. So it must be something between the device and the service. And since the packet can be traced on the PC that hosts the service, it is probably something between the network driver and the routing target.

Well, I might be totally wrong, but that's how it looks.