Indy UDP packets losses in duplex

[vladbfg]

Are you aware UDP is a potentionally lossy protocol? In other words it is fire and forget? Because that is basically everything where all your questions and possibly misunderstanding come from: UDP has no guaranteed delivery and is not designed like that. It is only fast, but not lossless.

With all due respect, you did not understand my question and apparently did not read the entire thread carefully.
I'm not looking for a lossless protocol. In my case, the losses were due to the buffer depth setting and slow reading.
In an abstract local network with sufficient bandwidth there should be no packet loss at all.

where a frame drop is not critical, UDP comes alive, but never use it for anything else

Sounds like too categorical a statement. And yes, you are right, RTP uses UDP as transport.
In turn, I can recommend that you read about such protocols as SNMP, SNTP/NTP, RIP, DHCP and may be google experimental QUIC and others. PS: DNS

[MarkMLl]

With all due respect, you did not understand my question and apparently did not read the entire thread carefully.
I'm not looking for a lossless protocol. In my case, the losses were due to the buffer depth setting and slow reading.
In an abstract local network with sufficient bandwidth there should be no packet loss at all.

I'll break the habit of a lifetime, and stick up for Thaddy here. With all due respect, /you/ haven't read the documentation: UDP is a lossy protocol with no guarantee of delivery order, even on a local network.

The fact that you've seen all the expected packets using Wireshark is meaningless, since it has a separate point at which it intercepts packets and is immune to kernel- and app-level buffering issues. It would also be irrelevant if Wireshark consistently showed packets arriving in the correct order by by the time you read them some were swapped: the protocol, considered end-to-end, does not guarantee order, and it is entirely possible for a system's buffering to dequeue them arbitrarily.

If you want something reliable, then either build sequence numbering and acknowledgements on to of UDP (which is a point somebody made earlier), use TCP, or use something like SCTP which should be OK locally even though many ISPs drop it.

MarkMLl

[vladbfg]

UDP is a lossy protocol with no guarantee of delivery order, even on a local network.

About order: I didn't say anything about order, I said all packs must be delivered.

So, network devices (swithes) decide to discard the UDP packet? May be NetCard decides drop the frame with UDP data and transfer the frame with TCP data?

I'll say it again: in a normal local network no packets should be lost.
PS: If your local network (swithes/PC/other HW) is losing packets in normal state (no overload bandwidth), look for hardware problems.

[rvk]

I'll break the habit of a lifetime, and stick up for Thaddy here. With all due respect, /you/ haven't read the documentation: UDP is a lossy protocol with no guarantee of delivery order, even on a local network.

All packages were received by the network card just fine. None were lost. Only because of the buffering in the network driver the packages were lost.

Agreed, udp doesn't guarantee delivery but that wasn't the problem here.

TS. You might want to remove the sending of packages from the receive event. Now the receive event takes too long (because you have the overload of sending). It would be better to, if you want a sort of echo, to push the data to a buffer, and do the sending in another thread. That way the receive end will stay clean and fast and you probably receive all data. Now you increased the buffer but that will only get you so far into it craps out again (with more data). Of course you need to determine a good size for your buffer but that might be better than adjusting the network driver buffer like it is done now.

[Khrys]

In an abstract local network with sufficient bandwidth there should be no packet loss at all.

UDP does not guarantee delivery, period. There's no way around it - that's how the protocol is defined, and anything else is essentialy relying on undefined behaviour.
Software is like a contract: don't be surprised when someone (metaphorically) sues you after you break its terms.

[Thaddy]

All packages were received by the network card just fine.

by accident....
Or the throw of a dice. Or living in the Benelux or the Baltic states. And even then it is by accident..

At least on this one I get overwealming support from others that are experts.

[rvk]

All packages were received by the network card just fine.

by accident....
Or the throw of a dice.

At least on this one I get overwealming support from others that are experts.

Or they are received because of good quality network (like we have here).

Guys, we know udp doesn't guarantee delivery (everyone agrees here) but if you loose 20% of your packages, which was the problem here, then there is something seriously wrong. In this case it was the buffering due to not reading the packages fast enough.

True, don't expect all packages to be delivered (for that you need other protocols or management) but when you loose that many, you need to find out why (which is discovered already).

(Same goes for voip which usually also uses udp.)

[Thaddy]

Rik, we are spoiled. Try Scotland Wales, Northen England or even France or Austria. Let alone the U.S. (where the infrastructure is happless, not helpless)
20 % package loss is actually quite good... allows for video streaming 

[rvk]

Rik, we are spoiled. Try Scotland Wales or even France or Austria. Let alone the U.S. (which is happless, not helpless)
 20 % package loss is actually quite good...

I wonder how voip performs there then.
Would be a nightmare I guess.

[Thaddy]

Depending on where you are.But thanks to udp it still is understandable in many cases.
You and me are really spoiled. ( I forgot about udp over tcp.)

[Warfley]

So, network devices (swithes) decide to discard the UDP packet? May be NetCard decides drop the frame with UDP data and transfer the frame with TCP data?

Yes every device in the network can decide to drop packets for any reason whatsoever. For example a switch can start dropping packages because it's overheating to protect the hardware from damage on a hot summer day. Also radiation, there is a low chance that any data in transmission will randomly get a bit flipped from background radiation. Also every device in the network has fixed size buffers, which when they overflow you get package loss.

You may think now why would you ever encounter package loss in a local network with low traffic? The answer to that is funnily enough TCP. TCP has a mechanism called congestion control. Basically TCP will increase it's sending rate until package loss is detected, in order to figure out the maximum capacity of the network and use this to adjust the sending rate. So TCP is implemented in a way that it will literally DOS your switches in order to figure out what the maximum transmission speed is. Smart routers like used by ISPs in the backend will treat TCP and UDP packets differently, and have different package loss rates for them. Your cheap home switches don't.
So a single TCP connection, e.g. someone using the web, can already lead to local network traffic loss.

[Warfley]

I wonder how voip performs there then.
Would be a nightmare I guess.

There is a whole area of research for this called Quality of Service. Basically how it works is that the internet backbones will look into the packages and categorize them according to their requirements. VoIP is one of the use cases which is relatively robust against package loss, but requires very small delays. So these packages will have usually a smaller queue that guarantees that a package is either dropped early or sent out early, to minimize delay.
TCP on the other hand requires a steady throughput, but as described in my previous post, relies on package loss for it's congestion control. So TCP packages are dropped very frequently, even before the buffers are full, each package will get a certain chance to be dropped when it arrives.
Online gaming usually requires both, low delay and low package loss and therefore are usually prioritized in general. Video Streaming requires low package loss but can live with a high delay, so it get's the largest buffers. And so much more.

I had a whole module in University about this stuff. Long story short it's very complicated and there are no generic statements like x% package loss possible/meaningful

[Remy Lebeau]

Thx for explanation. I use one binding (which is automatically created upon activation server idudpsrv.active:=true)

Are you aware that activating an Indy server without any bindings defined may create 2 bindings, not just 1?  If you want to ensure only 1 binding, you should define it yourself before activating.

Is it possible to pass one socket to 2(N) threads and each thread will be able to send data over the same socket?

Yes, provided adequate serialized access to the socket so multiple sends don't overlap each other.  For a message-oriented socket like UDP, the default serialization provided by the OS should suffice since sends are atomic.  This is not true for stream-oriented sockets like TCP, in which case you have to serialize in your own code, such as with a mutex.