cannot receive messages; udp server with lnet

[Warfley]

Timestamp will add small latency to the program. Receive them in time = receive when the datagram has arrived at the destination and the program reads it. I might have wrong assumptions on how things work. I thought that a udp datagram arrives, if you read it fast, then good, else the packet is gone/replaced by the next one. I set the server to sleep(1000), I still received all the datagrams.

If you are requiring these timing guarantees that the delay of checking a timestamp is crucial, then forget about working with the socket api all together. You either need to use user-space networking, or write your code as a kernel module.
Also you will probably run into scheduling problems as you don't use a real-time operating system

[MarkMLl]

The upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request).

Careful there. Some of this might be OS-specific, but generally speaking UDP broadcasts don't go through routers unless there's a special proxy or similar.

The overwhelming reason for UDP- apart from efficiency- is that (small) messages are received without artificial fragmentation or concatenation.

MarkMLl

[prodingus]

There is more than just UDP and TCP, there are things in between
TCP is a reliable ordered stream of data. UDP is for transmitting unreliable unordered packages. There is also RDM (Reliable Delivered Messages), which sends acks and has therefore delivery guarantees, but does not guarantee order, or SEQPACKET (Sequenced Packet), which also ensures ordering and is like TCP also connection based, but unlike TCP it does not provide a stream of data but only single packets, and therefore is more lightweight. Depending on how much guarantees you need and also how much overhead they imply, they can be useful choices as in-between options.

Nice to know all these info. Of course these can be done manually in order to fit the needs of the program.

If you are requiring these timing guarantees that the delay of checking a timestamp is crucial, then forget about working with the socket api all together. You either need to use user-space networking, or write your code as a kernel module.
Also you will probably run into scheduling problems as you don't use a real-time operating system

This goes deep into the rabbithole...

[rvk]

The upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request).

Careful there. Some of this might be OS-specific, but generally speaking UDP broadcasts don't go through routers unless there's a special proxy or similar.

With whole network I meant if course internal network. For example an internal messaging system.

Timestamp will add small latency to the program.

This could be as little as 4 bytes. So not really any problem considering the UDP is at least package size large.

I thought that a udp datagram arrives, if you read it fast, then good, else the packet is gone/replaced by the next one. I set the server to sleep(1000), I still received all the datagrams.

That's just how it works. An UDP package can even arive seconds later depending on network circumstances, buffering and OS priority.

If that's unacceptable you can send a few bytes extra (it really doesn't take any more space in a package) or leave the concept of UDP behind.

Then why are you using UDP and not TCP connection?

I am going to try the ping-pong. I want to use udp instead of tcp because I want to have as low lag as possible. What I am going to do if a packet is lost is up to me to make or not a custom ack/resend.

True but in that case you do need to handle receiving messages faster than the sender is sending them. You are not doing that.

You can just flush the buffer after receiving a message. In that case you (might) loose messages but you are more certain the message you do get are more recent.

[prodingus]

That's just how it works. An UDP package can even arive seconds later depending on network circumstances, buffering and OS priority.

If that's unacceptable you can send a few bytes extra (it really doesn't take any more space in a package) or leave the concept of UDP by behind.

True but in that case you do need to handle receiving messages faster than the sender is sending them. You are not doing that.

You can just flush the buffer after receiving a message. In that case you (might) loose messages but you are more certain the message you do get are more recent.

The whole point of the sleep(100) and sleep(1000) in the server was to simulate the "packet had arrived and time passed since then so you lost it". It doesn't happen (!). Of course the messages are very small, but unless there is something wrong with the code examples, it seems that udp is not behaving as udp 

[prodingus]

I just come across this nice post and I am going to drop it here:
https://stackoverflow.com/questions/7843644/how-long-does-a-udp-packet-stay-at-a-socket

So to reply to my last post, udp is acting as udp 

[rvk]

Of course the messages are very small, but unless there is something wrong with the code examples, it seems that udp is not behaving as udp 

No, it's not behaving like your notion of UDP.
But your notion is wrong 

The buffers are there to make UDP more reliable.
There is nothing in UDP protocol that says messages can't be buffered ir received out of order or after a delay.

Some reading https://jvns.ca/blog/2016/08/24/find-out-where-youre-dropping-packets/

You could just read the buffer until it's empty and ONLY use the last message. In that case you get what you want (or expected). The discarded messages are 'lost'.

[prodingus]

My bad, should have searched before my post #64.

[prodingus]

I would like to thank everyone in this thread for their contribution and help! I am sure that other dinguses, I mean noobs, I mean newbies would find all that info usefull. Let it be no doubt that I did search the references and examples, but they are not always easy to understand (but that's just me).

For the sake of the original post regarding the lnet and the handles, If someone can make an example without using classes, it would be welcome, for I know that you can write things with many different ways. Again, thank you guys!

Ps I am going to test all the examples in this thread, I just need time!

[rvk]

For the sake of the original post regarding the lnet and the handles, If someone can make an example without using classes, it would be welcome, for I know that you can write things with many different ways.

Like it was already said... you can't use lnet without classes.

But I'm curious. You say you want asynchronous communication... Do you know what that means?
Now you have a program with just one flow. Line by Line. You can't achieve asynchrony that way.

Asynchronous programming is a form of parallel programming that allows a unit of work to run separately from the primary application thread. When the work is complete, it notifies the main thread (as well as whether the work was completed or failed). There are numerous benefits to using it, such as improved application performance and enhanced responsiveness.

https://stackify.com/when-to-use-asynchronous-programming

So if you want to do asynchronous communication, you need to do some form of multiple program flow.

Coming back to your original lnet server (receiver).
Because lnet is event driven you DO need to signal it to do it's work in the background (well not actually background but you need to let it do it's work). You can do that with just one line. Sock.CallAction. So if you add line 13 to your original program, it will work.

(Note that this isn't real asynchronous communication though. You're just misusing the library to force synchronous communications.)

Code: Pascal  [Select][+][-]

1. program test1;
2. {$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
3. {$APPTYPE CONSOLE}
4. uses
5.   SysUtils, crt, lnet, lnetbase;
6. var
7.   Sock: TLUDP;
8.   tmp: string;
9. begin
10.   Sock := TLUdp.Create(nil);
11.   Sock.listen(1234);
12.   repeat
13.     Sock.CallAction;
14.     if Sock.GetMessage(tmp) > 0 then
15.       writeln(tmp);
16.     sleep(100);
17.   until False;
18. end.

Bare in mind... this is not the actual correct way to do things. You are still trying to do a GetMessage while there might not even be a message on the line. This could block the CallAction again and I'm not sure the code will work correctly. The correct way would be to use the sock.OnReceive because then you know there is a message to receive.

Did you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ??
It has a complete server/client udp example with a TLUDPTest class.

So the correct way to do it would be something like this:

Code: Pascal  [Select][+][-]

1. program test1;
2. {$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
3. {$APPTYPE CONSOLE}
4. uses
5.   SysUtils, crt, lnet, lnetbase;
6. var
7.   Sock: TLUDP;
8.   tmp: string;
9.  
10. type
11.   TMyDummy = class
12.     class procedure OnReceive(aSocket: TLSocket);
13.   end;
14.  
15. class procedure TMyDummy.OnReceive(aSocket: TLSocket);
16. begin
17.   if Sock.GetMessage(tmp) > 0 then
18.     writeln(tmp);
19. end;
20.  
21. begin
22.   Sock := TLUdp.Create(nil);
23.   Sock.OnReceive := TMyDummy.OnReceive;
24.   Sock.listen(1234);
25.   repeat
26.     Sock.CallAction;
27.     sleep(100);
28.   until False;
29. end.

You see, now when there is no message, CallAction keeps getting called. When there is a message the OnReceive from TMyDummy is called and the message is read and printed. In the example ludp.pp you see you can also send a reply back.

(The TMyDummy can be called something else of course, this is just to show you can use classes, and class procedures, without creating an instance. But that might be something too advanced for now  )

[prodingus]

But I'm curious. You say you want asynchronous communication... Do you know what that means?
Now you have a program with just one flow. Line by Line. You can't achieve asynchrony that way.

I read here about it and thougt it will be a good feature.

(Note that this isn't real asynchronous communication though. You're just misusing the library to force synchronous communications.)

I hope you mean ..misusing the library to force asynchronous communications..

Let's just leave the sync/async for now, as the

Code: Pascal  [Select][+][-]

1. program test1;
2. {$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
3. {$APPTYPE CONSOLE}
4. uses
5.   SysUtils, crt, lnet, lnetbase;
6. var
7.   Sock: TLUDP;
8.   tmp: string;
9. begin
10.   Sock := TLUdp.Create(nil);
11.   Sock.listen(1234);
12.   repeat
13.     Sock.CallAction;
14.     if Sock.GetMessage(tmp) > 0 then
15.       writeln(tmp);
16.     sleep(100);
17.   until False;
18. end.

works fine even if there is no messages. I just tried it and it works (at least in the testing; if any problem arrives I'll let you know).

Did you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ??
It has a complete server/client udp example with a TLUDPTest class.

Yes, but I find it "difficult" because of the classes. Your latest example seems quite easier to understand and use (and more compact)

Side question:

as this works (even if it might have problems that I don't see in this stage of the testing):

Code: Pascal  [Select][+][-]

1. rogram pointer_test;
2.  
3. var
4.         p: pointer;
5.  
6. procedure a;
7.  
8. var
9.         x:uint64;
10. begin
11.         x:=5;
12.         p:=@x;
13. end;
14.  
15. begin
16.         a;
17.  
18.         writeln( uint64(p^));
19.         readln;
20.  
21. end.  

this doesn't:

Code: Pascal  [Select][+][-]

1. program multi_1;
2. uses
3.         SysUtils, crt, lnet;
4. var
5.  
6.         tmp: string;
7.         psock:^TLUDP;
8.  
9. procedure connect(port:integer);
10. var
11.         Sock: TLUDP;
12. begin
13.  
14.         Sock := TLUdp.Create(nil);
15.         psock:=@sock;
16.         Sock.listen(port);
17. end;
18.  
19. begin
20.  
21.     connect(1234);
22.  
23.     repeat
24.         pSock^.CallAction;
25.         pSock^.GetMessage(tmp);
26.         writeln(tmp + ' ' + pSock^.host);
27.         sleep(500);
28.     until False;
29. end.

Also, I tried various ways to get the host but no success (it can be implemented inside the meagges)

[rvk]

Did you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ??
It has a complete server/client udp example with a TLUDPTest class.

Yes, but I find it "difficult" because of the classes. Your latest example seems quite easier to understand and use (and more compact)

As your programs get more advanced you eventually will need to go to procedures and classes.

as this works (even if it might have problems that I don't see in this stage of the testing):

Your first example works but is very dangerous. You have a pointer to some variable that doesn't exist anymore and can get overwritten at any time.

You second example has that same problem that the sock variable is local and is overwritten once you exit the procedure.
So you shouldn't use pointer p anymore.

It's better to declare sock as global variable in this case.
(or in case of a class you can define it as local class variable/property)

Also, I tried various ways to get the host but no success (it can be implemented inside the meagges)

Isn't the ip address of the sender in aSocket.PeerAddress in the OnReceive event?

Code: Pascal  [Select][+][-]

1. program project1;
2. {$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
3. {$APPTYPE CONSOLE}
4. uses
5.   SysUtils, crt, lnet, lnetbase;
6. var
7.   Sock: TLUDP;
8.   tmp: string;
9.  
10. type
11.   TMyDummy = class
12.     class procedure OnReceive(aSocket: TLSocket);
13.   end;
14.  
15. class procedure TMyDummy.OnReceive(aSocket: TLSocket);
16. begin
17.   if Sock.GetMessage(tmp) > 0 then
18.     writeln(tmp + ' from ' + aSocket.PeerAddress);
19. end;
20.  
21. begin
22.   Sock := TLUdp.Create(nil);
23.   Sock.OnReceive := TMyDummy.OnReceive;
24.   Sock.listen(1234);
25.   repeat
26.     Sock.CallAction;
27.     sleep(100);
28.   until False;
29. end.

[prodingus]

Isn't the ip address of the sender in aSocket.PeerAddress in the OnReceive event?

I was trying to add it in your first of the examples.

I stumbled uppon it but couldn't find any reference of the lifetime of local variables for pascal (missed it?) but for other languages (they gone as the function ends?). In the pointer test this *seems* to not be the case; even adding sleep it gives output. The solution is obvious as you said, but less compact/neat (thats what I was trying to avoid). Is there any "control" over lifetime?

[rvk]

In the pointer test this *seems* to not be the case; even adding sleep it gives output.

I can guarantee that this will go wrong at some point.
And so it did in your second example with sock 
With just one integer as variable it might go ok for one procedure but if your program gets more complex it WILL definitely go wrong.

The solution is obvious as you said, but less compact/neat (thats what I was trying to avoid). Is there any "control" over lifetime?

No. If you define a local variable... that's just what it is... local. You shouldn't access it outside of that procedure.

The "control" you have over the lifetime is if you make the variable global
Why do you think it is less neat? You also already have a pointer p which you defined global. Why not just define Sock: TLUDP global instead of the pointer p???

BTW. TLUDP is already a class. So any variable Sock: TLUDP is already a pointer. What you are doing with psock: ^TLUDP is creating a pointer to a pointer (class variable). Really not needed and inefficient.

Sock doesn't take any more memory than psock does.

So this would work just fine:

Code: Pascal  [Select][+][-]

1. program multi_1;
2.  
3. uses
4.   SysUtils, crt, lnet;
5.  
6. var
7.   tmp: string;
8.   Sock: TLUDP;
9.  
10.   procedure connect(port: integer);
11.   begin
12.     Sock := TLUdp.Create(nil);
13.     Sock.listen(port);
14.   end;
15.  
16. begin
17.   connect(1234);
18.   repeat
19.     Sock.CallAction;
20.     Sock.GetMessage(tmp);
21.     writeln(tmp + ' ' + Sock.host);
22.     sleep(500);
23.   until False;
24. end.

You can also do something like this:

Code: Pascal  [Select][+][-]

1. program multi_1;
2.  
3. uses
4.   SysUtils, crt, lnet;
5.  
6.   function CreateSockAndConnect(port: integer): TLUDP;
7.   var
8.     localSock: TLUDP;
9.   begin
10.     localSock := TLUdp.Create(nil);
11.     localSock.listen(port);
12.     Result := localSock;
13.   end;
14.  
15. var
16.   Sock: TLUDP;
17.   tmp: string;
18.  
19. begin
20.   Sock := CreateSockAndConnect(1234);
21.   repeat
22.     Sock.CallAction;
23.     Sock.GetMessage(tmp);
24.     writeln(tmp + ' ' + Sock.host);
25.     sleep(500);
26.   until False;
27. end.

[prodingus]

You are (again) correct. The local variable lives for one repeat loop, for loop, even if you copy-paste the lines multiple times, then it zeroes in my pointer test.  I just try to avoid the main "var" hell that I am getting every time that I start a project.