how to scan a network ?

[ProfHarry]

Hello,

I'm french so my english is not so good and my friend Deepl help me  ;-)

I have a set of ESP32 microcontrollers which are HTTP servers and DHCP clients on a wifi network.
The only user interface is the web page, so it is difficult for the user to know the IP addresses of the various microcontrollers. In general, the DHCP server and the list of leases are not accessible to users.

Each ESP32 responds to the URL http://IP/info.txt with a line containing, among other things, its IP address and hostname.

I therefore tried to develop a Lazarus application to scan the network and make this request to all the addresses.
But it's very, very slow... Because of the timeout for most addresses, it takes more than 10 minutes to scan an entire /24 network. I can't even imagine on a /16 network.

I use this code called in a loop on all IP addresses:

Code: Pascal  [Select][+][-]

1. function GetInfos(IP : string) : string;
2. Var
3.   URL : String;
4.   SL : TStringList;
5. begin
6.   URL := 'http://' + IP + '/infos.txt';
7.   SL := TStringList.Create();
8.  
9.   try
10.     with TFPHttpClient.Create(Nil) do
11.       try
12.         AllowRedirect := True;
13.         Get(URL,SL);
14.         application.processmessages;
15.       finally
16.         Free;
17.       end;
18.     Result := SL[0];
19.   finally
20.     SL.Free;
21.   end;
22. end;
23.  

Do you see a solution to speed up the scan and make it possible on a larger network?
Thanks for your help.

[alpine]

Couldn't you just send an UDP broadcast packet with the server IP and then all nodes to reply with their info back?
This is pretty trivial, I've done it before but not in FPC ie. don't have a source to show it.

[ProfHarry]

Yes, it's a great idea! Instead of searching for ESP32s from the lazarus program, I wait for the ESP32s to show up.

It's simpler in principle. All that's left is to implement it.

[dsiders]

Yes, it's a great idea! Instead of searching for ESP32s from the lazarus program, I wait for the ESP32s to show up.

It's simpler in principle. All that's left is to implement it.

Sounds like multicast DNS (mDNS) to me.

https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/protocols/mdns.html
https://lastminuteengineers.com/esp32-mdns-tutorial/

[ProfHarry]

Sounds like multicast DNS (mDNS) to me.

I tried to use mDNS but :
- you need to know the name
- it doesn't work on all browsers/OS

[bobby100]

Using blcksock from Synapse. ListIP contains the IP addresses I want to check. It is fast.

Code: Pascal  [Select][+][-]

1. var
2.   ipSeg: integer;
3.   sock: TTCPBlockSocket;
4.   ip: string;
5.  
6. begin
7.     sock := TTCPBlockSocket.Create;
8.     for ipSeg := 0 to ListIP.Count - 1 do
9.     begin
10.       ip := ListIP[ipSeg];
11.       sock.ConnectionTimeout := 100;
12.       sock.Connect(ip, '80');
13.       if sock.GetRemoteSinIP = ip then
14.       begin
15.         //I have found a http server on this address
16.       end;
17.       sock.CloseSocket;
18.     end;
19.     sock.Free;
20. end;

[MarkMLl]

I have a set of ESP32 microcontrollers which are HTTP servers and DHCP clients on a wifi network.
The only user interface is the web page, so it is difficult for the user to know the IP addresses of the various microcontrollers. In general, the DHCP server and the list of leases are not accessible to users.

In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes). Having done that, investigate linking a local DNS server to DHCP allocations so that the names are fixed.

Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

If you really do have to scan, since the DHCP server will be allocating an RFC1918 address from a fairly small range, and since you know what ports (plus probably a ping responder) each ESP32 is exposing, scanning shouldn't be that slow particularly if you run say 16 threads in parallel... you could obviously induct your students into the mysteries of e.g. Nmap thus setting them up for a career of hac^H^H^H honourable curiosity.

I've done this sort of thing in the context of having a known IP address (allocated to a drone) and finding what ports responded in a specific way (correctly-structured messages with a non-standard checksum), and the key is (a) to restrict your targets as far as possible and (b) to understand things like MSG_DONTWAIT (applied to fpRecv etc.) and the no-linger option.

MarkMLl

[ProfHarry]

In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes)

End users (teachers) don't have access to the DHCP server (BOFH)

Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

Nice and simple ! Thanks

[MarkMLl]

In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes)

End users (teachers) don't have access to the DHCP server (BOFH)

One of the hats I've worn in the past was building a university lab network: this was early-80s and you can work out the details from my sig.

I would expect that your local BOFH would welcome being told that you'll be running sessions with lots of transient connections from non-PC equipment, since his job will be far easier if he could reserve an address subrange for these devices. Few things are really quite as irritating as assuming that you've kept a few addresses reserved for servers, and then finding that your assumption is unfounded and various unknown systems are living there.

Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

Nice and simple ! Thanks

I've not tried this, but a UDP broadcast of a zero-length message to port zero might work. UDP broadcasts don't normally get routed, zero-length messages are usually dropped by firewalls, and because it's a broadcast any device on the same network segment should be able to view it on Wireshark. /However/, assuming that you're using WiFi that does make the assumption that the access point is set up so that local devices can see each other.

If you look at that, please let me know how you get on.

MarkMLl