Recent

Author Topic: how to scan a network ?  (Read 1174 times)

ProfHarry

  • New member
  • *
  • Posts: 9
how to scan a network ?
« on: May 19, 2024, 10:06:14 pm »
Hello,

I'm french so my english is not so good and my friend Deepl help me  ;-)

I have a set of ESP32 microcontrollers which are HTTP servers and DHCP clients on a wifi network.
The only user interface is the web page, so it is difficult for the user to know the IP addresses of the various microcontrollers. In general, the DHCP server and the list of leases are not accessible to users.

Each ESP32 responds to the URL http://IP/info.txt with a line containing, among other things, its IP address and hostname.

I therefore tried to develop a Lazarus application to scan the network and make this request to all the addresses.
But it's very, very slow... Because of the timeout for most addresses, it takes more than 10 minutes to scan an entire /24 network. I can't even imagine on a /16 network.

I use this code called in a loop on all IP addresses:

Code: Pascal  [Select][+][-]
  1. function GetInfos(IP : string) : string;
  2. Var
  3.   URL : String;
  4.   SL : TStringList;
  5. begin
  6.   URL := 'http://' + IP + '/infos.txt';
  7.   SL := TStringList.Create();
  8.  
  9.   try
  10.     with TFPHttpClient.Create(Nil) do
  11.       try
  12.         AllowRedirect := True;
  13.         Get(URL,SL);
  14.         application.processmessages;
  15.       finally
  16.         Free;
  17.       end;
  18.     Result := SL[0];
  19.   finally
  20.     SL.Free;
  21.   end;
  22. end;
  23.  

Do you see a solution to speed up the scan and make it possible on a larger network?
Thanks for your help.
« Last Edit: May 19, 2024, 10:09:21 pm by ProfHarry »

alpine

  • Hero Member
  • *****
  • Posts: 1132
Re: how to scan a network ?
« Reply #1 on: May 19, 2024, 11:16:24 pm »
Couldn't you just send an UDP broadcast packet with the server IP and then all nodes to reply with their info back?
This is pretty trivial, I've done it before but not in FPC ie. don't have a source to show it.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

ProfHarry

  • New member
  • *
  • Posts: 9
Re: how to scan a network ?
« Reply #2 on: May 19, 2024, 11:46:38 pm »
Yes, it's a great idea! Instead of searching for ESP32s from the lazarus program, I wait for the ESP32s to show up.

It's simpler in principle. All that's left is to implement it.

dsiders

  • Hero Member
  • *****
  • Posts: 1147
Re: how to scan a network ?
« Reply #3 on: May 20, 2024, 12:13:10 am »
Yes, it's a great idea! Instead of searching for ESP32s from the lazarus program, I wait for the ESP32s to show up.

It's simpler in principle. All that's left is to implement it.

Sounds like multicast DNS (mDNS) to me.

https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/protocols/mdns.html
https://lastminuteengineers.com/esp32-mdns-tutorial/
Preview Lazarus 3.99 documentation at: https://dsiders.gitlab.io/lazdocsnext

ProfHarry

  • New member
  • *
  • Posts: 9
Re: how to scan a network ?
« Reply #4 on: May 20, 2024, 12:20:12 am »
Sounds like multicast DNS (mDNS) to me.

I tried to use mDNS but :
- you need to know the name
- it doesn't work on all browsers/OS

bobby100

  • Full Member
  • ***
  • Posts: 196
    • Malzilla
Re: how to scan a network ?
« Reply #5 on: May 20, 2024, 01:25:43 am »
Using blcksock from Synapse. ListIP contains the IP addresses I want to check. It is fast.
Code: Pascal  [Select][+][-]
  1. var
  2.   ipSeg: integer;
  3.   sock: TTCPBlockSocket;
  4.   ip: string;
  5.  
  6. begin
  7.     sock := TTCPBlockSocket.Create;
  8.     for ipSeg := 0 to ListIP.Count - 1 do
  9.     begin
  10.       ip := ListIP[ipSeg];
  11.       sock.ConnectionTimeout := 100;
  12.       sock.Connect(ip, '80');
  13.       if sock.GetRemoteSinIP = ip then
  14.       begin
  15.         //I have found a http server on this address
  16.       end;
  17.       sock.CloseSocket;
  18.     end;
  19.     sock.Free;
  20. end;

MarkMLl

  • Hero Member
  • *****
  • Posts: 6872
Re: how to scan a network ?
« Reply #6 on: May 20, 2024, 09:25:04 am »
I have a set of ESP32 microcontrollers which are HTTP servers and DHCP clients on a wifi network.
The only user interface is the web page, so it is difficult for the user to know the IP addresses of the various microcontrollers. In general, the DHCP server and the list of leases are not accessible to users.

In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes). Having done that, investigate linking a local DNS server to DHCP allocations so that the names are fixed.

Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

If you really do have to scan, since the DHCP server will be allocating an RFC1918 address from a fairly small range, and since you know what ports (plus probably a ping responder) each ESP32 is exposing, scanning shouldn't be that slow particularly if you run say 16 threads in parallel... you could obviously induct your students into the mysteries of e.g. Nmap thus setting them up for a career of hac^H^H^H honourable curiosity.

I've done this sort of thing in the context of having a known IP address (allocated to a drone) and finding what ports responded in a specific way (correctly-structured messages with a non-standard checksum), and the key is (a) to restrict your targets as far as possible and (b) to understand things like MSG_DONTWAIT (applied to fpRecv etc.) and the no-linger option.

MarkMLl
« Last Edit: May 20, 2024, 10:54:58 am by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

ProfHarry

  • New member
  • *
  • Posts: 9
Re: how to scan a network ?
« Reply #7 on: May 20, 2024, 02:30:08 pm »
In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes)

End users (teachers) don't have access to the DHCP server (BOFH)

Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

Nice and simple ! Thanks

MarkMLl

  • Hero Member
  • *****
  • Posts: 6872
Re: how to scan a network ?
« Reply #8 on: May 20, 2024, 04:20:59 pm »
In that case set up the DHCP server to allocate a fixed IP address to each ESP32 based on the MAC address, or at the very least reserve a block for all MAC addresses from that given manufacturer (indicated by the first few bytes)

End users (teachers) don't have access to the DHCP server (BOFH)

One of the hats I've worn in the past was building a university lab network: this was early-80s and you can work out the details from my sig.

I would expect that your local BOFH would welcome being told that you'll be running sessions with lots of transient connections from non-PC equipment, since his job will be far easier if he could reserve an address subrange for these devices. Few things are really quite as irritating as assuming that you've kept a few addresses reserved for servers, and then finding that your assumption is unfounded and various unknown systems are living there.

Quote
Another approach would be to put something at the start of each ESP32 program that sent a dummy UDP message or possibly a ping to some other machine which could list what it saw.

Nice and simple ! Thanks

I've not tried this, but a UDP broadcast of a zero-length message to port zero might work. UDP broadcasts don't normally get routed, zero-length messages are usually dropped by firewalls, and because it's a broadcast any device on the same network segment should be able to view it on Wireshark. /However/, assuming that you're using WiFi that does make the assumption that the access point is set up so that local devices can see each other.

If you look at that, please let me know how you get on.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018