[Solved] Lazarus 2.2.4, Indy 10 (latest), OSX Ventura 13.4 on an M1 iMac

[JimKueneman]

I have a component set I created for my use.  I converted to Indy because Synapse was not working in 64 Bit OSX and did not look like it was very active.  I did this late last year on this M1 Mac and I swear it was working back then.  I am trying to work on this project again and I can't get TCP clients to get to the TCP server running on the M1. 

If I run the client on the M1 OSX (same IP) then I can connect no problem with clients that come from various sources including an Indy based one.  If I run the client from a VMWare operating system (different IP address but on the same local domain) or from an iPhone on the local domain with a Delphi app I wrote and a Swift app from a friend it never detects a connection.  I have turned the firewall off as well as select the app for incoming with it on. 

I can run a different server application for this port (it was written in Java) and I can connect no problem from the iOS apps and the VMWare other OS clients so it does not seem like it is an underlying firewall issue as I did not add that application to the firewall list and it worked.

When I look in WireShark I can see the first connection frame coming from all these client apps but my Lazarus/Indy app never responds at all.

I swear this was working last fall with the VMWare clients and the iOS apps but it is not now.  It is driving me nuts...  I also wrote a simple program in Lazarus creating a IdTCPServer (eliminating all my higher level wrappers in my components) and that will not connect with anything but a server running on the same IP address.

Does anyone have any idea where to look?
Jim

[JimKueneman]

So I compiled the simple app in Delphi and compiled for M1 and ran it in OSX.  When trying to connect it threw a ton of exceptions and never connected.  I now recall trying to get it to work on a Raspberry Pi and I could not get the server to work either so I am starting to think there is something not quite right with the ARM code in Indy (there are a number of ARM only conditional defines I stepped through in Delphi)

[Remy Lebeau]

I don't know what to tell you.  Indy uses standard Posix socket functions, and doesn't do anything special for M1, so it should work, I think.  You will just have to debug into Indy's code to see where the failure is actually happening.

[JimKueneman]

I don't know what to tell you.  Indy uses standard Posix socket functions, and doesn't do anything special for M1, so it should work, I think.  You will just have to debug into Indy's code to see where the failure is actually happening.

Spent a few hours digging around trying to figure it out.  So I found a couple things

1) I was using a bad example from somewhere and I was creating 2 bindings instead of one for the listener....

What I am doing now

Code: Pascal  [Select][+][-]

1.    IdSocketHandle := IdTCPServer.Bindings.Add;
2.    IdSocketHandle.Port := 12021;
3.    IdSocketHandle.IP := (ConnectionInfo as TLccEthernetConnectionInfo).ListenerIP;  
4.  

What I was doing based on an example from somewhere..

Code: Pascal  [Select][+][-]

1.    IdTCPServer.Bindings.Add.Port := 12021;
2.    IdTCPServer.Bindings.Add.IP := (ConnectionInfo as TLccEthernetConnectionInfo).ListenerIP;  
3.  

So connection was made when the client was local on the correct Port but the IP was zero

Then I forgot last fall about this:

Code: Pascal  [Select][+][-]

1. procedure TIdThread.Cleanup;
2. var
3.   LScheduler: TIdScheduler;
4.   LList: TIdYarnList;
5. begin
6.   Exclude(FOptions, itoReqCleanup);
7.  
8.   // RLebeau 9/20/2019: there is a race condition here with TIdScheduler.TerminateAllYarns().
9.   // Notify TIdScheduler of the Yarn being freed here, otherwise, a double free of the Yarn
10.   // can happen if TIdThread.Cleanup() and TIdSchedulerOfThread.TerminateYarn() try to destroy
11.   // the Yarn at the same time.  TerminateYarn() destroys the Yarn inside the ActiveYarns lock,
12.   // so the destroy here needs to be done inside of the same lock...
13.  
14.   //IdDisposeAndNil(FYarn);
15.   if FYarn is TIdYarnOfThread then
16.   begin
17.     LScheduler := TIdYarnOfThreadAccess(FYarn).FScheduler;  << If you don't have checking off this throws and exception....
18.    if Assigned(LScheduler) then
19.    begin
20.      LList := LScheduler.ActiveYarns.LockList;
21.      try
22.        // if the Yarn is still in the list, remove and destroy it now.
23.        // If not, assume TIdScheduler has already done so ...
24.        if LList.Remove(FYarn) <> -1 then begin
25.          IdDisposeAndNil(FYarn);
26.        end;
27.      finally
28.        LScheduler.ActiveYarns.UnlockList;
29.      end;
30.    end;
31.  

Is there anyway to do something to the code to allow procedure checking to be enabled?

I had high hopes after finding this as now I had only one binding with the right IP and Port but no joy... 

The Select function is called with the correct binding handle but it never returns that it was triggered when I try to connect from some other local IP address...

Jim

[JimKueneman]

Can anyone take a look at this and tell me if I am doing something wrong... simple server application.   If you change the path to your Indy install folders you can run it.  I don't install the packages I always dynamically create components.

Thanks
Jim

[Remy Lebeau]

1) I was using a bad example from somewhere and I was creating 2 bindings instead of one for the listener....

Yes, that was wrong.  A lot of those bad examples are floating around.

What I am doing now

That is correct.

Then I forgot last fall about this:
...
Is there anyway to do something to the code to allow procedure checking to be enabled?

I don't understand what "procedure checking" is.  Please clarify.

The Select function is called with the correct binding handle but it never returns that it was triggered when I try to connect from some other local IP address...

If select() is not triggering then it means the OS is not seeing any clients trying to handshake with that socket's IP/Port.  Use a packet sniffer like Wireshark to verify.

If you bind a listener to a specific local IP, clients can connect only to that IP.  If it is a local loopback IP, then the clients have to come from that same IP.  But if it is a network accessible IP, then the clients can come from any IP that is on the same network.

If you bind the listener to a blank IP (aka, '0.0.0.0' for IPv4 and '::0' for IPv6), then clients can connect to any IP that belongs to the server machine, and they can come from any IP that is on the local machine or the network.

This is basic TCP 101 stuff, it is not limited to just Indy.

[JimKueneman]

 The runtime error checking of type casts in the debugging dialog. 

Says this is a bad typecast.

LScheduler := TIdYarnOfThreadAccess(FYarn).FScheduler;

Jim

[Remy Lebeau]

The runtime error checking of type casts in the debugging dialog. 

Says this is a bad typecast.

LScheduler := TIdYarnOfThreadAccess(FYarn).FScheduler;

It is a perfectly fine typecast in this context.  FYarn is a pointer to a TIdYarn object.  TIdYarnOfThread derives from TIdYarn, and TIdYarnOfThreadAccess derives from TIdYarnOfThread.  FScheduler is a protected member of TIdYarnOfThread, hence the use of an accessor class to reach it.  This works perfectly fine, is a pretty common practice in Delphi, and there are quite a number of similar accessor classes used in Indy.

Is this the -CR (Verify method calls) compiler option at work?  From what I've read elsewhere, that option can actually silently convert an object hard-cast into a call to the as operator, which would be very bad in this case, since the as operator is not compatible with accessor classes. FYarn does not point at an object which actually derives from TIdYarnOfThreadAccess, so the as operator will fail such a cast.  An accessor is just meant to bring protected members into scope of the calling unit.  This silent behavior would break every other accessor class being used in Indy.  That compiler option needs to be disabled if it is enabled.  I'll have to see if there is a way to detect/disable it in code.

[JimKueneman]

Yes, back at the computer with Lazarus, yes the -CR option.  That would be great, the exact exception is:

Project "projectxxx" raised exception class EInvalidCast with message: Invalid type cast At address 100015BE0

Jim

[JimKueneman]

So it turns out other have had issues with OSX locking down all ports (other than what the OS needs) and not being able to get them unblocked... 

[Remy Lebeau]

Yes, back at the computer with Lazarus, yes the -CR option.

I was never aware of that option before.  If it's converting object type-casts into the as operator, that is going to wreak havoc with several areas of Indy's internals.  Not good.

All I can suggest right now is, do not compile Indy with -CR enabled.  But, since you are not installing packages, you are just compiling Indy directly into your project, that might not be a viable option for you.

I'll have to figure out a workaround for this.  In the meantime, I've opened a ticket for Indy about this issue:

#492: EInvalidCast in Freepascal/Lazarus when -CR option is enabled

[JimKueneman]

Thank you Remy... Yes if I uncheck it without packages it still solves the issue but it would be nice to leave it on for checking other code development.

Jim

[JimKueneman]

Well that was painful... needed to write a rule for the packet filter (pf) to unblock the port for the TCP connection... I could not get it to be not blocked any other way.

[Remy Lebeau]

Thank you Remy... Yes if I uncheck it without packages it still solves the issue but it would be nice to leave it on for checking other code development.

Can you do the following test for me?  On the line that is raising the error, change this:

Code: [Select]

LScheduler := TIdYarnOfThreadAccess(FYarn).FScheduler;
To this instead:

Code: [Select]

{$push}
{$objectChecks off}
LScheduler := TIdYarnOfThreadAccess(FYarn).FScheduler;
{$pop}
And see if the problem goes away.  If it does, then I can apply that workaround to everywhere else that Indy is using similar accesses.

[JimKueneman]

Yes that solves it. I had to make that same change here:

Code: Pascal  [Select][+][-]

1. procedure TIdSchedulerOfThreadDefault.ReleaseYarn(AYarn: TIdYarn);
2. //only gets called from YarnOf(Fiber/Thread).Destroy
3. var
4.   LThread: TIdThreadWithTask;
5. begin
6.   //take posession of the thread
7.   LThread := TIdYarnOfThread(AYarn).Thread;
8.   {$push}
9.   {$objectChecks off}
10.   TIdYarnOfThreadAccess(AYarn).FThread := nil;
11.   {$pop}
12.   //Currently LThread can =nil. Is that a valid condition?
13.   //Assert(LThread<>nil);    
14.  

Then I could disconnect without an exception.