Recent

Author Topic: IBX 2.3.3 a tale of two servers  (Read 2172 times)

RedOctober

  • Sr. Member
  • ****
  • Posts: 450
IBX 2.3.3 a tale of two servers
« on: August 24, 2019, 05:46:18 pm »
Server 1: Named JLD1
Used for: Software Development. Current project called "RUBI"
Platform:  Windows Server 2016, Firebird 3.0.4.33054
Dev Tools:  Lazarus 1.8.4 64 bit, FPC 3.0.4, IBX 2.3.3
Other: Has Delphi XE3 installed, 32 bit. IBDAC by DevArt, was used to build app called "JET"
Status:  My app is bullet proof. Never crashes when running within IDE or as stand alone .exe

Server 1: Named JLS1
Used for: Production server for about 20 users of "JET", also has on-going versions of RUBI for test
Platform:  Windows Server 2016, Firebird 3.0.4.33054  64bit
Dev Tools:  None
Other: Citrix, WiFi manager software
Status: JET runs bullet proof. RUBI always crashes sometimes immediately, some times after clicking around to different tabs that cause data fetching.  The error is most often "Unable to write data to the connection", and after that, "Access Error".

The two servers are hardware identical.  Another Lazarus built application called FireBolt, built with Laz 1.8.2 and IBX 2.3.0 works perfectly on both servers.

I have checked:
- Ensured DB is not corrupt. Ran a backup from Dev machine, restore to Production machine, then ran a full scope Validate.  No errors found.
- Ensured fbclient.dll is the correct version and bitness, even explicitly set the TIBDatabase.FirebirdLibraryPathName property to the correct fbclient.dll using a .ini file setting to point to a copy of fbclient.dll in the same folder as the .exe

(Setting: from the .ini file:
[DBConnSets]
FBClientDll_Path=C:\Program Files\Firebird\Firebird_3_0\fbclient.dll
Database=JLD1/3050:RUBI
);
(JLD1 is a path set in databases.conf

#
# Live Databases:
#
RUBI = D:\app_vlt\db\fdb\RUBI_TXT.FDB
RBLC = D:\app_vlt\db\fdb\RUBI_BLC.FDB
JET = D:\app_vlt\db\fdb\JET_TXT.FDB
)
- RUBI has 3 TIBDatabase components and all have their .FirebirdLibraryPathName property to the same fbclient.dll.  Two of the TIBDatabase components are used only for log-in purposes and are closed once the user is logged in.  After that, only one TIBDatabase component is used for all database access.
- Ensured that the character set of the TIBDatabase is set correctly to match the default character set of the DB, which is WIN1252  (using LC_CTYPE=WIN1252)

Both Firebird installations are the same architecture, but I forget which I used. I can find no documentation on how to determine what FB architecture is installed.  If it was FB SuperServer installed then FBguardian wd also be installed and running in Windows Services, but it's not in the Services list, so I don't think I used SuperServer.  I used either SuperClassic or Classic instead.

Here is a listing of error messages from the firebird.log I found in the folder with the fbclient.dll:


JLS1   Fri Aug 23 22:44:34 2019
   INET/inet_error: read errno = 10054, aux server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/55461


JLS1   Fri Aug 23 22:44:34 2019
   INET/inet_error: send errno = 10054, server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/3050


JLS1   Fri Aug 23 22:45:54 2019
   INET/inet_error: read errno = 10054, aux server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/55469


JLS1   Fri Aug 23 22:45:54 2019
   INET/inet_error: send errno = 10054, server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/3050




I need help with determining the cause of the "Unable to write data to the connection" error.  This is a show-stopper bug and my development is stalled until I can prove that RUBI will be as reliable as JET on the production server.

It "feels" very much like a DLL problem, but as far as I can tell, I'm using the correct one and pointing to it and loading the correct one.  I can't determine what is different between the two servers that causes IBX to fail 80% of the time within 30 seconds, and fail 100% of the time within 5 minutes of use.


Thank you in advance for any help you can provide:  I may need Tony W.  himself to have a look.



« Last Edit: August 24, 2019, 06:11:53 pm by RedOctober »

RedOctober

  • Sr. Member
  • ****
  • Posts: 450
Re: IBX 2.3.3 a tale of two servers
« Reply #1 on: August 24, 2019, 07:03:56 pm »
Following a suggestion from Tony W. (via email response to my original email to him) I did the following:

- Add rubi_core.exe (my application .exe name) to the file exclusion list of Windows Defender on the Windows Server 2016 JLS1 server.  So far, I have been unable to re-create the error.  RUBI on JLS1 seems stable now.  I'll test further during the week, when JLS1 is more heavily loaded.

Why JLD1 is able to reliably run rubi_core.exe WITHOUT the Windows Defender Exclusion is a mystery.

rubi_core.exe has two DB transactions in immediate succession. They are:

- Query a small "change recording table" of about 20 records, to see if a "pull" (SELECT...) of the larger table is required.  If no changes have taken place since the last pull, don't do another.  If a change is detected, pull another subset from the larger table.  The whole I idea of this is to *not* update TDBGrids from big tables if it's not necessary.  My users will typically bounce between tabs within a time span of about 1 second to 3 minutes.  Always refreshing the grids, regardless of need (like the way JET does it) is too slow.

I have noticed that in RUBI, I can now see a tiny "SQL Wait" cursor flash for about 100 milliseconds between tab changes.  That did not happen before I excluded the .exe in Windows Defender.  RUBI wd just crash.  So, maybe I just solved this problem with Tony's generous help.  I will come back here in a panic if the "Unable to send data to connection" error rears it's ugly head again.

« Last Edit: August 24, 2019, 07:05:51 pm by RedOctober »

valdir.marcos

  • Hero Member
  • *****
  • Posts: 1106
Re: IBX 2.3.3 a tale of two servers
« Reply #2 on: August 24, 2019, 07:29:09 pm »
Server 1: Named JLD1
Used for: Software Development. Current project called "RUBI"
Platform:  Windows Server 2016, Firebird 3.0.4.33054
Dev Tools:  Lazarus 1.8.4 64 bit, FPC 3.0.4, IBX 2.3.3
Other: Has Delphi XE3 installed, 32 bit. IBDAC by DevArt, was used to build app called "JET"
Status:  My app is bullet proof. Never crashes when running within IDE or as stand alone .exe

Server 1: Named JLS1
Used for: Production server for about 20 users of "JET", also has on-going versions of RUBI for test
Platform:  Windows Server 2016, Firebird 3.0.4.33054  64bit
Dev Tools:  None
Other: Citrix, WiFi manager software
Status: JET runs bullet proof. RUBI always crashes sometimes immediately, some times after clicking around to different tabs that cause data fetching.  The error is most often "Unable to write data to the connection", and after that, "Access Error".

The two servers are hardware identical.  Another Lazarus built application called FireBolt, built with Laz 1.8.2 and IBX 2.3.0 works perfectly on both servers.

I have checked:
- Ensured DB is not corrupt. Ran a backup from Dev machine, restore to Production machine, then ran a full scope Validate.  No errors found.
- Ensured fbclient.dll is the correct version and bitness, even explicitly set the TIBDatabase.FirebirdLibraryPathName property to the correct fbclient.dll using a .ini file setting to point to a copy of fbclient.dll in the same folder as the .exe

(Setting: from the .ini file:
[DBConnSets]
FBClientDll_Path=C:\Program Files\Firebird\Firebird_3_0\fbclient.dll
Database=JLD1/3050:RUBI
);
(JLD1 is a path set in databases.conf

#
# Live Databases:
#
RUBI = D:\app_vlt\db\fdb\RUBI_TXT.FDB
RBLC = D:\app_vlt\db\fdb\RUBI_BLC.FDB
JET = D:\app_vlt\db\fdb\JET_TXT.FDB
)
- RUBI has 3 TIBDatabase components and all have their .FirebirdLibraryPathName property to the same fbclient.dll.  Two of the TIBDatabase components are used only for log-in purposes and are closed once the user is logged in.  After that, only one TIBDatabase component is used for all database access.
- Ensured that the character set of the TIBDatabase is set correctly to match the default character set of the DB, which is WIN1252  (using LC_CTYPE=WIN1252)

Both Firebird installations are the same architecture, but I forget which I used. I can find no documentation on how to determine what FB architecture is installed.  If it was FB SuperServer installed then FBguardian wd also be installed and running in Windows Services, but it's not in the Services list, so I don't think I used SuperServer.  I used either SuperClassic or Classic instead.

Here is a listing of error messages from the firebird.log I found in the folder with the fbclient.dll:


JLS1   Fri Aug 23 22:44:34 2019
   INET/inet_error: read errno = 10054, aux server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/55461


JLS1   Fri Aug 23 22:44:34 2019
   INET/inet_error: send errno = 10054, server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/3050


JLS1   Fri Aug 23 22:45:54 2019
   INET/inet_error: read errno = 10054, aux server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/55469


JLS1   Fri Aug 23 22:45:54 2019
   INET/inet_error: send errno = 10054, server host = JLS1, address = fe80::34c3:aee1:b467:95fc%6/3050




I need help with determining the cause of the "Unable to write data to the connection" error.  This is a show-stopper bug and my development is stalled until I can prove that RUBI will be as reliable as JET on the production server.

It "feels" very much like a DLL problem, but as far as I can tell, I'm using the correct one and pointing to it and loading the correct one.  I can't determine what is different between the two servers that causes IBX to fail 80% of the time within 30 seconds, and fail 100% of the time within 5 minutes of use.


Thank you in advance for any help you can provide:  I may need Tony W.  himself to have a look.

Following a suggestion from Tony W. (via email response to my original email to him) I did the following:

- Add rubi_core.exe (my application .exe name) to the file exclusion list of Windows Defender on the Windows Server 2016 JLS1 server.  So far, I have been unable to re-create the error.  RUBI on JLS1 seems stable now.  I'll test further during the week, when JLS1 is more heavily loaded.

Why JLD1 is able to reliably run rubi_core.exe WITHOUT the Windows Defender Exclusion is a mystery.

rubi_core.exe has two DB transactions in immediate succession. They are:

- Query a small "change recording table" of about 20 records, to see if a "pull" (SELECT...) of the larger table is required.  If no changes have taken place since the last pull, don't do another.  If a change is detected, pull another subset from the larger table.  The whole I idea of this is to *not* update TDBGrids from big tables if it's not necessary.  My users will typically bounce between tabs within a time span of about 1 second to 3 minutes.  Always refreshing the grids, regardless of need (like the way JET does it) is too slow.

I have noticed that in RUBI, I can now see a tiny "SQL Wait" cursor flash for about 100 milliseconds between tab changes.  That did not happen before I excluded the .exe in Windows Defender.  RUBI wd just crash.  So, maybe I just solved this problem with Tony's generous help.  I will come back here in a panic if the "Unable to send data to connection" error rears it's ugly head again.
Long running state-full transactions on wireless connections might become a big problem for any RDBMS.
So, having Network SysAdmin, RDBMS DBA and Developing Team working together is always recommended.

RedOctober

  • Sr. Member
  • ****
  • Posts: 450
Re: IBX 2.3.3 a tale of two servers
« Reply #3 on: August 24, 2019, 08:28:08 pm »
You make a good point Valdir.  I threw in the fact that JLS1 has WiFi on it just for completeness. There are no database clients moving database data over the wireless.  It's there for a separate file transfer process done by tablets tablets and ES File Explorer.

rvk

  • Hero Member
  • *****
  • Posts: 6056
Re: IBX 2.3.3 a tale of two servers
« Reply #4 on: August 24, 2019, 09:47:33 pm »
It's there for a separate file transfer process done by tablets tablets and ES File Explorer.
10054 and the error you mentioned still point to a connection issue.

Maybe related to this issue
https://support.microsoft.com/en-us/help/981344/an-application-may-receive-the-10054-error-when-the-application-receiv

Do you have any virusscanners running on the server or client?
(Likely cause)


<offtopic>
Did you know that ES File Explorer has been removed from the playstore due to some controversies about adware? As far as I know it hasn't come back.
https://www.androidpolice.com/2019/04/27/es-file-manager-vanishes-from-play-store-possibly-part-of-do-global-scandal/
</offtopic>

RedOctober

  • Sr. Member
  • ****
  • Posts: 450
Re: IBX 2.3.3 a tale of two servers
« Reply #5 on: August 24, 2019, 09:57:17 pm »
Hi rvk.  My problem definitely seems to be caused by a virus scanner.  In this case Windows Defender.  I added my .exe name to the Excluded list and now it seems to be working correctly.

About ES FM, thanks for the link.  The tablets were loaded years ago and are on a secure system that prevents any traffic from coming in or going out.  We also disconnect them from the Google Play Store.  No ads ever appear on it.  Since then one tablet has failed and I seem to remember ES was no longer available so the staff used the next best thing.. the name of which I forget at the moment.

RedOctober

  • Sr. Member
  • ****
  • Posts: 450
Re: IBX 2.3.3 a tale of two servers
« Reply #6 on: October 23, 2020, 11:29:40 pm »
I'm the original poster.  The error has returned, even worse this time.  Adding the .exe name to the "File" exclusions AND the "Process" exclusions in Windows Defender has not helped.  RUBI crashes so often it is unusable, at least via .Rdp over the internet, 10 KM away.  I have not yet done any extensive testing using RUBI over .Rdp within the clinic.


tonyw

  • Sr. Member
  • ****
  • Posts: 319
    • MWA Software
Re: IBX 2.3.3 a tale of two servers
« Reply #7 on: October 24, 2020, 10:06:12 am »
Has Windows updated anything relevant recently. This seems like a return of our old friend the "stealth downgrade".

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11352
  • FPC developer.
Re: IBX 2.3.3 a tale of two servers
« Reply #8 on: October 24, 2020, 12:42:52 pm »
In August, Microsoft disabled disabling Windows Defender by Group Policy.


RedOctober

  • Sr. Member
  • ****
  • Posts: 450
Re: IBX 2.3.3 a tale of two servers
« Reply #9 on: October 24, 2020, 11:52:01 pm »
Update:  I did some testing right at the console of JLS1.  RUBI crashes the same way as it does when using RemoteDekstop.  So the problem is with the port, nothing to do with RemoteDesktop.

 

TinyPortal © 2005-2018