IBX 2.3.2 is now available for download

[tonyw]

There is more to the Wire Compression issue than I first thought. The Firebird documentation is confusing and needed some interpretation and testing.

Points of note:

1. My initial post was correct in itself and setting "WireCompression = true" in both client and Server "firebird.conf" files does force wire compression to be on.

2. Adding an isc_dpb_config item to the connection DPB set to "WireCompression=true" will force wire compression from the client side event when the local "firebird.conf" does not include "WireCompression = true".

3. If have no idea where the suggestion came from that executing

select RDB$GET_CONTEXT('SYSTEM', 'WIRE_COMPRESSED') as st from rdb$database

will do anything other than return an error message.

4. According to the "readme" with the Firebird source

"When compression is turned on Z flag is shown in client/server version info – for example: LI-T3.0.0.31451 Firebird 3.0 Beta 1/tcp (fbs)/P13:Z."

I had to search the Firebird-devel list to understand what this meant. In a very useful post back in 2016, the implementor (of wire compression) said "This is requested by isc_version() in legacy API and IUtil::getFbVersion() in new one."

5. I have as a result added the following function to the Firebird Pascal API IAttachment interface:

procedure getFBVersion(version: TStrings);

This seems to be the only way to check whether wire compression is in use, and returns the client/server version information.

6. Referring to item 2 above, adding "WireCompresion=true" to the TIBDatabase params is not going to be a natural way for the client to request wire compression. I have thus added a new published property to TIBDatabase (and to TIBXServicesConnection). This is

WireCompression: boolean;

If this property is set to true then WireCompression will be requested by the client. Note: it is still necessary to include "WireCompression = true" in the firebird.conf file on the server.

7. For an example of how to determine if wire compression is in used, see the ibx/examples/dbInfo example. This prints out the result of calling getFBVersion as part of the database information. Note: this information is the same as IBSQL returns with the "-z" option.

An updated version of IBX incorporating the above is now available from svn. Note: documentation update is to follow. Checkout using

svn co https://svn.mwasoftware.co.uk/public/ibx/trunk

[dsiders]

There is more to the Wire Compression issue than I first thought. The Firebird documentation is confusing and needed some interpretation and testing.

Hi, Tony.

Thanks for your hard work. I'd be glad to help in testing the changes.

3. If have no idea where the suggestion came from that executing

select RDB$GET_CONTEXT('SYSTEM', 'WIRE_COMPRESSED') as st from rdb$database

will do anything other than return an error message.

I'm the source of that misconception. I found it in a StackOverflow conversation about using WireCompression in FireDAC. If I had read a little closer, I would have seen that it is a feature added in Firebird 4.X. I assume to provide a better way of checking it than using the version information. My bad...

An updated version of IBX incorporating the above is now available from svn. Note: documentation update is to follow. Checkout using

svn co https://svn.mwasoftware.co.uk/public/ibx/trunk

Hmm, guess I need to brush some dust off of the SVN skills I haven't used for 15 years. Is Tortoise still the SVN client of choice for Windows?

[tonyw]

There is more to the Wire Compression issue than I first thought. The Firebird documentation is confusing and needed some interpretation and testing.

Hi, Tony.

Thanks for your hard work. I'd be glad to help in testing the changes.

3. If have no idea where the suggestion came from that executing

select RDB$GET_CONTEXT('SYSTEM', 'WIRE_COMPRESSED') as st from rdb$database

will do anything other than return an error message.

I'm the source of that misconception. I found it in a StackOverflow conversation about using WireCompression in FireDAC. If I had read a little closer, I would have seen that it is a feature added in Firebird 4.X. I assume to provide a better way of checking it than using the version information. My bad...

An updated version of IBX incorporating the above is now available from svn. Note: documentation update is to follow. Checkout using

svn co https://svn.mwasoftware.co.uk/public/ibx/trunk

Hmm, guess I need to brush some dust off of the SVN skills I haven't used for 15 years. Is Tortoise still the SVN client of choice for Windows?

Tortoise works but I prefer to use the command line svn for Windows.

[incendio]

As I wrote in http://forum.lazarus-ide.org/index.php?topic=43682.0, I use Firebird 3.0.4 to run this command :

Code: Pascal  [Select][+][-]

1. select RDB$GET_CONTEXT('SYSTEM', 'WIRE_COMPRESSED') as st from rdb$database

Previous version, will return an error.

I have IBDAC + C++ Builder, this IBDAC support WireCompression and it returns TRUE.

I also test it in action, retrieved about 10K rows by 15 columns via internet. zlib1.dll location is the same wtih app, performance difference between with WireCompression and without WireCompression, is quite noticeable.
 

[dsiders]

I finally got my install updated to IBX-trunk, and tested with the DBInfo demo.

Success!

Code: [Select]

Firebird/Windows/AMD/Intel/x64 (access method), version "WI-V3.0.4.33054 Firebird 3.0"
Firebird/Windows/AMD/Intel/x64 (remote server), version "WI-V3.0.4.33054 Firebird 3.0/tcp (HP17Z)/P15:CZ"
Firebird/Windows/AMD/Intel/x64 (remote interface), version "WI-V3.0.4.33054 Firebird 3.0/tcp (HP17Z)/P15:CZ"

But it did take a bit of fiddling.

I modified the firebird.conf file for the server to include

Code: [Select]

WireCompression = true.

I also copied zlib1.dll to the directory for the project (dbinfo). I did this because (thanks to Windows DLL-Hell) I already have four other versions of zlib1.dll on my system. I was afraid adding another version to the system path would hose one of the other applications.

I changed the WireCompression property in my TIBDatabase instance to True.

It will not actually use zlib compression until you add zlib to the uses clause in your app. I tried both the .lpk file and the .pas where the TIBDatabase unit resides. Both worked.

Zlib compression cannot be enabled for local connections (XNET). It works for IP addresses (INET). Haven't tried NETBEUI (WNET) yet.

[dsiders]

As I wrote in http://forum.lazarus-ide.org/index.php?topic=43682.0, I use Firebird 3.0.4 to run this command :
 

Code: Pascal  [Select][+][-]

1. select RDB$GET_CONTEXT('SYSTEM', 'WIRE_COMPRESSED') as st from rdb$database

I can confirm that this works on Firebird 3.0.4. Documented in Firebird doc/sql.extensions/README.context_variables2.txt.

[tonyw]

I finally got my install updated to IBX-trunk, and tested with the DBInfo demo.

Success!

Code: [Select]

Firebird/Windows/AMD/Intel/x64 (access method), version "WI-V3.0.4.33054 Firebird 3.0"
Firebird/Windows/AMD/Intel/x64 (remote server), version "WI-V3.0.4.33054 Firebird 3.0/tcp (HP17Z)/P15:CZ"
Firebird/Windows/AMD/Intel/x64 (remote interface), version "WI-V3.0.4.33054 Firebird 3.0/tcp (HP17Z)/P15:CZ"

But it did take a bit of fiddling.

I modified the firebird.conf file for the server to include

Code: [Select]

WireCompression = true.

I also copied zlib1.dll to the directory for the project (dbinfo). I did this because (thanks to Windows DLL-Hell) I already have four other versions of zlib1.dll on my system. I was afraid adding another version to the system path would hose one of the other applications.

I changed the WireCompression property in my TIBDatabase instance to True.

It will not actually use zlib compression until you add zlib to the uses clause in your app. I tried both the .lpk file and the .pas where the TIBDatabase unit resides. Both worked.

I have tried connecting from Windows to a Linux server. In this case, I had no problem enabling wire compression - and with no zlib included in the application. Maybe something about the location of your zlib. I have zlib1.dll in the same folder as the fbclient.dll (default Firebird install).

Zlib compression cannot be enabled for local connections (XNET). It works for IP addresses (INET). Haven't tried NETBEUI (WNET) yet.

[tonyw]

Looking again at the Firebird 3 release notes, it looks like I overlooked the extent of the use of the new DPB item code "isc_dpb_config". This is used at database connection time to provide a list of client side configuration overrides in the same syntax as "firebird.conf". Wire Compression is only one of the overrides that may be applied - e.g. use of Wire Encryption is another. I have added a new property to both TIBDatabase and TIBXServicesConnection to allow client side configuration overrides to be provided. This is now included in the development version of IBX which may be viewed at:

https://svn.mwasoftware.co.uk/viewvc/public/ibx/trunk/

To check out, install subversion and enter the following at a command line (should work on both Windows and Linux).

svn co https://svn.mwasoftware.co.uk/public/ibx/trunk/ ibx

Lazarus 2.0.0 RC3 is recommended for this update, but earlier versions should work.

The corresponding changelog entry is:

IBX Change Log  version (2.3-3 Build 11288) Fri, 28 Dec 2018 10:04:17 +0000

1. New property for TIBDatabase and TIBXServicesConnection. This is "ConfigOverrides"
   and is used to override the default settings in the client side "firebird.conf" This
   includes "WireCompression" and "WireCrypt". The value of this property is a "TStrings".
   Each line of the string list should be a setting in the same format as "firebird.conf".
   For example "WireCompression = true" in order to request wire compression on the client side.

   Note: explicit settings in the client side "firebird.conf" appear to take precedence
   over settings given in TIBDatabase and TIBXServicesConnection. Many client side settings
   also require a compatible setting in the server side "firebird.conf". For example
   "WireCompression = true" must also be set in the server's "firebird.conf" for wire
   compression to be used.

   Whether or not WireCompression or WireCrypt are in use can be checked using the information
   returned with the Client/Server protocol version. See ibx/examples/dbadmin and
   ibx/examples/dbinfo for examples of how to display this information.

2. New property for TIBDatabase and TIBXServicesConnection: "WireCompression". This
   is a "convenience" property. If checked then the setting "WireCompression=true" is added to the
   ConfigOverrides. If unchecked then this setting is removed.

Documentation should now be up-to-date. The TIBDatabase and TIBXServicesConnection property editors are also updated.

Note: with Firebird 3 before 3.0.4, the only way to check whether Wire Compression or encryption is in use is to get the Client/Server version and protocol information using the IAttachment interface method "getFBVersion", and inspect the second and third lines. Both the DBAdmin and DBInfo example applications provide an example of use. The information returned is the same as calling isql with the "-z" option. The second and third lines will end in "Z" is compression is used, "CZ" if both encryption and compression is in use and "C" if encryption only is in use.

[dsiders]

I also copied zlib1.dll to the directory for the project (dbinfo). I did this because (thanks to Windows DLL-Hell) I already have four other versions of zlib1.dll on my system. I was afraid adding another version to the system path would hose one of the other applications.

It will not actually use zlib compression until you add zlib to the uses clause in your app.

I have tried connecting from Windows to a Linux server. In this case, I had no problem enabling wire compression - and with no zlib included in the application. Maybe something about the location of your zlib. I have zlib1.dll in the same folder as the fbclient.dll (default Firebird install).

Updated to 2.3-3 Build 11288.

I tried it with zlib1.dll in the Firebird Library Path and in the application directory. Both worked. And neither required adding zlib to the uses clause.

My database server is on a LAN, so I can't really judge how Wire Compression is affecting throughput... but it is working.

[incendio]

I can test it Wire Encription performance on remote server, but how to download the entire folder ibx/trunk ?

And should old IBX 2.3.2 uninstall before install ver 2.3.3?

[dsiders]

I can test it Wire Encription performance on remote server, but how to download the entire folder ibx/trunk ?

I used the instructions in Reply # 22 above. Requires an SVN client. I used Tortoise SVN (for Windows) initially, but decided to use the command line client (Slik SVN).

And should old IBX 2.3.2 uninstall before install ver 2.3.3?

I always remove a previous version, completely scrub the directory structure, and re-install.  Just to avoid any issues. That may be overkill, but it works for me.

[incendio]

Have tested new feature WireCompression.

Data size is about 24K rows x 2 col (width = varchar(125)), loading took

• Compression Off : about 7 secs
• Compression On : about 5 secs

[gerardus]

Hi,
I have a set of apps and services wich rely on a framework with DB drivers to access databases.
So far the IBX driver worked flawlessly with IBX 2.2.0 compiled with Lazarus 1.8.4.
I installed Lazarus 2.0 and the Online Package manager installed IBX 2.3.2.
Now I get a SIGSEV error when the app closes.
It happens when the DB driver checks if the transaction is active and tries to commit:

Code: Pascal  [Select][+][-]

1. procedure TIBXDBConnection.InternalCommit;
2. begin
3.   try
4.     if FTransaction.Active then
5.       FTransaction.Commit;
6.   except
7.   end;
8. end;
9.  

TIBXDBConnection is a DB connection wrapper that holds a TIBDatabase and a TIBTransaction.
FTransaction is a TIBTransaction and is valid when this code is executed.

Going back to Lazarus 1.8.4 and IBX 2.2.0 solves the problem, though.
Any idea of what may be wrong?

[tonyw]

I am not sure how Online Package Manager installed IBX 2.3.2 as Online Package Manager is not supported by IBX.

IBX is anyway at 2.3.3. I suggest that you remove all copies of IBX from your PC - as my guess is that you have an inconsistent installation and the install IBX 2.3.3.

Hi,
I have a set of apps and services wich rely on a framework with DB drivers to access databases.
So far the IBX driver worked flawlessly with IBX 2.2.0 compiled with Lazarus 1.8.4.
I installed Lazarus 2.0 and the Online Package manager installed IBX 2.3.2.
Now I get a SIGSEV error when the app closes.
It happens when the DB driver checks if the transaction is active and tries to commit:

Code: Pascal  [Select][+][-]

1. procedure TIBXDBConnection.InternalCommit;
2. begin
3.   try
4.     if FTransaction.Active then
5.       FTransaction.Commit;
6.   except
7.   end;
8. end;
9.  

TIBXDBConnection is a DB connection wrapper that holds a TIBDatabase and a TIBTransaction.
FTransaction is a TIBTransaction and is valid when this code is executed.

Going back to Lazarus 1.8.4 and IBX 2.2.0 solves the problem, though.
Any idea of what may be wrong?

[gerardus]

Hi Toni,
I cleaned up everything, downloaded 2.3.3 and installed it (On Lazarus 1.8.4, Win32)
Same SIGSEGV on exit.
So it must be some change between 2.2.0 and 2.3.2.
For the moment it's back to 2.2.0 as I can't afford to spend more time investigating what's going on.