Recent

Author Topic: New float fields contain 6E-154  (Read 7125 times)

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #15 on: October 22, 2021, 05:58:26 pm »
Record buffer is incorrectly initialized for field type 'B' (double). It is padded with spaces instead of binary zero. The solution is easy. I can create a patch.

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #16 on: October 22, 2021, 06:19:04 pm »
I attach a patch. Please test it.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: New float fields contain 6E-154
« Reply #17 on: October 22, 2021, 06:25:30 pm »
Thanks! I'll test it extensively.

It's what XBase++ (a Clipper-successor, written in Delphi) produces, if you tell it to create a FoxPro level 31 database (with autonums). Perhaps the error is on their side?

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #18 on: October 22, 2021, 06:40:55 pm »
No, this is definitely a problem with the TDbf component.

winni

  • Hero Member
  • *****
  • Posts: 2802
Re: New float fields contain 6E-154
« Reply #19 on: October 22, 2021, 07:53:44 pm »
Hi!

The Employee data is not correct.

Floats in dBase are saved as ASCII and padded to the left with spaces $20.
But the whole  field is filled with zeros $00.

Winni

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #20 on: October 22, 2021, 08:07:04 pm »
This is true for numeric fields (native type 'N'). We are talking about binary float field ('B') which was introduced in FoxPro (I think).

winni

  • Hero Member
  • *****
  • Posts: 2802
Re: New float fields contain 6E-154
« Reply #21 on: October 22, 2021, 08:34:03 pm »
Hi!

Okay. If it is a 'B' then the dBase definition is:

B    Binary, a string    10 digits representing a .DBT block number. The number is stored as a string, right justified and padded with blanks.

This collides with your info about FoxPro.

Winni

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #22 on: October 22, 2021, 08:40:38 pm »
The nature of DBF files - unified standard has never been developed and each vendor has freely implemented it.
Here is the list of fields supported by Visual FoxPro:
https://docs.microsoft.com/en-us/sql/odbc/microsoft/visual-foxpro-field-data-types?view=sql-server-ver15

winni

  • Hero Member
  • *****
  • Posts: 2802
Re: New float fields contain 6E-154
« Reply #23 on: October 22, 2021, 09:07:40 pm »
Hi!

Gosh!  This collides with dBase definition:

O    Double    8 bytes - no conversions, stored as a double.

But:
If we assume that this is Foxpro the "TESTFLOAT" with definition "B" has a hexvalue of

$ 08 00 04 00

If you move this value into a double it results in

6.6....E-316

which might be a "special kind of zero"

Winni






korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #24 on: October 22, 2021, 09:21:38 pm »
But:
If we assume that this is Foxpro the "TESTFLOAT" with definition "B" has a hexvalue of

$ 08 00 04 00

If you move this value into a double it results in

6.6....E-316

which might be a "special kind of zero"
This means that the field is 8 bytes long and has a precision of 4 decimal places. I am not sure what the number of decimal places means, it may be for formatting purposes.

winni

  • Hero Member
  • *****
  • Posts: 2802
Re: New float fields contain 6E-154
« Reply #25 on: October 22, 2021, 09:48:18 pm »
Hi!

I assume that Foxpro does it like dBase and saves the binary value.

So we need another 4 bytes for a double.

If we pad the zeros to the right we get

$0800040000000000

This results in a double of

3.78....E-270

Also not so far away from zero ...

But who knows.
It is 30 years ago that I did this stuff the last time.

Winni

korba812

  • Full Member
  • ***
  • Posts: 224
Re: New float fields contain 6E-154
« Reply #26 on: October 22, 2021, 09:59:16 pm »
Yes, the values are stored as double so they are 8 bytes size. Note that the attached dbf file does not contain any data. The values you provide come from definition of table structure.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: New float fields contain 6E-154
« Reply #27 on: October 22, 2021, 10:01:24 pm »
In XBase++, if you specify the precision of a float ("F"), it gives an error message. Floats always seem to be 8 bytes as well. It's just a double.

While a numeric value ("N") is more flexible, it has strict rules about the amount of bytes you need for a specific precision and amount of decimals.

And a binary value ("B") seems to have been a blob, which was different form a memo ("M"), which is a blob that always contains text. Both reside in their own file.

FoxPro first introduced the float ("F"), which we would call a double, and later (Visual FoxPro?), the "B" data type was repurposed as a double (?) and memos could contain blobs as well. Or, that's what I understood.

dBase IV and V, XBase++ and numerous other databases that use variants of DBF files, seem to more or less disagree with the above.

« Last Edit: October 22, 2021, 10:03:22 pm by SymbolicFrank »

winni

  • Hero Member
  • *****
  • Posts: 2802
Re: New float fields contain 6E-154
« Reply #28 on: October 22, 2021, 10:29:14 pm »
In XBase++, if you specify the precision of a float ("F"), it gives an error message. Floats always seem to be 8 bytes as well. It's just a double.

Hi!

No, the dBase definition is more funny. The width definition connects to - in Pascal terms - floatToStr(myFloat)  :

F    Float    Number stored as a string, right justified, and padded with blanks to the width of the field.


I can only advise what I did 30 years ago:

Take the original software, export it as ASCII, ANSI or IBM8. Or whatever.
Import it to SQL. And never touch it again.

The only good part of dBase and friends was the index file with a balanced  Beyer tree.
The rest was a lot of hacks.

Winni

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: New float fields contain 6E-154
« Reply #29 on: October 22, 2021, 10:40:11 pm »
So, is the proposed fix going to work for all the dBase-alike DBF files? Or would it only work for XBase and/or (Visual) FoxPro? How should we fix TDbf?

 

TinyPortal © 2005-2018