Recent

Author Topic: Why Can't I add a new record ? (!! Thread drift !!)  (Read 15046 times)

sky_khan

  • Guest
Re: Why Can't I add a new record ?
« Reply #15 on: February 19, 2017, 02:14:48 pm »
Well, I was trying to say,humorously
this is self-documenting
Code: Pascal  [Select][+][-]
  1. const
  2.   GasRatesFile = 6;
  3. ...
  4. begin
  5.   Openfile(GasRatesFile);
  6.   ...
  7.  

OpenFile(6) is not.

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #16 on: February 19, 2017, 02:18:43 pm »
Well, I was trying to say,humorously
this is self-documenting
Code: Pascal  [Select][+][-]
  1. const
  2.   GasRatesFile = 6;
  3. ...
  4. begin
  5.   Openfile(GasRatesFile);
  6.   ...
  7.  

OpenFile(6) is not.

Whilst I can see your point - and have to agree to a certain extent -  my mind works slightly differently  :D
FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Why Can't I add a new record ?
« Reply #17 on: February 19, 2017, 03:31:26 pm »
@ Thaddy
Being 'Old School', I've never declared a record 'Packed' and I'm sure I could find volumes as to what benefits doing so might have.   
In TP 1.0 times everything was byte aligned, so there was no necessity for it.
Packed is necessary to achieve byte aligned records. Otherwise the compiler will try and naturally align a record, optionally depending on the {$packrecords X} directive.
Note that {$packrecords} has no influence on a packed record: the latter is ALWAYS byte aligned.
For File IO always declare your records as packed.
Small example I used on the forum this week:
Code: Pascal  [Select][+][-]
  1. program testrecords;
  2. {$mode objfpc}
  3. {$packrecords 2}
  4. type
  5.    Ta = record
  6.      b:byte;
  7.      c:integer;
  8.   end;
  9.   Td = packed record
  10.     e:byte;
  11.     f:integer;
  12.   end;
  13. begin
  14.    writeln(SizeOf(Ta));//prints 6... word aligned through {$packrecords 2}
  15.    writeln(SizeOf(Td));//prints 5..  packed records are always byte aligned.
  16. end.

 Note in TP mode it prints 4 and 3

Also note that TP55 has packed records as per screenshot (just for fun I tested if you really were right. Well, you were wrong ;) even in TP55 it can be essential! Options|Align Data)
« Last Edit: February 19, 2017, 03:52:42 pm by Thaddy »
Specialize a type, not a var.

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #18 on: February 19, 2017, 07:30:55 pm »
@ Thaddy
Being 'Old School', I've never declared a record 'Packed' and I'm sure I could find volumes as to what benefits doing so might have. 
For File IO always declare your records as packed.

Also note that TP55 has packed records as per screenshot (just for fun I tested if you really were right. Well, you were wrong ;)
It's not the first time and it'll not be the last !  :)

I've just changed the declaration to see the effect and, as expected, it totally screwed the data. That's no big deal since there are only 14 historic records and the data is available to re-create the files automatically anyway  -  but will it make a difference to the issue of saving a modified record to the middle of a file using the seek(f,8) call?  Or am I barking up the wrong tree ?

FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Why Can't I add a new record ?
« Reply #19 on: February 19, 2017, 07:46:44 pm »
Well, if the original data contains slack space it was never saved correctly in the first place...
If the data is written to disk without packed, its alignment can be either byte aligned or word aligned in the case of TP.
Byte aligned is the same as packed. If you stored the data word aligned you have to read it back word aligned of course, but that was programmer error even then.
The data should have been stored as packed.

The morale is, that if you stored it wrong back then (word aligned) you can only read and write it with a file of records that are word aligned.
That is also documented in the TP55 User manual.

That has nothing to do with FPC, but with how you used TP to create the file in the first place.

packed was important even back then....

I would simply test what alignment was used by simple using {$packrecords 2} and see if you can read the records sequentially..
If that works correct, you know you made a mistake in the past....and wasted disk space, which was expensive back then  O:-)

One other pitfall is of course the size of integer. Hence I showed that my example records would be size 4 and 3 in TP mode (integer =  2 bytes) and not 6 and 5 (integer = 4 bytes) like in objfpc or Delphi mode.
Working with old code and old files is not always easy if you do not know what you are doing. It needs careful consideration and knowledge about those old formats.
Other pitfalls include real and comp. Real is now an alias for double, so 8 bytes, not 6! There is a helper unit btw to mitigate that for files that stored a real as 6 bytes, Real48Utils.
So if your records contained reals... Boom...

You should really need to learn to use packed for records on disk. You should have learned that back then....
« Last Edit: February 19, 2017, 08:08:12 pm by Thaddy »
Specialize a type, not a var.

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Why Can't I add a new record ?
« Reply #20 on: February 19, 2017, 08:18:32 pm »
You may try and adapt molly's code like so:

Code: Pascal  [Select][+][-]
  1. uses real48utils;
  2. {$packrecords 2} // test, otherwise it is $packrecords 1
  3. type
  4.   // the Record is a String[12], two reals and two 'Dates' with two bytes and a word each.
  5.   TDate   =  Record
  6.     B1: Byte;
  7.     B2: Byte;
  8.     W1: Word;
  9.   end;
  10.  
  11.   TRecord = record
  12.     One : String[12];
  13.     Two : Real48;        // Note this
  14.     Three : Real48;     // And this
  15.     Four : TDate;
  16.     Five : TDate;
  17.   end;

« Last Edit: February 19, 2017, 08:23:08 pm by Thaddy »
Specialize a type, not a var.

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #21 on: February 19, 2017, 08:40:23 pm »
Sorry Thaddy, I seem to have inadvertently given you the wrong impression.

I'm not working with old (historic) code or files   -  the data is historical  ie. it is information about energy use and cost in 2014, 2015 and 2016  which I will ultimately compare with current use.

For simplicity I said that there were two 'reals' but in fact they are declared as 'single' --- as I've intimated, advice from this forum has made me stop using 'real', replacing it with single, double, extended etc.  as dictated by the data.

I've already recreated the three pertinent files as files of Packed Record and they display correctly.

Although I never used Packed Records in the 80s, 90's etc. under TP I did manually make the records that I created then multiples of 4 bytes which I consider effectively aligned them on longword boundries.

This of course is all academic (though interesting) but doesn't answer my real issue :

Can I write a modified record back to the middle of a disc file ?

Even with 'packed record' the   seek(f,8);    write(f,Record);    doesn't work.

« Last Edit: February 19, 2017, 08:52:40 pm by J-G »
FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Why Can't I add a new record ?
« Reply #22 on: February 19, 2017, 09:45:38 pm »
Can I write a modified record back to the middle of a disc file ?
Yes, of course you can.

Quote
Even with 'packed record' the   seek(f,8);    write(f,Record);    doesn't work.
It should not matter one bit whether your record is packed or not (well.... it does matter in filesize so bits do change  :) ).

See the following quick test example:
Code: Pascal  [Select][+][-]
  1. program test2;
  2.  
  3. {$MODE OBJFPC}{$H+}
  4.  
  5. // using file of record, see also http://www.delphicorner.f9.co.uk/articles/apps6.htm
  6.  
  7. uses
  8.   SysUtils;
  9.  
  10. type
  11.   // the Record is a String[12], two reals and two 'Dates' with two bytes and
  12.   // a word each.
  13.   // note: apparantly the types changed a bit
  14.   TDate   = Packed Record
  15.     B1: Byte;
  16.     B2: Byte;
  17.     W1: Word;
  18.   end;
  19.  
  20.   TRecord = Packed record
  21.     One : String[12];
  22.     Two : Single;
  23.     Three : Single;
  24.     Four : TDate;
  25.     Five : TDate;
  26.   end;
  27.  
  28. procedure display(R: TRecord; nr: integer);
  29. begin
  30.   WriteLn('.');
  31.   WriteLn(nr,'.One     = ', R.One);
  32.   WriteLn(nr,'.Two     = ', R.Two:4:4);
  33.   WriteLn(nr,'.Three   = ', R.Two:4:4);
  34.   WriteLn(nr,'.Four.B1 = ', HexStr(R.Four.B1,2));
  35.   WriteLn(nr,'.Four.B2 = ', HexStr(R.Four.B2,2));
  36.   WriteLn(nr,'.Four.W1 = ', HexStr(R.Four.W1,4));
  37.   WriteLn(nr,'.Five.B1 = ', HexStr(R.Five.B1,2));
  38.   WriteLn(nr,'.Five.B2 = ', HexStr(R.Five.B2,2));
  39.   WriteLn(nr,'.Five.W1 = ', HexStr(R.Five.W1,4));
  40. end;
  41.  
  42. var
  43.   F : File of TRecord;
  44.   FName : String = 'test.dat';
  45.  
  46.   Q : Array of TRecord;
  47.   R : TRecord;
  48.   I : Integer;
  49. begin
  50.   SetLength(Q, 0);
  51.  
  52.   // Create if necessary
  53.   if not FileExists(FName) then
  54.   begin
  55.     AssignFile(F, FName);
  56.     Rewrite(F);
  57.     CloseFile(F);
  58.   end;
  59.  
  60.   // Read
  61.   AssignFile(F, FName);
  62.   Reset(F);
  63.   i := 0;
  64.   while not eof(F) do
  65.   begin
  66.     SetLength(Q, Succ(Length(Q)));
  67.     Read(F, Q[High(Q)]);
  68.     inc(i);
  69.   end;  
  70.   CloseFile(F);
  71.   WriteLn('Number of records read = ', i);
  72.   WriteLn('length of array        = ', Length(Q));
  73.  
  74.   // Add a new record
  75.   WriteLn('Adding a new record');
  76.   SetLength(Q, Succ(Length(Q)));
  77.   With Q[high(Q)] do
  78.   begin
  79.     One   := IntToStr(high(Q));
  80.     Two   := High(Q) / 100*High(Q);
  81.     Three := High(Q) / 100*High(Q);
  82.     Four.B1 := high(Q);
  83.     Four.B2 := high(Q);
  84.     Four.W1 := high(Q);
  85.     Five.B1 := 0;  
  86.     Five.B2 := 0;  
  87.     Five.W1 := 0;  
  88.   end;
  89.   WriteLn('length of array        = ', Length(Q));
  90.  
  91.   // Append
  92.   AssignFile(F, FName);
  93.   Reset(F);
  94.   i := FileSize(F);
  95.   WriteLn('Positioning filepointer at record ', i);
  96.   Seek(F, i);
  97.   Write(F, Q[High(Q)]);    
  98.   CloseFile(F);
  99.  
  100.   // display
  101.   AssignFile(F, FName);
  102.   Reset(F);
  103.   i := 0;
  104.   while not EOF(F) do
  105.   begin
  106.     Read(F, R);
  107.     Display(R, i);
  108.     inc(i);
  109.   end;  
  110.   CloseFile(F);
  111.  
  112.   // change the third record in case exist
  113.   AssignFile(F, FName);
  114.   Reset(F);
  115.   i := FileSize(F);
  116.   if i > 3 then
  117.   begin
  118.     Writeln;
  119.     WriteLn('data file contains 3 records or more.');
  120.     WriteLn('attempting to modifiy the 3th record');
  121.     WriteLn('Positioning filepointer at record with index ', 2);
  122.     Seek(F, 2);
  123.     Read(F, R);
  124.     Writeln('This record was modified ', R.Five.W1, ' times');
  125.     // Modify contents.
  126.     WriteLn('Modify the contents of this record (again)');
  127.     R.Five.W1 := R.Five.W1 + 1;
  128.     Seek(F, 2);
  129.     Write(F, R);
  130.   end;
  131.   CloseFile(F);
  132.  
  133.   SetLength(Q, 0);
  134. end.
  135.  

btw: if you're going old-skool then i was more or less expecting that you would know what a shell/command-line is.

My programs should be launched from there in order to get proper feedback (you get that feedback now as well but the window closes automatically).

I refuse to write an readln at the end of my shell programs *period*

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #23 on: February 19, 2017, 11:38:37 pm »
BINGO !!  Thanks Molly!
Can I write a modified record back to the middle of a disc file ?
Yes, of course you can.
I knew that  --  it was just that since I couldn't (in the way that I thought I could) I was beginning to question my sanity!  %)

Quote from: molly
btw: if you're going old-skool then i was more or less expecting that you would know what a shell/command-line is.
Yes I ought to have appreciated that but this morning my mind was only part on Pascal and mostly on the upcoming Belshazar Rehearsal  :)
Quote from: molly
I refuse to write a readln at the end of my shell programs *period*
:D 

Your Code finally gave me the answer   -   ZERO based indexing  -  I was writing to the NEXT record, leaving a Blank Record after the previously 'Last'. The reason I couldn't see that was that I was testing the string[12] in the record to see if it contained more than ''. Since the 'Blank' record conformed, the reading stopped there and never got to the newly written record.
« Last Edit: February 20, 2017, 12:38:42 pm by J-G »
FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Why Can't I add a new record ?
« Reply #24 on: February 20, 2017, 12:20:15 pm »
BINGO !!  Thanks Molly!
You're most welcome.

Perhaps next time show some (faulty compilable) code in order to get a faster to the point response within one page of posts ?  ;D

I knew that  --  it's was just that since I couldn't (in the way that I thought I could) I was beginning to question my sanity!  %)
:)

There are some nice videos floating around the net of people that get so insane that eventually they snap and bash their hardware into smithereens  %)

Just a small tip: a computer always start counting from zero. It's very unnatural and therefor confuses a lot of people.

As another tip (and perhaps you do not even need this one): never try to go against zero indexing and for instance start your static indexes with one. It will confuse the hell out of you later on, especially when switching from static to dynamic arrays.


Quote
Yes I ought to have appreciated that but this morning my mind was only part on Pascal and mostly on the upcoming Belshazar Rehearsal  :)
Ah, i forgot already that was you. Nice. Handel ?

Quote
Your Code finally gave me the answer   -   ZERO based indexing  -
Yes, zero indexing is a pita if you're not used to it.

Reminds me of a small math riddle given as a child on elementary school: You are going to walk down a lane which is 50 yards long and at every yard there is a tree planted. Then the teacher asked: When you walked down the lane how many trees have you seen ?

My answer obviously was: i don't care much about the trees mr. but have you seen all them round delicious apples that where hanging from those trees...   :D

Good to hear you have your 'bug' found and your code 'fixed' now

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Why Can't I add a new record ?
« Reply #25 on: February 20, 2017, 12:28:13 pm »
So real was really a single after all ....  >:( Good find molly. (we discussed real48? a bit... :-*))

But I am glad it is solved
Then never call it real.... ever...that's C speak.... >:D

More old code? :-[
« Last Edit: February 20, 2017, 12:34:11 pm by Thaddy »
Specialize a type, not a var.

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #26 on: February 20, 2017, 01:57:41 pm »
Perhaps next time show some (faulty compilable) code in order to get a faster to the point response within one page of posts ?  ;D
You've asked me this before :(  and I do try to comply  -  but trying to select a few pertinent lines of code from a few thousand often eludes me.

My confusing Thaddy contributed most to the topic extending to two pages I'm sure.      Apart from my stupidity in not seeing the ZERO earlier that is !! 
Quote from: molly
Just a small tip: a computer always start counting from zero. It's very unnatural and therefor confuses a lot of people.

As another tip (and perhaps you do not even need this one): never try to go against zero indexing and for instance start your static indexes with one. It will confuse the hell out of you later on, especially when switching from static to dynamic arrays.
Yes, I'm still not weened off declaring arrays as [1..X]  - ( Note to self - 'Must try harder':)

Quote from: molly
Quote
Yes I ought to have appreciated that but this morning my mind was only part on Pascal and mostly on the upcoming Belshazar Rehearsal  :)
Ah, i forgot already that was you. Nice. Handel ?
Not Handel, William Walton, and yes 'nice' though difficult.  There is an irony here in that the work is based on the story in the Book of Daniel about the Babylonians  - -  the Babylonians were the first to use the concept of ZERO  -  a fascinating subject in its own right, the book 'The Nothing That Is' by Robert Kaplan is worth a read.
Quote from: molly
Quote
Your Code finally gave me the answer   -   ZERO based indexing  -
Yes, zero indexing is a pita if you're not used to it.
I ought to be used to it  -  but occasionally knowledge that hasn't been used for some time refuses to re-surface  :(
Quote from: molly
Good to hear you have your 'bug' found and your code 'fixed' now
You're not half a relieved as I am. :)
FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

J-G

  • Hero Member
  • *****
  • Posts: 953
Re: Why Can't I add a new record ?
« Reply #27 on: February 20, 2017, 02:12:36 pm »
So real was really a single after all ....  >:( Good find molly. (we discussed real48? a bit... :-*))
Being pedantic, a 'single' is a 'real' - just better defined.
Quote from: Thaddy
But I am glad it is solved
Not half a glad as I am!
Quote from: Thaddy
Then never call it real.... ever...that's C speak.... >:D
I consider my hand well and truly slapped!  :-[
Quote from: Thaddy
More old code? :-[
Although I have many thousands of lines of TP code from the 80's, 90's and early 00's, any code I've discussed on this forum is all newly created in Lazarus  -  albeit still using 'old school' calls that may well have been superceded and improved in the interim. Eventually I will learn the new conventions I'm sure and quicker by reference to the knowledge available on this forum.
FPC 3.0.0 - Lazarus 1.6 &
FPC 3.2.2  - Lazarus 2.2.0 
Win 7 Ult 64

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Why Can't I add a new record ?
« Reply #28 on: February 20, 2017, 02:51:21 pm »
Perhaps next time show some (faulty compilable) code in order to get a faster to the point response within one page of posts ?  ;D
You've asked me this before :(  and I do try to comply  -  but trying to select a few pertinent lines of code from a few thousand often eludes me.
In case it is not possible to extract but you do know what is going on, try to 'mimic' the behavior into simple examples as i've shown. Step by step, trying to understand how and/or if it works at all.

That might seem a tedious job todo at first but it usually is a very good method of getting to know your own code and/or particular aspects of your code.

A simple example with file of record, studying it workings, insert records, delete record, removing empty records and doing all sort of those other playful things let's you understand.

The knowledge you've gained with such 'simple' example can be stuffed back into your 'big' project. And in case a 'simple' step is not working for you then you could show us that code instead and ask simple question like i'm trying to insert a record, this code does not do it, what am i doing wrong.

But, that is of course just a pointer that i often use myself. Not something you would have to obey or something. Just keep it in mind and think about it. All i can say is that from the simple examples i write before asking a question on the forum, i already figured out what the problem actually is (well mostly that is, otherwise the forum, ML or wiki often provides me an answer) ;D

Quote from: molly
Quote from: J-G
Ah, i forgot already that was you. Nice. Handel ?
Not Handel, William Walton, and yes 'nice' though difficult.  There is an irony here in that the work is based on the story in the Book of Daniel about the Babylonians  - -  the Babylonians were the first to use the concept of ZERO  -  a fascinating subject in its own right, the book 'The Nothing That Is' by Robert Kaplan is worth a read.
Oh, Walton's feast... that's indeed something different :-), what do you play (if i might be so impolite) ?. Not that i have a something special with cantatas but music is generally speaking a universal language  :) The only thing i ever touched myself is a keyboard, and some drums so basically a sort of a caveman music-wise  :)

Quote
You're not half a relieved as I am. :)
:D

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Why Can't I add a new record ?
« Reply #29 on: February 20, 2017, 03:15:41 pm »
Quote
Being pedantic, a 'single' is a 'real' - just better defined.
Being pedanticly pedantic: c-speak is c-speak and Pascal is much older than C. Reals aren't really real at all in that sense, In C they are not even defined!!!. They are not part of the language up-until this date.  They are a macro.

Real reals,they are NUMERIC's or CALC's - for noobs, this is no shouting, this is the correct spelling, uppercased -. (or 48 bit reals.... those are the real reals)
If you can't define what you really mean with a real real number than what are you? Not Pascal.

It is also pedantic that you seem to know how to express base 10 math with sufficient accuracy to call it real... in a base 2 system. Really?...... in 4 bytes?
That's why we abolished them and gave our approximations as a single and a half and later as double...   >:D >:D >:( Honest language, heh? And generous: better approximation of what a real is supposed to really mean)

Note that the historically real (which wasn't real, just 48 bits of it) really defined  real was invented by Borland to provide more accurate calculations on systems that did not have a hardware co-processor. It worked at the time.

Ergo: A defined real stems from Wirthian languages, An old  Pascal user should know that. And not write records with singles in it and explain them as real  >:(. It really isn't a real, a single.

I hate single and double. I think  it should be interpreted as "Single:completely accurate, one way only." and "Double:well we may be a bit off." There's more logic to that.
Let's call it real48. Then it is defined. Again, a Pascal real is actually still the only defined real that I know of. Up until recently.... that is....
« Last Edit: February 20, 2017, 04:21:16 pm by Thaddy »
Specialize a type, not a var.

 

TinyPortal © 2005-2018