[SOLVED] Save/Load Series data to/from file (fastest way)

[ahmadian]

Hi all
Is there a fast way to save/load LineSeries (x,y) to a file instead of writing them in a for loop?
My line Series have more than 10000 points and my Hardware is Raspberry Pi.
My file can't be Typed file, because I want to add a Record before LineSeries data.
This is my record:

Code: [Select]

type
    TNewSample = Packed Record
      ID               : PtrUint;
      Name             : string[100];
      StandardName     : string[100];
      Material         : string[100];
      TestType         : PtrUint;
      TestSpeed        : PtrUint;
      Shape            : PtrUint;
      Length           : real;
      Width            : real;
      Height           : real;
      Weight           : real;
      SampleUserParam1 : string[100];
      SampleUserParam2 : string[100];
      SampleUserParam3 : string[100];
    end;

    TTest = Packed Record
      TestSample       : TNewSample;
      TestDate         : string[20];
      TestUserParam1   : string[100];
      TestUserParam2   : string[100];
      TestUserParam3   : string[100];
    end;

I want to save TTest record plus a LineSeries (array of x,y) to a file and load them again. My hardware is slow and I want to use fastest method. (If available on Linux)
Thanks for any help.

[Leledumbo]

Using untyped file and big memory buffer, you can reduce I/O call.

[ahmadian]

Thanks Leledumbo
Is this true:

Code: [Select]

procedure TForm1.Svae_test;
var
  MyFile: file;
  Data: TTest;
begin
  AssignFile(MyFile, 'test1.bin');
  Rewrite(MyFile);
  try
    BlockWrite(MyFile, Data, SizeOf(Data));
    BlockWrite(MyFile, Chart1LineSeries1.Source, SizeOf(Chart1LineSeries1.Source));
  finally
    CloseFile(MyFile);
  end;
end;


[wp]

I don't understand in your posting which one of the records is the header and which contains the data.

You can mix different record types by writing them into a binary file or a TFileStream. Let me suppose that the header is a TNewSample record that you give and the data points are in the ListSource of a TLineSeries. Then you can write them in this way:

Code: [Select]

type
  TDataRecord = record x,y: Double; end;

var
  header: TNewSample;

procedure SaveToFile(ASeries: TChartSeries; AFileName: String);
var
  MS: TMemoryStream;
  i: Integer;
  data: TDataRecord;
begin
  MS := TMemoryStream.Create;
  try
    MS.WriteBuffer(header, SizeOf(header));
    for i:=0 to ASeries.ListSource.Count-1 do begin
      data,X] := ASeries.ListSource.Item[i]^.X;
      data[Y] := ASeries.ListSource.Item[i]^.Y;
      MS.WriteBuffer(data, SizeOf(data));
    end;
    MS.SaveToFile(AFileName);
  finally
    MS.Free;
  end;
end;

procedure ReadDataFromFile(ASeries: TChartSeries; AFileName: String);
var
  FS: TFileStream;
  MS: TMemoryStream;
  data: TDataRecord;
begin
  ASeries.Clear;
  MS := TMemoryStream.Create;
  try
    FS := TFileStream.Create(AFileName, fmOpenRead);
    try
      MS.CopyFrom(FS);
    finally
      FS.Free;
    end;
    MS.Read(header, SizeOf(header));
    while MS.Position < MS.Size do begin
      MS.ReadBuffer(data, Sizeof(data));
      ASeries.AddXY(data.x, data.y);
    end;
  finally
    MS.Free;
  end;
end;


[ahmadian]

Thanks wp
I test your code.The CopyFrom procedure need size (maybe filesize).

[wp]

The size of the filestream, FS.Size

[ahmadian]

Hi wp
I attached test program(on windows).
It saves data to file but don't load it from file.

[ahmadian]

Thanks a lot 'wp'
I changed the code and remove the FileStream.
Now it works OK 
I attached code for others.
I don't know how to add (Solved) to topic name!
Regards

(Attached program Save and Load a LineSeries with Record header to/from a Untyped file)

[wp]

I don't know how to add (Solved) to topic name!

Just edit the title of the first posting

[wp]

A remark on your code of adding data to the series:

  for i := 0 to 100000 do
  begin
    LineSeries1.BeginUpdate;
    LineSeries1.AddXY ( i/100 , SIN(pi*i / 10000) );
    LineSeries1.EndUpdate;
  end;

Calling Begin/EndUpdate inside the loop has no effect. Your chart will be much faster if you call them before/after the loop. BeginUpdate just blocks the 100000 unnecessary redrawing operations after adding new data points until the EndUpdate occurs. Your code blocks the drawing, adds one data point and redraws, and repeats this 100000 times. My code below blocks redrawing, adds all data without redrawing and redraws only afterwards.

And don't forget the try/finally block - if the AddXY crashes for some reason your series would never be redrawn without it in this case:

Code: [Select]

  LineSeries1.BeginUpdate;
  try
    for i := 0 to 100000 do
      LineSeries1.AddXY ( i/100 , SIN(pi*i / 10000) );
  finally
    LineSeries1.EndUpdate;
  end;


[ahmadian]

Thanks wp
That was fast test program and my unaware mistake  (I will change last attached code for others).
But a bit question:
What is difference between "TMemoryStream.Write" and  "TMemoryStream.WriteBuffer" ?
So for Read and ReadBuffer?

[wp]

From looking at the source code (you can do this too by clicking at the "WriteBuffer" of "MS.WriteBuffer" with the CTRL-Key down) I'd guess that there is not difference on this level. In detail, however, there is a big difference because Read and Write are virtual, i.e. they are responsible for the correct read/write behaviors of the various stream types.  Reading and writing has different requirements when you use a Memorystream or a FileStream. When you write an Integer, for example, four bytes are written to the stream my means of the method implemented by "Write". The same for "WriteBuffer" which writes a given number of bytes. From this aspect, I'd say that "WriteBuffer" is more "correct" than "Write", but in practice there is no difference.

If you want to learn more about streams read the short article http://delphi.about.com/od/vclusing/l/aa110803a.htm

[ahmadian]

Thanks wp
I read the article and it compare Arrays and Stream. But the Stream is look like Pointer. Do you see any difference?

[wp]

But the Stream is look like Pointer

No, a pointer is a variable that holds the address of something. A stream is a very generalized file, a collection of data stored in a way defined by the stream type.

[howardpc]

A stream is a class, so whatever kind of stream it is (file, string, memory etc.) an instantiated stream has not only its own data, but its own methods for interacting with that data and with other classes etc.
A stream variable is a pointer in that it is a reference variable. It is a reference that identifies the particular memory location where that instance of the stream has been set up (or will be set up if it is presently unassigned).