Threaded file writer

[mas steindorff]

I'm in need of a threaded file writer that will buffer the output data while the disk is busy with another app (thank you dropbox  ). I think I remember this is already done but I can't find either a unit or a built-in.
Otherwise... The file stuff is ez but whats the best way to pass data from the main program into the threads FIFO?  I also remember reading about some sort of threaded FIFO
Thanks
Mr forgetful ... mas

[mas steindorff]

so I created my own unit but I'm running into some SIGSEGV exceptions when I try and modify my FIFO variables.
I though the issue was my access of the fifo's "count" variable but removed it and I get it when I modify the in variable.
here are the relative snippets of code

Code: Pascal  [Select][+][-]

1. const RawBuffSize = 8192*4;
2.  
3. Type
4.  
5.  TThreadFileSave = class(TThread)
6.   private     { private declarations }
7.      Raw :record
8.        inpos, outpos:integer;
9.        count :integer;
10.        Buff  :Array[-2..RawBuffSize] of byte; // two lead bytes for rap fixes
11.        end;
12.  
13.  
14. ...
15.    end;
16.  

I get the exception when I access raw.inpos with the following after I have copied the new data into my raw.buff[]

Code: Pascal  [Select][+][-]

1. function TThreadFileSave.AddToBuff(p: pbyte; Buflen: integer): Boolean;
2. var pbuf:pbyte;
3.     len, limit,count:integer;
4. begin
5.   if not assigned(self) then
6.       exit;
7.   result := FALSE;   //
8.   if (Buflen < 1) then begin
9.      Showstatus('Buffer add of 0 bytes?');
10.      exit;
11.      end;
12. ...
13.      pbuf := @(raw.Buff[raw.inpos]);
14.      move(p, pbuf, len);                          // move data into local FIFO works
15.     count := raw.count;                          // works
16.     inc(count, len);                              
17.     if (count >= RawBuffSize) then
18.         begin
19.         ShowStatus(' Thread Says buffer overrun.');
20.         count := RawBuffSize;
21.         end;
22.     raw.count = count;                              // <== exception here
23.     inc(raw.inpos, len);                              // <== or here if I comment out the line above
24.     if (raw.inpos = RawBuffSize+1) then
25.         raw.inpos := 0  // this works here due to reduction of len before call
26.     else if (raw.inpos > RawBuffSize) then begin
27.         raw.inpos := raw.inpos - RawBuffSize;
28.         ShowStatus('Warning: was not expecting FIFO to wrap in one pass.');
29.         end;
30. result := true;
31. end;
32.  
33.  

I've tried using EnterCriticalsection and LeaveCriticalSection without success. the error hits the on the 1st call. 

I define the thread within the VSD class

Code: Pascal  [Select][+][-]

1.    T_VSD = class
2.      my : record
3.         tag               :longint;
4.         BufFile  : TThreadFileSave;
5.      ...
6.    public
7.      constructor Create;
8.      destructor Destroy;  override;
9. end;
10.  

my thread init code and I am also starting the thread before I make the call to AddToBuff above

Code: Pascal  [Select][+][-]

1.  
2. constructor vsd.create;
3. begin
4. ...
5.   my.BufFile           := TThreadFileSave.Create(true);  // suspended until file name is set
6. ...
7.   my.BufFile.OpenFile('rawTest1.bin');  // which has Start built in.
8. ...
9.   my.BufFile.AddToBuff(parameters)          // later after testing that OpenFile was successful
10.  
11.  

any ideals?
MAS

[marcov]

    move(p, pbuf, len);     

If you move pointers you do realize that you copy len bytes from the "p" pointer( the location with the address, not where it points to) to pbuf pointer location.

Unless len <> sizeof(pointer) this will go horribly wrong.

What you want is probably

move(p^,raw.Buff[raw.inpos],len);

or making p a formal parameter.

[mas steindorff]

thanks for the error check.  I've now commented out the move command and retested with the same results.

[marcov]

Hard to say from these fragments. But I would throughly check buff[] indexing. Keep in mind an buff[inpos] with inpos =0 means the THIRD character because of you funky array definiton!

[mas steindorff]

Thanks for looking.  I'll generate a simple program later if I can't track my issue down.
I've done threads that push data into main just fine.  I was thinking and figured this is my first where main pushes data into the thread.  the Addtobuff() function  is called in the main thread and modifies thread variables.  This should be OK AFAIK but is there any hidden memory access issues I've may have forgotten? it's only these inpos, count and outpos variables that are having issues.  I can read them in the same function without any issues.
I guess I'll try moving them out of the "raw" sub record to see if that does anything.

[mas steindorff]

WOW I found the issue
it looks like the thread can not access sub records correctly when accessed from main.  all I did was to change the FIFO variables from

Code: Pascal  [Select][+][-]

1.  TThreadFileSave = class(TThread)
2.   private     { private declarations }  
3.     Raw :record
4.        inpos, outpos:integer;
5.        count :integer;
6.        Buff  :Array[-2..RawBuffSize] of byte; // two lead bytes for rap fixes
7.        end;
8. ...
9.  

to

Code: Pascal  [Select][+][-]

1.  TThreadFileSave = class(TThread)
2.   private     { private declarations }  
3.      Raw :record
4.        Buff  :Array[-2..RawBuffSize] of byte; // two lead bytes for rap fixes
5.        end;
6.      raw_count :integer;
7.      raw_inpos,
8.      raw_outpos :integer;  
9.  

and it works  (with same name changes done in code).  Is this a bug?

I'm using rev 2.0.0.r60307