* * *

Author Topic: Filestream.size lot s of lseek  (Read 872 times)

valerife

  • New member
  • *
  • Posts: 10
Filestream.size lot s of lseek
« on: March 28, 2017, 11:27:21 am »
Hello,

I was wondering is there a way to get the size of a file stream without the method .size of TFileStream ?
My concern about this method is how the size of the filestream will be calculated.

Code: Pascal  [Select]
  1. While (true) do
  2. begin
  3.    if (FileStream.Size > 0) then
  4.    begin
  5.      //Do something with the File Stream
  6.    end
  7.    else
  8.    begin
  9.      // do something else
  10.    end;
  11. end;
  12.  

When i use strace i can see the following syscall made :
Quote
     lseek(7, 0, SEEK_CUR)                   = 0
     lseek(7, 0, SEEK_END)                   = 0
     lseek(7, 0, SEEK_SET)                    = 0 

I find it a bit heavy and too many syscalls are made (5 min of program running -> 100k+ lseek)

Does anyone faced the same issue ? Did i misuse the TFileStream class ?

PS : Should i get the size of the file through stat or another syscall ?

Thank you!

Valerife

Thaddy

  • Hero Member
  • *****
  • Posts: 4413
Re: Filestream.size lot s of lseek
« Reply #1 on: March 28, 2017, 11:44:10 am »
This is the same number of calls other programming languages use: checkout C++ streams for example.
It is also the minimum number of calls to determine it. If your stream has a fixed size, you can buffer the size in a var.
"Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory, but a single counterexample is logically decisive."

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5640
Re: Filestream.size lot s of lseek
« Reply #2 on: March 28, 2017, 12:54:01 pm »
No.  It is possible to get it for a filename using findfirst/findnext (or unportable fpstat) of course.

Thaddy

  • Hero Member
  • *****
  • Posts: 4413
Re: Filestream.size lot s of lseek
« Reply #3 on: March 28, 2017, 01:09:34 pm »
That's cheating and probably does just the same, if not slower. I think he means to speed up things.
And .. not to put too fine a point to it... That's only for FILE streams. Ok, he asked explicitly about  TFilestream,
but you will loose abstraction with regard to other stream types.
« Last Edit: March 28, 2017, 01:12:59 pm by Thaddy »
"Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory, but a single counterexample is logically decisive."

rvk

  • Hero Member
  • *****
  • Posts: 2648
Re: Filestream.size lot s of lseek
« Reply #4 on: March 28, 2017, 01:17:56 pm »
Did i misuse the TFileStream class ?
Is that your exact code? No delay in your loop?
That's going to eat up a lot of CPU and HD cycles  ;D

In this case, yes, I feel you are misusing TFileStream.
Isn't it better to use some monitoring facility in the OS? (like Inotify)

http://www.digitalinternals.com/unix/linux-monitor-file-system-changes/397/

It also kinda depends on how you use this code. If it's in a thread, to monitor changes to a file you are going to change yourself elsewhere in your program, there are also other options.

valerife

  • New member
  • *
  • Posts: 10
Re: Filestream.size lot s of lseek
« Reply #5 on: March 28, 2017, 01:46:56 pm »
Oh i see,

I think i will go for a var that i inc or dec with the stream size as Thaddy suggested. (Why don't the TFileStream have this as basic implementation ? What are the wrongs of doing so ?).
As far as i can see it would be an overflow issue ?

Indeed there is a sleep inside this loop, i summed it up to show you my problem. 
Wouldn't inotify be a bit too much implementation to fix a trivial problem like this one? (However wouldn't i do lot s of syscall by using inotify in a loop like this).

Thank you very much for all your answers !

rvk

  • Hero Member
  • *****
  • Posts: 2648
Re: Filestream.size lot s of lseek
« Reply #6 on: March 28, 2017, 02:02:39 pm »
Wouldn't inotify be a bit too much implementation to fix a trivial problem like this one? (However wouldn't i do lot s of syscall by using inotify in a loop like this).
It depends on what you want to use this for.

If you want to monitor the size-changes of an external file it might just be the way to go.

If you want to monitor file-changes inside your own program you could also build in some events in places where you change that file.

You just showed a small snippet without explaining how, when and why you are using it. If explained, maybe there are alternatives how to do certain things.

The use of inotify would be much better for changes by external programs because you don't need to poll the system for changes, you get a callback from the OS itself when the files change. So there are no syscalls in your own program. I don't have experience with this on Linux but Windows has a similar feature.


https://www.ibm.com/developerworks/linux/library/l-ubuntu-inotify/index.html
http://man7.org/linux/man-pages/man7/inotify.7.html
http://www.freepascal.org/docs-html/rtl/linux/inotify_add_watch.html

Maybe there is some more explanation how to do this in FPC in this PDF:
http://www.freepascal.org/~michael/articles/dirwatch/dirwatch.pdf
« Last Edit: March 28, 2017, 02:07:30 pm by rvk »

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus