Recent

Author Topic: Close/reopen text file  (Read 5173 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Close/reopen text file
« on: January 09, 2022, 01:46:10 pm »
Is it possible to close and then reopen the "Output" device on Linux?

The situation is that I've got potential output to the unnamed "Output" device, to a named file, to a unix-domain pipe or to a FIFO (named pipe).

The output format does not, as standard, have any form of EOF marker so any cooperating programs have to use e.g. the inotify (FAM) API to monitor the writer's state.

If the writer program is looping, e.g. to process successive runs of an instrument capture, then it can close output routed to any of the named file types and then reopen it immediately before starting another run.

For the purpose of orthogonality, I would like to be able to handle "Output" in the same way. I appreciate that if this is being redirected or piped (by the shell) that the behaviour might be undefined, but for the moment I'm focussing on the writer and the relationship between "Output" and the "stdout" handle: any signals or error states seen by the reader is for another day.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Close/reopen text file
« Reply #1 on: January 09, 2022, 02:38:20 pm »
but for the moment I'm focussing on the writer and the relationship between "Output" and the "stdout" handle: any signals or error states seen by the reader is for another day.

MarkMLl
FYI stdout is simply an alias for output. They are interchangeble.
https://www.freepascal.org/docs-html/rtl/system/stdout.html
So relation part solved   :D

As for eof, dunno. Afaik that is indeed not standard for output/stdout.
« Last Edit: January 09, 2022, 02:42:13 pm by Thaddy »
Specialize a type, not a var.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Close/reopen text file
« Reply #2 on: January 09, 2022, 02:47:43 pm »
FYI stdout is simply an alias for output.
https://www.freepascal.org/docs-html/rtl/system/stdout.html
So relation part solved   :D

As for eof, dunno. Afaik that is indeed not standard.

No, stdout is the unix handle with the default value 1.

I anticipate that closing/reopening "Output" hence stdout will break something if e.g. it's being piped to another program. I'm not all that bothered, "somebody else's problem".

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Kays

  • Hero Member
  • *****
  • Posts: 569
  • Whasup!?
    • KaiBurghardt.de
Re: Close/reopen text file
« Reply #3 on: January 09, 2022, 03:00:01 pm »
Is it possible to close and then reopen the "Output" device on Linux?
No, not the way you want it: Once /dev/stdout has been closed it is not connected to the respective terminal [or file myProgram > outfile] anymore. You cannot, at least not trivially, regain access to user’s console (or the file specified on the command line upon program invocation).
Code: Pascal  [Select][+][-]
  1. uses
  2.         baseUnix;
  3. var
  4.         FD: text;
  5. begin
  6.         fpClose(stdOutputHandle);
  7.         assign(FD, '/dev/stdout');
  8.         rewrite(FD); { causes RTE 2 "File not found" }
  9. end.
Yours Sincerely
Kai Burghardt

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Close/reopen text file
« Reply #4 on: January 09, 2022, 03:14:08 pm »
No, not the way you want it: Once /dev/stdout has been closed it is not connected to the respective terminal [or file myProgram > outfile] anymore. You cannot, at least not trivially, regain access to user’s console (or the file specified on the command line upon program invocation).

In practice, I think that's exactly what I want: I'd forgotten that special device.

I anticipate that either the inotify API, a signal or a runtime error condition (EPIPE?) will alert the reader to the fact that the writer's closed its end. If in practice the reader can't recover... that's not the writer's problem, the user should have set things up to use a unix-domain socket or a FIFO.

It might be possible for the reader to examine the writer's /proc/something/fd/1 and to reconnect, but again that's not the writer's problem.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Close/reopen text file
« Reply #5 on: January 09, 2022, 05:07:15 pm »
No, stdout is the unix handle with the default value 1.
Am I right that you suggest that the official documentation is incorrect?  >:D ;D
You can also reverse it: Output is really an alias for stdout. Clear?

If you really feel bad about it, file a bug against documentation.

The point is that from the language point of view the documentation is correct, but from the platform point of view it is the other way around. I would stick to the language.
You will find that the handle - on unix flavors - is indeed 1 as expected in most cases.
« Last Edit: January 09, 2022, 05:14:44 pm by Thaddy »
Specialize a type, not a var.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Close/reopen text file
« Reply #6 on: January 09, 2022, 05:37:29 pm »
If you really feel bad about it, file a bug against documentation.

If /you/ really feel bad about it, file a bug against unix.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Close/reopen text file
« Reply #7 on: January 10, 2022, 02:17:03 pm »
You two are talking about two different things:
Thaddy says that StdOut is an alias of Output which in the context of FPC is definitely correct.
MarkMLI says that StdOut is an alias for the Unix handle with value 1, which is also correct, but in the context of Unix. In the context of FPC that is named StdOutputHandle.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Close/reopen text file
« Reply #8 on: January 10, 2022, 02:40:11 pm »
I know that, and I suspect he knows that as well :-)

And I did carefully say at the start 'relationship between "Output" and the "stdout" handle'... and since this is the unix topic I'd have thought that the reference to "handle" needed no further expansion.

This does of course xref to my slightly older https://forum.lazarus.freepascal.org/index.php/topic,57706.0.html , and when considering closing/reopening it looks as though the preferred behaviour should be slightly different depending on whether output is to a regular file (where reopening will truncate) vs anything else.

However all I'm trying to do at present is rough out "reasonable" behaviour for the various types... and as discussed I anticipate that doing this to a named pipe will have undesirable consequences (which I'm documenting but not prohibiting).

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Kays

  • Hero Member
  • *****
  • Posts: 569
  • Whasup!?
    • KaiBurghardt.de
Re: Close/reopen text file
« Reply #9 on: January 10, 2022, 05:50:49 pm »
[…] I anticipate that either the inotify API,
Inotify does not work (reliably) on pseudo file systems such as procfs(5) or devfs/devtmpfs.

a signal or a runtime error condition (EPIPE?) will alert the reader to the fact that the writer's closed its end.
Files are only really deleted if they are not opened (confer output of lsof(8)) and their (hardlink) reference count hits zero. Similarly, as long as someone somewhere (e.g. another process) holds a valid FD to stdout, this stdout is not closed/“deleted”.
Code: Pascal  [Select][+][-]
  1. uses
  2.         baseUnix;
  3. var
  4.         FD: text;
  5. begin
  6.         assign(FD, '/dev/stdout');
  7.         rewrite(FD);
  8.         fpClose(stdOutputHandle);
  9.         writeLn(FD, 'Hi!'); { still works as expected }
  10.         writeLn('Bye!'); { causes RTE 101 }
  11. end.
On FreeBSD you could use revoke(2) to really close /dev/stdout for good.
« Last Edit: January 10, 2022, 09:50:50 pm by Kays »
Yours Sincerely
Kai Burghardt

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Close/reopen text file
« Reply #10 on: January 10, 2022, 09:46:03 pm »
Inotify does not work (reliably) on pseudo file systems such as procfs(5) or devfs/devtmpfs.

True, but what I'm trying to do is get the writer "doing the right thing", or at least "doing the best under the circumstances".

Quote
Files are only really deleted if they are not opened (confer output of lsof(8)) and their (hardlink) reference count hits zero. Similarly, as long as someone somewhere (e.g. another process) holds a valid FD to stdout, this stdout is not closed/“deleted”.

True, and that's one reason why unix doctrine has settled on a comparatively complex rename-and-copy sequence to handle reopening a file when a backup is to be saved. But I'd prefer not to rely on it when a Pascal-style Rewrite() is used, since this implies that a file is truncated and (potentially) that the space is reused.

So what I've coded for the time being is:

* When the instrument has generated data the file or pipe/FIFO/socket is closed.

* If it's a pipe/FIFO/socket it's reopened promptly, since this gives both ends the best chance of sorting things out.

* If it's a regular file it's not reopened until the next instrument run in case that causes data to be lost.

The interpretation of "promptly" will no doubt need to be adjusted, depending on the reliability of the inotify API and/or file timestamps etc.

As I've said though, this is really an attempt to "do the right thing" under different circumstances.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Kays

  • Hero Member
  • *****
  • Posts: 569
  • Whasup!?
    • KaiBurghardt.de
Re: Close/reopen text file
« Reply #11 on: January 10, 2022, 10:11:40 pm »
[…] Rewrite()
There is append (in Extended Pascal, ISO 10206, known as extend). For character devices such as your terminal it’s evidently irrelevant, I think,.
Yours Sincerely
Kai Burghardt

 

TinyPortal © 2005-2018