Recent

Author Topic: Custom low-level stuff for WriteLn() etc.  (Read 762 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Custom low-level stuff for WriteLn() etc.
« on: November 29, 2022, 06:01:35 pm »
Noting that Text -> TextRec are opaque, what's the least-overhead way of reimplementing the low-level stuff so that WriteLn() etc. may be used with custom I/O code?

I'm about to redo a Telnet server in a hurry which I anticipate will take me about ten minutes (building on something I've already got to hand), but am rather stumped by the best way of plumbing it into existing software.

I definitely don't need codepages, Unicode or anything like that, and am intent on keeping things as straightforward as possible.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 10498
  • FPC developer.
Re: Custom low-level stuff for WriteLn() etc.
« Reply #1 on: November 29, 2022, 06:08:17 pm »
Look at the streamio unit in fcl-base that redirects a textfile to a stream.

I use it a lot to upgrade haphazard scripts.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Re: Custom low-level stuff for WriteLn() etc.
« Reply #2 on: November 29, 2022, 06:21:15 pm »
I don't see how that helps, since by the time the Telnet server has split out the data and IAC flows it no longer looks anything like a file.

Is that where a TMemoryStream comes into it? Which set of documentation should I believe: https://www.freepascal.org/docs-html/current/rtl/classes/tmemorystream.html or https://www.freepascal.org/docs-html/current/rtl/objects/tmemorystream.html ?

Slightly later: I think that using AssignPipe() directly is probably the best way of doing what I want.

MarkMLl
« Last Edit: November 29, 2022, 07:33:55 pm by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 10498
  • FPC developer.
Re: Custom low-level stuff for WriteLn() etc.
« Reply #3 on: November 29, 2022, 09:15:25 pm »
I meant it as an example of redirecting textfile I/O to something else. Not as a direct tool to use.

Another such thing is sock2text and other "text" functions in the sockets unit.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Re: Custom low-level stuff for WriteLn() etc.
« Reply #4 on: November 29, 2022, 09:37:21 pm »
I meant it as an example of redirecting textfile I/O to something else. Not as a direct tool to use.

Another such thing is sock2text and other "text" functions in the sockets unit.

Sockets won't work directly, because of the necessity of separating the text and control flow. I'm reasonably happy with the pipes idea, particularly since although TextRec is opaque there's a reasonable tradition of going direct to the handle which is something that I'm likely to need for input (read characters, lines etc. with timeout). Tinkering...

Application is a backdoor/debugging interface to various apps, something that happened a few days ago relating to a blown video card and the difficulty of an alternative brought this to the fore. Some code I wrote a couple of weeks ago to support an SCPI instrumentation interface is vastly more robust than LNet which I've used for this in the past, so while I've been thinking about how to structure this for some while it's time to actually do it.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

ccrause

  • Hero Member
  • *****
  • Posts: 689
Re: Custom low-level stuff for WriteLn() etc.
« Reply #5 on: November 30, 2022, 06:11:35 am »
The OpenIO procedure in the embedded & freertos consoleio units shows how to configure TextRect from scratch.  An example of configuring standard IO handles to use custom read and write functions.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Re: Custom low-level stuff for WriteLn() etc.
« Reply #6 on: November 30, 2022, 09:03:31 am »
The OpenIO procedure in the embedded & freertos consoleio units shows how to configure TextRect from scratch.  An example of configuring standard IO handles to use custom read and write functions.

Thanks very much for that. I'd already got much the same from the RTL and it really is extremely tempting to use it. I'm very concerned though about the documentation's explicit warning that TextRec should be treated as opaque: I'm still smarting from the way that FileExists() was changed a year or so ago and if the developers do that what are they likely to do to something that they've warned is an internal implementation mechanism?

I wonder whether there's any way of verifying that the data structure is unchanged (i.e. in an updated version of the compiler/runtimes) and in particular that the functions being pointed to still take the expected parameters etc.?

Have you every had any comment from the core developers on this, or would any of them like to comment now?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

ccrause

  • Hero Member
  • *****
  • Posts: 689
Re: Custom low-level stuff for WriteLn() etc.
« Reply #7 on: November 30, 2022, 02:03:18 pm »
The OpenIO procedure in the embedded & freertos consoleio units shows how to configure TextRect from scratch.  An example of configuring standard IO handles to use custom read and write functions.

Thanks very much for that. I'd already got much the same from the RTL and it really is extremely tempting to use it. I'm very concerned though about the documentation's explicit warning that TextRec should be treated as opaque: I'm still smarting from the way that FileExists() was changed a year or so ago and if the developers do that what are they likely to do to something that they've warned is an internal implementation mechanism?

I wonder whether there's any way of verifying that the data structure is unchanged (i.e. in an updated version of the compiler/runtimes) and in particular that the functions being pointed to still take the expected parameters etc.?

Have you every had any comment from the core developers on this, or would any of them like to comment now?

Very valid concern.  Perhaps the OpenIO procedure (maybe renamed to RegisterIO?) and TWriteCharFunc/TReadCharFunc types should be exposed to enable user defined IO.  Using a configuration helper then hides the low level definition of TextRec, making it a bit more future proof.

I haven't discussed this before, so compiler devs are welcome to comment on this.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Re: Custom low-level stuff for WriteLn() etc.
« Reply #8 on: November 30, 2022, 05:40:55 pm »
Very valid concern.  Perhaps the OpenIO procedure (maybe renamed to RegisterIO?) and TWriteCharFunc/TReadCharFunc types should be exposed to enable user defined IO.  Using a configuration helper then hides the low level definition of TextRec, making it a bit more future proof.

I haven't discussed this before, so compiler devs are welcome to comment on this.

I've been out most of the day, but am just about to start taking a look at this. This is not necessarily something that should be exposed to application programmers, but I think that TWriteCharFunc/TReadCharFunc could usefully be defined with a commitment that code using them will be broken if that definition changes (i.e. so that the compiler would raise an error, rather than compiled code bombing).

MarkMLl

Somewhat later:

Code: Pascal  [Select][+][-]
  1. function TextRecValid(var t: Text; testHandle: boolean= true): boolean;
  2.  
  3. var
  4.   a, b, c, d: ptruint;
  5.  
  6. begin
  7.   result := false;
  8.  
  9. (* If we're not actually being asked to test this then return true.             *)
  10.  
  11.   if not Assigned(@t) then
  12.     exit(true);
  13.  
  14. (* If the parameter is INPUT or OUTPUT then we can reasonably test the mode and *)
  15. (* possibly handle.                                                             *)
  16.  
  17.   if @t = @INPUT then begin
  18.     if TextRec(t).mode <> fmInput then
  19.       exit;
  20.     if testHandle and (TextRec(t).Handle <> 0) then
  21.       exit
  22.   end;
  23.   if @t = @OUTPUT then begin
  24.     if TextRec(t).mode <> fmOutput then
  25.       exit;
  26.     if testHandle and (TextRec(t).Handle <> 1) then
  27.       exit
  28.   end;
  29.  
  30. (* By default, expect the buffer to be internal to the TextRec.                 *)
  31.  
  32.   if TextRec(t).bufPtr <> @(TextRec(t).buffer) then
  33.     exit;
  34.  
  35.   a := ptruint(@(TextRec(t).buffer));
  36.   b := ptruint(@(TextRec(t).bufPtr));
  37.   c := a - b;
  38.  
  39. { define DUMP_TEXTREC }
  40. {$ifdef DUMP_TEXTREC }
  41.   with TextRec(t) do begin
  42.     d := ptruint(@bufptr);
  43.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16), ' ', HexStr(ptruint(bufptr), 16));
  44.     d := ptruint(@openfunc);
  45.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  46.     d := ptruint(@inoutfunc);
  47.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  48.     d := ptruint(@flushfunc);
  49.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  50.     d := ptruint(@closefunc);
  51.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  52.     d := ptruint(@userdata);
  53.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  54.     d := ptruint(@name);
  55.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  56.     d := ptruint(@LineEnd);
  57.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  58.     d := ptruint(@buffer);
  59.     WriteLn(ERROUTPUT, (d - b):5, ' ', HexStr(d, 16));
  60.     WriteLn(ERROUTPUT)
  61.   end;
  62. {$endif DUMP_TEXTREC }
  63.  
  64. (* If we believe we have a good understanding of the fields between the buffer  *)
  65. (* pointer and the buffer itself, then it is likely that the overall data       *)
  66. (* structure- and in particular the code pointers- behaves much as we expect.   *)
  67.  
  68.   d := SizeOf(Pointer); // bufptr
  69.   d += 4 * SizeOf(Pointer); // codepointers
  70.   d += 32; // userdata
  71.   d += textrecnamelength * SizeOf(TFileTextRecChar); // name
  72.   d += 1 + 3; // lineend
  73.   result := c = d                       (* Good place for a breakpoint          *)
  74. end { TextRecValid } ;
  75.  
  76.  
  77. type
  78.   PText= ^Text;
  79.  
  80.  
  81. begin
  82.   if TextRecValid(INPUT) then
  83.     WriteLn(ERROUTPUT, 'INPUT OK')
  84.   else
  85.     WriteLn(ERROUTPUT, 'Bad INPUT TextRec');
  86.  
  87.   if TextRecValid(OUTPUT) then
  88.     WriteLn(ERROUTPUT, 'OUTPUT OK')
  89.   else
  90.     WriteLn(ERROUTPUT, 'Bad OUTPUT TextRec');
  91.  
  92.   if not TextRecValid(PText(Nil)^) then
  93.     WriteLn(ERROUTPUT, 'Bad nil TextRec');
  94.  
  95. end.
  96.  
« Last Edit: November 30, 2022, 07:59:40 pm by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 4893
  • Compiler Developer
Re: Custom low-level stuff for WriteLn() etc.
« Reply #9 on: November 30, 2022, 10:24:26 pm »
Thanks very much for that. I'd already got much the same from the RTL and it really is extremely tempting to use it. I'm very concerned though about the documentation's explicit warning that TextRec should be treated as opaque:

It should be considered opaque from the outside. As a user of a Text variable you should not make any assumptions about its internal state (that's why you need to cast Text to TextRec). That is different for the functions implementing the functionality of the TextRec record.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5866
Re: Custom low-level stuff for WriteLn() etc.
« Reply #10 on: December 01, 2022, 09:53:39 am »
Thanks very much for that. I'd already got much the same from the RTL and it really is extremely tempting to use it. I'm very concerned though about the documentation's explicit warning that TextRec should be treated as opaque:

It should be considered opaque from the outside. As a user of a Text variable you should not make any assumptions about its internal state (that's why you need to cast Text to TextRec). That is different for the functions implementing the functionality of the TextRec record.

Thanks for that, it makes me feel a bit better particularly since my "sanity check" function doesn't look too bad.

Working at this level is much neater than by using AssignPipe() etc., and I'm reasonably confident that I'll be able to do a fairly neat and robust telnet handler in... well, I'm tempted to say a hundred lines or so but my code tends to have a lot of comments and whitespace.

Let's just say more compactly and intelligibly than LNet. But that's really a very low bar :-)

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018