Lazarus

Free Pascal => Beginners => Topic started by: ssliackus on September 18, 2018, 09:18:42 pm

Title: What is buffer and how to use it?
Post by: ssliackus on September 18, 2018, 09:18:42 pm
Hi guys,

That is me again  :)

What is buffer and how to use it? I know there is frame buffer, key buffer. For example Crt module has got readkey that reads from buffer, but I cannot find any tutorial or good explanation on that.

Thank you for help.

S.
Title: Re: What is buffer and how to use it?
Post by: Handoko on September 18, 2018, 10:29:49 pm
I'm good in codes not words so I'm not going to explain the definition of buffer. You can search the Internet for it if you want.

Here I have some explanations how to use buffer to reduce flickering effect when painting on the screen:
https://forum.lazarus.freepascal.org/index.php/topic,38136.msg263143.html#msg263143

And here has explanation about using memory buffer to load and store font for use in game programming:
https://forum.lazarus.freepascal.org/index.php/topic,33909.msg220914.html#msg220914
Title: Re: What is buffer and how to use it?
Post by: lucamar on September 18, 2018, 10:38:29 pm
Simplified explanation

In the sense you mean, a buffer is nothing more than a zone of memory where one process deposits values and from which another process retrieves them.

For example, a frame buffer is a piece of memory where p.e. your program constructs a representation of the screen (as it wants it to be) and which is subsequently "transferred" to the actual video memory, by the same or other function, system call or actual hardware.

The same for the keyboard buffer: the system software responds to physical key presses/releases by reading the keyboard and storing a representation in a buffer which afterwards is read by other system processes or applications (in a Pascal program, with ReadKey, v.g.).

Note that buffer, in a extense sense, is also used to name almost any region of memory dedicated to this or other kinds of process communication: file buffers, string buffers, stream buffers, etc.

Just think of what the word buffer means IRW and apply it to computer programs' behaviour..
Title: Re: What is buffer and how to use it?
Post by: jamie on September 18, 2018, 10:49:29 pm
Normally a buffer is a chunk of memory set aside to accumulate data in the background until
at some point the main application reads from it there by emptying the buffer/memory chunk.

 There are the most common types:

    1. First In, First Out  (FIFO);
         
          The first data item in the buffer will be read first when the buffer is read.

    2. First In, Last out (FILO);

         The First data item in the buffer will be read last, meaning its like a stack of papers, you
      need to remove the top sheets before you get to the bottom sheets, the first one.


 etc.

  A buffer is just a storage space with indexes to that when written to or read from these indexes
are managed to so the system will know where to put the next data item and where to read to
get the next data item.

 In most systems, when both of these indexes are the same, it means the buffer is empty..

Title: Re: What is buffer and how to use it?
Post by: Handoko on September 19, 2018, 07:54:15 am
I've just remembered, here I ever wrote the explanation how to use buffer for keyboard input:
http://forum.lazarus.freepascal.org/index.php/topic,38136.msg261965.html#msg261965

That example uses First In, First Out buffer technique to solve the missing keypress problem, which is a common issue in game programming.
Title: Re: What is buffer and how to use it?
Post by: ssliackus on September 24, 2018, 01:39:51 pm
Thank you, guys, for your time and answers. I will as soon as I have time for that :-)
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 11, 2022, 04:13:29 am
keyboard must have some kind of buffer? If key is pressed and held, how to limit the number of executions then? Say, to 3 or 4.

Currently, in my code, if I press and hold for more than a split second, say, zoom in button, graphs keeps being re-plotted.  As my graphs are quite slow, when I release the key, I still have to wait while graphs continues being replotted number of times, whatever number of keystrokes is in the buffer I guess.

How to control that?

or is it controlled by OS?
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 11, 2022, 06:58:01 am
while waiting for your suggestions I found windows keyboard setting that does the trick. Decreasing the "repeat character rate" makes graph response so much more manageable, some plots are built in exact rate as key repeat, feels like direct control. I can also increase the delay for repeat character, which feels like one click for each graph refresh.

Meanwhile I also played with "repeat until not keypressed", "While keypressed Do readkey"  etc, all kinds of logic constructs. Results are not good.
Title: Re: What is buffer and how to use it?
Post by: Zvoni on November 11, 2022, 09:31:39 am
Normally a buffer is a chunk of memory set aside to accumulate data in the background until
at some point the main application reads from it there by emptying the buffer/memory chunk.

 There are the most common types:

    1. First In, First Out  (FIFO);
         
          The first data item in the buffer will be read first when the buffer is read.

    2. First In, Last out (FILO);

         The First data item in the buffer will be read last, meaning its like a stack of papers, you
      need to remove the top sheets before you get to the bottom sheets, the first one.


 etc.

  A buffer is just a storage space with indexes to that when written to or read from these indexes
are managed to so the system will know where to put the next data item and where to read to
get the next data item.

 In most systems, when both of these indexes are the same, it means the buffer is empty..
I'll add LIFO here: Last in, First Out. a real life example would be loading a Truck. --> This is (in a way) the same as FILO, but it's more known as LIFO
a programmers example of LIFO would be a Stack (Push/Pop).

The way i explain Buffer: a real life example: WhatsApp.
When you start writing a message to someone, that TextField you're writing in is your "Buffer".
The other person doesn't see what you're writing while you're writing.
The moment you tap "send", your buffer gets read, sent to the receiver, and emptied out on your side.
Without buffer, your message would be sent character by character, and the receiver would see the message appearing as you're typing
Title: Re: What is buffer and how to use it?
Post by: Leledumbo on November 11, 2022, 11:13:08 am
keyboard must have some kind of buffer? If key is pressed and held, how to limit the number of executions then? Say, to 3 or 4.

Currently, in my code, if I press and hold for more than a split second, say, zoom in button, graphs keeps being re-plotted.  As my graphs are quite slow, when I release the key, I still have to wait while graphs continues being replotted number of times, whatever number of keystrokes is in the buffer I guess.

How to control that?

or is it controlled by OS?
OS only has generic controls, like the repeat rate that you've found, but it might not be sufficient for your own needs. In this case, you can make your own buffer, a queue for instance, where instead of directly processing all ReadKey directly, you enqueue it instead. Then, somewhere else another code dequeues key from this queue one by one, at the rate you control (a simple SysUtils.Sleep before next iteration should do). As a means of preventing DDoS, you can limit the size of the queue that when it's reached, any ReadKey result is simply discarded / ignored.
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 16, 2022, 08:06:23 pm

OS only has generic controls, like the repeat rate that you've found, but it might not be sufficient for your own needs. In this case, you can make your own buffer, a queue for instance, where instead of directly processing all ReadKey directly, you enqueue it instead. Then, somewhere else another code dequeues key from this queue one by one, at the rate you control (a simple SysUtils.Sleep before next iteration should do). As a means of preventing DDoS, you can limit the size of the queue that when it's reached, any ReadKey result is simply discarded / ignored.

you are right, slowing the repeat rate wasn't good idea. I forgot that same computer is used by many technicians for some other needs too, and everybody is used to a certain speed.

I tried to play with keyboard unit (TKeyEvent etc.) but this doesn't work with GRAPH unit. How else I can make my custom logical buffer?

The easiest solution turned out to be "while KeyPressed Do Key := ReadKey; "  where "Key" is declared as char type.  This way there is no stored repeat keys, direct control.

I don't understand how it works, but it has desirable effect.  I am getting used to not understanding things.


Title: Re: What is buffer and how to use it?
Post by: Leledumbo on November 16, 2022, 10:43:27 pm
you are right, slowing the repeat rate wasn't good idea. I forgot that same computer is used by many technicians for some other needs too, and everybody is used to a certain speed.

I tried to play with keyboard unit (TKeyEvent etc.) but this doesn't work with GRAPH unit. How else I can make my custom logical buffer?

The easiest solution turned out to be "while KeyPressed Do Key := ReadKey; "  where "Key" is declared as char type.  This way there is no stored repeat keys, direct control.

I don't understand how it works, but it has desirable effect.  I am getting used to not understanding things.
The downside of graph unit (alongside crt) is that they're not friendly with the KVM (keyboard, video, mouse) units. Both units manipulate everything in their own incompatible way. So indeed if you have to use graph and crt, stick to everything they have.

I don't think it's possible to escape threads, though. So this is approximately what you need to do:
Code: Pascal  [Select][+][-]
  1. {$mode objfpc}{$H+}
  2.  
  3. uses
  4.   {$ifdef unix}
  5.   cthreads,
  6.   {$endif}
  7.   SysUtils,
  8.   crt,
  9.   gqueue;
  10.  
  11. type
  12.   TCharQueue = specialize TQueue<Char>;
  13. var
  14.   cq: TCharQueue;
  15.  
  16. function PrintChar(AData: Pointer): PtrInt;
  17. begin
  18.   while true do begin
  19.     if cq.Size > 0 then begin
  20.       Write(cq.Front); // this should be your char handler, i.e. what to do with the char
  21.       cq.Pop; // remove char from the buffer
  22.       // this is your rate control, i.e. it won't handle the next char after 1 second has passed since the last handled char
  23.       // change it as you need
  24.       SysUtils.Sleep(1000);
  25.     end;
  26.   end;
  27. end;
  28.  
  29. var
  30.   c: Char;
  31. begin
  32.   cq := TCharQueue.Create;
  33.   BeginThread(@PrintChar, nil);
  34.   WriteLn('Press and hold any key then see how they''re written slowly');
  35.   repeat
  36.     c := ReadKey;
  37.     if UpCase(c) = #27 then Break; // ESC to stop the program, even before all the chars have been handled
  38.     if cq.Size < 5 then cq.Push(c); // maximum 5 chars in buffer
  39.   until false;
  40.   cq.Free;
  41. end.
  42.  
You can press as many keys as you want, but if the buffer has 5 chars already where they're not yet processed, the 6th and following chars will be ignored. Likewise, in the middle of processing, if you press ESC, the program will terminate anyway without waiting for the rest of the char in the buffer. Try pressing a-z sequentially one by one with normal speed (e.g.: 100ms difference), you will see not all characters will be printed.
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 19, 2022, 04:28:14 am
thank you Leledumbo. It works. You are using some syntax FPC compiler doesn't accept though. Like SysUtils need to be declared, can't call SysUtils.Sleep() directly. And then  since it is declared, I can just call Sleep().

I remember C# is like that, no need to declare if you don't mind typing the whole thing every time. FPC apparently not. Maybe in Delphi it is possible?

A bit of mind-bending though, how it works. In your example, in my mind, as soon as you handle it to function PrintChar(), user is stuck in infinite loop. But, keyboard input is still handled, in parallel?
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 19, 2022, 04:37:17 am
aha, found it, it is threads. Another thing I didn't know exist.
Title: Re: What is buffer and how to use it?
Post by: Bogen85 on November 19, 2022, 09:45:04 am
thank you Leledumbo. It works. You are using some syntax FPC compiler doesn't accept though. Like SysUtils need to be declared, can't call SysUtils.Sleep() directly. And then  since it is declared, I can just call Sleep().

I just tried Leledumbo's program with FPC 3.3.1 and 3.2.2. Both compiled it and the resulting executables ran as expected.
Title: Re: What is buffer and how to use it?
Post by: Leledumbo on November 19, 2022, 01:53:03 pm
thank you Leledumbo. It works. You are using some syntax FPC compiler doesn't accept though. Like SysUtils need to be declared, can't call SysUtils.Sleep() directly. And then  since it is declared, I can just call Sleep().
What version are you using? I only test on latest stable and latest trunk, nothing prior to them. And surely both work as expected.
I remember C# is like that, no need to declare if you don't mind typing the whole thing every time. FPC apparently not. Maybe in Delphi it is possible?
It's just for clarity, FPC and Delphi are the same in this regard, no need to qualify if you're sure which one to call of you want to override through the uses clause order.
A bit of mind-bending though, how it works. In your example, in my mind, as soon as you handle it to function PrintChar(), user is stuck in infinite loop. But, keyboard input is still handled, in parallel?
Yep, more or less is like that.
Title: Re: What is buffer and how to use it?
Post by: Bogen85 on November 19, 2022, 04:35:03 pm
What version are you using? I only test on latest stable and latest trunk, nothing prior to them. And surely both work as expected.

Which is what I did to confirm.

I just tried Leledumbo's program with FPC 3.3.1 and 3.2.2. Both compiled it and the resulting executables ran as expected.

However, there was an illegal character I had to remove close to the top.


The 0xC2 at offset 0x14.


Code: Text  [Select][+][-]
  1. $ hexdump -C 42592.pas | head -n 3
  2. 00000000  7b 24 6d 6f 64 65 20 6f  62 6a 66 70 63 7d 7b 24  |{$mode objfpc}{$|
  3. 00000010  48 2b 7d 0a c2 a0 0a 75  73 65 73 0a 20 20 7b 24  |H+}....uses.  {$|
  4. 00000020  69 66 64 65 66 20 75 6e  69 78 7d 0a 20 20 63 74  |ifdef unix}.  ct|
  5.  

Code: Text  [Select][+][-]
  1. $ fpc 42592.pas
  2. Free Pascal Compiler version 3.2.2 [2022/08/17] for x86_64
  3. Copyright (c) 1993-2021 by Florian Klaempfl and others
  4. Target OS: Linux for x86-64
  5. Compiling 42592.pas
  6. 42592.pas(2,1) Fatal: illegal character "'�'" ($C2)
  7. 42592.pas(2,1) Fatal: Compilation aborted
  8. Error: /usr/bin/ppcx64 returned an error exitcode
  9.  

Code: Text  [Select][+][-]
  1. $ fpc 42592.pas
  2. Free Pascal Compiler version 3.3.1 [2022/11/18] for x86_64
  3. Copyright (c) 1993-2022 by Florian Klaempfl and others
  4. Target OS: Linux for x86-64
  5. Compiling 42592.pas
  6. 42592.pas(2,1) Fatal: Illegal character "'�'" ($C2)
  7. Fatal: Compilation aborted
  8. Error: /home/dev/fpc_usr/lib/fpc/3.3.1/ppcx64 returned an error exitcode
  9.  
Title: Re: What is buffer and how to use it?
Post by: Bogen85 on November 19, 2022, 05:19:29 pm
@Leledumbo this is the first time I've had in this happen in copying pasting code from the Forums here.

However, there was an illegal character I had to remove close to the top.
The 0xC2 at offset 0x14.

https://www.codetable.net/hex/c2

Not sure if this was an extra character from your side, or something in the forum software.

It is strange though, it is only when I copy/paste your code example. I can't find it in the HTML source either as the illegal character or the above HTML entity.

And it is not always showing up in the same location, if I use the [Select] from the forum I get this:
Code: Text  [Select][+][-]
  1. 00000000  20 20 20 20 7b 24 6d 6f  64 65 20 6f 62 6a 66 70  |    {$mode objfp|
  2. 00000010  63 7d 7b 24 48 2b 7d 0a  20 20 20 20 c2 a0 0a 20  |c}{$H+}.    ... |
  3. 00000020  20 20 20 75 73 65 73 0a  20 20 20 20 20 20 7b 24  |   uses.      {$|
  4.  

Whereas If I just copy the code:
Code: Text  [Select][+][-]
  1. 00000000  7b 24 6d 6f 64 65 20 6f  62 6a 66 70 63 7d 7b 24  |{$mode objfpc}{$|
  2. 00000010  48 2b 7d 0a c2 a0 0a 75  73 65 73 0a 20 20 7b 24  |H+}....uses.  {$|
  3. 00000020  69 66 64 65 66 20 75 6e  69 78 7d 0a 20 20 63 74  |ifdef unix}.  ct|
  4.  

But I digress....
I only tried to compile/run it in the first place because @Weiss said FPC would not accept it...





Title: Re: What is buffer and how to use it?
Post by: Weiss on November 20, 2022, 02:15:24 am
What version are you using?

3.2.2.

I went over your example again and noticed that you DO have SysUtils in uses clause. 

I can't copy-paste, have to read from screen and re-type on my other little laptop. Must have missed it while typing.

Or probably, omitted SysUtils as soon as I saw the syntax SysUtils.Sleep(), kind of presumed it is possible in FPC. But it is not.

So, the call SysUtils.Sleep() is not possible without including SysUtils in uses. I checked other calls, too. Mouse.GetMouseY does not compile without including mouse in uses. I definitely misunderstood what you were talking about, please expand on what you meant here:

It's just for clarity, FPC and Delphi are the same in this regard, no need to qualify if you're sure which one to call of you want to override through the uses clause order.
Title: Re: What is buffer and how to use it?
Post by: Leledumbo on November 20, 2022, 10:13:24 pm
So, the call SysUtils.Sleep() is not possible without including SysUtils in uses. I checked other calls, too. Mouse.GetMouseY does not compile without including mouse in uses. I definitely misunderstood what you were talking about, please expand on what you meant here:
<unit name>.<identifier> is a way to explicitly say where the compiler should look the identifier for, regardless of uses clause order. So if you have:
Code: Pascal  [Select][+][-]
  1. uses
  2.   SysUtils,Windows;
  3. ...
  4. Sleep();
  5.  
It will refer to Windows.Sleep() as Windows is placed after SysUtils. But if you have:
Code: Pascal  [Select][+][-]
  1. uses
  2.   Windows, SysUtils;
  3. ...
  4. Sleep();
  5.  
It will be SysUtils.Sleep() to call. So if you want to be sure that an identifier you call is from a specific unit, regardless of the order in the uses clause, you qualify it with the unit name.
Code: Pascal  [Select][+][-]
  1. uses
  2.   SysUtils,Windows;
  3. ...
  4. SysUtils.Sleep();
  5.  
will always call SysUtils.Sleep(), even if Windows unit also has Sleep() identifier. OTOH, you might want the totally opposite case. You want to control which identifier to call solely from the uses clause. This is a common practice in Pascal, one notable example is from fcl-web. You can have this:
Code: Pascal  [Select][+][-]
  1. uses
  2.   fphttpapp;
  3. ...
  4. begin
  5.   Application.Initialize;
  6.   Application.Run;
  7. end.
  8.  
Just by changing the fphttpapp to fpCGI or fpFCGI, you web app magically switches to CGI or FastCGI, respectively, from embedded server.
Title: Re: What is buffer and how to use it?
Post by: Weiss on November 21, 2022, 12:39:23 am
appreciated! Thank you Leledumbo.
TinyPortal © 2005-2018