* * *

Author Topic: TLazSerial: reading and displaying multiple characters  (Read 3240 times)

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #15 on: April 28, 2017, 01:43:02 am »
The RecvTerminated function is in synser:
Code: [Select]
Str := LazSerial1.synser.RecvTerminated(150,'?');That may work for the test string, but that is only one function of the serial port. When I send a "Start" command, which is Ctrl_A, the device will send a continuous stream of byte pairs, which can be anything from 0x00 to 0xff. I want to store them in a circular queue for a time period of 100 to 300 mSec, which corresponds to 480 to 1440 bytes. When the timer event fires, I will be parsing the data as byte pairs which contain 6 bits of a 12 bit ADC value, and two bits which increment as 00, 01, 10, 11, 00, ... These values are used to synchronize the data stream. Each valid pair will start with 00 or 10 and the second byte of the pair will be 01 or 11. If not, it means that a data byte has been missed, and the data pair will be discarded. The errors are tracked and usually are very uncommon.

Using my previous serial port components, SerialNG and ComDrv32, the serial port continues to run and fills the main buffer, while the timer tick processes the data received up to the time of the timer tick. This is what I am trying to do with the "OnTimer" event. The application error you experienced was probably because of the "SetFocus" when the TMemo had not yet been created. I removed it and the try..except caught and handled the attempt to open a non-existent COM port.

After removing the "Memo1.SetFocus" from the timer tick, the Open and Close buttons seem to be more responsive. So perhaps they need to have focus on the main form rather than the memo, in order to work.

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 620
Re: TLazSerial: reading and displaying multiple characters
« Reply #16 on: April 28, 2017, 03:34:24 am »
you can also try something like this :
Code: Pascal  [Select]
  1. procedure TfmSerialTest.LazSerial1RxData(Sender: TObject);
  2. var Str : string;
  3. begin
  4. if   LazSerial1.SynSer.WaitingDataEx < 479 then exit;
  5. CharCount := LazSerial1.SynSer.WaitingDataEx;
  6. eCharCount.Text:= IntToStr(CharCount);
  7. CharCount := 0;
  8. Str :=  LazSerial1.ReadData;
  9. if LazSerial1.SynSer.LastError = 0 then
  10. begin
  11.     Memo1.Lines.BeginUpdate;
  12.     Memo1.Lines.Add(Str);
  13.     Memo1.Lines.EndUpdate;
  14.     Memo1.SelStart := Length(Memo1.Lines.Text)-1;
  15.     Memo1.SelLength:=0;
  16. end
  17. Else
  18.   begin
  19.      Memo1.Lines.Add('Error : ' + LazSerial1.SynSer.LastErrorDesc );
  20.   end;
  21.  
  22. end;  

Wait for a number of bytes  in the receive buffer  before to do something

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #17 on: April 28, 2017, 06:44:10 am »
That might be useful for the purpose of reading the 240 characters of the test response. But if there is a problem and characters are skipped, it would always exit on the first line. There needs to be a timeout so that the receive buffer is emptied and the number of characters will determine if the COM port and device are working properly.

My application is actually rather complex, and it uses the functions in the attached CommLib unit. Actually my test of the received data is very simple:
Code: Pascal  [Select]
  1. //************************ ReadTestString (CommTest) **************************
  2. function TfmCommLib.ReadTestString: Boolean;
  3. begin
  4.   if (fmCommLib.CommCharCount > 200) and (fmCommLib.CommBuffer[5] = '1') then
  5.     Result := True
  6.   else begin
  7.     Result := False;
  8.     end;
  9. end;
The conversion of the received ADC data is parsed, converted, and checked for errors like this:
Code: Pascal  [Select]
  1. //************************ ReadBuffer *******************************
  2. // ReadBuffer converts two bytes to an integer representing current data
  3. // The CommRead pointer and RxDataCount increments and the CommCharCount decrements
  4. // Actual data 0-4095 is offset by 2048 for bipolar current
  5. // Returns signed integer data
  6.  
  7. function TfmCommLib.ReadBuffer : Integer;
  8. var
  9.   ch1, ch2: Char;
  10.   RecdData: Word;
  11.   DataChk: SmallInt;
  12.  
  13. begin
  14.      RecdData := 2048;  //Default for zero
  15. //     CommCriticalSection.Enter;
  16.      if CommCharCount >= 2 then begin
  17.        ch1 := CommBuffer[ CommReadPtr ];  // Read first character
  18.        dec( CommCharCount );
  19.        inc( CommReadPtr );
  20.        if CommReadPtr > MAXCOMM then CommReadPtr := 0;
  21.        if (Byte(ch1) and $01 <> 0) then begin
  22.          // Wrong sequence, exit and read next byte
  23.          Inc(RxError);                    // Bit 0 of LSB must be 0
  24.          CommError := CommError or E_FIRSTCHAR; //eFirstChar;
  25. //         fmComm.Visible := True;
  26.          result := 0;
  27.          CommCriticalSection.Leave;
  28.          If CommError > 0 then
  29.            FirstChar := ch1;
  30.          exit;
  31.          end;
  32.  
  33.        ch2 := CommBuffer[ CommReadPtr ];  // Read second character
  34.        dec( CommCharCount );
  35.        inc( CommReadPtr );
  36.        if CommReadPtr > MAXCOMM then CommReadPtr := 0;
  37.        if (Byte(ch2) and $01 <> 1) then begin
  38.          // Wrong sequence, exit and read next byte
  39.          Inc(RxError);                    // Bit 0 of LSB must be 0
  40.          CommError := CommError or E_SECONDCHAR;  //eSecondChar;
  41. //         fmComm.Visible := True;
  42.          result := 0;
  43. //         CommCriticalSection.Leave;
  44.          If CommError > 0 then
  45.            SecondChar := ch2;
  46.          exit;
  47.          end;
  48.  
  49.        DataChk := BYTE(ch1) AND $02;
  50.        If (BYTE(ch2) AND $02) <> DataChk then begin
  51.          Inc(RxError);                        // Two low bits must be equal
  52.          CommError := CommError or E_CHARMATCH;  //eCharMatch;
  53. //         fmComm.Visible := True;
  54.          Result := 0;
  55.          end
  56.        else begin
  57.          RecdData := BYTE(ch2) AND $FC;      // MSB
  58.          RecdData := RecdData shl 6;
  59.          RecdData := RecdData OR BYTE(ch1);  // LSB
  60.          RecdData := RecdData AND $0FFC;     // 10 bit precision
  61.          Result := Integer(RecdData) - 2048;   // Actual signed data
  62.          end;
  63.        end
  64.      else // If CommCharCount < 2 (ERROR)
  65.        Result := 0;                          // Default if error
  66. //  CommCriticalSection.Leave;
  67. end;
I was curious to learn more about implementing serial communication in Windows, and I found the following article:
http://www.codeguru.com/cpp/i-n/network/serialcommunications/article.php/c5425/Serial-Communication-in-Windows.htm

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #18 on: April 28, 2017, 09:18:00 am »
I have now connected my other serial port device, which sends a character stream when Ctrl-A is sent, and stops when Ctrl-C is sent. The hardware works correctly, and stops sending the data, but the OnRxData event continues to fire. I see that this code is also being executed:
Code: Pascal  [Select]
  1. { TComPortReadThread }
  2.  
  3. procedure TComPortReadThread.CallEvent;
  4. begin
  5.   if Assigned(Owner.FOnRxData) then begin
  6.     Owner.FOnRxData(Owner);
  7.   end;
  8. end;    

Apparently it comes from:
Code: Pascal  [Select]
  1. procedure TComPortReadThread.Execute;
  2. begin
  3.   try
  4.     while not MustDie do begin
  5.       if Owner.FSynSer.CanReadEx(100) then
  6.         Synchronize(@CallEvent);
  7.     end;
  8.   finally
  9.     Terminate;
  10.   end;

and:
Code: Pascal  [Select]
  1. function TLazSerial.DataAvailable: boolean;
  2. begin
  3.   if FSynSer.Handle=INVALID_HANDLE_VALUE then begin
  4.     result:=false;
  5.     exit;
  6.   end;
  7.   result:=FSynSer.CanReadEx(0);
  8. end;      
I am now getting continuous firing of OnRxData and even the following code does not stop reading when no more is being transmitted:
Code: Pascal  [Select]
  1.   if not LazSerial1.DataAvailable then
  2. //  if not LazSerial1.SynSer.CanRead(10) then
  3.     exit;
  4.  
It also seems to hang up the program and the buttons are non-responsive.

avra

  • Hero Member
  • *****
  • Posts: 1048
    • Additional info
Re: TLazSerial: reading and displaying multiple characters
« Reply #19 on: April 28, 2017, 11:15:48 am »
When it is in this state, the buttons on the form are very sluggish and don't respond except after multiple taps. Once the port is closed, the buttons act immediately, and the CanEvent code is no longer being called. This is a serious problem that renders the component unusable in my application.
This will explain why you have such problems and what would be the solution:
http://forum.lazarus.freepascal.org/index.php/topic,36656.msg244520.html
ct2laz - Easily convert components and projects between Lazarus and CodeTyphon

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #20 on: April 28, 2017, 06:12:46 pm »
I don't think that is the problem. I am not introducing any intentional delays, nor do I use the sleep() function. If I use the following for normal processing of fixed length data, and put a breakpoint on exit, execution stops after the breakpoint at RecvByte(), with either of the tests for buffer empty;
Code: Pascal  [Select]
  1.   if not LazSerial1.DataAvailable then
  2. //  if not LazSerial1.SynSer.CanRead(10) then
  3.     exit;
  4.   RecvData := LazSerial1.synser.RecvByte(0);
  5.  
It seems that the trouble occurs when I send the Ctrl-A to initiate the stream of bytes. In the RxData procedure I added:
Code: Pascal  [Select]
  1.     if CharCount > 500 then
  2.         LazSerial1.SynSer.SendByte(Ctrl_D);
  3.  
This stops the stream of bytes by sending a Ctrl-D, and my hardware device shows this. But the stream of bytes does not stop, even though the Rx/Tx LED on my device is off. However, sometimes it flashes. The data I see after sending the STOP command sometimes repeats some of the data from the TEST and CAL commands, so I think the COM buffer is not recognizing that it is empty, and continues to loop through its data from previously received bytes. It may even be sending data to the device - otherwise I don't know why it would show Tx/Rx activity.

[edit] If I close the port, transmission stops. Then if I open the port, with more than 500 characters in the CharCount, and click the START button, only about four bytes are received before it stops.

[edit2] I have verified that my hardware device is working properly. My application that uses ComDrv32 works as it should, as does a previous version that uses SerialNG. But SerialNG does not work on Windows10, and ComDrv32 often throws serious access violation errors. So I am in need of a Serial component that is reliable and proven to work on Win10 and other Windows platforms.
« Last Edit: April 28, 2017, 06:31:33 pm by PStechPaul »

avra

  • Hero Member
  • *****
  • Posts: 1048
    • Additional info
Re: TLazSerial: reading and displaying multiple characters
« Reply #21 on: April 28, 2017, 11:04:13 pm »
I don't think that is the problem.
I thought it will be clear from the link, but since it is not let me rephrase it... Whenever your application is executing code in your methods or timers, your main application thread is blocked. When it is blocked, messages are not processed and your GUI is not updated. Users see this as erratic mouse move or form delayed update or not responding behavior. Opening or closing ports takes time. Receiving or sending hundreds of bytes takes even more time. GUI can not be updated when the only thread it has is executing your code. So putting Application.ProcessMessages in loops or long time operations can help in some situations, but the only long term solution would be to create communication thread and synchronize it with your main thread to update GUI. Everything you need for both solutions is in the link I provided.

This stops the stream of bytes by sending a Ctrl-D, and my hardware device shows this. But the stream of bytes does not stop, even though the Rx/Tx LED on my device is off.
You command PIC to stop sending bytes and that's what you see. What you don't see is that already sent bytes are already in serial receive buffer and that you did nothing to empty this buffer so you will receive characters until buffer is empty.

I am in need of a Serial component that is reliable and proven to work on Win10 and other Windows platforms.
I mostly use Synaser with several types of microcontrollers, from WinXP to Win10 and several Linuxes and architectures. It is very reliable. I played a little with LazSerial (which is based on Synaser) and had no problems, but I have no 24/7 LazSerial application so I can't talk about it's reliability. If you don't like any of these maybe you will want to take a look at https://github.com/serbod/dataport. Haven't looked at it, but you might like it so I mention it.

I see that you control both sides of the communication. If you can change PIC side firmware then you can ease your self a lot. Your data would be much easier to receive properly if for example you have start and stop characters (like STX and ETX), or at least a termination string (like CRLF), and send all bytes as 2 hex ascii chars (like MODBUS ASCII). In such case you will probably have to raise from 57600 to 115200 to receive all bytes in time, but everything else would be much easier.
« Last Edit: April 28, 2017, 11:53:41 pm by avra »
ct2laz - Easily convert components and projects between Lazarus and CodeTyphon

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #22 on: April 29, 2017, 12:29:28 am »
I added "Application.ProcessMessages" to the RxData loop, as well as the onTimer event handler, and even the TBlockSerial.CanEvent . It was no more responsive than before, and eventually I got an error: "Project SerialTest raised exception class 'External: SIGSEGV'. In file 'lazserial.pas' at line 230: Application.ProcessMessages;" This was actually in "TLazSerial.DeviceClose". Same thing happened again after I removed the line from TBlockSerial.CanEvent.

AFAIK, and from my experience, the only time Application.ProcessMessages is needed is within a sleep loop.

Previously I had put a line to send the Ctrl-D STOP command when the CharCount exceeded 500. At 4800 bytes per second this would be about 100 mSec. If the receive buffer is at least 500 bytes deep, the onRxData handler should receive these bytes and put them into the application's 5000 byte buffer. Then when the timer tick occurs (now set at 200 mSec), it should empty the buffer and display the 500 bytes on the Memo. My application reports 2178 bytes received and the COM port is OFF. But the error prevents the application from closing normally.

I added an edit box where I can change the value of MaxChar, which when CharCount > MaxChar, the Ctrl_D command is sent. With it set at 250 or less, it works OK. But with 254 or more, I get the continuous stream of bytes even though the PIC has stopped sending. So it looks like the COM buffer is "byte sized".

I calculated that 240 bytes will be received in 50 mSec, so I changed the timer tick to 40 mSec. This still has problems, although the CLOSE button seems more responsive.

I can't change the way the communication is set up, because there are over 50 units in the field, and all of them have been working fine using the old SerialNG component. The ComDrv32 component also usually works. But there are problems with Win10. Perhaps I can troubleshoot and fix the problems with ComDrv32. I'll also look at the component you suggest. Thanks.


PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #23 on: April 29, 2017, 01:58:52 am »
I installed the DataPort component and modified my application to use it instead of TLazSerial. I had to do the following to make it work:
Code: Pascal  [Select]
  1. procedure TfmSerialTestDP.DataPortSerial1DataAppear(Sender: TObject);
  2.   var       RecvData: byte;
  3.             Str: String;
  4.   begin
  5.     while DataPortSerial1.SerialClient.Serial.CanRead(100) do begin
  6.     RecvData := DataPortSerial1.SerialClient.Serial.RecvByte(100);
  7.     if (not Test) or (RecvData > CR) then begin
  8.       CommBuffer[CommBufferPtr] := RecvData;
  9.       if CommBufferPtr < MAXBUFF then
  10.         inc( CommBufferPtr )
  11.       else
  12.         CommBufferPtr := 0;
  13.       inc(CharCount);
  14.       if CharCount > MaxChar then
  15.           DataPortSerial1.SerialClient.Serial.SendByte(Ctrl_D);
  16.       end;
  17.     end;
  18.   end;

It seems to work marginally better, but still will not respond to the STOP button. If I set MaxChar=1000, it runs for a long time, and reports 1004 characters. Maybe it just takes that long for the Memo to update?

Project attached.

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #24 on: April 30, 2017, 08:44:57 am »
I now have a working serial test application using TLazSerial that can handle 48,000 characters per second. It was a problem with TMemo being too slow. Now I am just processing the characters as bytes and making a total value, and I can start the stream, observe the total increasing, and then stopping the stream.

Project files attached:

avra

  • Hero Member
  • *****
  • Posts: 1048
    • Additional info
Re: TLazSerial: reading and displaying multiple characters
« Reply #25 on: May 01, 2017, 02:52:30 am »
You have done it so wrong that I don't know where to start. No wonder you have so many problems. I have attached a simple PIC message simulator, which will send 240 bytes when Ctrl+T is received. I use 2 virtual ports, but serial null cable would also be good. You have real PIC so you don't need it, but others might. Then in your project I have set DataPortSerial1.MinDataBytes to 1 and replaced DataPortSerial1DataAppear method with this:

Code: Pascal  [Select]
  1. procedure TfmSerialTestDP.DataPortSerial1DataAppear(Sender: TObject);
  2. var
  3.   Str: string;
  4.   Cnt: word;
  5. begin
  6.   Str := '';
  7.   Cnt := 0;
  8.   while DataPortSerial1.PeekSize > 0 do
  9.   begin
  10.     Str := Str + HexStr(byte(DataPortSerial1.Pull(1)[1]), 2) + ' ';
  11.     Inc(Cnt);
  12.     if (Cnt mod 20) = 0 then
  13.       Str := Str + #13 + #10;
  14.     Application.ProcessMessages; // not used just for sleep loops as you can see :-)
  15.   end;
  16.   Memo1.Append(Str + 'Received ' + IntToStr(Cnt)+' bytes' + #13 + #10);
  17. end;
  18.  

No mather how many times in a row I send Ctrl+T, I will always get the same 240 bytes as expected.

This is quick and dirty code, with strings instead of byte arrays for simplicity (a big no-no in production code), and without any threads (which would certainly not use event driven DataPort component but plain Synaser instead). Simple demonstration that I hope will help you to see that there is nothing wrong with Lazarus serial components.

If you stick to event driven serial application without threads, then I advice you to take a look at attached PicSim project for serial status monitoring, and to change the way you use timer. For example, a button could send ^T command and start a timer for timeout error handling. In DataAppear method you could then reset timer, and stop it if number of received bytes and parsing would confirm it's a good and complete message.

Hope this helps a little in your quest  ;)

UPDATE: I have just seen that you uploaded another project. I haven't looked at it but if it now works good for you then I wish you best with your application.
« Last Edit: May 01, 2017, 03:06:58 am by avra »
ct2laz - Easily convert components and projects between Lazarus and CodeTyphon

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #26 on: May 01, 2017, 08:11:35 am »
That looks like a very valuable alternative to what I have come up with. It seems that each serial port component has some common and some unique ways to open, set baud rates and other properties, notify reception of data, and accessing the data in the buffer. It may be helpful to define my own component (object) with only the properties and methods and events I need, and then use conditional compilation to allow use of various components like TLazSerial, DataPortSerial, ComDrv32, and even the original SerialNG that I used for 10 years until Win10 broke it.

There is much more that I really should do to clean up my applications. Basically they function as storage oscilloscopes that read current and store samples so the waveform can be analyzed to determine true RMS amplitude, to better than 1% accuracy, of continuous current as well as pulses of 60 Hz as short as 1 cycle (16 mSec) or as long as 30 seconds. It must also be able to read the time duration within about 1 mSec.

The application can be downloaded from www.ortmaster.com, and will run in demo mode (if the special USB serial port is not found), using proprietary format waveform files of actual tests.

Thanks!

avra

  • Hero Member
  • *****
  • Posts: 1048
    • Additional info
Re: TLazSerial: reading and displaying multiple characters
« Reply #27 on: May 01, 2017, 12:04:44 pm »
That looks like a very valuable alternative to what I have come up with.
I am glad my small demo turned out to be useful after all.

Basically they function as storage oscilloscopes that read current and store samples so the waveform can be analyzed to determine true RMS amplitude, to better than 1% accuracy, of continuous current as well as pulses of 60 Hz as short as 1 cycle (16 mSec) or as long as 30 seconds. It must also be able to read the time duration within about 1 mSec.
Well, FYI if much higher sampling rates are needed then you can achieve up to 40 Msps on BeagleBone wit PRUDAQ cape. Everything is open source and open hardware. There are Beaglebone schematics with affordable SBCs and SOMs available, and PRUDAQ cape with full schematics if you want to customize it. Beaglebone is much more capable then Raspberry Pi in this area because of 2 PRUs that run in parallel to CPU at up to 200MHz. Open source drivers fully use PRUs so CPU usage is very low, and these 40Msps can be further divided between up to 8 ADC channels. More info here:
http://linuxgizmos.com/group-buy-site-launches-40msps-adc-beaglebone-cape/

Thanks!
You're most welcome!  :D
ct2laz - Easily convert components and projects between Lazarus and CodeTyphon

PStechPaul

  • Jr. Member
  • **
  • Posts: 72
    • P S Technology, Inc.
Re: TLazSerial: reading and displaying multiple characters
« Reply #28 on: May 01, 2017, 08:55:28 pm »
This device is used only for reading current at power line frequencies (60 Hz), although it must also be able to resolve transients equivalent to perhaps 20 times that (1200 Hz), so the 2400 samples/second seemed reasonable. Sending 10 or 12 bit ADC data from the PIC required two bytes per sample, or 4800 bytes/second. With 1 start and 1 stop bit, each byte requires 10 bits, or 48000 bits/sec. The 57,600 baud rate is just barely adequate, but has been working well until the problems with Win10.

If I were designing this system today, I would certainly consider platforms such as the BeagleBone, or 32 bit processors, and perhaps do most of the real-time level sensing, data conversion, storage, and analysis in the same package, perhaps even with a display and controls, so the computer would be needed only for test result and waveform storage and report generation. But this is a $4000 industrial device used in niche industries, and the market is pretty much saturated. One can get general purpose storage scopes for well under $100.

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus