Recent

Author Topic: Porting Delphi application to Lazarus then Linux, what about serial comm?  (Read 7010 times)

BosseB

  • Sr. Member
  • ****
  • Posts: 468
I am trying to port a Windows Service application written in Delphi7/2007 back in 2004-2012 to FPC/Lazarus so it can later be moved to Linux.
This application talks via RS232 to a data collection system for control and data retrieval.

Unfortunately there was no built-in support in Delphi for serial comm so we used external components for this, at first a very simple TSerial component from a developer in the UK and later Turbo Async Pro.
I have conditionals in the code to switch between the two components.

The TSerial component is contained in a single source file whereas AsyncPro is splitting its serial component in a large set of units, very hard to navigate.

In both cases the application is written using the event model for receiving data and non-blocking send functions.

I would rather not port these components (at least not Async Pro) since they are inherently Windows centric.
Instead I want to use the FPC built-in Serial component, but it needs to be packaged with a wrapper such that an OnRxData event is added and such that there is a non-blocking transmit.

It must be possible to set the RX/Tx buffer size too.

What is the best way to accomplish the move of serial comm to FPC/Lazarus in such a way that it is portable to Linux?

Is there a way to add receive events and non-blocking send function to the built-in serial object in FPC?
I mean if I make a custom component by inheriting from the built-in serial, how can these extra features be added?

Maybe it has already been done?

EDIT:
After posting I found LazSerial, which can be installed using OnLine PackageManager.
But when I look at it it has these read and write functions defined like this:

Code: Pascal  [Select][+][-]
  1.     // read data from port
  2.     function DataAvailable: boolean;
  3.     function ReadData: string;
  4. //    function ReadBuffer(var buf; size: integer): integer;
  5.  
  6.     // write data to port
  7.     function WriteData(data: string): integer;
  8.     function WriteBuffer(var buf; size: integer): integer;

As you can see the ReadBuffer method is not implemented.
Since I will read/write non-textual binary data I need a Read method that can handle binary buffers and not work with strings. Strings are hard to use since there is inherent interpretation of unicode and such, which is not allowed on binary data.

How is this solved for binary transfers? On the send side there is a WriteBuffer() method but not for receiving data....



« Last Edit: April 25, 2020, 01:10:43 am by BosseB »
--
Bo Berglund
Sweden

PaulRowntree

  • Full Member
  • ***
  • Posts: 132
    • Paul Rowntree
I've written a wrapper around LazSerial that allows string or fixed character count receiving, and it will fire events either on line end or N bytes .... seems to work when communicating with an embedded device I use Windows, but it should port.  Does this sound like what you want?
Paul Rowntree
- coding for instrument control, data acquisition & analysis, CNC systems

BosseB

  • Sr. Member
  • ****
  • Posts: 468
Quote
Does this sound like what you want?
Might be!
What I am after is a replacement for the external serial components that will allow porting to Linux and still will work on Windows 10. So it should be part of Lazarus itself or an accepted contribution that can be expected to stay available and supported.

What I have in Delphi works on Windows XP..7 but has issues on Win10.

Since I moved on to Delphi XE5 I have grown very cautious with using string as buffers of any kind since they are originally designed as containers for text and for example in Delphi9 were silently changed from AnsiString (array of bytes) to unicode where the characters can occupy 1 or several bytes each. Not usable as a byte buffer then.

So whenever I move a piece of code that uses strings as buffers (we have many around) I try to change that to use TBytes or similar byte buffers.

That is why I got suspicious when the LazSerial Write method has two versions, one using string and one using a buffer of bytes, but the same is not true for incoming data. Only a single ReadData method.
And a commented out ReadBuffer method! Looks like they were trying to create a byte size read function but failed....
I also note that the declaration:
Code: Pascal  [Select][+][-]
  1. function WriteBuffer(var buf; size: integer): integer;
uses a var declaration of the buf argument!
Why should one have var for a container that holds outgoing data?

--
Bo Berglund
Sweden

wp

  • Hero Member
  • *****
  • Posts: 11855
What I am after is a replacement for the external serial components that will allow porting to Linux and still will work on Windows 10. So it should be part of Lazarus itself or an accepted contribution that can be expected to stay available and supported.
FPC has a serial unit. People, however, usualy use Synaser from which is part of the synapse libarary (use OPM to get it). LazSerial is based on it.

Since I moved on to Delphi XE5 I have grown very cautious with using string as buffers of any kind since they are originally designed as containers for text and for example in Delphi9 were silently changed from AnsiString (array of bytes) to unicode where the characters can occupy 1 or several bytes each. Not usable as a byte buffer then.

So whenever I move a piece of code that uses strings as buffers (we have many around) I try to change that to use TBytes or similar byte buffers.
When Lazarus talks of "string" it usually means 1-byte string elements (char), not 2-byte elements (widechar) like Delphi. Although a "string" buffer is basically equivalent to a TBytes buffer, but I'd prefer to use a TBytes or an array of bytes buffer or at least RawByteString, because otherwise FPC may perform undesired code page conversions with strings when bytes > 128 are encountered.

Code: Pascal  [Select][+][-]
  1. function WriteBuffer(var buf; size: integer): integer;
uses a var declaration of the buf argument!
Why should one have var for a container that holds outgoing data?
The untyped var parameter buf is a way to allow for an arbitrary type here. The calling routine can have an array[0..1000] of byte have here, or a Qword (size = 4) or whatever. It is the Pascal way of passing a pointer to a buffer of any type. Be careful with strings, though, because they are pointers; you should use the first character.

BosseB

  • Sr. Member
  • ****
  • Posts: 468
What I am after is a replacement for the external serial components that will allow porting to Linux and still will work on Windows 10. So it should be part of Lazarus itself or an accepted contribution that can be expected to stay available and supported.
FPC has a serial unit. People, however, usually use Synaser from which is part of the synapse libarary (use OPM to get it). LazSerial is based on it.
Yes, I saw that it is based on SynaSer, which is a blocking library if I am not wrong.
To port the application I need to use an event driven receive which seems to be available in LazSerial.
Quote
When Lazarus talks of "string" it usually means 1-byte string elements (char), not 2-byte elements (widechar) like Delphi. Although a "string" buffer is basically equivalent to a TBytes buffer, but I'd prefer to use a TBytes or an array of bytes buffer or at least RawByteString, because otherwise FPC may perform undesired code page conversions with strings when bytes > 128 are encountered.
Yes that is what I was afraid of when using string. Widechar conversion and codepage interactions with my binary data...
Quote
Code: Pascal  [Select][+][-]
  1. function WriteBuffer(var buf; size: integer): integer;
uses a var declaration of the buf argument!
Why should one have var for a container that holds outgoing data?
The untyped var parameter buf is a way to allow for an arbitrary type here. The calling routine can have an array[0..1000] of byte have here, or a Qword (size = 4) or whatever. It is the Pascal way of passing a pointer to a buffer of any type. Be careful with strings, though, because they are pointers; you should use the first character.

OK, then I understand. It is still a write to the serial output function then...

So a followup question on the handling:
I looked at how the ReadData is implemented:
Code: Pascal  [Select][+][-]
  1. function TLazSerial.ReadData: string;
  2. begin
  3.   result:='';
  4.   if FSynSer.Handle=INVALID_HANDLE_VALUE then
  5.     ComException('can not read from a closed port.');
  6.   if FRcvLineCRLF then
  7.   result:=FSynSer.RecvString(0)
  8.   else
  9.   result:=FSynSer.RecvPacket(0);
  10. end;
  11.  
From this it seems like it uses the property FRcvLineCRLF to select between two ways of reading the actual data, String vs Packet.
I don't know what FRcvLineCRLF actually does but to me it seems like it handles the data as string if this property is set true and by the name it also implies that somehow it will look for CRLF in the data stream...
Seems like it waits for CRLF before returning the data, not what I want..

Since I am using TBytes as internal buffers for processing, I just briefly tested the following code, and it compiles OK:

Code: Pascal  [Select][+][-]
  1.   TForm1 = class(TForm)
  2.     SerialConn: TLazSerial;
  3.     procedure SerialConnRxData(Sender: TObject);
  4.   private
  5.     FMyBuffer: TBytes;
  6.   public
  7.  
  8.   end;
  9.  
  10. var
  11.   Form1: TForm1;
  12.  
  13. implementation
  14.  
  15. {$R *.lfm}
  16.  
  17. { TForm1 }
  18.  
  19. procedure TForm1.SerialConnRxData(Sender: TObject);
  20. {MyBuffer is a TBytes object}
  21. var
  22.   tmp: RawByteString;
  23.   len, p: integer;
  24. begin
  25.   tmp := SerialConn.ReadData();
  26.   len := Length(tmp);
  27.   if len = 0 then exit;
  28.   //Extend buffer, then add incoming data to end
  29.   p := Length(FMyBuffer);
  30.   SetLength(FMyBuffer, p + len);
  31.   Move(tmp[1], FMyBuffer[p], len);
  32. end;
  33.  

Is this OK, does tmp typed as RawByteString prohibit any codepage expansions inside the handling in SynaSer and elsewhere?
--
Bo Berglund
Sweden

BosseB

  • Sr. Member
  • ****
  • Posts: 468
Back after some testing on windows, now moving to Linux (Raspbian Buster on RPi4).
I am getting problems installing the package available in On Line Package Manager as
LazSerial.

During rebuild of Lazarus I get this error, it is repeated many times with a different identifier:
Code: Text  [Select][+][-]
  1. lazsynaser.pas(252,15) Error: Identifier not found "B500000"
  2. lazsynaser.pas(252,22) Error: Illegal expression
  3. lazsynaser.pas(253,14) Error: Identifier not found "B576000"
  4. lazsynaser.pas(253,21) Error: Illegal expression
  5.  

The code section it appears in is this inside lazsynaser:
Code: Pascal  [Select][+][-]
  1.   {$IFDEF UNIX}
  2.     ,(500000, B500000),
  3.     (576000, B576000),
  4.     (921600, B921600),
  5.     (1000000, B1000000),
  6.     (1152000, B1152000),
  7.     (1500000, B1500000),
  8.     (2000000, B2000000),
  9.     (2500000, B2500000),
  10.     (3000000, B3000000),
  11.     (3500000, B3500000),
  12.     (4000000, B4000000)
  13.   {$ENDIF}

If I rightclick an identifier hat does not trigger the problem then I can go to definition and wind up inside an inc file in FPC!
It is the fpc/3.0.4/rtl/linux/termios.inc file

So each of these generate a pair of errors.
I first tried to install the package into Lazarus 2.0.6, which I had on my Raspberry Pi4 already. When that failed I installed 2.0.8 like I have on Windows instead.
But no go, exactly the same error when I try to install it into Lazarus 2.0.8.

So is LazSerial not compatible with Raspbian or what else could be wrong here?
How can I debug this, it is way outside my knowledge to dig into package sources...
« Last Edit: April 28, 2020, 11:50:54 pm by BosseB »
--
Bo Berglund
Sweden

BosseB

  • Sr. Member
  • ****
  • Posts: 468
You need to open that file to get a full view of the fpc Processor script.
I edited the post above to mention the file that should hold the definition but does not.
It is located in the fpc 3.0.4 rtl/linux directory and named termios.inc

I do not understand what you mean by "get a full view of the fpc Processor script"
Quote
Looks like it isn't seeing the target correctly and is getting lost in its define.

In other words, those constants don't exists anywhere prior to..
Above the constants I showed are those that do not trigger the error and these are located in termios.inc in the fpc rtl/linux directory. It exists on Raspbian but I don't believe it is in a Windows installation (not needed there apparently).

Am I the first user who is trying to use LazSerial on Linux???
--
Bo Berglund
Sweden

tetrastes

  • Sr. Member
  • ****
  • Posts: 473
Raspian is ARM?
From rtl/linux/termios.inc
Code: Pascal  [Select][+][-]
  1. {$ifdef cpuarm}
  2. ...
  3.    CBAUD   = $000100F;
  4.    B0      = $0000000;
  5.    B50     = $0000001;
  6.    B75     = $0000002;
  7.    B110    = $0000003;
  8.    B134    = $0000004;
  9.    B150    = $0000005;
  10.    B200    = $0000006;
  11.    B300    = $0000007;
  12.    B600    = $0000008;
  13.    B1200   = $0000009;
  14.    B1800   = $000000A;
  15.    B2400   = $000000B;
  16.    B4800   = $000000C;
  17.    B9600   = $000000D;
  18.    B19200  = $000000E;
  19.    B38400  = $000000F;
  20.    EXTA    = B19200;
  21.    EXTB    = B38400;
  22.    CSIZE   = $0000030;
  23.      CS5   = $0000000;
  24.      CS6   = $0000010;
  25.      CS7   = $0000020;
  26.      CS8   = $0000030;
  27.    CSTOPB  = $0000040;
  28.    CREAD   = $0000080;
  29.    PARENB  = $0000100;
  30.    PARODD  = $0000200;
  31.    HUPCL   = $0000400;
  32.    CLOCAL  = $0000800;
  33.    CBAUDEX = $0001000;
  34.    B57600  = $0001001;
  35.    B115200 = $0001002;
  36.    B230400 = $0001003;
  37.    B460800 = $0001004;
  38.  
  39.    CIBAUD  = $100F0000;
  40.    CMSPAR  = $40000000;
  41.    CRTSCTS = $80000000;
  42. ...
  43. {$endif cpuarm}
  44.  
You can see that the last is B460800.
So edit synaser.pas smth like
Code: Pascal  [Select][+][-]
  1. {$IFDEF UNIX}
  2. {$ifndef cpuarm}
  3.    ,(500000, B500000),
  4.     (576000, B576000),
  5.     (921600, B921600),
  6.     (1000000, B1000000),
  7.     (1152000, B1152000),
  8.     (1500000, B1500000),
  9.     (2000000, B2000000),
  10.     (2500000, B2500000),
  11.     (3000000, B3000000),
  12.     (3500000, B3500000),
  13.     (4000000, B4000000)
  14. {$endif}
  15. {$ENDIF}
  16.  
And also for MaxRates little above. 
« Last Edit: April 29, 2020, 12:57:25 am by tetrastes »

BosseB

  • Sr. Member
  • ****
  • Posts: 468
@tetrastes MANY THANKS!

I edited the following file:
~/.lazarus_2.0.8/onlinepackagemanager/packages/LazSerial/lazsynaser.pas

And added conditionals for cpuarm as you described above.
What you did not show was the fix for MaxRates, which I modified as follows:

Code: Pascal  [Select][+][-]
  1.   const
  2.   {$IFDEF UNIX}
  3.     {$IFDEF DARWIN}
  4.       MaxRates = 18;  //MAC
  5.     {$ELSE}
  6.       {$ifdef cpuarm} //Raspberry Pi
  7.          MaxRates = 19;
  8.       {$else}
  9.          MaxRates = 30; //UNIX
  10.       {$endif}
  11.     {$ENDIF}
  12.   {$ELSE}
  13.     MaxRates = 19;  //WIN
  14.   {$ENDIF}
  15.  

After these two changes were done I could successfully rebuild my Lazarus with the LazSerial package included. :)

QUESTION:
This solution should probably be notified to the maintainer of the OLPM package LazSerial, how do I get in touch with the maintainer?
--
Bo Berglund
Sweden

BosseB

  • Sr. Member
  • ****
  • Posts: 468
You think you are the first to use it on Linux ?
It seems like I am the first to do it on Raspberry Pi, which uses an ARM CPU and Raspbian, which is based on Debian Buster....
--
Bo Berglund
Sweden


BosseB

  • Sr. Member
  • ****
  • Posts: 468
It seems like I am the first to do it on Raspberry Pi
No. You should work on your forum search skills.
https://forum.lazarus.freepascal.org/index.php/topic,20481.msg299747.html#msg299747
https://forum.lazarus.freepascal.org/index.php/topic,43935.msg308730.html#msg308730
https://forum.lazarus.freepascal.org/index.php/topic,44339.msg311738.html#msg311738
https://forum.lazarus.freepascal.org/index.php/topic,19636.msg141722.html#msg141722
OK, thanks for the linked pages, they explain a lot.

I assumed that since I installed the component via OPM today in April 2020 it would be fixed if someone else had used it before and failed like me on RPi ARM platform.

Your linked pages suggest that it is a known issue since at least April 2018 and yet it is not updated in the LazSerial OPM package...

I would not have thought this possible....

How do I get in contact with the maintainer of the OPM LazSerial package so it can be fixed for all new users as well?

I have made notes to myself for future installations, but I guess others will bump into this even if I am in the clear.
--
Bo Berglund
Sweden

wp

  • Hero Member
  • *****
  • Posts: 11855
Your linked pages suggest that it is a known issue since at least April 2018 and yet it is not updated in the LazSerial OPM package...

I would not have thought this possible....
OPM is just distributing the packages, it is impossible for a single person to maintain the huge number of the packages provided. When the original developer of a package has decided that the package is "complete" and he leaves it alone and focusses on something else, but some changes in hardware, FPC or Lazarus cause problems later he probably may not notice it, and the things that you describe will happen. That's life in a community project... If you want it to be fixed try to contact the author: search the forum for "LazSerial", oldest topics first, you'll find that the original author is "Jurassic Pork". Send him a PM, and hopefully he will fix it.

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
I use TComport on both Delphi (XE10-seattle) and FPC, but in both cases only on win32/64

BosseB

  • Sr. Member
  • ****
  • Posts: 468
the original author is "Jurassic Pork". Send him a PM, and hopefully he will fix it.
THanks,
I have just sent him a message about this.
Hopefully he will correct the file and post a package update.
--
Bo Berglund
Sweden

 

TinyPortal © 2005-2018