Recent

Author Topic: Linux serial comms: a heads-up  (Read 5266 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Linux serial comms: a heads-up
« on: October 08, 2021, 08:59:53 am »
Everybody agrees that the days of the RS232 serial port are past, but serial comms APIs still tend to be used heavily when talking to USB-connected equipment: bench meters and so on, often via an FTDI chip. In the particular case I've got here I've got an HP/IB/GP-IB/IEEE-488 interface that looks like a CDC device.

In conjunction with the standard serial.pp unit, this code may be used to check how much data the OS has buffered prior to a SerRead() etc.:

Code: Pascal  [Select][+][-]
  1. function SerAvailable(handle: TSerialHandle): integer;
  2.  
  3. begin
  4.   if handle < 0 then
  5.     result := -1
  6.   else begin
  7.     if fpIoctl(handle, FIONREAD, @result) <> 0 then
  8.       result := -1
  9.   end
  10. end { SerAvailable } ;
  11.  

However the result appears to be limited to the range 0..4095, and this appears to be a strict limit rather than a truncated number of bits. Obviously this can have implications if trying to deal with large blocks efficiently.

Above on Linux 4.19 x86_64, other platforms untested but probably affected.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

geraldholdsworth

  • Jr. Member
  • **
  • Posts: 96
Re: Linux serial comms: a heads-up
« Reply #1 on: October 18, 2021, 02:01:33 pm »
Everybody agrees that the days of the RS232 serial port are past
Not everybody - we regularly install AV equipment which communicates over RS232. OK, it may be past in the PC world, but is still very much still around.

PascalDragon

  • Hero Member
  • *****
  • Posts: 3505
  • Compiler Developer
Re: Linux serial comms: a heads-up
« Reply #2 on: October 19, 2021, 09:01:23 am »
Everybody agrees that the days of the RS232 serial port are past,

At work we use this day in day out for debugging our software (which runs standalone without an OS) on various systems. So I can't agree here. ;)

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #3 on: October 19, 2021, 09:25:18 am »
:-) I've got RS-232 interfaces to modems and possibly a UPS, but apart from that I find embedded USB serial chips (including many fake FTDI) much more common. Even the HP-IB interface to my logic analyser and 'scope looks like one.

In any event, it would probably be wisest to assume that this is a general serial API limitation, rather than being driver-specific. And while 4K serial data blocks are probably fairly rare, I noticed this when parsing a data dump arriving from an instrument that didn't use handshaking... the transfer itself is fine, it's only that one ioctl that is a problem.

Edited after consideration: As far as I know it's only that call that's a problem. Buffered incoming data is definitely not being lost, and I presume that somebody would have spotted if Linux syscalls were size-limited.

This only came to light when I wrote some functions to allow backtracking, i.e. to allow me to put data back into a local buffer for a different parser to try. So if you've just put a few bytes back, you might need to be able to find out if you can read a 5K block yet.

MarkMLl
« Last Edit: October 19, 2021, 09:43:21 am by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: Linux serial comms: a heads-up
« Reply #4 on: October 19, 2021, 10:19:58 am »
You can do: TStringList.WriteToFile("COM1").

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #5 on: October 19, 2021, 11:08:58 am »
No I can't.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

af0815

  • Hero Member
  • *****
  • Posts: 771
Re: Linux serial comms: a heads-up
« Reply #6 on: October 19, 2021, 11:32:14 am »
I see at https://www.freepascal.org/docs-html/rtl/baseunix/fpioctl.html and https://man7.org/linux/man-pages/man4/tty_ioctl.4.html the argument is int. So this is not the limitation.

is the limitation for 0..4095 by system ? Can the Hardware/inputbuffer have more than 4095 Bytes ?!




regards
Andreas

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #7 on: October 19, 2021, 12:00:00 pm »
I see at https://www.freepascal.org/docs-html/rtl/baseunix/fpioctl.html and https://man7.org/linux/man-pages/man4/tty_ioctl.4.html the argument is int. So this is not the limitation.

is the limitation for 0..4095 by system ? Can the Hardware/inputbuffer have more than 4095 Bytes ?!

Only tested on x86_64 (Debian Linux) at the moment, and it appears to be that one specific ioctl: as I've said, no incoming data is lost. If I were making a wild guess I's say that it means "I'm using more than one page in a linked list for this, and you surely don't want me to walk the whole kirottu thing?".

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: Linux serial comms: a heads-up
« Reply #8 on: October 19, 2021, 12:39:05 pm »
No I can't.

MarkMLl

Well, ok, like this:

Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. uses Sysutils, Classes;
  4.  
  5. var
  6.   MyList: TStringList;
  7. begin
  8.   MyList := TStringList.Create;
  9.   MyList.Add('Bla');
  10.   MyList.SaveToFile('COM1');
  11. end.    
  12.  

And only on Windows, if the port is used and opened.


MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #9 on: October 19, 2021, 12:46:25 pm »
@SymbolicFrank, with apologies to the mods I'm about to waste far more time on you than you deserve.

Did you bother to read the OP, where was made quite clear it was a Linux problem?

Did you bother to read the OP, where was made quite clear it was a problem with one specific ioctl?

Did you bother to read the OP, where was made quite clear that the ioctl was used to read the amount of input data currently buffered?

If you can't be bothered to read the OP, then I suggest that reading https://monologues.co.uk/004/Owl-Critic.htm would be appropriate.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: Linux serial comms: a heads-up
« Reply #10 on: October 19, 2021, 12:50:17 pm »
Hi Mark,

I did. There was a discussion about RS-232 being past, to which I also contributed.

Yes, the last time I tried to communicate with a specific USB device in Linux, it was definitely non-trivial. And no, I cannot help you with the bug.

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #11 on: October 19, 2021, 01:13:21 pm »
Yes, the last time I tried to communicate with a specific USB device in Linux, it was definitely non-trivial.

The communications as such isn't a problem: serial.pp works well for serial devices and in general libusb works well... in both cases subject to access rights.

If I had time to do a bit of maintenance (or more specifically, the required testing on multiple OSes) on serial.pp I'd change it to (a) lock the port by default since that's Windows behaviour and is generally what's wanted and (b) add that "what's buffered?" function if non-Linux unix variants supported it.

Walking the /sys tree is a hassle, since different drivers can put things in slightly different places... cdc_acm appearing to be one specific problem. However I've now checked over the code I've previously posted at e.g. https://github.com/MarkMLl/hp2671/blob/main/hp2671/locatecdcacmport.pas to centralise its handling of different types of chip, and it's definitely possible- albeit arduous- to have a general solution.

I believe that I've mentioned in another thread that FTDI chips have a unique serial number in their EEPROM, which makes them particularly useful since e.g. two ends of a comms link can be easily distinguished during testing. Unfortunately the majority of "FTDI" chips are actually counterfeits, and in at least some cases the standard utilities can't rewrite the configuration... I need to take a debugger to the code involved (it's C, which is not something I'm particularly looking forward to) to see if that's a mere software problem or if the counterfeit firmware had its R/W capability removed after FTDI bricked a whole lot of questionable devices.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #12 on: October 20, 2021, 01:14:55 pm »
If I had time to do a bit of maintenance (or more specifically, the required testing on multiple OSes) on serial.pp I'd change it to (a) lock the port by default since that's Windows behaviour and is generally what's wanted and (b) add that "what's buffered?" function if non-Linux unix variants supported it.

Also I've just spotted that (c) SerSetParams() starts off from a zeroed state rather than getting the current line state which is considered to be correct practice. Somehow my system's in a state where by default incoming CR is changed into NL, which sent me looking at initialisation...

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 736
Re: Linux serial comms: a heads-up
« Reply #13 on: October 20, 2021, 03:10:18 pm »
Yes, locating the right USB device, accessing and locking it is the main problem.

IIRC, long ago, with Unix, you had serial and block devices. You could directly copy to and from them, like they were files. With some restrictions, the serial devices were like pipes. Linux didn't do that at first, although they made an effort to make it like that about a decade ago. The status- and info-files are a great example. And a few years ago they abandoned that idea again and made separate device classes for each type of interface. Flexible, yes, but you need different tools and libraries for each one.

The only thing that survived it all is the Berkeley TCP/IP stack and sockets, which are part of every OS that has internet that I know of.

Ok, the main culprit for the complexity of USB devices is Microsoft, who wanted everything to be their vision of Plug-and-Play. And it is clearly designed by a committee. But I think KISS should be used for all layers, not just the top one.

MarkMLl

  • Hero Member
  • *****
  • Posts: 3515
Re: Linux serial comms: a heads-up
« Reply #14 on: October 20, 2021, 03:32:54 pm »
No, Linux has always had character and block devices: it's a fundamental unix-like feature. It's obviously also always had an IP stack with TCP and UDP on top (I can't say when it stopped badging that as Swansea-derived NET4), and I think it had unix-domain sockets etc. fairly early as well. I'm pretty sure that its serial handling was orthodox, except that setserial had some PC-only facilities which would quite simply have been meaningless on e.g. an Ethernet-connected terminal server.

The two areas where it is rather odd are excessively-heavy use of the /proc tree (compare that with SunOS etc. where it is far more process-oriented) and increasingly-heavy emphasis on named device files being there purely to provide a handle for subsequent ioctl calls. The latter in particular is very much "un-unix" and in particular "un-Plan9", since it moves the APIs progressively further from the "everything is a file" model. Then there are things that have never been in the filesystem namespace at all, like network interfaces.

I'm all for purity and uniformity, although it gets very difficult to imagine how you could implement a rich API like OpenGL as a file. Unless, of course, you reverted to something like Display Postscript... which would probably be redefined these days in terms of XML.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018