Lazarus

Free Pascal => Unix => Topic started by: TCH on May 14, 2022, 03:58:42 pm

Title: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 14, 2022, 03:58:42 pm

I have two processes, which have these lines:

Code: Pascal  [Select][+][-]

1. msgp.mtype := 1;
2. e := msgsnd(msqid, @msgp, 4, IPC_NOWAIT);

and

Code: Pascal  [Select][+][-]

1. sz := msgrcv(msqid, @msgp, 4, 1, IPC_NOWAIT);

The variable msgp is the type of

Code: Pascal  [Select][+][-]

1. type mymsg1 = record
2.         mtype: longword;
3.         mtext: array[0..7] of byte;
4. end;

Now, msgsnd() always reports no problems, that the buffer is sent. However, msgrcv() always gives back -1 with errno of ESysENOMSG, except if the first four byte of msgp.mtext are zeros. Then it succeeds, else it fails. Regardless the size of the sent buffer.

The very same algorithm and code works flawlessly with C. Any ideas?

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 17, 2022, 11:58:24 am

No ideas?

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 17, 2022, 12:13:49 pm

What library are you using? How is the socket opened?

Give us a compilable example to demonstrate the problem.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: marcov on May 17, 2022, 01:31:09 pm

• OS
• compiler target and version
• c code
• pascal code

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 17, 2022, 02:09:15 pm

If we really are obliged to read minds, I suggest that https://linux.die.net/man/2/msgrcv might be relevant, which *explicitly* says:

Code: Text  [Select][+][-]

1. The msgp argument is a pointer to caller-defined structure of the following
2. general form:
3.  
4.     struct msgbuf {
5.         long mtype;       /* message type, must be > 0 */
6.         char mtext[1];    /* message data */
7.     };
8.  
9. The mtext field is an array (or other structure) whose size is specified by
10. msgsz, a nonnegative integer value. Messages of zero length (i.e., no
11. mtext field) are permitted. The mtype field must have a strictly positive
12. integer value. This value can be used by the receiving process for message
13. selection (see the description of msgrcv() below).
14.  

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 18, 2022, 06:36:02 pm

I use no library and i open no socket. The OS is Linux AMD64, the target is the same. The FPC is of version 3.2.2 The C and Pascal code are here:
http://oscomp.hu/depot/msgtest.c (http://oscomp.hu/depot/msgtest.c)
http://oscomp.hu/depot/msgtest.pas (http://oscomp.hu/depot/msgtest.pas)
And i've read the manual; the C code works as intended. But in the Pascal code, only a longint of zero can be transmitted, all else will yield a result of -1 and ESysENOMSG in errno (fpgeterrno).

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 18, 2022, 06:47:45 pm

Quote from: TCH on May 18, 2022, 06:36:02 pm

I use no library and i open no socket. The OS is Linux AMD64, the target is the same. The FPC is of version 3.2.2 The C and Pascal code are here:
http://oscomp.hu/depot/msgtest.c (http://oscomp.hu/depot/msgtest.c)
http://oscomp.hu/depot/msgtest.pas (http://oscomp.hu/depot/msgtest.pas)
And i've read the manual; the C code works as intended. But in the Pascal code, only a longint of zero can be transmitted, all else will yield a result of -1 and ESysENOMSG in errno (fpgeterrno).

You are using the Pascal ipc library as documented at https://www.freepascal.org/docs-html/current/rtl/ipc/index-5.html.

I suggest you start off with the example give there, without making any assumptions that it is directly compatible with the low-level unix API.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 18, 2022, 07:26:35 pm

Isn't that a unit? Also, even if it's not compatible with the UNIX API, why does it work with a buffer of zeros and not elseway? This is the most basic existing usage: one send and one receive. It should work.
Besides i need to be compatible with the UNIX API. The programs will not necessarily be all written in Pascal. Maybe either process will be C, the other will be Pascal. (Not this one example.)

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: Lutz Mändle on May 20, 2022, 12:32:21 am

Hi,
in your demo program change message type record to:

Code: Pascal  [Select][+][-]

1. type mymsg_t = record
2.         mtype: clong;
3.         mtext: array[0..7] of byte;
4. end;
5.  

Also change the line

Code: Pascal  [Select][+][-]

1.         msqid := msgget(IPC_PRIVATE, 0600 or IPC_CREAT);

to

Code: Pascal  [Select][+][-]

1.         msqid := msgget(IPC_PRIVATE, &0600 or IPC_CREAT);

Then it works.

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 09:18:36 am

Changing the type of mtype from longword to int64 did the trick, thank you very much!
You are right about the missing & character too, but that was only a mistake in the example code i provided here; in the actual code, the & was there to sign the octal representation: the erroneous type caused the problem.

I don't get this though. These functions are calling the libc functions, aren't they? (At least according to packages/rtl-extra/src/unix/ipc.pp:656, or FPC_USE_LIBC is not set?) The needed structure should be identical and the C versions use 32-bit for that field everywhere...

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 09:33:49 am

Quote from: TCH on May 20, 2022, 09:18:36 am

These functions are calling the libc functions, aren't they?

Not a safe assumption in the general case. Libc is called by some stuff but my understanding is that FPC tries to avoid it as much as possible; even when it does call it you can find that there are unexpected changes e.g. what libc would expect to be a zero-terminated string is now a Pascal-type string.

You /can/ go direct to the syscalls or low-level kernel libraries, but it's very easy to fall foul of e.g. alignment issues.

So the first thing to do is *always* test the example provided in a package first, then to read the documentation, then to look at the wiki, and finally to start writing your own code.

But I'm as bad as anybody at trying to bypass some of those steps :-)

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 09:45:00 am

I was to follow FP guidelines, but i was deceived by Wikipedia (https://en.wikipedia.org/wiki/C_data_types#Main_types) where it was told, that long is 32-bit. But that turned out to be erroneous, it is 64-bit on 64-bit OS-es (http://www.tutorialspoint.com/cprogramming/c_data_types.htm).
The FreePascal manual says however, that clong is always 64-bit. (https://www.freepascal.org/docs-html/current/rtl/unixtype/clong.html) Does that mean, that if FPC_USE_LIBC is set, then these functions will fail if the TMSGbuf derivate is defined with clong as mtype?

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 09:52:45 am

Quote from: TCH on May 20, 2022, 09:45:00 am

I was to follow FP guidelines, but i was deceived by Wikipedia (https://en.wikipedia.org/wiki/C_data_types#Main_types) where it was told, that long is 32-bit. But that turned out to be erroneous, it is 64-bit on 64-bit OS-es (http://www.tutorialspoint.com/cprogramming/c_data_types.htm).
The FreePascal manual says however, that clong is always 64-bit. (https://www.freepascal.org/docs-html/current/rtl/unixtype/clong.html) Does that mean, that if FPC_USE_LIBC is set, then these functions will fail if the TMSGbuf derivate is defined with clong as mtype?

Well, that's your fault for relying on Wikipedia and on other random URLs. But using a random URL also give me this:

https://fpc-pascal.freepascal.narkive.com/3JsSIyvf/usage-of-fpc-use-libc

So the bottom line is that if you rebuilt FPC you might be able to use it... and the core developers would probably be interested in your experience.

But I say again:

* Test programs. If one of these doesn't work as expected raise a bug.

* Documentation. If this is incomplete discuss it here or on the MLs.

* FPC/Lazarus wiki (and I /don't/ mean Wikipedia here). If this is wrong fix it.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 10:27:18 am

Quote from: MarkMLl on May 20, 2022, 09:52:45 am

Well, that's your fault for relying on Wikipedia and on other random URLs. But using a random URL also give me this:

And if i would relying on the FPC documentation, then would it be my fault too, if i would have ended up the same position, but from a different angle?
The FPC manual is actually erroneously states, that clong is int64 (https://www.freepascal.org/docs-html/current/rtl/unixtype/clong.html). I've just compiled the test program on a 32-bit Linux, with the mtype as int64. It produced the same error as before. If i change it back to longword, then it works perfectly. So, it needs to be 32-bit on 32-bit OS and 64-bit on 64-bit OS. With clong, it works on both. Therefore the bitwidth of clong also depends on the bitwidth of the underlying arch and thus the FPC manual is erroneous.

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 10:37:29 am

Quote from: TCH on May 20, 2022, 10:27:18 am

The FPC manual is actually erroneously states, that clong is int64 (https://www.freepascal.org/docs-html/current/rtl/unixtype/clong.html).

I agree, raise it as a bug.

Code: Pascal  [Select][+][-]

1.      52 {$ifdef cpu64}
2.      53   clong                  = int64;              pclong                 = ^clong;
3.      54   cslong                 = int64;              pcslong                = ^cslong;
4.      55   culong                 = qword;              pculong                = ^culong;
5.      56 {$else}
6.      57   clong                  = longint;            pclong                 = ^clong;
7.      58   cslong                 = longint;            pcslong                = ^cslong;
8.      59   culong                 = cardinal;           pculong                = ^culong;
9.      60 {$endif}
10.  

Not to mince words, this isn't the first time I've seen this sort of thing.

But if you'd started off with the example program- and this is the third- sorry, FOURTH- time I've told you this- you wouldn't have been caught by that one or at least would have had a much better idea of where to look.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 10:48:15 am

Quote from: MarkMLl on May 20, 2022, 10:37:29 am

But if you'd started off with the example program- and this is the third- sorry, FOURTH- time I've told you this- you wouldn't have been caught by that one or at least would have had a much better idea of where to look.

You can tell me that a few more times, but it will not change what happened in the past.
BTW:

Code: Bash  [Select][+][-]

1. grep -r 'msgrcv' /usr/share/fpcsrc/3.2.2/

Code: [Select]

rtl/dragonfly/sysnr.inc: syscall_nr_msgrcv                      = 227;
rtl/beos/ostypes.inc:        ru_msgrcv   : clong;            { messages received }
rtl/linux/arm/sysnr.inc:  syscall_nr_msgrcv                     = syscall_nr_base+302;
rtl/linux/sysnr-gen.inc:        syscall_nr_msgrcv                       = 188;
rtl/linux/x86_64/sysnr.inc:  syscall_nr_msgrcv                             = 70;
rtl/openbsd/sysnr.inc:(* 227 *)    syscall_nr_msgrcv = 227;
rtl/netbsd/sysnr.inc:  { syscall: "msgrcv" ret: "ssize_t" args: "int" "void  " "size_t" "long" "int"  }
rtl/netbsd/sysnr.inc:     syscall_nr_msgrcv = syscall_nr_base + 227;
rtl/netbsd/sysnr.inc:  { 227 is excluded msgrcv  }
rtl/bsd/ostypes.inc:        ru_msgrcv   : clong;            { messages received }
packages/pthreads/src/pthraix.inc:    ru_msgrcv,
packages/rtl-extra/src/linux/ipccall.inc:function msgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:clong; msgflg:cint):cint;
packages/rtl-extra/src/linux/ipccall.inc:  msgrcv:=ipccall(CALL_MSGRCV,msqid,msgsz,msgflg,@tmp,0);
packages/rtl-extra/src/linux/ipcsys.inc:function msgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:clong; msgflg:cint):cint;
packages/rtl-extra/src/linux/ipcsys.inc:msgrcv:=do_syscall(syscall_nr_MSGRCV,TSysParam(msqid),TSysParam(msgp),TSysParam(msgsz),TSysParam(msgtyp),TSysParam(msgflg));
packages/rtl-extra/src/unix/ipc.pp:Function msgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:clong; msgflg:cint): {$if defined(Darwin) or defined(aix) or defined(Solaris)}ssize_t;{$else}cint;{$endif} {$ifdef FPC_USE_LIBC} cdecl; external clib name 'msgrcv'; {$endif}
packages/rtl-extra/src/unix/ipccdecl.inc:Function cmsgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:cint; msgflg:cint):cint;  cdecl; external;
packages/rtl-extra/src/unix/ipccdecl.inc:Function msgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:cint; msgflg:cint):cint;
packages/rtl-extra/src/unix/ipccdecl.inc:  msgrcv:=cmsgrcv(msqid, msgp, msgsz, msgtyp, msgflg);
packages/rtl-extra/src/bsd/ipcbsd.inc:Function msgrcv(msqid:cint; msgp: PMSGBuf; msgsz: size_t; msgtyp:clong; msgflg:cint):cint;
packages/rtl-extra/src/bsd/ipcbsd.inc:  msgrcv:=(do_syscall(syscall_nr_msgrcv, msqid, TSysParam(msgp), msgsz, msgtyp, msgflg));
packages/rtl-extra/src/bsd/ipcbsd.inc:  msgrcv:=(do_syscall(syscall_nr_msgsys,3, msqid, TSysParam(msgp), msgsz, msgtyp, msgflg));
packages/libc/src/bresourceh.inc:       ru_msgrcv : longint;
packages/libc/src/smsgh.inc:function msgrcv(__msqid:longint; __msgp:pointer; __msgsz:size_t; __msgtyp:longint; __msgflg:longint):longint;cdecl;external clib name 'msgrcv';
packages/libc/src/smsgh.inc:function msgrcv(__msqid:longint; var __msgp; __msgsz:size_t; __msgtyp:longint; __msgflg:longint):longint;cdecl;external clib name 'msgrcv'; Where are the example programs? I've installed FPC and FPC source from the Debian packages they provided.

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 11:04:34 am

If you can't find the examples, why didn't you say that when I first mentioned them?

Start off at

https://www.freepascal.org/docs.html

Ignore the PDF links, they're still broken. Yes, I have been complaining about it.

Select "Run-Time Library (RTL) units reference manual" hence

https://www.freepascal.org/docs-html/current/rtl/index.html

Scroll down to "ipc" hence

https://www.freepascal.org/docs-html/current/rtl/ipc/index.html

Click on "Procedures and functions" at the top. Go to "msgrcv" which among other things says "For an example, see msgctl." which has an example.

Alternatively

Code: [Select]

$ cd /usr/local/src_fpc/fpcbuild-3.2.2
$ find -iname '*msgtool*'
./fpcdocs/ipcex/msgtool.pp

...and so on.

Look, I'm not going to apologise for the difficulty finding some of this stuff: I've been making myself unpopular with the core team for years by complaining about it. But the fact is that the information /is/ in there, and a simple "is there a reliable example for..." or "I can't find..." would have saved you- and everybody else- a lot of grief.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 12:26:01 pm

Quote from: MarkMLl on May 20, 2022, 11:04:34 am

Click on "Procedures and functions" at the top. Go to "msgrcv" which among other things says "For an example, see msgctl." which has an example.

Did you check this certain example of msgctl (https://www.freepascal.org/docs-html/current/rtl/ipc/msgctl.html)? It says:

Code: Pascal  [Select][+][-]

1. Type
2.   PMyMsgBuf = ^TMyMsgBuf;
3.   TMyMsgBuf = record
4.     mtype : Longint;
5.     mtext : string[255];
6.   end;

which is - as it just turned out - erroneous, because longint is always 32-bit and i have a 64-bit system; this should be clong. Tell me please, what difference it would made, if i would start based on this example and not converting my C code? Because for me, it seems that no difference would occur. My code was erroneous because i relied an erroneous source of C types which stated that the type of mtype is always 32-bit and not arch-register wide. Using this example would result the same. Is it still my fault?

Quote from: MarkMLl on May 20, 2022, 11:04:34 am

Look, I'm not going to apologise for the difficulty finding some of this stuff: I've been making myself unpopular with the core team for years by complaining about it.

Chill out dude, i blame you for nothing.
I just pointed out that it is a bit funny that i get chastised once for using flawed random sources when the official sources are also flawed and next for not using the examples when those vaunted examples are not just missing from the default install, but they are also faulty...

Quote from: MarkMLl on May 20, 2022, 11:04:34 am

But the fact is that the information /is/ in there, and a simple "is there a reliable example for..." or "I can't find..." would have saved you- and everybody else- a lot of grief.

The fact is that the information what is there is actually as erroneous as some random sources can be and thus, not reliable. The random source you had chastised me for using contained one flaw, while in the official documentation i have already found two with the very same characteristics (erroneous bitwidth) as in the random source.
As for the "lot of grief for everybody else"...aren't you overdramatizing it a bit? Aside from you, two other folks posted in this topic, one post each; one asked for some more information and the other one simply delivered the solution. I don't see the grief here... Only you are making a scene here by blaming me for not listening to you, while all the sources you have shown me contains the same bitwidth errors as Wikipedia.
I mean: C'mon man.

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 01:15:36 pm

Look, I'm trying to do some work here.

You've uncovered a bug- a serious, showstopping bug IMO- in the documentation process. Report it.

Apropos the example programs: that in part is my mistake for which I apologise, I was assuming that they were run and checked with the formal test programs as part of the build process. Again, raise a bug report.

But my earlier points apply: you should only use Wikipaedia etc. for background information.

MarkMLl

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: TCH on May 20, 2022, 02:40:17 pm

I've reported them:
https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39311 (https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39311)
https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39312 (https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39312)

Title: Re: msgrcv only succeeds if msgsnd sends a buffer with only zeros.
Post by: MarkMLl on May 20, 2022, 03:48:26 pm

Well done :-)

MarkMLl