Lazarus

Programming => General => Topic started by: MarkMLl on February 17, 2021, 10:55:24 am

Title: Equivalent to stacking varargs
Post by: MarkMLl on February 17, 2021, 10:55:24 am
The attached project is a small fragment illustrating static and dynamic linkage to the Python 2.7 runtimes, allowing a Python script to be embedded in a Pascal program. The python2/python.pas and python2/python_dynamic.pas files are machine-generated from ./python2/PythonDefs.inc, the full version of this file is currently about 325Kb since as well as active declarations it includes documentation in a form which works with Lazarus hover-over hints.

Static linkage works fine. Dynamic linkage works fine /provided/ that functions marked as varargs are commented out, see python2/python_dynamic.pas but in particular this:

Code: Pascal  [Select][+][-]
  1. function TPython.PyArg_ParseTuple(args: PPyObject; format: PAnsiChar {;...}): Integer; cdecl varargs;
  2.  
  3. begin
  4.   LoadRoutine(pointer(FPyArg_ParseTuple), 'PyArg_ParseTuple');
  5.   result := FPyArg_ParseTuple(args, format)
  6. end { TPython.PyArg_ParseTuple } ;
  7.  

which if compiled using the makefile results in


lazbuild-trunk PythonDemo.lpi
/usr/local/share/lazarus-trunk/lazbuild --pcp=/home/markMLl/.lazarus-trunk PythonDemo.lpi
SetPrimaryConfigPath NewValue="/home/markMLl/.lazarus-trunk" -> "/home/markMLl/.lazarus-trunk"
Hint: (11030) Start of reading config file /etc/fpc.cfg
Hint: (11031) End of reading config file /etc/fpc.cfg
Free Pascal Compiler version 3.2.0 [2020/11/20] for x86_64
Copyright (c) 1993-2020 by Florian Klaempfl and others
(1002) Target OS: Linux for x86-64
(3104) Compiling PythonDemo.lpr
/home/markMLl/varargs/trunk/PythonDemo.lpr(8,2) Note: (2025) User defined: ===== Intending to link to Python v2 =====
(3104) Compiling ./python2/python_dynamic.pas
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(103,1) Error: (3178) VarArgs directive (or '...' in MacPas) without CDecl/CPPDecl/MWPascal/StdCall and External
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(104,3) Error: (5000) Identifier not found "LoadRoutine"
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(104,23) Error: (5000) Identifier not found "FPyArg_ParseTuple"
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(105,3) Error: (5000) Identifier not found "result"
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(105,13) Error: (5000) Identifier not found "FPyArg_ParseTuple"
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(105,31) Error: (5000) Identifier not found "args"
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(105,37) Error: (3026) Wrong number of parameters specified for call to "Format"
/usr/local/lib/fpc/3.2.0/units/x86_64-linux/rtl/sysutils.ppu:sysstr.inc(1095,10) Error: (5088) Found declaration: Format(const AnsiString;const Array Of Const):AnsiString;
/usr/local/lib/fpc/3.2.0/units/x86_64-linux/rtl/sysutils.ppu:sysstr.inc(1087,10) Error: (5088) Found declaration: Format(const AnsiString;const Array Of Const;const TFormatSettings):AnsiString;
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(106,34) Fatal: (2003) Syntax error, "." expected but ";" found
Fatal: (1018) Compilation aborted
Error: (lazarus) Compile Project, Target: PythonDemo-x86_64-linux: stopped with exit code 1
Error: (lazbuild) failed compiling of project /home/markMLl/varargs/trunk/PythonDemo.lpi
make: *** [Makefile:2: all] Error 2


The makefile invokes lazbuild-trunk hence FPC 3.2.0 on Debian Linux x86_64, but broadly speaking I get the same if using FPC directly.

I've obviously read the description of varags in the Reference, but would appreciate any suggestions as to how I should be working round this. I'd obviously prefer it if code that calls the relevant function could be unchanged between static linkage (where the varargs syntax appears OK) and dynamic linkage (where the above problem occurs), but so far I don't have an example of how that function is actually used to test against.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 17, 2021, 11:25:06 am
Later: I've tried converting the varargs modifier to an extra array-of-const parameter, but I'm still getting an error


(3104) Compiling ./python2/python_dynamic.pas
/home/markMLl/varargs/trunk/./python2/PythonDefs.inc(64,63) Error: (3178) VarArgs directive (or '...' in MacPas) without CDecl/CPPDecl/MWPascal/StdCall and External
/home/markMLl/varargs/trunk/./python2/python_dynamic.pas(80,1) Fatal: (10026) There were 1 errors compiling module, stopping
Fatal: (1018) Compilation aborted


where the offending line reads like

Code: Pascal  [Select][+][-]
  1.     {$macro on }
  2.     {$define LIBPYTHON__:= }
  3. function PyArg_ParseTuple(args: PPyObject; format: PAnsiChar; aoc: array of const): Integer; cdecl; LIBPYTHON__
  4.  

I presume that what's happening here is that the compiler is finding the combination of an array-of-const and cdecl and saying that it can't be used for an internal (Pascal-language) function.

I think that what I should be doing is defining the interface to the external library as cdecl but the internal shim function as using the Pascal calling convention, or (better) the external one as array-of-const+cdecl and the shim as array-of-const which would allow the caller to use the Object Pascal [] parameter list.

It's obviously a bit of extra work in the program that generates the interface units, but nothing insuperable since it's reasonable to expect the writer of the input file to be fairly careful with his definitions.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: PascalDragon on February 17, 2021, 01:34:13 pm
I think that what I should be doing is defining the interface to the external library as cdecl but the internal shim function as using the Pascal calling convention, or (better) the external one as array-of-const+cdecl and the shim as array-of-const which would allow the caller to use the Object Pascal [] parameter list.

It's obviously a bit of extra work in the program that generates the interface units, but nothing insuperable since it's reasonable to expect the writer of the input file to be fairly careful with his definitions.

That is the most useful solution. FPC does not support this, because every C compiler handles the implementation side differently, so we didn't want to bother with this. (And you can be sure if we'd provide support for implementing varargs routines that users would want their code to be useable from C and in Pascal we have array of const already anyway)
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 17, 2021, 02:30:13 pm
I think that what I should be doing is defining the interface to the external library as cdecl but the internal shim function as using the Pascal calling convention, or (better) the external one as array-of-const+cdecl and the shim as array-of-const which would allow the caller to use the Object Pascal [] parameter list.

It's obviously a bit of extra work in the program that generates the interface units, but nothing insuperable since it's reasonable to expect the writer of the input file to be fairly careful with his definitions.

That is the most useful solution. FPC does not support this, because every C compiler handles the implementation side differently, so we didn't want to bother with this. (And you can be sure if we'd provide support for implementing varargs routines that users would want their code to be useable from C and in Pascal we have array of const already anyway)

Thanks for that. So if you could double-check my philosophy:

* Leave the static interface unchanged, i.e. the declarations have cdecl and possibly varargs.

* The field (entry point pointer) in the dynamic interface keep cdecl/varargs.

* The class declaration and implementation in the dynamic interface lose cdecl/varargs.

* Where necessary, the class declaration and implementation have varargs rewritten as array of const.

The simplest way of doing that would be to declare new macros CDECL__ and CDECL_VARARGS___ or similar.

Can you confirm that these examples from the Reference really are equivalent in all situations:

Code: Pascal  [Select][+][-]
  1. Function PrintF1(fmt : pchar                       ); cdecl; varargs; external ’c’ name ’printf’;  
  2. Function PrintF2(fmt : pchar; Args : Array of const); cdecl;          external ’c’ name ’printf’;
  3.  

In MacPas mode, is "..." a precise equivalent of "array of const"?

The preprocessor is handling stuff fairly naively, but it works and saves a vast amount of manual editing. In the past I've used pretty much the same approach to also define the exports from a dynamically-linkable unit.

MarkMLl



Title: Re: Equivalent to stacking varargs
Post by: PascalDragon on February 18, 2021, 08:59:10 am
I think that what I should be doing is defining the interface to the external library as cdecl but the internal shim function as using the Pascal calling convention, or (better) the external one as array-of-const+cdecl and the shim as array-of-const which would allow the caller to use the Object Pascal [] parameter list.

It's obviously a bit of extra work in the program that generates the interface units, but nothing insuperable since it's reasonable to expect the writer of the input file to be fairly careful with his definitions.

That is the most useful solution. FPC does not support this, because every C compiler handles the implementation side differently, so we didn't want to bother with this. (And you can be sure if we'd provide support for implementing varargs routines that users would want their code to be useable from C and in Pascal we have array of const already anyway)

Thanks for that. So if you could double-check my philosophy:

* Leave the static interface unchanged, i.e. the declarations have cdecl and possibly varargs.

* The field (entry point pointer) in the dynamic interface keep cdecl/varargs.

* The class declaration and implementation in the dynamic interface lose cdecl/varargs.

* Where necessary, the class declaration and implementation have varargs rewritten as array of const.

The simplest way of doing that would be to declare new macros CDECL__ and CDECL_VARARGS___ or similar.

Can you confirm that these examples from the Reference really are equivalent in all situations:

Code: Pascal  [Select][+][-]
  1. Function PrintF1(fmt : pchar                       ); cdecl; varargs; external ’c’ name ’printf’;  
  2. Function PrintF2(fmt : pchar; Args : Array of const); cdecl;          external ’c’ name ’printf’;
  3.  

In MacPas mode, is "..." a precise equivalent of "array of const"?

The preprocessor is handling stuff fairly naively, but it works and saves a vast amount of manual editing. In the past I've used pretty much the same approach to also define the exports from a dynamically-linkable unit.

In MacPas mode ... is an equivalent for declaring the function as varargs, not array of const. Also the two PrintFx functions behave the same on the ABI level, but they need to be called differently as the example in the Language Reference Guide showed further. All other points are correct assumptions however (at least as far as I understood them).
Title: Re: Equivalent to stacking varargs
Post by: nanobit on February 18, 2021, 10:19:35 am
Can anyone explain the found issue in shorter words?
I only know that in Delphi and FPC this should work without "external" (static):
var pyArg_ParseTuple: function( args: PPyObject; format: PAnsiChar {;...}): Integer; cdecl varargs;
result := pyArg_ParseTuple( args, pFmtPoint, @x, @y);
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 18, 2021, 12:11:29 pm
In MacPas mode ... is an equivalent for declaring the function as varargs, not array of const. Also the two PrintFx functions behave the same on the ABI level,

Thanks for that. Allowing that it's strictly the array of const ... cdecl case that I'm interested in, I think that's adequate.

In the end I've not attempted to convert cdecl varargs; to array of const ... cdecl but have simply inserted a $warning that the usage is deprecated in that context... I think that covers all possibilities.

Quote
but they need to be called differently as the example in the Language Reference Guide showed further. All other points are correct assumptions however (at least as far as I understood them).

The calling issue doesn't bother me at all since it's consistent behaviour, much as I argue that the community should be receptive to ideas from other languages on occasion.

(Somewhat later...)

Well, I'm fairly sure that the preprocessor output is as I want it but I'm left with


python_dynamic.pas(108,1) Error: Wrong type "Array Of Const" in array constructor
python_dynamic.pas(154) Fatal: There were 1 errors compiling module, stopping


with the indicated line being

Code: Pascal  [Select][+][-]
  1.     103 function TPython.PyArg_ParseTuple(args: PPyObject; format: PAnsiChar; vaoc: array of const): Integer;
  2.     104
  3.     105 begin
  4.     106   LoadRoutine(pointer(FPyArg_ParseTuple), 'PyArg_ParseTuple');
  5.     107   result := FPyArg_ParseTuple(args, format, vaoc)
  6.     108 end { TPython.PyArg_ParseTuple } ;
  7.  

I've found http://free-pascal-general.1045716.n5.nabble.com/passing-quot-array-of-const-quot-into-cdecl-procvar-td5725304.html and a linked StackOverflow question which suggests that while it's an issue with Delphi it shouldn't be with FPC... I tack on my current files in case anybody wants to tell me I'm being silly and will come back to it slightly later.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 18, 2021, 12:25:59 pm
Can anyone explain the found issue in shorter words?
I only know that in Delphi and FPC this should work without "external" (static):
var pyArg_ParseTuple: function( args: PPyObject; format: PAnsiChar {;...}): Integer; cdecl varargs;
result := pyArg_ParseTuple( args, pFmtPoint, @x, @y);

It's not so much an "issue" but a question of coding style. What you show in your example assumes that the variable pyArg_ParseTuple has been preinitialised, which can be done either by doing them all when the program starts or by loading each routine on demand. In the latter case that can best be done by wrapping the library call in a method i.e. something like Python.pyArg_ParseTuple(), but that means passing the method's array of const parameter to the library routine i.e. "stacking vararg" or "stacking array of const" parameters.

The bigger picture is that I'm attempting to automate generating interface units, which can get to be a helluva job for even a small external library. I worked out a layout which could be used for both static and dynamic linkage (and also for generating a library) years ago, but it still meant that I had to edit the entry point definition, type definition, variable, variable assignment etc. manually... most of that's just hack work and there's absolutely no reason why it shouldn't be automated. It works in almost all cases...

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: PascalDragon on February 18, 2021, 02:01:15 pm
Well, I'm fairly sure that the preprocessor output is as I want it but I'm left with


python_dynamic.pas(108,1) Error: Wrong type "Array Of Const" in array constructor
python_dynamic.pas(154) Fatal: There were 1 errors compiling module, stopping


with the indicated line being

Code: Pascal  [Select][+][-]
  1.     103 function TPython.PyArg_ParseTuple(args: PPyObject; format: PAnsiChar; vaoc: array of const): Integer;
  2.     104
  3.     105 begin
  4.     106   LoadRoutine(pointer(FPyArg_ParseTuple), 'PyArg_ParseTuple');
  5.     107   result := FPyArg_ParseTuple(args, format, vaoc)
  6.     108 end { TPython.PyArg_ParseTuple } ;
  7.  

I've found http://free-pascal-general.1045716.n5.nabble.com/passing-quot-array-of-const-quot-into-cdecl-procvar-td5725304.html and a linked StackOverflow question which suggests that while it's an issue with Delphi it shouldn't be with FPC... I tack on my current files in case anybody wants to tell me I'm being silly and will come back to it slightly later.

I should have mentioned this: it's not possible (for the compiler) to convert a Pascal-style array of const for a call to a varargs function. You can use code like the one in the mentioned StackOverflow question, but you'll have to implement it for each platform yourself.
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 18, 2021, 02:37:51 pm
I should have mentioned this: it's not possible (for the compiler) to convert a Pascal-style array of const for a call to a varargs function. You can use code like the one in the mentioned StackOverflow question, but you'll have to implement it for each platform yourself.

Ah yes... you have an enviable record of hitting the nail on the head :-)

OK, I'll mark that bit of generated code with $error for the moment. It appears to only affect a half dozen of the (so-far translated) Python entry points although there might obviously be more in the debugging etc. stuff which I hope to be looking at presently, and is not something that I'd be likely to put into a locally-written plugin or backend.

I'll comment here when I have a better fix, in case anybody wants a copy of the preprocessor.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: nanobit on February 19, 2021, 07:45:16 am
What you show in your example assumes that the variable pyArg_ParseTuple has been preinitialised, which can be done either by doing them all when the program starts or by loading each routine on demand. In the latter case that can best be done by wrapping the library call in a method i.e. something like Python.pyArg_ParseTuple(), but that means passing the method's array of const parameter to the library routine i.e. "stacking vararg" or "stacking array of const" parameters.

Ah, I see now, the thread is about a compiler limitation: varargs call cannot be wrapped nicely.
(Pascal-ish would be only "case length(vaoc) of", thus only for a limited choice of calls )

Then I would not write wrappers for them, but initialize the function pointers earlier,
actually I would initialize all pointers in the common first call (probably in a wrapper of Py_Initialize or earlier)
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 19, 2021, 09:41:23 am
Ah, I see now, the thread is about a compiler limitation: varargs call cannot be wrapped nicely.

Maybe. I definitely don't want to appear to be criticising the compiler developers here.

In practical terms I can almost certainly work round it using a locally-defined Pascal function that takes a fixed number of parameters and then calls the C library entry point via the initialised variable, but that might end up making the static- and dynamically-linked programs different. I'll take more of a look at this presently.

Of course, the joke here is that the sort of language preprocessing I'm doing is easy enough with Pascal or C/C++, but /far/ more hairy with Python since it doesn't have begin/end or any reasonable equivalent.

Quote
Then I would not write wrappers for them, but initialize the function pointers earlier,
actually I would initialize all pointers in the common first call (probably in a wrapper of Py_Initialize or earlier)

See the example I posted. The pointers (strictly, procedure variables since they're typed) can be initialised during wrapper object creation, or it can be deferred.

Once the variable is initialised it can obviously be called by local helper functions.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: nanobit on February 19, 2021, 03:17:59 pm
Ok, but I considered to provide the variable/property directly (without wrapper method),
thus clients can call it without predefined number of arguments.
property PyArg_ParseTuple: TPyArg_ParseTuple read FPyArg_ParseTuple;
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on February 19, 2021, 03:40:22 pm
Ok, but I considered to provide the variable/property directly (without wrapper method),
thus clients can call it without predefined number of arguments.
property PyArg_ParseTuple: TPyArg_ParseTuple read FPyArg_ParseTuple;

Thus losing all type checking :-/

OK, that sort of thing has to be done on occasion, but it's a strong incentive for having something like Modula-3's scheme where unsafe operations could be isolated.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on March 03, 2021, 11:06:51 pm
I should have mentioned this: it's not possible (for the compiler) to convert a Pascal-style array of const for a call to a varargs function. You can use code like the one in the mentioned StackOverflow question, but you'll have to implement it for each platform yourself.

I've modified my preprocessor to treat procedures/functions with varargs specially: if dynamic linkage is being used they're converted to properties but one has to call LoadVarargsRoutine() explicitly.

Without intending to be in the least critical, I note for completeness that declarations like

Code: Pascal  [Select][+][-]
  1. (* Write the output string described by format to sys.stdout. No exceptions are
  2. raised, even if truncation occurs (see below).
  3.  
  4. format should limit the total size of the formatted output string to 1000 bytes
  5. or less – after 1000 bytes, the output string is truncated. In particular, this
  6. means that no unrestricted “%s” formats should occur; these should be limited
  7. using “%.<N>s” where <N> is a decimal number calculated so that <N> plus the
  8. maximum size of other formatted text does not exceed 1000 bytes. Also watch out
  9. for “%f”, which can print hundreds of digits for very large numbers.
  10.  
  11. If a problem occurs, or sys.stdout is unset, the formatted message is written
  12. to the real (C level) stdout.
  13. *)
  14. procedure PySys_WriteStdout(format: pchar); CDECL_VARARGS__ LIBPYTHON__
  15. // procedure PySys_WriteStdout(format: pchar; vaoc: array of const); CDECL__ LIBPYTHON__
  16. // void PySys_WriteStdout(const char *format, ...)
  17.  
  18. (* As above, but write to sys.stderr or stderr instead.
  19. *)
  20. // procedure PySys_WriteStderr(format: pchar); CDECL_VARARGS__ LIBPYTHON__
  21. procedure PySys_WriteStderr(format: pchar; vaoc: array of const); CDECL__ LIBPYTHON__
  22. // void PySys_WriteStderr(const char *format, ...)
  23.  

result in Pascal-style entry points which need to be called differently:

Code: Pascal  [Select][+][-]
  1.   Python.LoadVarargsRoutine('PySys_WriteStdout');
  2.   if Assigned(Python.PySys_WriteStdout) then
  3.     Python.PySys_WriteStdout('%s %s\n', 'Test message to', 'StdOut');
  4.   Python.LoadVarargsRoutine('PySys_WriteStderr');
  5.   if Assigned(Python.PySys_WriteStderr) then
  6.     Python.PySys_WriteStderr('%s %s\n', ['Test message to', 'StdErr']);
  7.  

This results in output like


Python is dynamically linked, and has been successfully loaded.
2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0]
Test message to StdErr\nTest message to StdOut\n['/usr/local/src/pythonForFpc/trunk/PythonDemo-x86_64-linux']
Hello, World!


Since I'm not going via a shim procedure I can't convert the \n to an EOL: I don't see that as a significant issue since the parameter is being parsed by Pascal (count the quotes) rather than Python or C/C++.

Hadjab MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: nanobit on April 17, 2021, 05:57:33 am
Your solution looks very good. If you want to extend this later,
you could add a wrapper on top of your function pointer, conceptionally like:

procedure TPython.writeStdout( const format: string; const a: variant);
begin ... uniCall.run( FPySys_WriteStdout, format, a); end;

I see no easy uniCall Pascal-implementation, but it could
use some ffi-library (foreign function interface), like this:
https://dyncall.org/pub/dyncall/dyncall/file/default/test/ellipsis/main.cc
DC_CALL_C_ELLIPSIS for format-parameter, DC_CALL_C_ELLIPSIS_VARARGS for array
Title: Re: Equivalent to stacking varargs
Post by: Thaddy on April 17, 2021, 08:48:46 am
Note libffi already comes as standard package with the fpc distribution.
It is in packages/libffi.
The necessary binaries are in most linux distro's.
For windows the dll's are in both mingw and cygwin.
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on April 17, 2021, 09:22:12 am
@nanobit @Thaddy thanks, I'll check libffi presently.

The gist of the program I've written is that it takes a .inc file which is pretty much exactly what you'd use for static linkage, except that the last bit of each procedure/function declaration is replaced by a macro name. It generates two unit source files:

* A static unit which includes the .inc file expanding the macro.

* A dynamic unit which doesn't expand the macro, but declares an object with methods each of which is a thin wrapper around (a pointer to) an imported function (@nanobit, this is probably equivalent to what you've suggested). This also "automagically" generates all procedure/function types, fields of those types to hold function pointers, the implementation of each thin wrapper and so on, and as icing on the cake keeps comments in a format compatible with Lazarus hinting.

There's also provision for a file containing locally-written replacements for C/C++ macros.

The nice thing is that by naming the static unit and the dynamic object the same, the code at which imported functions are used is completely oblivious as to whether static or dynamic linkage is currently being used (except that dynamic vararg functions have to be imported explicitly, since wrappers can't be written).

The overall layout of the .inc file is something I hacked together quite some years ago, based on a Delphi interface somebody gave me which might have originated at NAG. But I also find that with care one can use the same file when *creating* a library, hence one can have common definitions (plus magic number protection etc.) when defining a frontend/backend architecture.

I've tacked on an example: the .inc files are created manually but the others by the program. The advantage doesn't really show for something this small, but by the time one has a dozen or so functions (e.g. for even a minimal ALSA interface https://github.com/MarkMLl/asound ) the work involved if it's done by hand is substantial. I'm not sure that the program is ready for "general release" yet, or who'd be interested, or whether I want to commit myself to unending maintenance as is apparently necessary for e.g. fpcupdeluxe.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: nanobit on April 17, 2021, 02:14:40 pm
Oh, I've not seen libffi available, then some info in this example thread:
https://sourceware.org/pipermail/libffi-discuss/2021/002607.html
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on April 17, 2021, 09:09:50 pm
I'm left with the nagging feeling that because of the way that I'm generating code there's probably some obvious solution which I'm overlooking. Allowing the ease with which C code can crash if the format string is wrong I presume that the variadic parameters are pointers with no associated type information, but I think I really need to read up on this (e.g. via libffi).

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: jamie on April 18, 2021, 12:32:12 am
I've been reading this thread and I still don't really know which direction  you are heading ?

are you trying to use a Lib function that expects specific C style variant parameters or are you trying to simulate the same in your code?


Anyways, Its most likely using the current style of  variant parameters
"..." in the last parameter of the function prototype...

 That style of calling is a pointer to a va_list which uses the va_start, va_arg and va_stop or end, I can't remember now, its been a while
but the those are just macros..

  Also for the C/C++ spec's you can get the record lay out of the current spec's.. I can't say if they are the same or not but if you are trying to simulate that to call an external function from fpc then I would think you need to create the third parameter as a pointer to an va_list

etc.
Title: Re: Equivalent to stacking varargs
Post by: nanobit on April 18, 2021, 08:12:47 am
The thread is about calling external varargs routines, which is possible already,
but due to clientside technical reasons not most flexible.

1) Such library could allow: support fname( ..., format, variantArray),
where the user can create any (format, variantArray) pair at runtime.
Special case: If the c-api wants as format only a typelist (array signature),
then fname( ..., variantArray) would suffice, because the wrapper
can deduce the formatString from the array.

(Background: The formatString (between fixed args and var args) is function specific,
the c-api defines a relation between format and corresponding type,
for example 'i' could mean integer or Pinteger (if defined as returnvalue)).

2) Currently the array signature (types (thus count too)) has to be fixed
at compile time. This is minimal support, but mostly this can suffice.
But it's not pascal-ish and less than the library allows at runtime.
Also, the library user (because mostly he defines the signature) needs to interface the library directly,
because a generic wrapper (accepting an array) around the function pointer is not possible.
No wrapper also means no wrapper which could do parameter-validation (if wanted) or something else.

A ffi library is a wrapper which allows full use of the wrapped library.
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on April 18, 2021, 08:46:01 am
1) Such library could allow: support doCall( ..., format, variantArray),
where the user can create any (format, variantArray) pair at runtime.
Special case: If the c-api wants as format only a typelist (array signature),
then doCall( ..., variantArray) would suffice, because the wrapper
can deduce the formatString from the array.

OTOH that would move away from the situation where I can at least call something that looks- in terms of parameter list)- exactly like what the C library documents :-/

What I'll do in the interim is make sure that there's some reliable way that a program can detect that it needs to initialise vararg imports as a special case, rather than its being done automatically.

MarkMLl
Title: Re: Equivalent to stacking varargs
Post by: nanobit on April 18, 2021, 09:12:56 am
Ah, "doCall" (now I have changed the post) was not chosen well in my text,
I meant specific functionnames, for example:
writeStdout( format, variantArray); in addition to:
PySys_WriteStdout( format, ....); // this is supported already
Title: Re: Equivalent to stacking varargs
Post by: MarkMLl on April 18, 2021, 09:19:14 am
Whatever... I'll investigate presently (but not right now :-)

Many thanks for the thoughts.

MarkMLl
TinyPortal © 2005-2018