Lazarus
Free Pascal => General => Topic started by: Fred vS on July 13, 2014, 11:27:46 pm
-
Hello great fpc people.
Here the story :
A program uses a library.
That library is fpc and can be modified.
Is it possible to access procedures of the library with a variable from the main application ?
Something like this :
=> Library side :
library mylib;
...
procedure one_of_library_procedure(x, y, z : integer); /// NOT EXPORTED
begin
// something...
end;
procedure run_procedure(theproc : string) ; cdecl; /// EXPORTED
begin
run(theproc) ; //// this the thing...
end;
...
export
run_procedure name 'run_procedure';
end.
=> Main program side :
...
run_procedure('one_of_library_procedure(2,4,5)') ;
...
Many thanks.
-
the described approach will work, as long as you're able to parse the passed script (theproc : string).
Thinks about the case
run_procedure('one_of_library_procedure((2+5)*getsomeval() div 4,4,5)')
is there a problem exporting the procedure?
btw, OpenGL is using "extensions" mechanism, to describe supported extensions. Each extensions comes with certain set of variables and functions - available as library exported functions.
Thus the main program would parse the extensions list and import functions based of the list. Is this is something you're looking for?
-
Very hypothetical it could work, but of course FPC in library form would hit FPC's GPL license that means that your main program also should be GPL compatible.
There are some practical problems to fix though:
- FPC has no "to memory" abilities only to disc. On most targets external assemblers and linkers are used that expect their input on disc. All this disc writing would be slow.
- FPC generates a new EXE or DLL, and has no facilities for running at all. You would have to engineer for loading and interfacing the compiled result into your own environment.
All of which is non trivial, which is probably why it is not done already.
-
@ marcov and skalogryz => many thanks => ;)
is there a problem exporting the procedure?
Hum, exporting the procedures is not the problem, but there are so much to declare that i would prefer to do it via variable.
That library uses AggPas and i want to find a trick to not export all the AggPas functions...
So if the developer want to use some AggPas procedures, he can call it via a variable.
Thus the main program would parse the extensions list and import functions based of the list. Is this is something you're looking for?
Maybe, not sure i understood what you explained...
Very hypothetical it could work, but of course FPC in library form would hit FPC's GPL license that means that your main program also should be GPL compatible.
Aaargh, not sure i understood that too, why this "calling procedures via variable" would alter the license ?
So, to resume, if a library uses AggPas, must each AggPas procedure be exported inside the library or is there a trick to uses that Aggpas procedures without exporting all the procedures ?
Many thanks.
Fred
-
Very hypothetical it could work, but of course FPC in library form would hit FPC's GPL license that means that your main program also should be GPL compatible.
Aaargh, not sure i understood that too, why this "calling procedures via variable" would alter the license ?
No integrating the compiler into your application would. As DLL or not.
So, to resume, if a library uses AggPas, must each AggPas procedure be exported inside the library or is there a trick to uses that Aggpas procedures without exporting all the procedures ?
You are mixing too many things. In Windows it must be exported. But while on Linux you might not need to, you only could get the function that way, and still need a declaration to call it.
But calling code in a string is near impossible.
-
But calling code in a string is near impossible.
But that's scripting task. That's what PascalScript is doing right now - not 100% Pascal compatible, but at least could do the trick.
However, using a pascal script or a built-in compiler would be to sluggish and an over-engineering for the task of exporting functions.
That library uses AggPas and i want to find a trick to not export all the AggPas functions...
So if the developer want to use some AggPas procedures, he can call it via a variable.
I certainty recommend the following:
* export all the AggPas libraries, and a way to determine what is the AggPas library version number. The version number will allow to identify what functions and what parameters are available.
* do not require a developer to use exported functions. If they find the need to use AggPas function directly they could always load the function reference in run-time.
-
Another alternative:
procedure RunProc(const nm: string; const params: array of const)
begin
if (nm = 'one_of_library_procedure') and length(params=3) then
one_of_library_procedure ( params[0], params[1], params[2])
else if nm = 'another_library_procedure') and length(params=2) then
another_library_procedure ( params[0], params[1] )
...
end;
completely eliminates a need of building and parsing of "the call string".
-
But calling code in a string is near impossible.
But that's scripting task. That's what PascalScript is doing right now - not 100% Pascal compatible, but at least could do the trick.
If Fred was asking for ways to let the user plug-in or parametrize the program with code, I'd steer him in the direction of scripting.
I didn't get that impression though, so I didn't want to confuse the picture with scripting.
-
Wow, lot ot very interesting things here... many thanks.
Now, with your help, i see better what i want and i will try to explain it.
In a normal world, when you create a library, you have to export each function/procedure you want be accessible.
Here a traditional library :
library mylib;
...
begin
...
procedure proc1(x, y, z : integer); /// EXPORTED
begin
// something...
end;
procedure proc2(x : string); /// EXPORTED
begin
// something...
end;
procedure proc3(x : boolean); /// EXPORTED
begin
// something...
end;
...
export
proc1 name 'proc1';
proc2 name 'proc2';
proc3 name 'proc3';
end.
You need a wrapper/header for your main program, with declaration of the exported procedures..
And your main application will use the library like this :
program myprog;
uses
...
mylib, // the header of the library
...
begin
proc1(1,2,3);
proc2('yep');
proc3(false);
end.
In the not classical library, only one procedure will be exported .
And that alone procedure will run script that call procedure who are part of the library, but NOT exported.
library mynotnormallib;
...
begin
...
procedure proc1(x, y, z : integer); /// NOT EXPORTED
begin
// something...
end;
procedure proc2(x : string); /// NOT EXPORTED
begin
// something...
end;
procedure proc3(x : boolean); ///NOT EXPORTED
begin
// something...
end;
procedure run_lib_script(thescript : string) ; cdecl; /// THIS IS EXPORTED
begin
fpc_run_script(thescript) ; //// here run the script and HOW to DO that ?
end;
...
export
run_lib_script name 'run_lib_script'; /// only one procedure exported
end.
And your main application will use the library like this :
program myprog;
uses
...
mynotnormallib, // the header of the library with only one exported proc.
...
begin
run_lib_script('proc3(false)');
run_lib_script('proc2('yep')');
run_lib_script('proc1(1,2,3)');
end.
Hum, is it possible ?
-
Hum, is it possible ?
it is possible.
-
it is possible.
Yep, great news (but i knew it, with fpc, everything is possible.) ;D
So => much easier because same syntax for all languages who use the library (C, python, delphi, basic, java,...) => unique universal library (only 2 exported procedures, one for classic native and one for java native) => possibility to use text-files for script => lot of other great things...
Hum, other little question...=> how to do it, what fpc command to use to run the script ? :-[
Thanks.
-
let me reiterate the bitter truth.
There's no built-in pascal language features to do that. (yet... let's give Embracradero a few more releases).
Reason is simple - Pascal is compiled language.
Thus, in order to execute a code passed in runtime via string, you'd need to use additional library that can do the script evaluation.
It can be:
FP Expression Parser (http://wiki.lazarus.freepascal.org/How_To_Use_TFPExpressionParser)
Pascal Script (http://wiki.lazarus.freepascal.org/Pascal_Script)
Lua (http://lua-users.org/wiki/LuaInFreePascal) (extremely popular these days)
or a self written parser (as simple as get the name of routine, get list of parameters).
-
So => much easier because same syntax for all languages who use the library (C, python, delphi, basic, java,...) => unique universal library (only 2 exported procedures, one for classic native and one for java native) => possibility to use text-files for script => lot of other great things.
Some thoughts here. Languages would still need to import at least 2 functions. So if they can do it for two, they could do it for X-number of functions.
Would other language developers be happy to learn a new syntax? I doubt that.
Please condider debugging abilities for thr scripting feature. If a script hard to debug, it will be avoided
-
(I don't think he wants scripting. Rather maybe one exported procedure that sets a record with procedure variables?)
-
There's no built-in pascal language features to do that.
Aaaargh...
It can be:
FP Expression Parser
Pascal Script
Lua (extremely popular these days)
or a self written parser (as simple as get the name of routine, get list of parameters).
Ok, i will take a look (but it will be much heavier)...
Some thoughts here. Languages would still need to import at least 2 functions. So if they can do it for two, they could do it for X-number of functions.
More or less... Languages will need to import only one (of 2 available), one for Java programs (because of personal-exotic-native Java library syntax), one for all others.
But, yes, if it can be for 1, it can be for thousands too. ;)
Would other language developers be happy to learn a new syntax? I doubt that.
Hum, even if all the procedures are exported, they need to study the syntax of that new procedures => with the "script" way, of course they have to learn that syntax... but it will be the same for all languages.
Please condider debugging abilities for thr scripting feature. If a script hard to debug, it will be avoided
I agree, with scripting feature => difficult to debug the script.
(I don't think he wants scripting.)
Yep, i was hoping that script will be the miracle-solution, but if i must install external tools, that is not a light solution...
Rather maybe one exported procedure that sets a record with procedure variables
Hum, maybe that way ( how to ? ).... ?
PS : This is prototyping, maybe that idea is totally utopia and the "traditional way" (exporting all needed procedures) is the only realistic way.
PS2: I have perfect results with the "traditional way". I only try to find a more "universal" way.
Many thanks.
Fred
Many thanks.
-
Hum, maybe that way ( how to ? ).... ?
library mylib;
type
TMyLibAPI = record
version : Integer;
proc1 : procedure(x, y, z : integer);
proc2 : procedure (x : string);
proc3 : procedure (x : boolean);
end;
procedure proc1(a,b)
begin
..
end;
procedure proc2(a,b)
begin
..
end;
procedure proc3(a,b)
begin
..
end;
procedure GetMyLIPAPI( var apirec: TMyLibAPI )
begin
apirec.version = VERSION_OF_MYLIB;
apirec.proc1:=@proc1;
apirec.proc2:=@proc2;
apirec.proc3:=@proc3;
end;
program main;
type
TMyLibAPI = record
version : Integer;
proc1 : procedure(x, y, z : integer);
proc2 : procedure (x : string);
proc3 : procedure (x : boolean);
end;
procedure GetMyLIPAPI( var apirec: TMyLibAPI ); external;
var
api : TMyLibAPI;
begin
GetMyLIPAPI(api);
api.proc1(1,2,3);
api.proc2('blah', 'blah');
api.proc3(x);
Something similar is used in C-style libraries. For example zlib, uses this approach to interface dataread and memory routines.
Eventually (for better cross-language and cross-process compatibility) the idea of passing records evolved to (com/corba) Interfaces. From the low level point of view, they are still same records with function references.
-
@ skalogryz => :-* XXL.
I will study your code and try it.
Write you later....
Many thanks.
Fred
-
I will study your code and try it.
You need to research if this approach is friendly to high-level scripting languages such as Python, Java and (Visual?) Basic.
Once the research is done, ask yourself: what's the benefit of this approach over a regular library export.
-
Ok, im back...
Nice code but... this one =>
program main;
type
TMyLibAPI = record
version : Integer;
proc1 : procedure(x, y, z : integer);
proc2 : procedure (x : string);
proc3 : procedure (x : boolean);
end;
With my (little) experience of library, i note that, to avoid problems, you have to ask the less possible to the program who will use the library.
The only easy-accessible variables for each languages are:
- Integers.
- Float.
- Strings (but not so easy).
And, it is really difficult to deal with:
- Classes.
- Procedures.
The only universal way i see is to use strings as command...
Like this one :
program main;
var
thecom : string;
begin
thecom := 'proc1(1,2,3)';
run_lib_command(thecom);
end;
But it seems that is not possible with fpc directly...
-
- Strings (but not so easy).
That's possible the hardest type to be passed for different languages.
As each language implements them in various ways (multiple times)
Need to mention, that Microsoft has achieved some good solution here with COM interfaces and OLE Strings (WideStrings on Pascal, but not UnicodeStrings).
OleStrings works flawlessly (from my experience) throughout all languages that support COM Interfaces on Windows
- Procedures
- Classes
Both are the easiest!
Procedures come naturally (as long as a language has an understanding of a routine/function, in the same manner as C-does.)
Classes are typically replaced with opaque "Pointer" or pointer-sized integer. With proper methods exposed and functions.
-
That's possible the hardest type to be passed for different languages.
As each language implements them in various ways (multiple times)
Yep, maybe, but i have easy solutions for C, Python, Pascal, Java and Basic.
With classes and procedures (method) => difficult with Java and Basic.
And, using Com interface will give problems with some OS...
But nice to open and clarify the way...
Thanks
-
That's possible the hardest type to be passed for different languages.
As each language implements them in various ways (multiple times)
Yep, maybe, but i have easy solutions for C, Python, Pascal, Java and Basic.
But isn't a Java string a class?
-
But isn't a Java string a class?
Hum, i only try Java by testing my "universal" fpc libraries...
So, i do not know...
-
Hello.
Other way...
Does it exist a equivalent of TProcess for Internal process ?
=> TProcess :
...
var
VProcess: TProcess;
begin
VProcess.CommandLine := 'gdb';
VProcess.Execute;
=> TInternalProcess :
...
var
VProcess: TInternalProcess;
begin
VProcess.CommandLine := 'proc1(1,2,3)';
VProcess.Execute;
?
-
Well of course! TInternalProcess! how could I forgot about that!
Great catch, Fred !
here's an example!
-
=> TProcess :
...
var
VProcess: TProcess;
begin
VProcess.CommandLine := 'gdb';
VProcess.Execute;
Searches on disc already compiled program, if found starts already compiled program "gdb"
=> TInternalProcess :
...
var
VProcess: TInternalProcess;
begin
VProcess.CommandLine := 'proc1(1,2,3)';
VProcess.Execute;
?
No. Since that would require a compiler, and that was already answered in the first question. EXECUTING CODE IN STRINGS IS NOT POSSIBLE UNLESS YOU PULL IT THROUGH AN (EXTERNAL) COMPILER. The only escape is using scripting languages, but that is a entirely different matter. In general performance and control suffers that way.
-
here's an example!
Hum, it seams exactly what i want... :o
So i do not understand... =>
EXECUTING CODE IN STRINGS IS NOT POSSIBLE UNLESS YOU PULL IT THROUGH AN (EXTERNAL) COMPILER.
I have to re-study skalogryz-code... (if it is not a joke-code then it is a bomb-code...).
-
Yep, nice code skalogryz ;)
The only part i do not like too much is this one =>
procedure HiddenProc(Var Result : TFPExpressionResult; Const Args : TExprParameterArray);
begin
Proc1(args[0].ResInteger, args[1].ResInteger, args[2].ResInteger);
Result.ResInteger:=0;
end;
initialization
RegisterProc('proc1', 'III', @HiddenProc);
But, yes, very, very promising way...
Write you later...
-
Hum, it seams that skalogryz-code is not a joke, so be careful, it could be a bomb...
Look at that :
The program =>
program project1;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}{$IFDEF UseCThreads}
cthreads,
{$ENDIF}{$ENDIF}
Classes,
libunit; // the pseudo-library
begin
ProcExported('Proc1(1,2,3)');
end.
The pseudo-library =>
unit libunit;
{$mode objfpc}{$H+}
interface
uses
Classes, SysUtils, internalprocess;
procedure Proc1(a,b,c: Integer);
procedure ProcExported(theproc: string); /// Exported in library
implementation
procedure Proc1(a,b,c: Integer); /// not exported in library
begin
writeln(a);
writeln(b);
writeln(c);
end;
procedure ProcExported(theproc: string); /// Exported in library
var
VProcess: TInternalProcess;
begin
VProcess:= TInternalProcess.Create;
try
VProcess.CommandLine := theproc;
VProcess.Execute;
finally
VProcess.Free;
end;
end;
procedure HiddenProc(Var Result : TFPExpressionResult; Const Args : TExprParameterArray);
begin
Proc1(args[0].ResInteger, args[1].ResInteger, args[2].ResInteger);
Result.ResInteger:=0;
end;
initialization
RegisterProc('proc1', 'III', @HiddenProc);
end.
And the bomb =>
unit internalprocess;
{$mode delphi}{$H+}
interface
uses
Classes, SysUtils, fpexprpars;
Type
TFPExpressionResult = fpexprpars.TFPExpressionResult;
TExprParameterArray = fpexprpars.TExprParameterArray;
TFPExprFunctionCallBack = fpexprpars.TFPExprFunctionCallBack;
{ TInternalProcess }
TInternalProcess = class(TObject)
private
fCommandLine : string;
public
LastError : string;
procedure Execute;
property CommandLine: string read fCommandLine write fCommandLine;
end;
procedure RegisterProc(const name, types: string; ptr: TFPExprFunctionCallBack);
implementation
var
IntProc : TStringList;
procedure RegisterProc(const name, types: string; ptr: TFPExprFunctionCallBack);
begin
IntProc.AddObject(name+'|'+types, TObject(@ptr));
end;
{ TInternalProcess }
procedure TInternalProcess.Execute;
var
exp : TFPExpressionParser;
i : Integer;
nm : string;
j : Integer;
ptr : TFPExprFunctionCallBack;
begin
try
exp:=TFPExpressionParser.Create(nil);
try
for i:=0 to IntProc.Count-1 do begin
ptr:=TFPExprFunctionCallBack(IntProc.Objects[i]);
nm:=IntProc[i];
j:=Pos('|',nm);
exp.Identifiers.AddFunction(Copy(nm, 1, j-1), 'I', Copy(nm, j+1, length(nm)), ptr);
end;
exp.Expression:=fCommandLine;
exp.Evaluate;
finally
exp.Free;
end;
except
on E: Exception do
LastError := E.Message;
end;
end;
initialization
IntProc := TStringList.Create;
finalization
IntProc.Free;
end.
With that code, i get exactly what i want ( but, yes, some "hidden" code to add to original code...).
=> @ Marcov and Skalogryz : ok, you win, where is the bug (if there is one) ?
-
=> @ Marcov and Skalogryz : ok, you win, where is the bug (if there is one) ?
There's no bug. You need to understand and accept the following:
* pascal has no built-in features for "compiling" in run-time, because it's a compiled language. Most of programs written with pascal doesn't need the feature... except for the compiler itself.
* in order to be able to "compile" a code in run-time you'll need the proper code -> either use a pascal compiler code or a interpretting (scripting) code.
* either solution will come with it's own price: size - your library will grow, performance - executing a script in run-time always comes with a (small) penalty.
You also need to make sure that this "compile" time won't affect your library. Multimedia code is typically time critical.
That's it. if you really need scripting - you must accept to pay the price.
Or on the other hand, just expose the address of functions (by either exporting them as library functions / procedural variables in a records / interfaces). In that case, there's no price, but some (none?) issues for other languages. And yes, it might require you to do some extra wrappers for your internal library structure.
The choise is yours, and you can take either approach and just see how either works.
The price of "size" and "time" will depend on the choice of a "scripting" implementation you choose.
----
...oh, and feel free to post P-code (http://en.wikipedia.org/wiki/P-code_machine) request with LGPL license to the bugtracker :) and yes - send patches
-
@ brilliant and clear => ok, i understand now...
I will study, compare, test, the plus and minus of using your TInternalProcess.
PS : Is there a plan for Pascal built-in features for "compiling" in run-time" ?
Many thanks.
-
PS : Is there a plan for Pascal built-in features for "compiling" in run-time" ?
Yes and no, as said the license of the compiler makes it nearly impossible to base that on the compiler. Afaik Lazarus is packaging a simple scripting engine in trunk though.
-
@ Marcov => thanks for all that clear and honest explanations.
Fred