Recent

Author Topic: Unwanted Underscores Added To External Names  (Read 1444 times)

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Unwanted Underscores Added To External Names
« on: May 14, 2020, 06:50:45 pm »
I'm working in MSys on 64-Bit Windows 7, running "Free Pascal Compiler version 3.0.4 [2017/10/06] for i386". I get the same behavior if I work at the DOS prompt, issuing the same FPC commands. Here is a library program "d.pas" that compiles with "fpc d" to produce "d.dll".

Code: Pascal  [Select][+][-]
  1. library d;
  2.  
  3. function answer:longint; cdecl;
  4. begin
  5.         writeln('Hello from answer');
  6.         answer:=42;
  7. end;
  8.  
  9. exports
  10.         answer name '_answer';
  11. end.

The Dependency Walker (http://dependencywalker.com) tells me that my exported routine is declared in the d.dll under the name "_answer", as I expect. I have main program "m.pas" in the same directory as "d.dll" that compiles with "fpc m" to produce executable "m.exe", which calls "_answer".

Code: Pascal  [Select][+][-]
  1. program m;
  2.  
  3. {$linklib d}
  4.  
  5. function answer:integer; cdecl; external name 'answer';
  6.  
  7. begin
  8.         writeln('Answer: ',answer);
  9. end.

This works, see below for output.

Code: Bash  [Select][+][-]
  1. $ ./m.exe
  2. Hello from answer
  3. Answer: 42

The external declaration of "_answer" in "m.pas" does not include the underscore. It's just "answer". The underscore is added by FPC at compile time. If I want "m.pas" to link to a routine "answer" with no underscore in front of it, I am unable to do so with the "cdecl". And that's a problem for me, because the Tcl86.dll library I'm trying to link to does not have underscores in front of its routine names, and uses the "cdecl" calling convention.

If I switch to "stdcall" in "d.pas" and export the routine as "answer", then switch to "stdcall" in "m.pas" and once again give the external name as just "answer", no underscore is added by FPC when it compiles "m.pas", and the two programs work together just fine. But that does not help me, because the Tcl86 library is compiled with the "cdecl" convention.

How can I stop FPC from adding an underscore to "cdecl" external names?

jiaxing2

  • Full Member
  • ***
  • Posts: 164
Re: Unwanted Underscores Added To External Names
« Reply #1 on: May 14, 2020, 07:02:29 pm »
Nope. It's designed to do so. Nothing wrong. You should learn more about calling convention. A single Google search is enough.

If the library you want to link use cdecl, use cdecl with your declaration. Just that.
« Last Edit: May 14, 2020, 07:13:33 pm by jiaxing2 »

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Re: Unwanted Underscores Added To External Names
« Reply #2 on: May 14, 2020, 07:43:12 pm »
Dear Jiaxing2,

>  You should learn more about calling convention. A single Google search is enough.

Ha! You sure can talk the talk Jiaxing2. Let's see if you can walk the walk as well. I challenge you to call "answer", declared in the following library, from an FPC main program on Windows, and using only the "cdecl" calling convention in the main program.

Code: Pascal  [Select][+][-]
  1. library d;
  2.  
  3. function answer:longint; cdecl;
  4. begin
  5.         writeln('Hello from answer');
  6.         answer:=42;
  7. end;
  8.  
  9. exports
  10.         answer name 'answer';
  11. end.

And no: you are not allowed to add an underscore to the export name in "d.pas". I look forward to seeing your masterful solution, and being impressed.

Best, Kevan

jiaxing2

  • Full Member
  • ***
  • Posts: 164
Re: Unwanted Underscores Added To External Names
« Reply #3 on: May 14, 2020, 07:55:36 pm »
Dear Jiaxing2,

>  You should learn more about calling convention. A single Google search is enough.

Ha! You sure can talk the talk Jiaxing2. Let's see if you can walk the walk as well. I challenge you to call "answer", declared in the following library, from an FPC main program on Windows, and using only the "cdecl" calling convention in the main program.

Code: Pascal  [Select][+][-]
  1. library d;
  2.  
  3. function answer:longint; cdecl;
  4. begin
  5.         writeln('Hello from answer');
  6.         answer:=42;
  7. end;
  8.  
  9. exports
  10.         answer name 'answer';
  11. end.

And no: you are not allowed to add an underscore to the export name in "d.pas". I look forward to seeing your masterful solution, and being impressed.

Best, Kevan

Because export answer; is enough. No need the other part.

I have follow this guide for a long time, and have never fail.

https://macpgmr.github.io/MacXPlatform/PascalDynLibs.html

p/s: I'm on Linux and having abandoned Windows for years, I will not install Windows again just to win a challenge of you.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8780
  • FPC developer.
Re: Unwanted Underscores Added To External Names
« Reply #4 on: May 14, 2020, 07:55:53 pm »
This is governed by the CPREFIX of the said target. (see compiler/systems/i_win.pas  line 66)

These per target settings fixate normal per target conventions (usually afaik cprefix is mainly a binary format thing. A.OUT and COFF have _, ELF not).

However that is from the declaration side, your problem is on the referencing side. However your reference is a bit werid, since it is supposed to have the dll name, and not use {$linklib??}.

The default way works fine, so I think your main problem is in the way you manually hacked together an import lib or so?

Quote

D:\Testing\2345>m
Hello from answer
Answer: 42

Code: [Select]
    program m;
     
    function answer:integer; cdecl; external 'd' name 'answer';
     
    begin
            writeln('Answer: ',answer);
    end.

jiaxing2

  • Full Member
  • ***
  • Posts: 164
Re: Unwanted Underscores Added To External Names
« Reply #5 on: May 14, 2020, 08:10:35 pm »
The default way works fine, so I think your main problem is in the way you manually hacked together an import lib or so?

I think so, too.

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Re: Unwanted Underscores Added To External Names
« Reply #6 on: May 14, 2020, 08:14:54 pm »
Jiaxing2:

Quote
I'm on Linux and having abandoned Windows for years, I will not install Windows again just to win a challenge of you.

The problem arises only on Windows. On Linux, FPC's implementation of "cdecl" does not add underscores to external file names.

Marcov:

Thanks for the code reference. There is this line in i_win.pas.

Code: Pascal  [Select][+][-]
  1.             Cprefix      : '_';

It looks like I need to change that, because it's the character that is added to all eternal names.

Fred vS

  • Hero Member
  • *****
  • Posts: 1871
    • StrumPract is the musicians best friend
Re: Unwanted Underscores Added To External Names
« Reply #7 on: May 14, 2020, 08:22:11 pm »
Quote
It looks like I need to change that, because it's the character that is added to all eternal names.

Or, maybe, use LoadLibrary() instead of external...

Ok, ok, I'm going out, sorry for the noise.
I use Lazarus 2.0.6 32/64 and FPC 3.2.0 32/64 on Debian 10.2 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64 and Mac OS X Snow Leopard 32.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt, Carbon.

https://github.com/fredvs
https://gitlab.com/fredvs

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Re: Unwanted Underscores Added To External Names
« Reply #8 on: May 14, 2020, 08:23:29 pm »
Quote
The default way works fine, so I think your main problem is in the way you manually hacked together an import lib or so?

No, that's not my problem. I composed some test code so that it would be easy for you to duplicate the problem and see it for yourself. The code I gave you has the fewest possible lines to illustrate the problem.

The problem arises in practice because the TclTk libraries on Windows are 'cdecl' and the routine names have no underscores in front of them. But FPC on Windows with 'cdecl' will not allow me to declare an external 'cdecl' routine name "Tcl_PckgProvide" because it always converts that name to "_Tcl_PckgProvide". At that point, the linker gives an unhelpful message, it says, "can't find d.dll". But that's not the actual problem. The problem is that it can't find "_Tcl_PckgProvide".

If I switch to 'stdcall', then no underscore is added by FPC and I can link to the Tcl libraries, but of course the main program crashes when I actually call the Tcl routine, because the calling conventions are different.

I wanted to spare you the the whole "Tcl" business, so made you some simple code to illustrate the problem.

Now, how can I change "Cprefix"? Do I need to compile FPC from sources on Windows?

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Re: Unwanted Underscores Added To External Names
« Reply #9 on: May 14, 2020, 08:42:45 pm »
Quote
Or, maybe, use LoadLibrary() instead of external...

I just tried with LoadLibrary, using your sample code, for which I thank you again. Even with LoadLibrary, the fact that we declare the function as 'cdecl' is sufficient to make FPC add an underscore to the external name when it tries to look up the routine in the dynamic library.

Code: Pascal  [Select][+][-]
  1. type
  2.   TMyMethod=function :integer; cdecl;

So the routine has to have an underscore prefix in the dynamic library, or else there is no way to call it.

jiaxing2

  • Full Member
  • ***
  • Posts: 164
Re: Unwanted Underscores Added To External Names
« Reply #10 on: May 14, 2020, 08:43:11 pm »
Quote
The default way works fine, so I think your main problem is in the way you manually hacked together an import lib or so?

No, that's not my problem. I composed some test code so that it would be easy for you to duplicate the problem and see it for yourself. The code I gave you has the fewest possible lines to illustrate the problem.

The problem arises in practice because the TclTk libraries on Windows are 'cdecl' and the routine names have no underscores in front of them. But FPC on Windows with 'cdecl' will not allow me to declare an external 'cdecl' routine name "Tcl_PckgProvide" because it always converts that name to "_Tcl_PckgProvide". At that point, the linker gives an unhelpful message, it says, "can't find d.dll". But that's not the actual problem. The problem is that it can't find "_Tcl_PckgProvide".

If I switch to 'stdcall', then no underscore is added by FPC and I can link to the Tcl libraries, but of course the main program crashes when I actually call the Tcl routine, because the calling conventions are different.

I wanted to spare you the the whole "Tcl" business, so made you some simple code to illustrate the problem.

So you think this is bug. Let's the FPC devs invest on it.

Quote
Now, how can I change "Cprefix"? Do I need to compile FPC from sources on Windows?

What about compile Tcl/Tk from source? It's open source.

From what you said I think you installed them from MSYS2. Try get the official version from their homepage.

BTW, some FPC devs here told me on Windows x64 there is only one calling convention and regardless of how you declared them as cdecl or stdcall it would just work. You said your program crashed so I think it's more likely a bug of FPC.

jiaxing2

  • Full Member
  • ***
  • Posts: 164
Re: Unwanted Underscores Added To External Names
« Reply #11 on: May 14, 2020, 08:47:14 pm »
Quote
Or, maybe, use LoadLibrary() instead of external...

I just tried with LoadLibrary, using your sample code, for which I thank you again. Even with LoadLibrary, the fact that we declare the function as 'cdecl' is sufficient to make FPC add an underscore to the external name when it tries to look up the routine in the dynamic library.

Code: Pascal  [Select][+][-]
  1. type
  2.   TMyMethod=function :integer; cdecl;

So the routine has to have an underscore prefix in the dynamic library, or else there is no way to call it.

I don't understand. An underscore is normal for cdecl. I think FPC does everything right. Check your Tcl/Tk dll.

Fred vS

  • Hero Member
  • *****
  • Posts: 1871
    • StrumPract is the musicians best friend
Re: Unwanted Underscores Added To External Names
« Reply #12 on: May 14, 2020, 09:11:28 pm »
Quote
Or, maybe, use LoadLibrary() instead of external...

I just tried with LoadLibrary, using your sample code, for which I thank you again. Even with LoadLibrary, the fact that we declare the function as 'cdecl' is sufficient to make FPC add an underscore to the external name when it tries to look up the routine in the dynamic library.

Code: Pascal  [Select][+][-]
  1. type
  2.   TMyMethod=function :integer; cdecl;

So the routine has to have an underscore prefix in the dynamic library, or else there is no way to call it.

I don't understand. An underscore is normal for cdecl. I think FPC does everything right. Check your Tcl/Tk dll.

IMHO, fpc should give more liberty to developers, why add by force that "_"?.

It is the same for so number with Unix libraries, the so number is arbitrary deleted, so you cannot link libraries with name 'libmylib.so.2" for example.

We are not all stupid and let us the choice to do things . (even if you find it stupid).

I use Lazarus 2.0.6 32/64 and FPC 3.2.0 32/64 on Debian 10.2 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64 and Mac OS X Snow Leopard 32.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt, Carbon.

https://github.com/fredvs
https://gitlab.com/fredvs

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8780
  • FPC developer.
Re: Unwanted Underscores Added To External Names
« Reply #13 on: May 14, 2020, 09:14:15 pm »
Quote
The default way works fine, so I think your main problem is in the way you manually hacked together an import lib or so?

No, that's not my problem. I composed some test code so that it would be easy for you to duplicate the problem and see it for yourself. The code I gave you has the fewest possible lines to illustrate the problem.

Please do try to use the syntax that I specified:

Code: Pascal  [Select][+][-]
  1.     function answer:integer; cdecl; external 'sometcltk.dll' name 'Tcl_PckgProvide';

remove the $linklib and move away all .a's, and see what happens.

For me it worked, and I tried with Dependencies (the successor to dependency walker) that the DLL that I tested with has no underscore.
« Last Edit: May 14, 2020, 09:17:55 pm by marcov »

Kevan S. Hashemi

  • New Member
  • *
  • Posts: 23
Re: Unwanted Underscores Added To External Names
« Reply #14 on: May 14, 2020, 09:18:09 pm »
Quote
So you think this is bug. Let's the FPC devs invest on it.

No, it's not a bug. It's deliberate policy. Here is the code from pdecvar.pas, routine name read_public_and_external:

Code: Pascal  [Select][+][-]
  1. { Add C _ prefix }
  2.       if is_cdecl or
  3.          (
  4.           is_dll and
  5.           (target_info.system in systems_darwin)
  6.          ) then
  7.         C_Name := target_info.Cprefix+C_Name;

On Windows and MacOSX, Cprefix is "_". You can call external "cdecl" routines only if their names begin with underscores. On Linux, Cprefix is empty, so you have complete control over the names.

Meanwhile, the "export" section of a "library" in FPC does not force the addition of an underscore for "cdecl" routines, so we must add the underscore ourselves on MacOSX and Windows, but not on Linux, by use of OS-dependent compiler directives. Otherwise, the "export" and "external" don't agree.

Quote
An underscore is normal for cdecl.

The Linux and Windows Tcl libraries don't have underscores in front of their names. I'm calling the Linux Tcl libraries with "cdecl" just fine. I just examined half a dozen Linux kernel libraries with the "nm" utility, and there are many names that have no underscores in front of them.

 

TinyPortal © 2005-2018