Recent

Author Topic: Statically link object files from GoLang  (Read 917 times)

jimako

  • Newbie
  • Posts: 4
Statically link object files from GoLang
« on: May 30, 2020, 09:53:16 am »
I've been looking at this all day and am stumped. I want to statically link a few procedures written in Go into a Freepascal (Lazarus) program. Now, you can make Go generate object files, but it is not clear what particular format these are in. Also, it is not clear what format FP can handle. I've tried to reduce this to the simplest possible example. Here's the Golang code:

Code: C  [Select][+][-]
  1. package main
  2.  
  3. import "C"
  4.  
  5. //export AddTwo
  6. func AddTwo(a, b int) int {
  7.         return a + b
  8. }
  9.  
  10. func main () {}
  11.  

and here's the FP code:

Code: Pascal  [Select][+][-]
  1. program usego;
  2.  
  3. {$link go/golib.a}  // or .o or .obj or .lib -- tried all sorts of options here
  4.  
  5. uses sysutils;
  6.  
  7. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';  
  8.  
  9. begin
  10.   Writeln('Hello World');
  11.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  12. end.
  13.  

Now, the problem is there are several ways to create object files in Go. I've tried all of these:

go build -buildmode=c-archive -o golib3.a golib3.go
go tool compile -o golib3.a golib3.go
go tool compile -linkobj golib3.o -o golib3.a golib3.go


The "tool compile" options require that you remove the import "C" or it doesn't compile.

The FP (Lazarus) build generates the error:

Error: Illegal COFF Magic while reading (...path to...)\go\golib.(whatever)

The object file is found (you get a not found message otherwise), so the problem is that the format of the file is not as expected. It's not clear to me whether the message is saying
  • it can't handle COFF format (in which case, what format is it expecting?)
  • it needs and is not getting a COFF file
  • the file is a COFF file but contains some unexpected "magic"
Has anyone managed to get this working? Note that I am specifically looking to statically link the Go object code -- I know I could create a DLL/SO file, but that's not going to work in this instance.

Oh, working on Win10 at the moment, but will also need to get this to work on at least Mac OSX and (ideally) Linux.

Any insights or pointers would be greatly appreciated!

PascalDragon

  • Hero Member
  • *****
  • Posts: 2391
  • Compiler Developer
Re: Statically link object files from GoLang
« Reply #1 on: May 30, 2020, 10:45:43 am »
The $Link directive is for linking object files, however .a files are static libraries, so you need to use $LinkLib.

Additionally you might want to post the first 255 Byte or so of the library. I might be able to tell you whether the format is compatible.

jimako

  • Newbie
  • Posts: 4
Re: Statically link object files from GoLang
« Reply #2 on: May 30, 2020, 04:05:12 pm »
Thanks for the input. I've tried different compile options -- don't assume that .a actually means library (although it might) as it is just the extension I have specified. I think that the first compile does indeed result in a library but the Go docs are a little sketchy on the details. If you could shed some light, that would be awesome. Anyway, here is the info. To be clear, the first compile uses a different source to the other two, as it needs the 'import "C"' which fails in the other two.

Here are the two sources:

golib.go:

Code: C  [Select][+][-]
  1. package main
  2.  
  3. import "C"
  4.  
  5. //export AddTwo
  6. func AddTwo(a, b int) int {
  7.         return a + b
  8. }
  9.  
  10. func main () {}

golib2.go and golib3.go (identical)

Code: C  [Select][+][-]
  1. package main
  2.  
  3. //export AddTwo
  4. func AddTwo(a, b int) int {
  5.         return a + b
  6. }
  7.  
  8. func main () {}

So, here are dumps of the first part of the output object files / libraries, none of which link and all of which result in the same error:

COMPILE WITH: go build -buildmode=c-archive -o golib.a golib.go

Code: Text  [Select][+][-]
  1. hexdump -C golib.a
  2. 000000  21 3c 61 72 63 68 3e 0a 2f 20 20 20 20 20 20 20  !<arch>./
  3. 000010  20 20 20 20 20 20 20 20 31 35 39 30 38 34 35 36          15908456
  4. 000020  35 31 20 20 30 20 20 20 20 20 30 20 20 20 20 20  51  0     0
  5. 000030  30 20 20 20 20 20 20 20 34 39 32 33 34 20 20 20  0       49234
  6. 000040  20 20 60 0a 00 00 06 aa 00 00 c0 96 00 00 c0 96    `.............
  7. 000050  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  8. 000060  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  9. 000070  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  10. 000080  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  11. 000090  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  12. 0000a0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  13. 0000b0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  14. 0000c0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  15. 0000d0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  16. 0000e0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................
  17. 0000f0  00 00 c0 96 00 00 c0 96 00 00 c0 96 00 00 c0 96  ................

COMPILE WITH: go tool compile -linkobj golib2.o -o golib2.a golib2.go

Code: Text  [Select][+][-]
  1. hexdump -C golib2.a
  2. 000000  21 3c 61 72 63 68 3e 0a 5f 5f 2e 50 4b 47 44 45  !<arch>.__.PKGDE
  3. 000010  46 20 20 20 20 20 20 20 30 20 20 20 20 20 20 20  F       0
  4. 000020  20 20 20 20 30 20 20 20 20 20 30 20 20 20 20 20      0     0
  5. 000030  36 34 34 20 20 20 20 20 31 38 39 20 20 20 20 20  644     189
  6. 000040  20 20 60 0a 67 6f 20 6f 62 6a 65 63 74 20 77 69    `.go object wi
  7. 000050  6e 64 6f 77 73 20 61 6d 64 36 34 20 67 6f 31 2e  ndows amd64 go1.
  8. 000060  31 34 2e 33 20 58 3a 66 72 61 6d 65 70 6f 69 6e  14.3 X:framepoin
  9. 000070  74 65 72 0a 6d 61 69 6e 0a 0a 0a 24 24 42 0a 69  ter.main...$$B.i
  10. 000080  01 42 2a 26 43 3a 5c 55 73 65 72 73 5c 6a 69 6d  .B*&C:\Users\jim
  11. 000090  5c 63 6f 64 65 5c 6c 69 62 74 65 73 74 5c 67 6f  \code\libtest\go
  12. 0000a0  5c 67 6f 6c 69 62 33 2e 67 6f 01 61 01 62 00 04  \golib3.go.a.b..
  13. 0000b0  6d 61 69 6e 09 2e 69 6e 69 74 74 61 73 6b 06 41  main..inittask.A
  14. 0000c0  64 64 54 77 6f 7d 0a 16 00 06 24 01 27 01 29 98  ddTwo}....$.'.).
  15. 0000d0  01 98 01 46 1a 12 00 02 1c 27 01 0c 29 01 01 1c  ...F.....'..)...
  16. 0000e0  2b 01 00 2b 2b 2b 05 55 08 56 0a 16 00 07 2b 01  +..+++.U.V....+.
  17. 0000f0  2b 2c 00 02 31 24 3b 0e 01 2b 01 3b 00 0a 24 24  +,..1$;..+.;..$$
  18. 000100  0a 00                                            ..
  19.  
  20. hexdump -C golib2.o
  21. 000000  21 3c 61 72 63 68 3e 0a 5f 67 6f 5f 2e 6f 20 20  !<arch>._go_.o
  22. 000010  20 20 20 20 20 20 20 20 30 20 20 20 20 20 20 20          0
  23. 000020  20 20 20 20 30 20 20 20 20 20 30 20 20 20 20 20      0     0
  24. 000030  36 34 34 20 20 20 20 20 31 30 34 35 20 20 20 20  644     1045
  25. 000040  20 20 60 0a 67 6f 20 6f 62 6a 65 63 74 20 77 69    `.go object wi
  26. 000050  6e 64 6f 77 73 20 61 6d 64 36 34 20 67 6f 31 2e  ndows amd64 go1.
  27. 000060  31 34 2e 33 20 58 3a 66 72 61 6d 65 70 6f 69 6e  14.3 X:framepoin
  28. 000070  74 65 72 0a 6d 61 69 6e 0a 0a 0a 21 0a 00 67 6f  ter.main...!..go
  29. 000080  31 31 34 6c 64 01 00 04 2e 67 6f 66 69 6c 65 2e  114ld....gofile.
  30. 000090  2e 3c 61 75 74 6f 67 65 6e 65 72 61 74 65 64 3e  .<autogenerated>
  31. 0000a0  5c 67 6f 66 69 6c 65 2e 2e 43 3a 2f 55 73 65 72  \gofile..C:/User
  32. 0000b0  73 2f 6a 69 6d 2f 63 6f 64 65 2f 6c 69 62 74 65  s/jim/code/libte
  33. 0000c0  73 74 2f 67 6f 2f 67 6f 6c 69 62 33 2e 67 6f fe  st/go/golib3.go.
  34. 0000d0  12 22 22 2e 41 64 64 54 77 6f 02 fe 54 67 63 6c  ."".AddTwo..Tgcl
  35. 0000e0  6f 63 61 6c 73 c2 b7 33 33 63 64 65 63 63 63 63  ocals..33cdecccc
  36. 0000f0  65 62 65 38 30 33 32 39 66 31 66 64 62 65 65 37  ebe80329f1fdbee7

COMPILE WITH: go tool compile -o golib3.a golib3.go

Code: Text  [Select][+][-]
  1. hexdump -C golib3.a
  2. 000000  21 3c 61 72 63 68 3e 0a 5f 5f 2e 50 4b 47 44 45  !<arch>.__.PKGDE
  3. 000010  46 20 20 20 20 20 20 20 30 20 20 20 20 20 20 20  F       0
  4. 000020  20 20 20 20 30 20 20 20 20 20 30 20 20 20 20 20      0     0
  5. 000030  36 34 34 20 20 20 20 20 31 38 39 20 20 20 20 20  644     189
  6. 000040  20 20 60 0a 67 6f 20 6f 62 6a 65 63 74 20 77 69    `.go object wi
  7. 000050  6e 64 6f 77 73 20 61 6d 64 36 34 20 67 6f 31 2e  ndows amd64 go1.
  8. 000060  31 34 2e 33 20 58 3a 66 72 61 6d 65 70 6f 69 6e  14.3 X:framepoin
  9. 000070  74 65 72 0a 6d 61 69 6e 0a 0a 0a 24 24 42 0a 69  ter.main...$$B.i
  10. 000080  01 42 2a 26 43 3a 5c 55 73 65 72 73 5c 6a 69 6d  .B*&C:\Users\jim
  11. 000090  5c 63 6f 64 65 5c 6c 69 62 74 65 73 74 5c 67 6f  \code\libtest\go
  12. 0000a0  5c 67 6f 6c 69 62 33 2e 67 6f 01 61 01 62 00 04  \golib3.go.a.b..
  13. 0000b0  6d 61 69 6e 09 2e 69 6e 69 74 74 61 73 6b 06 41  main..inittask.A
  14. 0000c0  64 64 54 77 6f 7d 0a 16 00 06 24 01 27 01 29 98  ddTwo}....$.'.).
  15. 0000d0  01 98 01 46 1a 12 00 02 1c 27 01 0c 29 01 01 1c  ...F.....'..)...
  16. 0000e0  2b 01 00 2b 2b 2b 05 55 08 56 0a 16 00 07 2b 01  +..+++.U.V....+.
  17. 0000f0  2b 2c 00 02 31 24 3b 0e 01 2b 01 3b 00 0a 24 24  +,..1$;..+.;..$$

If it helps, these are the sizes of the files:

2,688,846 golib.a
      258 golib2.a
    1,114 golib2.o
    1,364 golib3.a


Thanks in advance for any light you can shed on this!

PascalDragon

  • Hero Member
  • *****
  • Posts: 2391
  • Compiler Developer
Re: Statically link object files from GoLang
« Reply #3 on: May 30, 2020, 06:57:13 pm »
Those three are definitely library archive files, cause those start with !<arch>#10. Thus you need to use $LinkLib file.a. If the file does not end with .a you need to use $LinkLib file.o static as otherwise the compiler will treat it as a dynamic library.

One further problem might be that the libraries are in the MSVC archive format and not the GCC archive format. FPC's internal linker currently only supports the GCC archive format. You could try to use the external linker using -Xe, but this will probably slow down compilation and there will be problems if you combine smartlinking with DWARF debug information (namely that smartlinking won't work then).

jimako

  • Newbie
  • Posts: 4
Re: Statically link object files from GoLang
« Reply #4 on: May 31, 2020, 03:22:37 am »
Thanks for that, but I suspect that the archive is in fact created by GCC, specifically MinGW. I created a little dummy C program to mimic the Go one, and compiled it to an object file and library:

add2.c

Code: C  [Select][+][-]
  1. int AddTwo(int a, int b) {
  2.         return a + b;
  3. }

add2.h

Code: C  [Select][+][-]
  1. int AddTwo(int a, int b);

Code: Text  [Select][+][-]
  1. gcc -c addtwo.c -o addtwo.o
  2. ar rcs addtwo.a addtwo.o

So, dumping the two files in hex:

hexdump -C addtwo.a

Code: Text  [Select][+][-]
  1. 000000  21 3c 61 72 63 68 3e 0a 2f 20 20 20 20 20 20 20  !<arch>./
  2. 000010  20 20 20 20 20 20 20 20 31 35 39 30 38 38 34 33          15908843
  3. 000020  34 32 20 20 30 20 20 20 20 20 30 20 20 20 20 20  42  0     0
  4. 000030  30 20 20 20 20 20 20 20 31 36 20 20 20 20 20 20  0       16
  5. 000040  20 20 60 0a 00 00 00 01 00 00 00 54 41 64 64 54    `........TAddT
  6. 000050  77 6f 00 00 61 64 64 74 77 6f 2e 6f 2f 20 20 20  wo..addtwo.o/
  7. 000060  20 20 20 20 31 35 39 30 38 38 34 32 36 38 20 20      1590884268
  8. 000070  30 20 20 20 20 20 30 20 20 20 20 20 31 30 30 36  0     0     1006
  9. 000080  36 36 20 20 37 32 30 20 20 20 20 20 20 20 60 0a  66  720       `.
  10. 000090  64 86 06 00 00 00 00 00 96 01 00 00 10 00 00 00  d...............
  11. 0000a0  00 00 04 00 2e 74 65 78 74 00 00 00 00 00 00 00  .....text.......
  12. 0000b0  00 00 00 00 20 00 00 00 04 01 00 00 00 00 00 00  .... ...........
  13. 0000c0  00 00 00 00 00 00 00 00 20 00 50 60 2e 64 61 74  ........ .P`.dat
  14. 0000d0  61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  a...............
  15. 0000e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  16. 0000f0  40 00 50 c0 2e 62 73 73 00 00 00 00 00 00 00 00  @.P..bss........
  17. 000100  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  18. 000110  00 00 00 00 00 00 00 00 80 00 50 c0 2e 78 64 61  ..........P..xda
  19. 000120  74 61 00 00 00 00 00 00 00 00 00 00 08 00 00 00  ta..............
  20. 000130  24 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  $...............
  21. 000140  40 00 30 40 2e 70 64 61 74 61 00 00 00 00 00 00  @.0@.pdata......
  22. 000150  00 00 00 00 0c 00 00 00 2c 01 00 00 78 01 00 00  ........,...x...
  23. 000160  00 00 00 00 03 00 00 00 40 00 30 40 2f 34 00 00  ........@.0@/4..
  24. 000170  00 00 00 00 00 00 00 00 00 00 00 00 40 00 00 00  ............@...
  25. 000180  38 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  8...............
  26. 000190  40 00 50 40 55 48 89 e5 89 4d 10 89 55 18 8b 55  @.P@UH...M..U..U
  27. 0001a0  10 8b 45 18 01 d0 5d c3 90 90 90 90 90 90 90 90  ..E...].........
  28. 0001b0  90 90 90 90 01 04 02 05 04 03 01 50 00 00 00 00  ...........P....
  29. 0001c0  14 00 00 00 00 00 00 00 47 43 43 3a 20 28 78 38  ........GCC: (x8
  30. 0001d0  36 5f 36 34 2d 70 6f 73 69 78 2d 73 65 68 2d 72  6_64-posix-seh-r
  31. 0001e0  65 76 30 2c 20 42 75 69 6c 74 20 62 79 20 4d 69  ev0, Built by Mi
  32. 0001f0  6e 47 57 2d 57 36 34 20 70 72 6f 6a 65 63 74 29  nGW-W64 project)
  33. 000200  20 38 2e 31 2e 30 00 00 00 00 00 00 04 00 00 00   8.1.0..........
  34. 000210  03 00 04 00 00 00 04 00 00 00 03 00 08 00 00 00  ................
  35. 000220  0a 00 00 00 03 00 2e 66 69 6c 65 00 00 00 00 00  .......file.....
  36. 000230  00 00 fe ff 00 00 67 01 61 64 64 74 77 6f 2e 63  ......g.addtwo.c
  37. 000240  00 00 00 00 00 00 00 00 00 00 41 64 64 54 77 6f  ..........AddTwo
  38. 000250  00 00 00 00 00 00 01 00 20 00 02 01 00 00 00 00  ........ .......
  39. 000260  00 00 00 00 00 00 00 00 00 00 00 00 00 00 2e 74  ...............t
  40. 000270  65 78 74 00 00 00 00 00 00 00 01 00 00 00 03 01  ext.............
  41. 000280  14 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  42. 000290  00 00 2e 64 61 74 61 00 00 00 00 00 00 00 02 00  ...data.........
  43. 0002a0  00 00 03 01 00 00 00 00 00 00 00 00 00 00 00 00  ................
  44. 0002b0  00 00 00 00 00 00 2e 62 73 73 00 00 00 00 00 00  .......bss......
  45. 0002c0  00 00 03 00 00 00 03 01 00 00 00 00 00 00 00 00  ................
  46. 0002d0  00 00 00 00 00 00 00 00 00 00 2e 78 64 61 74 61  ...........xdata
  47. 0002e0  00 00 00 00 00 00 04 00 00 00 03 01 08 00 00 00  ................
  48. 0002f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 2e 70  ...............p
  49. 000300  64 61 74 61 00 00 00 00 00 00 05 00 00 00 03 01  data............
  50. 000310  0c 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00  ................
  51. 000320  00 00 00 00 00 00 0f 00 00 00 00 00 00 00 06 00  ................
  52. 000330  00 00 03 01 3f 00 00 00 00 00 00 00 00 00 00 00  ....?...........
  53. 000340  00 00 00 00 00 00 1a 00 00 00 2e 72 64 61 74 61  ...........rdata
  54. 000350  24 7a 7a 7a 00 2e 72 64 61 74 61 24 7a 7a 7a 00  $zzz..rdata$zzz.

hexdump -C addtwo.o

Code: Text  [Select][+][-]
  1. 000000  64 86 06 00 00 00 00 00 96 01 00 00 10 00 00 00  d...............
  2. 000010  00 00 04 00 2e 74 65 78 74 00 00 00 00 00 00 00  .....text.......
  3. 000020  00 00 00 00 20 00 00 00 04 01 00 00 00 00 00 00  .... ...........
  4. 000030  00 00 00 00 00 00 00 00 20 00 50 60 2e 64 61 74  ........ .P`.dat
  5. 000040  61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  a...............
  6. 000050  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  7. 000060  40 00 50 c0 2e 62 73 73 00 00 00 00 00 00 00 00  @.P..bss........
  8. 000070  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  9. 000080  00 00 00 00 00 00 00 00 80 00 50 c0 2e 78 64 61  ..........P..xda
  10. 000090  74 61 00 00 00 00 00 00 00 00 00 00 08 00 00 00  ta..............
  11. 0000a0  24 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  $...............
  12. 0000b0  40 00 30 40 2e 70 64 61 74 61 00 00 00 00 00 00  @.0@.pdata......
  13. 0000c0  00 00 00 00 0c 00 00 00 2c 01 00 00 78 01 00 00  ........,...x...
  14. 0000d0  00 00 00 00 03 00 00 00 40 00 30 40 2f 34 00 00  ........@.0@/4..
  15. 0000e0  00 00 00 00 00 00 00 00 00 00 00 00 40 00 00 00  ............@...
  16. 0000f0  38 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  8...............
  17. 000100  40 00 50 40 55 48 89 e5 89 4d 10 89 55 18 8b 55  @.P@UH...M..U..U
  18. 000110  10 8b 45 18 01 d0 5d c3 90 90 90 90 90 90 90 90  ..E...].........
  19. 000120  90 90 90 90 01 04 02 05 04 03 01 50 00 00 00 00  ...........P....
  20. 000130  14 00 00 00 00 00 00 00 47 43 43 3a 20 28 78 38  ........GCC: (x8
  21. 000140  36 5f 36 34 2d 70 6f 73 69 78 2d 73 65 68 2d 72  6_64-posix-seh-r
  22. 000150  65 76 30 2c 20 42 75 69 6c 74 20 62 79 20 4d 69  ev0, Built by Mi
  23. 000160  6e 47 57 2d 57 36 34 20 70 72 6f 6a 65 63 74 29  nGW-W64 project)
  24. 000170  20 38 2e 31 2e 30 00 00 00 00 00 00 04 00 00 00   8.1.0..........
  25. 000180  03 00 04 00 00 00 04 00 00 00 03 00 08 00 00 00  ................
  26. 000190  0a 00 00 00 03 00 2e 66 69 6c 65 00 00 00 00 00  .......file.....
  27. 0001a0  00 00 fe ff 00 00 67 01 61 64 64 74 77 6f 2e 63  ......g.addtwo.c
  28. 0001b0  00 00 00 00 00 00 00 00 00 00 41 64 64 54 77 6f  ..........AddTwo
  29. 0001c0  00 00 00 00 00 00 01 00 20 00 02 01 00 00 00 00  ........ .......
  30. 0001d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 2e 74  ...............t
  31. 0001e0  65 78 74 00 00 00 00 00 00 00 01 00 00 00 03 01  ext.............
  32. 0001f0  14 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  33. 000200  00 00 2e 64 61 74 61 00 00 00 00 00 00 00 02 00  ...data.........
  34. 000210  00 00 03 01 00 00 00 00 00 00 00 00 00 00 00 00  ................
  35. 000220  00 00 00 00 00 00 2e 62 73 73 00 00 00 00 00 00  .......bss......
  36. 000230  00 00 03 00 00 00 03 01 00 00 00 00 00 00 00 00  ................
  37. 000240  00 00 00 00 00 00 00 00 00 00 2e 78 64 61 74 61  ...........xdata
  38. 000250  00 00 00 00 00 00 04 00 00 00 03 01 08 00 00 00  ................
  39. 000260  00 00 00 00 00 00 00 00 00 00 00 00 00 00 2e 70  ...............p
  40. 000270  64 61 74 61 00 00 00 00 00 00 05 00 00 00 03 01  data............
  41. 000280  0c 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00  ................
  42. 000290  00 00 00 00 00 00 0f 00 00 00 00 00 00 00 06 00  ................
  43. 0002a0  00 00 03 01 3f 00 00 00 00 00 00 00 00 00 00 00  ....?...........
  44. 0002b0  00 00 00 00 00 00 1a 00 00 00 2e 72 64 61 74 61  ...........rdata
  45. 0002c0  24 7a 7a 7a 00 2e 72 64 61 74 61 24 7a 7a 7a 00  $zzz..rdata$zzz.

We can see that the .a file is indeed an archive (it starts with !<arch>) and that the .o is not. We can also see that in both, the complier/archiver has inserted a signature:

Built by MinGW-W64 project

So, I thought I would try to use those from the FP program, first trying the .o file with $LINK:

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$link addtwo.o}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Hello World');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.
 
As expected, this worked (compiled, and when run produced the expected output).

Next, tried the use the .a file, with $LINKLIB:

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$linklib addtwo.a}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Hello World');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.

This produced this error:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 1, Warnings: 1
  2. usego.lpr(11,1) Warning: Library addtwo.a not found, Linking may fail !
  3. usego.lpr(11,1) Error: Can't open object file: libaddtwo.a

Adding static to linklib:

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$linklib addtwo.a static}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Hello World');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.

Also produces an error:
     
Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 1, Warnings: 1
  2. usego.lpr(13,0) Warning: Library addtwo.a static not found, Linking may fail !
  3. usego.lpr(11,1) Error: Import library not found for addtwo.a static

Hmm. Looks kind'a familiar. Let's dump the Go archive -- the big one that I suspect is the one we want as it looks like it would contain the Go runtime. I piped the Hex dump to a file (because it is so large) then did a search for the string "Built". Lo and behold:

Code: Text  [Select][+][-]
  1. 289220  9e 20 20 02 01 00 01 01 5f 63 67 6f 5f 77 61 69  .  ....._cgo_wai
  2. 289230  74 5f 72 75 6e 74 69 6d 65 5f 69 6e 69 74 5f 64  t_runtime_init_d
  3. 289240  6f 6e 65 00 63 72 6f 73 73 63 61 6c 6c 32 00 5f  one.crosscall2._
  4. 289250  63 67 6f 5f 72 65 6c 65 61 73 65 5f 63 6f 6e 74  cgo_release_cont
  5. 289260  65 78 74 00 47 43 43 3a 20 28 78 38 36 5f 36 34  ext.GCC: (x86_64
  6. 289270  2d 70 6f 73 69 78 2d 73 65 68 2d 72 65 76 30 2c  -posix-seh-rev0,
  7. 289280  20 42 75 69 6c 74 20 62 79 20 4d 69 6e 47 57 2d   Built by MinGW-
  8. 289290  57 36 34 20 70 72 6f 6a 65 63 74 29 20 38 2e 31  W64 project) 8.1
  9. 2892a0  2e 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .0..............

So, we find the same string embedded in that archive (which is what I thought, as there is no copy of the MS toolchain that I am aware of on this computer). I think we can assume that this is indeed a GCC (specifically, MinGW) .a format lib file, but there is something not right in the setup to let it be linked to the FP program.

Unfortunately, I can't get at the .o files from the Go program library (I guess it MIGHT be possible, but linking in all the dependencies would be a nightmare and realistically unworkable) so I need to stick with getting FP to link to the library.  Happy to make the .a files (both from the C and the Go programs) available -- the C one is tiny but the Go one is around 2.5MB, so even compressed it exceeds the attachment limits here (comes out to over 800K).

Really appreciate your input -- please don't think I am being stubborn, but I think that we are not looking at an MSVC archive here.

TRon

  • Hero Member
  • *****
  • Posts: 536
Re: Statically link object files from GoLang
« Reply #5 on: May 31, 2020, 04:28:58 am »
This produced this error:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 1, Warnings: 1
  2. usego.lpr(11,1) Warning: Library addtwo.a not found, Linking may fail !
  3. usego.lpr(11,1) Error: Can't open object file: libaddtwo.a
Error: Can't open object file: libaddtwo.a ?

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1044
  • Former Delphi 1-7, 10.2 User
Re: Statically link object files from GoLang
« Reply #6 on: May 31, 2020, 09:01:20 am »
Oh, working on Win10 at the moment, but will also need to get this to work on at least Mac OSX and (ideally) Linux.
Any insights or pointers would be greatly appreciated!

For macOS I recently wrote this Wiki tutorial: macOS Static Libraries which includes linking FPC and C libraries along with some useful troubleshooting information.
o Lazarus v2.1.0 r63871, FPC v3.3.1 r47164, macOS 10.14.6 (with sup update), Xcode 11.3.1
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.1 amd64 (VMware Fusion VM)
o FPC 3.0.4, FreeBSD 12.2-STABLE r365646 amd64
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 20.04 (Parallels VM)

PascalDragon

  • Hero Member
  • *****
  • Posts: 2391
  • Compiler Developer
Re: Statically link object files from GoLang
« Reply #7 on: May 31, 2020, 10:57:49 am »
Thanks for that, but I suspect that the archive is in fact created by GCC, specifically MinGW.

That's good. The differences between GCC and MSVC libraries are rather subtle, but important...

Next, tried the use the .a file, with $LINKLIB:

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$linklib addtwo.a}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Hello World');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.

This produced this error:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 1, Warnings: 1
  2. usego.lpr(11,1) Warning: Library addtwo.a not found, Linking may fail !
  3. usego.lpr(11,1) Error: Can't open object file: libaddtwo.a

What if you rename addtwo.a to libaddtwo.a? Can you pass the path to the directory the library resides in using -Fl? (I don't know right now where FPC searches by default, so add it manually)

Adding static to linklib:

Adding static won't change anything here, cause as long as the file ends with .a (on Windows at least) the compiler will assume that the library is static.

Unfortunately, I can't get at the .o files from the Go program library (I guess it MIGHT be possible, but linking in all the dependencies would be a nightmare and realistically unworkable) so I need to stick with getting FP to link to the library.  Happy to make the .a files (both from the C and the Go programs) available -- the C one is tiny but the Go one is around 2.5MB, so even compressed it exceeds the attachment limits here (comes out to over 800K).

It will probably be smaller once everything is linked. Static library usually contain everything that may be needed and only the final linking will reduce that.

Really appreciate your input -- please don't think I am being stubborn, but I think that we are not looking at an MSVC archive here.

I'm not fixated on MSVC archives, it was just something I wanted to rule out which you did. Reduce the problem space so to speak.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8876
  • FPC developer.
Re: Statically link object files from GoLang
« Reply #8 on: May 31, 2020, 02:32:29 pm »
Or try {$LIBPREFIX ’’}

to remove auto librprefixing

jimako

  • Newbie
  • Posts: 4
Re: Statically link object files from GoLang
« Reply #9 on: May 31, 2020, 03:47:20 pm »
So, I tried using {$LIBPREFIX ''} with {$linklib addtwo.a} but that did not work -- FP complained that it could not find the library.

Copied the addtwo.a library to libaddtwo.a and changed the directive to {$linklib libaddtwo.a}, and it worked:

copy addtwo.a libaddtwo.a

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$linklib libaddtwo.a}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Static linked with {$linklib libaddtwo.a}');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.

Running it:

Code: Text  [Select][+][-]
  1. >usego
  2. Static linked with {$linklib libaddtwo.a}
  3.  2 + 3 = 5

The {$LIBPREFIX ''} does not seem to have any effect.

So, onwards. We've got the C archive linking correctly, so now we turn to the Go archive. I copied the large Go archive so as to create a copy named with a prefix of "lib":

copy golib.a libgolib.a

Compiling the FP code:

Code: Pascal  [Select][+][-]
  1. program usego;
  2. uses sysutils;
  3. {$linklib libgolib.a}
  4. function AddTwo(a, b: integer): integer; cdecl; external name 'AddTwo';
  5. begin
  6.   Writeln('Static linked with {$linklib libaddtwo.a}');
  7.   Writeln(' 2 + 3 = ' + IntToStr(AddTwo(2, 3)));
  8. end.

produces a different result to before. It still fails, but comes up with a whole lot of undefined symbols:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 50
  2. usego.lpr(11,1) Error: Undefined symbol: __imp_CreateEventA
  3. usego.lpr(11,1) Error: Undefined symbol: __imp_InitializeCriticalSection
  4. usego.lpr(11,1) Error: Undefined symbol: __imp___iob_func
  5. usego.lpr(11,1) Error: Undefined symbol: __imp_Sleep
  6. usego.lpr(11,1) Error: Undefined symbol: __imp__beginthread
  7. usego.lpr(11,1) Error: Undefined symbol: __imp__errno
  8. usego.lpr(11,1) Error: Undefined symbol: __imp_EnterCriticalSection
  9. usego.lpr(11,1) Error: Undefined symbol: __imp_LeaveCriticalSection
  10. usego.lpr(11,1) Error: Undefined symbol: __imp_WaitForSingleObject
  11. usego.lpr(11,1) Error: Undefined symbol: __imp_SetEvent
  12. usego.lpr(11,1) Error: Undefined symbol: fwrite
  13. usego.lpr(11,1) Error: Undefined symbol: abort
  14. usego.lpr(11,1) Error: Undefined symbol: fprintf
  15. usego.lpr(11,1) Error: Undefined symbol: SetUnhandledExceptionFilter
  16. usego.lpr(11,1) Error: Undefined symbol: VirtualQuery
  17. usego.lpr(11,1) Error: Undefined symbol: CreateEventA
  18. usego.lpr(11,1) Error: Undefined symbol: GetProcAddress
  19. usego.lpr(11,1) Error: Undefined symbol: SetEvent
  20. usego.lpr(11,1) Error: Undefined symbol: WaitForSingleObject
  21. usego.lpr(11,1) Error: Undefined symbol: WriteFile
  22. usego.lpr(11,1) Error: Undefined symbol: WriteConsoleW
  23. usego.lpr(11,1) Error: Undefined symbol: WaitForMultipleObjects
  24. usego.lpr(11,1) Error: Undefined symbol: VirtualFree
  25. usego.lpr(11,1) Error: Undefined symbol: VirtualAlloc
  26. usego.lpr(11,1) Error: Undefined symbol: SwitchToThread
  27. usego.lpr(11,1) Error: Undefined symbol: SuspendThread
  28. usego.lpr(11,1) Error: Undefined symbol: SetWaitableTimer
  29. usego.lpr(11,1) Error: Undefined symbol: SetProcessPriorityBoost
  30. usego.lpr(11,1) Error: Undefined symbol: SetErrorMode
  31. usego.lpr(11,1) Error: Undefined symbol: SetConsoleCtrlHandler
  32. usego.lpr(11,1) Error: Undefined symbol: ResumeThread
  33. usego.lpr(11,1) Error: Undefined symbol: PostQueuedCompletionStatus
  34. usego.lpr(11,1) Error: Undefined symbol: LoadLibraryA
  35. usego.lpr(11,1) Error: Undefined symbol: LoadLibraryW
  36. usego.lpr(11,1) Error: Undefined symbol: SetThreadContext
  37. usego.lpr(11,1) Error: Undefined symbol: GetThreadContext
  38. usego.lpr(11,1) Error: Undefined symbol: GetSystemInfo
  39. usego.lpr(11,1) Error: Undefined symbol: GetSystemDirectoryA
  40. usego.lpr(11,1) Error: Undefined symbol: GetStdHandle
  41. usego.lpr(11,1) Error: Undefined symbol: GetQueuedCompletionStatus
  42. usego.lpr(11,1) Error: Undefined symbol: GetProcessAffinityMask
  43. usego.lpr(11,1) Error: Undefined symbol: GetEnvironmentStringsW
  44. usego.lpr(11,1) Error: Undefined symbol: GetConsoleMode
  45. usego.lpr(11,1) Error: Undefined symbol: FreeEnvironmentStringsW
  46. usego.lpr(11,1) Error: Undefined symbol: ExitProcess
  47. usego.lpr(11,1) Error: Undefined symbol: DuplicateHandle
  48. usego.lpr(11,1) Error: Undefined symbol: CreateThread
  49. usego.lpr(11,1) Error: Undefined symbol: CreateIoCompletionPort
  50. usego.lpr(11,1) Error: Undefined symbol: CloseHandle
  51. usego.lpr(11,1) Error: Undefined symbol: AddVectoredExceptionHandler

I'm not sure whether these are Go or Pascal symbols that are missing.

I tried a similar process (that is, rename the different golibX.a to prefix with "lib", including the .o file (which still has the "!<arch>" prefix) to a new libXXX.a file name. In all those cases, the FP compile comes up with an interesting error:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 1
  2. Fatal: No memory left

We're obviously making some progress here. We need to name the archive libXXX.a -- no problem. I think it's a safe guess that the large .a file (created with the c-archive option in the Go toolchain) is the correct library, which is what you would infer from the documentation.

Anyone have any further ideas?

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8876
  • FPC developer.
Re: Statically link object files from GoLang
« Reply #10 on: May 31, 2020, 04:09:00 pm »
My bad, I think the prefix only works when generating libraries, not when linking to it

TRon

  • Hero Member
  • *****
  • Posts: 536
Re: Statically link object files from GoLang
« Reply #11 on: May 31, 2020, 04:12:09 pm »
I'm not sure whether these are Go or Pascal symbols that are missing.
Quote
I'm not sure whether these are Go or Pascal symbols that are missing.
That depends :-)

As it seems go library requires these functions to be present in order to function (and link) correctly.

It more or less means:
go always needs to be linked together with lib(s) that provide those (now missing) functionality.

.. and that can be interpreted as either a library provided by go (tool chain, usually relying on c libs or c -startup code) or that go expects the software that is linked against (free pascal) to provide this functionality (often used as/for startup/initialization -code).

It can be a mixture of both. Go documentation should be able to provide that answer. Most (if not all) libraries are pretty much c-oriented, so require c startup-code.

edit:
Ah, i overlooked something...
...
Code: C  [Select][+][-]
  1. package main
  2.  
  3. import "C"
  4.  
  5. //export AddTwo
  6. func AddTwo(a, b int) int {
  7.         return a + b
  8. }
  9.  
  10. func main () {}
  11.  
import "C"

That example code explicitly asks for it  :) 
(whatever that means in go)
« Last Edit: May 31, 2020, 04:42:07 pm by TRon »

PascalDragon

  • Hero Member
  • *****
  • Posts: 2391
  • Compiler Developer
Re: Statically link object files from GoLang
« Reply #12 on: May 31, 2020, 05:52:36 pm »
produces a different result to before. It still fails, but comes up with a whole lot of undefined symbols:

Code: Text  [Select][+][-]
  1. Compile Project, Target: usego.exe: Exit code 1, Errors: 50
  2. usego.lpr(11,1) Error: Undefined symbol: __imp_CreateEventA
  3. usego.lpr(11,1) Error: Undefined symbol: __imp_InitializeCriticalSection
  4. usego.lpr(11,1) Error: Undefined symbol: __imp___iob_func
  5. usego.lpr(11,1) Error: Undefined symbol: __imp_Sleep
  6. usego.lpr(11,1) Error: Undefined symbol: __imp__beginthread
  7. usego.lpr(11,1) Error: Undefined symbol: __imp__errno
  8. usego.lpr(11,1) Error: Undefined symbol: __imp_EnterCriticalSection
  9. usego.lpr(11,1) Error: Undefined symbol: __imp_LeaveCriticalSection
  10. usego.lpr(11,1) Error: Undefined symbol: __imp_WaitForSingleObject
  11. usego.lpr(11,1) Error: Undefined symbol: __imp_SetEvent
  12. usego.lpr(11,1) Error: Undefined symbol: fwrite
  13. usego.lpr(11,1) Error: Undefined symbol: abort
  14. usego.lpr(11,1) Error: Undefined symbol: fprintf
  15. usego.lpr(11,1) Error: Undefined symbol: SetUnhandledExceptionFilter
  16. usego.lpr(11,1) Error: Undefined symbol: VirtualQuery
  17. usego.lpr(11,1) Error: Undefined symbol: CreateEventA
  18. usego.lpr(11,1) Error: Undefined symbol: GetProcAddress
  19. usego.lpr(11,1) Error: Undefined symbol: SetEvent
  20. usego.lpr(11,1) Error: Undefined symbol: WaitForSingleObject
  21. usego.lpr(11,1) Error: Undefined symbol: WriteFile
  22. usego.lpr(11,1) Error: Undefined symbol: WriteConsoleW
  23. usego.lpr(11,1) Error: Undefined symbol: WaitForMultipleObjects
  24. usego.lpr(11,1) Error: Undefined symbol: VirtualFree
  25. usego.lpr(11,1) Error: Undefined symbol: VirtualAlloc
  26. usego.lpr(11,1) Error: Undefined symbol: SwitchToThread
  27. usego.lpr(11,1) Error: Undefined symbol: SuspendThread
  28. usego.lpr(11,1) Error: Undefined symbol: SetWaitableTimer
  29. usego.lpr(11,1) Error: Undefined symbol: SetProcessPriorityBoost
  30. usego.lpr(11,1) Error: Undefined symbol: SetErrorMode
  31. usego.lpr(11,1) Error: Undefined symbol: SetConsoleCtrlHandler
  32. usego.lpr(11,1) Error: Undefined symbol: ResumeThread
  33. usego.lpr(11,1) Error: Undefined symbol: PostQueuedCompletionStatus
  34. usego.lpr(11,1) Error: Undefined symbol: LoadLibraryA
  35. usego.lpr(11,1) Error: Undefined symbol: LoadLibraryW
  36. usego.lpr(11,1) Error: Undefined symbol: SetThreadContext
  37. usego.lpr(11,1) Error: Undefined symbol: GetThreadContext
  38. usego.lpr(11,1) Error: Undefined symbol: GetSystemInfo
  39. usego.lpr(11,1) Error: Undefined symbol: GetSystemDirectoryA
  40. usego.lpr(11,1) Error: Undefined symbol: GetStdHandle
  41. usego.lpr(11,1) Error: Undefined symbol: GetQueuedCompletionStatus
  42. usego.lpr(11,1) Error: Undefined symbol: GetProcessAffinityMask
  43. usego.lpr(11,1) Error: Undefined symbol: GetEnvironmentStringsW
  44. usego.lpr(11,1) Error: Undefined symbol: GetConsoleMode
  45. usego.lpr(11,1) Error: Undefined symbol: FreeEnvironmentStringsW
  46. usego.lpr(11,1) Error: Undefined symbol: ExitProcess
  47. usego.lpr(11,1) Error: Undefined symbol: DuplicateHandle
  48. usego.lpr(11,1) Error: Undefined symbol: CreateThread
  49. usego.lpr(11,1) Error: Undefined symbol: CreateIoCompletionPort
  50. usego.lpr(11,1) Error: Undefined symbol: CloseHandle
  51. usego.lpr(11,1) Error: Undefined symbol: AddVectoredExceptionHandler

I'm not sure whether these are Go or Pascal symbols that are missing.

Those are Windows and MSVCRT symbols. This is going to get ugly... You'll have to look amidst the MinGW libraries (maybe distributed with your Go installation?) for e.g. kernel32.a (or libkernel32.a?) and probably something like msvcrt.a and link against those as well.

 

TinyPortal © 2005-2018