Recent

Author Topic: Developing Python Modules  (Read 8084 times)

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Developing Python Modules
« on: October 16, 2017, 06:45:58 pm »
Hi, I've been playing around with Python modules and followed the example from the page http://wiki.freepascal.org/Developing_Python_Modules_with_Pascal
which worked and explained everything really well for Python 2.7
Problem I have is the corporate build only has Python 3.6.3 installed and to me, a novice, Python does not seem to be very backward compatible or maybe I have just been spoiled over the years with Pascal  :).
Anyway I found the following link https://macpgmr.github.io/MacXPlatform/PascalDynLibs.html again well written, the DLL/PYD complied fine with no issues but failed to work with Python 3.6.3
just wondering has anybody got any tips on creating modules for 3.6.3?

At the moment they are forced to use my external DLL, now they don't find this as convenient as just using the 'import' command.
They are also Python obsessed  %)



Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: Developing Python Modules
« Reply #1 on: October 16, 2017, 06:49:00 pm »
Anyway I found the following link https://macpgmr.github.io/MacXPlatform/PascalDynLibs.html again well written, the DLL/PYD complied fine with no issues but failed to work with Python 3.6.3
just wondering has anybody got any tips on creating modules for 3.6.3?

That should work fine with 3.6. Normally you need to adjust the Python library file name in PyAPI.pas.

Otherwise, you'll need to provide more information.

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #2 on: October 16, 2017, 07:09:03 pm »
Phil

I modified the PyAPI

Code: Pascal  [Select][+][-]
  1. const
  2. {$IFDEF MSWINDOWS}
  3.  {$IFDEF USE_PYTHON3}
  4.   //PythonLib = 'python35.dll';
  5.   PythonLib = 'python36.dll';  //changed to match version of Python
  6.  {$ELSE}
  7.   PythonLib = 'python27.dll';
  8.  {$ENDIF}
  9. {$ENDIF}


no other changes where made. I'm using Win10 X64 Python 3.6.3 X64 and Lazarus 1.8 RC4

the error python returns is "ImportError: DLL load failed: The specified procedure could not be found."

I'm at a loss

Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: Developing Python Modules
« Reply #3 on: October 16, 2017, 07:15:04 pm »

no other changes where made. I'm using Win10 X64 Python 3.6.3 X64 and Lazarus 1.8 RC4

the error python returns is "ImportError: DLL load failed: The specified procedure could not be found."


Make sure you defined USE_PYTHON3 when you built your extension module.

Note that the extension module Pascal code you write is different between 2.7 and 3.x, but you probably already noticed that. If not, look at the ndfdmod.pas example. See if the example works.

Once you get it working you might also test with IronPython. If anyone needs to work with .NET assemblies, that's an easy way to do it via Python.

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #4 on: October 16, 2017, 10:06:13 pm »
Phil

I've been using -dUSE_PYTHON, I'm sure it's something stupid I'm doing. I shall take a closer look at the ndfdmod example.

soerensen3

  • Full Member
  • ***
  • Posts: 213
Re: Developing Python Modules
« Reply #5 on: October 17, 2017, 12:02:01 am »
Quote
the error python returns is "ImportError: DLL load failed: The specified procedure could not be found."
I don't think that the DLL was not found. That's sounds more like it's looking for a procedure that isn't there.
I haven't used PyAPI but it probably loads each procedure and function of the DLL dynamically (If the error is generated at runtime). I would look for the loading procedure and step to each line with F7 and see where it complains. Then you know at least which procedure causes the problem and you can compare that with the procedure in the DLL header of Python 3.
Lazarus 1.9 with FPC 3.0.4
Target: Manjaro Linux 64 Bit (4.9.68-1-MANJARO)

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Developing Python Modules
« Reply #6 on: October 17, 2017, 05:57:52 am »
Phil

I've been using -dUSE_PYTHON, I'm sure it's something stupid I'm doing. I shall take a closer look at the ndfdmod example.

USE_PYTHON3

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #7 on: October 17, 2017, 04:31:55 pm »
@engkin I've been using 'USE_PYTHON3' was a typo.
@soerensen3 I shall give it a try.


I just at a loss with this one, I've got to be missing/doing something stupid!

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #8 on: October 18, 2017, 04:26:00 pm »
@soerensen3 I tested both DLL/PYD files with the follow simple test code
Code: Pascal  [Select][+][-]
  1. If not FileExists(mydll) then exit;
  2.      DLL:=dynlibs.NilHandle;
  3.      DLL:=LoadLibrary(MYDLL);
  4.      if DLL=dynlibs.NilHandle then Exit;  //DLL was not loaded successfully
  5.      ShowMessage('DLL Loaded');
  6.  

the DLL created for python 2.7 loads without issue the one for 3.6 returns a NilHandle
so looks like a DLL structure issue.

problem is I'm new to the python side of things and don't yet fully understand the structure 



Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: Developing Python Modules
« Reply #9 on: October 18, 2017, 04:33:16 pm »
the DLL created for python 2.7 loads without issue the one for 3.6 returns a NilHandle
so looks like a DLL structure issue.

I can't follow what you're trying to do.

The Python extension module is itself a dynamic library, with a .pyd extension on Windows. You create one for 2.7 and a different one for 3.6. These extension modules can themselves load the same dynamic library, as in the ndfd module example.

As I suggested earlier, make sure the examples work before attempting to solve your own problems.

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #10 on: October 18, 2017, 04:42:17 pm »
@Phil
the test was to try and narrow down where the issue lay. I've been unable to get any examples to work under 3.6

Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: Developing Python Modules
« Reply #11 on: October 18, 2017, 04:43:42 pm »
@Phil
the test was to try and narrow down where the issue lay. I've been unable to get any examples to work under 3.6

The ndfd example works with 3.6.

ahiggins

  • Jr. Member
  • **
  • Posts: 92
Re: Developing Python Modules
« Reply #12 on: October 18, 2017, 11:05:53 pm »
@Phil I have still been unable to get any examples to work, now this is all new to me, so I'm convinced I'm missing something obvious, I've listed the steps I took and the software/tool I'm using.

Windows 10 64bit
Free Pascal Compiler version 3.0.2 [2017/02/27] for x86_64
Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Source files extracted from p4g.zip

15/03/2015  22:18             1,826 ndfd_types.pas
14/02/2016  11:17             5,711 ndfdmod.pas
24/04/2016  16:35             4,531 PyAPI.pas

changed PyAPI to use python36.dll (3,610,776)

const
{$IFDEF MSWINDOWS}
 {$IFDEF USE_PYTHON3}
  PythonLib = 'python36.dll'; //'python35.dll';
 {$ELSE}
  PythonLib = 'python27.dll';
 {$ENDIF}
{$ENDIF}

compiled with
fpc -dUSE_PYTHON3 -ondfdmod64_3.pyd ndfdmod.pas

D:\Code\Python\library\x64\ndfd>C:\lazarus\fpc\3.0.2\bin\x86_64-win64\fpc -dUSE_PYTHON3 -ondfdmod64_3.pyd ndfdmod.pas
Free Pascal Compiler version 3.0.2 [2017/02/27] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others
Target OS: Win64 for x64
Compiling ndfdmod.pas
Compiling ndfd_types.pas
Compiling NdfdLib.pas
Compiling PyAPI.pas
Linking ndfdmod64_3.pyd
628 lines compiled, 0.9 sec, 73024 bytes code, 4916 bytes data

D:\Code\Python\library\x64\ndfd>

tried to import with python

Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ndfdmod64_3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed: The specified module could not be found.
>>>

Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: Developing Python Modules
« Reply #13 on: October 18, 2017, 11:23:36 pm »
tried to import with python

Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ndfdmod64_3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed: The specified module could not be found.
>>>

Did you also compile the DLL that the extension module loads? (ndfd64.dll) The idea here is a single library that can be used with a variety of languages. It's also possible to put the library code in the extension module, but that would kind of defeat the purpose of the exercise.

Also, run the example app:

python3 testndfd.py


ahiggins

  • Jr. Member
  • **
  • Posts: 92
[solved] Developing Python Modules
« Reply #14 on: October 19, 2017, 02:16:16 am »
@Phil thank you so much for your patience, looks like I could not see the wood for the trees, I can tell you this has been driving me crazy  %)

Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ndfdmod64_3
>>>

all I can say is I'd be lost without this forum for sure.






« Last Edit: October 20, 2017, 12:34:06 pm by ahiggins »

 

TinyPortal © 2005-2018