Lazarus

Miscellaneous => Other => Topic started by: MarkMLl on January 12, 2021, 03:25:56 pm

Title: RANT: Python (it's not funny)
Post by: MarkMLl on January 12, 2021, 03:25:56 pm
I find myself dithering over debugging some software written in Python, which hides behind Electron as the user interface. In part this is due to unfamiliarity with the available tools, and in part due to a nagging feeling that implementing a plugin- no matter how clever the algorithms- by embedding a web browser (and repeating this for /every/ plugin for a particular program) is a trifle profligate.

I find that I'm not the only person to have misgivings: https://drewdevault.com/2016/11/24/Electron-considered-harmful.html but Liam puts it fairly succinctly:

Quote
> You remind me of something I looked at a while ago.
>
> I noted some things that might be quite doable in the browser, or as a
> browser add-on. I had been looking for a reason to play with Python. I
> don't really want to learn Javascript. So I discovered someone had
> created a way to write Firefox plugins in Python.
>
> That sounds good, I thought. Just the ticket.
>
> I had a look.
>
> Yes, it was a tiny Javascript plugin that embedded an entire Python
> interpreter and then another, bigger Javascript program that translated
> Python calls to Firefox calls, and it would have to be embedded into
> your addon. So your 2 kB Python addin would need a 300MB wrapper to
> work, and would thus end up considerably larger than the browser itself.
>
> I am not sure now but they might have ported Python to Javascript in
> order to do it.
>
> "Perhaps not, eh?" I thought as I metaphorically backed away.

No matter how much we all loathe Python, it seems to me that using Lazarus's technology to provide an efficient wrapper UI instead of Javascript and HTML5 would be generally useful.

The same, potentially, could be said about a Lazarus wrapper about C++ code. And refusing to countenance this due to a difference in stackframe use etc. is a pretty lame excuse IMO and will contribute to Pascal's continued slide towards irrelevance.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 12, 2021, 03:53:03 pm
Well you can create a local server with Lazarus, create endpoints, and create the GUI with web technologies.

Start the .exe and that opens in your default browser. Yo can work with files and databases. All in the user PC.
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 12, 2021, 04:23:45 pm
Who in God's name needs a browser in order to display a progress bar, half a dozen buttons and some non-interactive vector graphics?

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 12, 2021, 04:28:21 pm
Who in God's name needs a browser in order to display a progress bar, half a dozen buttons and some non-interactive vector graphics?

MarkMLl

Well, no, not that simple UI. I mean an UI that's good =)
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 12, 2021, 07:43:25 pm
But in this context "good" UIs aren't needed... I'm talking about an Inkscape plugin here that really doesn't have a UI to speak of.

Oh, and I'd settle for something that takes less than a minute to start up. Because having to wait 70 secs or so for. every. operation. "get's old fast" as they say.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 12, 2021, 10:16:20 pm
I can see me in the future, why do you need 'X' stuff, if you can use Electron or a web app? Web apps are not old...  :D That 'X' stuff at least takes a GB of file download, just for uxing 'XZ' language.

Yep, we're all getting older, all will be replaced with something else.

Why do you need Inkscape? If there are web apps that does the same vector stuff online? Why downloading Inkscape...  :D

(Justk joking, I use Inkscape, but I've seen web apps that does vector too).
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 12, 2021, 10:20:46 pm
Because of the plugin (Inkstitch). Which I'm trying to debug, to find out why it takes 70 secs to run (short answer: CJK founts. Longer answer: what can be done about it?).

I've just had a thoroughly frustrating early-evening trying to get enough Python, Electron, NPM and GOK what else installed that I have some chance of debugging it. There's got to be a better way...

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 12, 2021, 10:37:42 pm
Do you have an SSD drive? Or a hard disk?

Usually Electron and all the like takes time to load from disk, if you use an SSD you have better startup times.

Edit: I can say that is true, because LazPaint uses python for scripting, when I had an HDD it was slower, now with an SSD runs really fast. Remember that scripting is usually running a command line process...
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 13, 2021, 09:59:00 am
Disk. Most of that particular problem is fixed by removing the CJK founts. All of the Python stuff is bundled up into some sort of binary with Electron, I'm currently trying to sort out the details.

Granted that the plugin author has pulled in a vast amount of stuff from other open source projects- I can even see some FORTRAN in there- but there has to be a better way.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 13, 2021, 04:47:42 pm
Too much megabytes in fonts being loaded into memory?

And not ever used?

Well yes that is a problem indeed.
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 13, 2021, 04:56:59 pm
And the rest. A vague recollection of the messages suggest that the two CJK packages were taking in excess of 300Mb on disc, and even somebody who like me appreciates pretty typefaces has to wonder why that lot's being installed for a European- sorry, BRITISH (now that we've Brexited) locale. Then there's the convenient-but-inefficient plugin mechanism which runs a program from scratch for every operation, and the inefficient-and-inconvenient architecture based on Python+Electron.

I can't comment on how difficult it would be to integrate a modified FPC+LCL runtime with Inkscape, since there appears to be no recent documentation of the interfaces.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: lainz on January 14, 2021, 02:00:37 am
Yeah so the problem was not the tools, but how these are being used.

Edit: you can bring some links so we can look at? I use inkscape for fun and sometimes for work, mostly icons.

What kind of plugin you want to make? The same plugin but in fpc?
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 14, 2021, 09:48:38 am
I'm inclined towards the viewpoint that this amount of overhead is a tools problem, not just one of inappropriate use. And I'm not trying to rewrite it, merely fix a couple of the problems. https://inkstitch.org/

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: PascalDragon on January 14, 2021, 09:53:57 am
I'm inclined towards the viewpoint that this amount of overhead is a tools problem, not just one of inappropriate use. And I'm not trying to rewrite it, merely fix a couple of the problems. https://inkstitch.org/

I really hope that this year I'll be able to build my "embroidery addon" for my sewing machine, then I'll probably play around with inkstitch as well ;)
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on January 14, 2021, 10:12:31 am
I really hope that this year I'll be able to build my "embroidery addon" for my sewing machine, then I'll probably play around with inkstitch as well ;)

https://github.com/MarkMLl/pesdump might have some useful stuff. I've since attempted to modify it to patch files but it turns out that the Brother machine I have access to is /very/ picky about having everything consistent... I can trivially crash its firmware resulting in a (watchdog?) reset.

A great deal of work's gone into Inkstitch... lots of people (including Torvalds) appear to have dipped their toe in the water but I think it's the most mature project by far. And compared with the cost of Brother's own software it's worth supporting (you can get "informal copies" cheaply, but they're NBG without one of Brother's dongles).

I'm still trying to work out how things hang together, but with the number of things that take plugins these days I'd have thought that there would be a niche for a good development too which allowed the appropriate DLL/so/binary to be customised.

(Slightly later) Extension invocation is described at https://inkscape.org/develop/extensions/ although I've yet to work out how an extension modifies the live image.

Considering some stuff I've done in the past where a communications program could invoke a nested shell, hence a copy of Lazarus, hence debug a nested copy of itself, I find myself wondering whether a facility whereby Lazarus could be invoked in lieu of a program to be debugged and would transparently pass on all (shell) variables, (command line) parameters and open handles (in particular stdin and stdout) would be useful.

That would allow somebody working on something like an Inkscape extension to switch between running his binary and running the full development environment with a one-line change to a configuration file (Inkscape .inx file in the current case). https://wiki.inkscape.org/wiki/index.php?title=Script_extensions

MarkMLl


Title: Re: RANT: Python (it's not funny)
Post by: vfclists on February 02, 2021, 05:24:41 pm
One of the problem Lazarus with its LCL GUI is that they don't have much support from scripting or embedded languages.

The only one I can think of is vclua and there is also Py4Delphi/Py4Lazarus.

Another is gabrielrcouto/php-gui] Extensionless PHP Graphic User Interface library (Github) (https://github.com/gabrielrcouto/php-gui), which interfaces with the LCL over XM-RPC, or is that JSON.
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on February 02, 2021, 07:16:30 pm
I've been using something based on Meta-2 for this sort of thing for a /long/ time... I mean "long" as in "predating Delphi". Alternatively there's Lua, plus a couple of Python implementations (at least three if you count the one I've been hacking).

I've got Python2 running with both static and dynamic linkage, although I've only transcribed enough of the API to do /exactly/ what I need so far. Organising it so that Python can call back into the Lazarus runtimes looks as though it will be pretty easy.

The .inc file I've edited to contain the API declarations and comments describing each function comes to about 300K, Lazarus picks the descriptions up properly as hover-over hints. Much as I loathe Python's syntax, its documentation is pretty good with notes of when functions were introduced and had major changes.

Code: Pascal  [Select][+][-]
  1. program PythonDemo;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. { define DYNAMIC }
  6. {$define PYTHON2 }
  7.  
  8. (* Assume that there is only superficial similarity between the Python v2 and   *)
  9. (* v3 script syntax and semantics. This demo is, more than anything else, the   *)
  10. (* "point of contact" between the various implementation possibilities at the   *)
  11. (* time of development.                                                         *)
  12.  
  13. {$ifdef PYTHON2 }
  14. {$unitpath ./python2 }
  15. {$note ===== Intending to link to Python v2 ===== }
  16. {$else          }
  17. {$unitpath ./python3 }
  18. {$note ===== Intending to link to Python v3 ===== }
  19. {$endif PYTHON2 }
  20.  
  21. (* WARNING: since the unit path is being set up during compilation, Lazarus's   *)
  22. (* build facility might not successfully retire unwanted files. If in doubt do  *)
  23. (* a "Clean directory".                                                         *)
  24.  
  25. uses
  26.   {$IFDEF UNIX}
  27.   cthreads,
  28.   {$ENDIF}
  29.   Classes,
  30. {$ifdef DYNAMIC }
  31.   python_dynamic;                       (* "Python" is a shared-library object  *)
  32. {$note ===== Intending to link dynamically ===== }
  33. {$else          }
  34.   python;                               (* "Python" is a unit file              *)
  35. {$note ===== Intending to link statically ===== }
  36. {$endif DYNAMIC }
  37.  
  38. var
  39.   localArgv: array of PChar;
  40.   debugging: boolean= false;
  41.  
  42.  
  43. (* Generate an array of pointers from the flat block of null-terminated strings,
  44.   excluding the --DEBUG-- option which is handled specially.
  45.  
  46.   This is temporary, and should be moved to the Python unit/object even if it's
  47.   not handled automatically.
  48. *)
  49. procedure initArgv;
  50.  
  51. var
  52.   i, j: integer;
  53.  
  54. begin
  55.   SetLength(localArgv, 1);
  56.   localArgv[0] := System.Argv^;         (* Program name                         *)
  57.   j := 1;
  58.   for i := 1 to ParamCount() do
  59.     if ParamStr(i) = '--DEBUG--' then
  60.       debugging := true
  61.     else begin
  62.       SetLength(localArgv, j + 1);
  63.       localArgv[j] := localArgv[i - 1] + StrLen(localArgv[i - 1]) + 1;
  64.       j += 1
  65.     end
  66. end { initArgv } ;
  67.  
  68.  
  69. begin
  70.   InitializeLibrary;                    (* Ensures Python.ModuleInMemory valid  *)
  71.   if Python.IsDynamic then
  72.     Write('Python is dynamically linked, ')
  73.   else
  74.     Write('Python is statically linked, ');
  75.   if Python.ModuleInMemory then
  76.     Write('and has ')
  77.   else
  78.     Write('but has not ');
  79.   WriteLn('been successfully loaded.');
  80.   initArgv;                             (* localArgv and debugging flag         *)
  81.   if debugging then
  82.     WriteLn('Debugging is enabled.');   (* Special --DEBUG-- option             *)
  83.  
  84. (* Report Python version etc., this calls into libpython but does not rely on   *)
  85. (* script execution. Note that while Py_SetProgramName() has to be called       *)
  86. (* before Py_Initialize in order to set up internal paths etc., PySys_SetArgv() *)
  87. (* has to be called afterwards.                                                 *)
  88.  
  89.   WriteLn(StrPas(Python.Py_GetVersion()));
  90.   Python.Py_SetProgramName(PChar(ParamStr(0))); (* Before _Initialize           *)
  91.  
  92. (* Initialise the embedded Python implementation and run the traditional        *)
  93. (* trivial script. For the moment at least, expect output to go to the console. *)
  94.  
  95.   Python.Py_Initialize;
  96.   try
  97. //    Py_InitModule("emb", EmbMethods);   // emb would be part of this Pascal program
  98.     Python.PySys_SetArgv(Length(localArgv), @localArgv[0]);
  99.     Python.PyRun_SimpleString(
  100.         'import sys' + LineEnding +     (* Just to check EOL convention         *)
  101.         'print(sys.argv[0:])' + LineEnding +
  102.         'print("Hello, World!")' + LineEnding
  103.     );
  104.   finally
  105.     Python.Py_Finalize
  106.   end
  107. end.
  108.  

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: vfclists on February 03, 2021, 12:59:52 am
I've been using something based on Meta-2 for this sort of thing for a /long/ time... I mean "long" as in "predating Delphi". Alternatively there's Lua, plus a couple of Python implementations (at least three if you count the one I've been hacking).

I've got Python2 running with both static and dynamic linkage, although I've only transcribed enough of the API to do /exactly/ what I need so far. Organising it so that Python can call back into the Lazarus runtimes looks as though it will be pretty easy.

The .inc file I've edited to contain the API declarations and comments describing each function comes to about 300K, Lazarus picks the descriptions up properly as hover-over hints. Much as I loathe Python's syntax, its documentation is pretty good with notes of when functions were introduced and had major changes.

Code: Pascal  [Select][+][-]
  1. program PythonDemo;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. <snip>
  6.   finally
  7.     Python.Py_Finalize
  8.   end
  9. end.
  10.  

MarkMLl

So the code above enables Python 2 execution from within Lazarus, but how about calling Lazarus LCL functions from Python?

Does that require Lazarus to export its functionality as DLLs, or a special build of Python that is linked with some Lazarus obj or DLL?
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on February 03, 2021, 08:43:56 am
So the code above enables Python 2 execution from within Lazarus, but how about calling Lazarus LCL functions from Python?

Does that require Lazarus to export its functionality as DLLs, or a special build of Python that is linked with some Lazarus obj or DLL?

I'm using a standard build of Python, linked either statically or dynamically.

Code: [Select]
//    Py_InitModule("emb", EmbMethods);   // emb would be part of this Pascal program

That's the magic line, hence https://docs.python.org/3/extending/embedding.html section 1.4.

In practical terms, anything that the LCL etc. wanted to export would need to be wrapped in a function of the appropriate form, and the Python runtimes advised of it. Once that was done a Python program could call the LCL or whatever via the emb module.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: PascalDragon on February 03, 2021, 09:59:04 am
So the code above enables Python 2 execution from within Lazarus, but how about calling Lazarus LCL functions from Python?

Does that require Lazarus to export its functionality as DLLs, or a special build of Python that is linked with some Lazarus obj or DLL?

Python4Delphi (https://github.com/pyscripter/python4delphi) provides various wrapper classes that provide access to the VCL for Python scripts. I don't know if these units are yet tested with FPC/Lazarus, but in theory they should work as well. And I think there is a general wrapper that uses published properties thus any published property can be used as well as long as you tell Python4Delphi about that type.

In addition to that the Extended RTTI (which FPC does not yet provide) could be used in theory to automatically provide access to types and their fields/methods/properties though of course a corresponding wrapper code needs to be written first (one needs to be careful with smartlinking then however).
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on February 03, 2021, 10:07:19 am
Python4Delphi (https://github.com/pyscripter/python4delphi) provides various wrapper classes that provide access to the VCL for Python scripts. I don't know if these units are yet tested with FPC/Lazarus, but in theory they should work as well. And I think there is a general wrapper that uses published properties thus any published property can be used as well as long as you tell Python4Delphi about that type.

Yes, there's a variant that works with FPC but it's walked away from Python2 which is what I need for the moment (as per OP, I've got some Python which doesn't work properly and I want to find out why without getting embroiled in converting the whole damn thing to v3). So I'm taking this opportunity to repackage.

I'd also observe that there is a tendency for Delphi... no, I /am/ going to say "Object Pascal" here... programmers to be excessively clever and wrap a whole lot of stuff in classes etc. simply because doing so suggests elegance. That can be extremely counter-productive, since the result bears absolutely no resemblance to the examples published and used as foundation works by the non-Pascal community. So what I put in the example I posted yesterday was very much a line-by-line transcript of what a C/C++ programmer approaching embedded Python would find, and I intend to keep it that way.

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: PascalDragon on February 03, 2021, 01:24:42 pm
Python4Delphi (https://github.com/pyscripter/python4delphi) provides various wrapper classes that provide access to the VCL for Python scripts. I don't know if these units are yet tested with FPC/Lazarus, but in theory they should work as well. And I think there is a general wrapper that uses published properties thus any published property can be used as well as long as you tell Python4Delphi about that type.

Yes, there's a variant that works with FPC but it's walked away from Python2 which is what I need for the moment (as per OP, I've got some Python which doesn't work properly and I want to find out why without getting embroiled in converting the whole damn thing to v3). So I'm taking this opportunity to repackage.

My answer was mainly to vfclists as I'm already aware that you also use Python 2 (though one could fork Python4Delphi before this (https://github.com/pyscripter/python4delphi/commit/0080c99ca1153482f5810b16d34d66d8d0439128) commit from middle of October).

I'd also observe that there is a tendency for Delphi... no, I /am/ going to say "Object Pascal" here... programmers to be excessively clever and wrap a whole lot of stuff in classes etc. simply because doing so suggests elegance. That can be extremely counter-productive, since the result bears absolutely no resemblance to the examples published and used as foundation works by the non-Pascal community. So what I put in the example I posted yesterday was very much a line-by-line transcript of what a C/C++ programmer approaching embedded Python would find, and I intend to keep it that way.

I don't agree here. A point that Delphi makes is convenient access for various functionality. Think about COM which is more a hazzle in (plain) C++, but which Delphi did in a rather strong way that even the C++ devs were envious back then. Similar for Python4Delphi: I most cases I won't even need the C tutorials. (And when I had first looked at Python4Delphi around 15 years ago I had no clue about C, so that wouldn't have helped me anyway)
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on February 03, 2021, 01:35:39 pm
Yes, I'm working from the October version.

I'd also observe that there is a tendency for Delphi... no, I /am/ going to say "Object Pascal" here... programmers to be excessively clever and wrap a whole lot of stuff in classes etc. simply because doing so suggests elegance. That can be extremely counter-productive, since the result bears absolutely no resemblance to the examples published and used as foundation works by the non-Pascal community. So what I put in the example I posted yesterday was very much a line-by-line transcript of what a C/C++ programmer approaching embedded Python would find, and I intend to keep it that way.

I don't agree here. A point that Delphi makes is convenient access for various functionality. Think about COM which is more a hazzle in (plain) C++, but which Delphi did in a rather strong way that even the C++ devs were envious back then. Similar for Python4Delphi: I most cases I won't even need the C tutorials. (And when I had first looked at Python4Delphi around 15 years ago I had no clue about C, so that wouldn't have helped me anyway)

In that case we'll have to agree to disagree. Oh, BTW: are you of the opinion that documentation should be written in Loglan because of its elegance and precision? :-)

MarkMLl
Title: Re: RANT: Python (it's not funny)
Post by: PascalDragon on February 04, 2021, 09:15:12 am
I'd also observe that there is a tendency for Delphi... no, I /am/ going to say "Object Pascal" here... programmers to be excessively clever and wrap a whole lot of stuff in classes etc. simply because doing so suggests elegance. That can be extremely counter-productive, since the result bears absolutely no resemblance to the examples published and used as foundation works by the non-Pascal community. So what I put in the example I posted yesterday was very much a line-by-line transcript of what a C/C++ programmer approaching embedded Python would find, and I intend to keep it that way.

I don't agree here. A point that Delphi makes is convenient access for various functionality. Think about COM which is more a hazzle in (plain) C++, but which Delphi did in a rather strong way that even the C++ devs were envious back then. Similar for Python4Delphi: I most cases I won't even need the C tutorials. (And when I had first looked at Python4Delphi around 15 years ago I had no clue about C, so that wouldn't have helped me anyway)

In that case we'll have to agree to disagree. Oh, BTW: are you of the opinion that documentation should be written in Loglan because of its elegance and precision? :-)

I have no idea what Loglan is. And like most everything documentation should be written in whatever is the most useful and convenient.
Title: Re: RANT: Python (it's not funny)
Post by: dsiders on February 04, 2021, 07:33:12 pm
I have no idea what Loglan is. And like most everything documentation should be written in whatever is the most useful and convenient.

Think Esperanto for the criminally insane.
Title: Re: RANT: Python (it's not funny)
Post by: MarkMLl on February 17, 2021, 09:54:46 am
I've got Python2 running with both static and dynamic linkage, although I've only transcribed enough of the API to do /exactly/ what I need so far. Organising it so that Python can call back into the Lazarus runtimes looks as though it will be pretty easy.

The .inc file I've edited to contain the API declarations and comments describing each function comes to about 300K, Lazarus picks the descriptions up properly as hover-over hints. Much as I loathe Python's syntax, its documentation is pretty good with notes of when functions were introduced and had major changes.

I'm now at the point where I've got a single .inc file describing the bulk of the Python v2.7 API plus a preprocessor which generates unit files from it: one for a static interface to an external library and another for a dynamic on-demand interface. No manual tweaks are necessary, any locally-defined functions (usually standing in for a C/C++ macro) go into another .inc file which is pulled in automatically if it exists.

A program calling this interface sees either a unit (python.pas) or an object in a unit (TPython in python_dynamic.pas) and is unchanged irrespective of the type of linkage.

I intend to extend this to replace some extremely cruddy UI stuff (hence the tenor of the start of this thread) and support Python debugging.

I've got a problem relating to functions marked as varargs which I'll raise in a separate thread. I'm not sure whether I need them but my eventual aim is to support the entire documented Python API, Python4Delphi is not at all complete.

MarkMLl
TinyPortal © 2005-2018