Recent

Author Topic: Microsoft Open Neural Network Exchange ( ONNXRuntime ) for Lazarus / Delphi  (Read 810 times)

hshatti

  • New member
  • *
  • Posts: 7
  • Don't be evil isn't enough, Be virtuous!
Greetings everyone, :)
I'm happy to share with you my Freepascal / Delphi implementation of Microsoft ONNXRuntime
Open Neural Network Exchange AI and Machine Learning library

Install
    Please visit
      https://github.com/hshatti/TONNXRuntime

    or
Code: Bash  [Select][+][-]
  1. git clone https://github.com/hshatti/TONNXRuntime.git

Usage

Code: Pascal  [Select][+][-]
  1. unit formUnit;
  2. {$h+}
  3.  
  4. interface
  5. uses onnxruntime, Classes etc... ;

Load a Model
Code: Pascal  [Select][+][-]
  1. var
  2.   session : TORTSession;
  3. begin
  4.   session := TORTSession.Create('./mymodel/filname.onnx');
  5. {
  6. *****************************************************************
  7.     Check your model requirements for input/output
  8.     names and value dimensions before preparing the inputs.
  9.     to explore the model before preparing use :
  10.       session.GetInputCount and session.GetOutputCount
  11.       session.GetInputName and session.GetOutputName
  12.       session.GetInputTypeInfo and session.GetOutputTypeInfo
  13.  ****************************************************************
  14. }

Prepare an input tensor with the desired shape using
Code: [Select]
TORTTensor<type> and your inputs using the library's name/value Dictionary
Code: [Select]
TORTNameValueList
Code: Pascal  [Select][+][-]
  1. var
  2.   x,y:integer;
  3.   imageData : array of array of single;
  4.   inTensor : TORTTensor<single> ;
  5.   inputs : TORTNameValueList  ;
  6. begin
  7.   // assuming the model input name is 'image' and the tensor shape is [width, height]
  8.   inTensor := TORTTensor<single>.Create([width, height{, depth ,etc...}]);
  9.   for y:=0 to inTensor.shape[1]-1 do
  10.       for x:=0 to inTensor.shape[0]-1 do
  11.           inTensor.index2[x, y]:= imageData[x, y];  // your float values
  12.   inputs['image'] := inTensor;

Inference
Code: Pascal  [Select][+][-]
  1. var
  2.     myDetection : array of single;
  3.     i:integer;
  4.     outputs : TORTNameValueList;
  5.     outTensor : TORTTensor<single>
  6.   bagin
  7.       outputs   := session.run(inputs);
  8.       outTensor := outputs['result']
  9.      
  10.       for i:=0 to outTensor.shape[0]-1 do
  11.         myDetection[i] := outTensor.index1[i]

Quick notes
If you are on Windows mostly you are good to go, ONNXRuntime libraries come shipped with most of modern Windows releases.
check :
Code: Text  [Select][+][-]
  1. %WINDIR%\SysWOW64\onnxruntime.dll
or
Code: Text  [Select][+][-]
  1. %WINDIR%\System32\onnxruntime.dll

  • MacOS / Linux users please download the dynamic link libraries https://github.com/microsoft/onnxruntime/releases/tag/v1.13.1 or compile static libraries from ONNXRuntime Github repository :  https://github.com/microsoft/onnxruntime

  • I have included a simple inference example for Object Detection using FasterRCNN-10 model, this should be straight forward to understand, if you need further help please let me know, more good Inference and Training examples to be uploaded soon.

  • Check Microsoft model zoo repository if you like to play around with AI and Neural Networks, Now! most of Python, C++ and C# examples can be easily ported to Lazarus : https://github.com/onnx/models

  • Further documentation on API and usage of GPUs can be found here : https://onnxruntime.ai/docs/api/

  • Suggestions, Criticism and Contributions are most welcome.

  • Please use it for peaceful purposes, quote my work if you find it useful.

« Last Edit: December 27, 2022, 01:18:13 pm by hshatti »
Don't be evil isn't enough, Be virtuous!
------------
Research-O-Holic .. Highly Skilled Delphi/Lazarus/FPC developer Oh C/C++ too - Solutions with Micro-Controllers (PIC+AVR +STM .. etc )- I.A - anything related to bind thinking and humanity with technology .. all the interesting ideas

Thaddy

  • Hero Member
  • *****
  • Posts: 14166
  • Probably until I exterminate Putin.
Very nice!

One initial remark: you use:
Code: Pascal  [Select][+][-]
  1. {$MACRO ON}
  2. {$define ORT_API_CALL:=stdcall}

It is probably better to use no macro at all, but:
Code: Pascal  [Select][+][-]
  1. const ORT_API_CALL=winapi;
because that will automagically switch to cdecl on linux/mac and stdcall on intel windows.
Saves you a headache in maintenance between Freepascal and Delphi, because both understand it.
I suspect linux to be cdecl... Normally MS sticks to the conventions. (That is: apart for their proprietary __fastcall now and then in C++)


I am curious how our forum member and specialist in the field Dr Schuler Phd will review your work!
His recent Phd is in AI, specifically neural networks.

(QRcode will jump to the discussion)
« Last Edit: December 25, 2022, 08:54:21 am by Thaddy »
Specialize a type, not a var.

hshatti

  • New member
  • *
  • Posts: 7
  • Don't be evil isn't enough, Be virtuous!
Thank you for the valuable comment and i'm glad that you like it.

although it's my bad habit inherited from coding with C++, I agree it's always better not to use macros.

one note to mention in fact I tried to use winapi before and it would work,
but the functions as per ONNXRuntime libraries were not nakedly exported
with C declarations as one would expect, thus to be used with cdecl (where the caller is
responsible for clearing the stack on return if i'm not wrong),
instead, they were wrapped inside a C struct with stdcall calling convention
which would imply that the callee function is responsible for clearing the stack on return,
the only cdecl exported function was OrtGetApiBase() which will return pointer to that struct of functions,
I'm not sure of the difference between Windows vs POSIX based systems in terms of
calling behavior in this particular case but I think it won't be necessary to let calls from
FPC to the struct wrapped functions be responsible for stack clearing.

I have checked Dr Schuler's NN library and I must say i'm so much impressed!,
I'm also researching with NN and was do something similar with pascal in a desperate attempt
to UnPython the world during the pandemic!, I would also be so much happy to contribute if the project is open to contributions.


In an unrelated matter i'm using a very old account here that I figured still working, but i cannot modify the profile data that was placed that time, is there any way to do so? the "Profile=>Account Settings" menu doesn't open all the profile information for editing!
Don't be evil isn't enough, Be virtuous!
------------
Research-O-Holic .. Highly Skilled Delphi/Lazarus/FPC developer Oh C/C++ too - Solutions with Micro-Controllers (PIC+AVR +STM .. etc )- I.A - anything related to bind thinking and humanity with technology .. all the interesting ideas

 

TinyPortal © 2005-2018