Microsoft Open Neural Network Exchange ( ONNXRuntime ) for Lazarus / Delphi

[hshatti]

Greetings everyone,
I'm happy to share with you my Freepascal / Delphi implementation of Microsoft ONNXRuntime
Open Neural Network Exchange AI and Machine Learning library

Install
    Please visit
      https://github.com/hshatti/TONNXRuntime

    or

Code: Bash  [Select][+][-]

1. git clone https://github.com/hshatti/TONNXRuntime.git

Usage

Code: Pascal  [Select][+][-]

1. unit formUnit;
2. {$h+}
3.  
4. interface
5. uses onnxruntime, Classes etc... ;

Load a Model

Code: Pascal  [Select][+][-]

1. var
2.   session : TORTSession;
3. begin
4.   session := TORTSession.Create('./mymodel/filname.onnx');
5. {
6. *****************************************************************
7.     Check your model requirements for input/output
8.     names and value dimensions before preparing the inputs.
9.     to explore the model before preparing use :
10.       session.GetInputCount and session.GetOutputCount
11.       session.GetInputName and session.GetOutputName
12.       session.GetInputTypeInfo and session.GetOutputTypeInfo
13.  ****************************************************************
14. }

Prepare an input tensor with the desired shape using

Code: [Select]

TORTTensor<type> and your inputs using the library's name/value Dictionary

Code: [Select]

TORTNameValueList

Code: Pascal  [Select][+][-]

1. var
2.   x,y:integer;
3.   imageData : array of array of single;
4.   inTensor : TORTTensor<single> ;
5.   inputs : TORTNameValueList  ;
6. begin
7.   // assuming the model input name is 'image' and the tensor shape is [width, height]
8.   inTensor := TORTTensor<single>.Create([width, height{, depth ,etc...}]);
9.   for y:=0 to inTensor.shape[1]-1 do
10.       for x:=0 to inTensor.shape[0]-1 do
11.           inTensor.index2[x, y]:= imageData[x, y];  // your float values
12.   inputs['image'] := inTensor;

Inference

Code: Pascal  [Select][+][-]

1. var
2.     myDetection : array of single;
3.     i:integer;
4.     outputs : TORTNameValueList;
5.     outTensor : TORTTensor<single>
6.   bagin
7.       outputs   := session.run(inputs);
8.       outTensor := outputs['result']
9.      
10.       for i:=0 to outTensor.shape[0]-1 do
11.         myDetection[i] := outTensor.index1[i]

Quick notes
If you are on Windows mostly you are good to go, ONNXRuntime libraries come shipped with most of modern Windows releases.
check :

Code: Text  [Select][+][-]

1. %WINDIR%\SysWOW64\onnxruntime.dll

or

Code: Text  [Select][+][-]

1. %WINDIR%\System32\onnxruntime.dll

• MacOS / Linux users please download the dynamic link libraries https://github.com/microsoft/onnxruntime/releases/tag/v1.13.1 or compile static libraries from ONNXRuntime Github repository :  https://github.com/microsoft/onnxruntime
  
  
• I have included a simple inference example for Object Detection using FasterRCNN-10 model, this should be straight forward to understand, if you need further help please let me know, more good Inference and Training examples to be uploaded soon.
  
  
• Check Microsoft model zoo repository if you like to play around with AI and Neural Networks, Now! most of Python, C++ and C# examples can be easily ported to Lazarus : https://github.com/onnx/models
  
  
• Further documentation on API and usage of GPUs can be found here : https://onnxruntime.ai/docs/api/
  
  
• Suggestions, Criticism and Contributions are most welcome.
  
  
• Please use it for peaceful purposes, quote my work if you find it useful.
  
  

[Thaddy]

Very nice!

One initial remark: you use:

Code: Pascal  [Select][+][-]

1. {$MACRO ON}
2. {$define ORT_API_CALL:=stdcall}

It is probably better to use no macro at all, but:

Code: Pascal  [Select][+][-]

1. const ORT_API_CALL=winapi;

because that will automagically switch to cdecl on linux/mac and stdcall on intel windows.
Saves you a headache in maintenance between Freepascal and Delphi, because both understand it.
I suspect linux to be cdecl... Normally MS sticks to the conventions. (That is: apart for their proprietary __fastcall now and then in C++)


I am curious how our forum member and specialist in the field Dr Schuler Phd will review your work!
His recent Phd is in AI, specifically neural networks.

(QRcode will jump to the discussion)

[hshatti]

Thank you for the valuable comment and i'm glad that you like it.

although it's my bad habit inherited from coding with C++, I agree it's always better not to use macros.

one note to mention in fact I tried to use winapi before and it would work,
but the functions as per ONNXRuntime libraries were not nakedly exported
with C declarations as one would expect, thus to be used with cdecl (where the caller is
responsible for clearing the stack on return if i'm not wrong),
instead, they were wrapped inside a C struct with stdcall calling convention
which would imply that the callee function is responsible for clearing the stack on return,
the only cdecl exported function was OrtGetApiBase() which will return pointer to that struct of functions,
I'm not sure of the difference between Windows vs POSIX based systems in terms of
calling behavior in this particular case but I think it won't be necessary to let calls from
FPC to the struct wrapped functions be responsible for stack clearing.

I have checked Dr Schuler's NN library and I must say i'm so much impressed!,
I'm also researching with NN and was do something similar with pascal in a desperate attempt
to UnPython the world during the pandemic!, I would also be so much happy to contribute if the project is open to contributions.

In an unrelated matter i'm using a very old account here that I figured still working, but i cannot modify the profile data that was placed that time, is there any way to do so? the "Profile=>Account Settings" menu doesn't open all the profile information for editing!