AI Vision ONNXRuntime Loading model issue

[Jonvy]

Hello everyone,

I tried  ONNXRuntime sample from github:https://github.com/hshatti/TONNXRuntime/tree/main/examples/Lazarus/FasterRCNN10

When I select the model, I get this error, can anyone help me what's wrong with the code?

Jonvy

[domasz]

Move you .EXE to the folder with your .ONNX

[Jonvy]

Yes,they're in same folder.

[Jonvy]

Looks like sample's onnx model format has problem, I get error information from Netron.

But if I choose my own correct model,best_simply.onnx, it will raise another error.

Can anyone succesfully use this example?

[myisjwj]

Your model looks like a YOLO. Instead of using the FasterRCNN10 example, you can refer to the YoloV7_GPU example to implement it. You can also look at this address
https://www.cnblogs.com/jwjss/p/18263154

[Jonvy]

Yes,my model is YOLOv5, I wonder why Haitham Shatti's example not working. I just use his FasterRCNN-10 model, still not working.

For YoloV7_GPU model, I can not use it, it's for Delphi, even convert from lazaurs, it will lost many units.

Follow your blog, I modified line 2487 in onnxruntime.pas, is that what your mean?

After this, still get error when loading model FasterRCNN-10:Protobuf parsing failed.

I download another model,yolov7-tiny_640x640, it can be loaded, after loading model, when I load source Image, it will get exception: Access violation.

[Jonvy]

exception when load image.

[myisjwj]

Lazarus calls the Yolo example

Code: Pascal  [Select][+][-]

1. {$mode delphi}{$H+}
2. interface
3. uses
4. onnxruntime_pas_api,onnxruntime;
5. var
6.   session:TORTSession;
7. InputNames,OutputNames:array of string;
8. input_w,input_h,output_w,output_h:Integer;
9. img: TBitmap;
10. input:TOrtTensor<single>;
11. tempp:PByte;
12. inputs,outputs:TORTNameValueList;
13. outTensor: TOrtTensor<single>;
14.  
15.  
16. //InitOnnxrun
17. session := TORTSession.Create(YouModFile);
18. setLength(InputNames, Session.GetInputCount());
19.     for i := 0 to High(InputNames) do
20.     begin
21.       Info := Session.GetInputTypeInfo(i);
22.       InputNames[i] := string(session.GetInputNameAllocated(i, DefaultAllocator)
23.         {$ifndef NO_SMARTPTR}
24.         .Instance
25.         {$endif}
26.         );
27.       tensorInfo := Info.GetTensorTypeAndShapeInfo();
28.       ints := tensorInfo.GetShape();
29.       input_w := ints[3];
30.       input_h := ints[2];
31.     end;
32.     setLength(OutputNames, Session.GetOutputCount());
33.     for i := 0 to High(OutputNames) do
34.     begin
35.       Info := Session.GetOutputTypeInfo(i);
36.       OutputNames[i] := string(session.GetOutputNameAllocated(i, DefaultAllocator)
37.         {$ifndef NO_SMARTPTR}
38.         .Instance
39.         {$endif}
40.         );
41.       if OutputNames[i].IndexOf('output') >= 0 then
42.       begin
43.         tensorInfo := Info.GetTensorTypeAndShapeInfo();
44.         ints := tensorInfo.GetShape();
45.         output_w := ints[2];
46.         output_h := ints[1];
47.         outst:=OutputNames[i];
48.       end;
49.     end;
50.  
51. //run
52. img:=TBitmap.Create;
53.   img.Width:=input_w;
54.   img.Height:=input_h;
55.   img.PixelFormat:=pf24bit;
56. .....
57. Input := TOrtTensor<single>.Create([input_w, input_h, 3, 1]);
58.     for yy := 0 to input_h - 1 do
59.     begin
60.       tempp := img.ScanLine[yy];
61.       for xx := 0 to input_w - 1 do
62.       begin
63.         input.Index3[xx, yy, 0] := tempp[xx * 3 ] / 255;
64.         input.Index3[xx, yy, 1] := tempp[xx * 3 + 1] / 255;
65.         input.Index3[xx, yy, 2] := tempp[xx * 3+ 2] / 255;
66.       end;
67.     end;
68.     inputs.AddOrSetValue('images', input);
69. Outputs:=session.Run(Inputs);

[Jonvy]

OK, I'll study this example,thanks myisjwj!

[hshatti]

Greetings,
I Apologize for not seeing this earlier, the error seems to be related to the ONNX library version compatibility,
ONNX library versions beyond 1.16 will raise an exception with unsigned .onnx files which may seems the case you are having.

The latest ONNX library version that was released when I published this library was 1.13.1.

Unfortunately I ran into a health related problem and with the financial stress I'm having, I couldn't find much focusing time or energy to maintain the library after that, nevertheless!, the last guaranteed successful test l did which was with the DirectML version 1.16.23 which I highly recommend to use.

not sure if you have successfully managed to port Delphi's YOLOV7 example to Lazarus, it should be possible but I'm happy to help.

recent versions may have compatibility problems with your model or my implementation, hopefully soon I will update this to match the recent release and maybe add more exotic samples such as image background removal, voice cancelation and chatGPT like Language models.

BTW I have released another library called LightNet which is 100% written pure in pascal, totally dependentless and works with YOLO V7 and can be used with other models as well, this was inspired by the original darknet YOLO C implementation and managed to make it even faster than the original C (on CPU using heavily optimized SIMD assembly instructions) have a look at it,
I will be releasing a better version soon with more training examples leveraging either OpenCL, of CUDNN (targeted for NVIDIA Jeston Embedded Systems).