Good GPU accelerated math library

[jollytall]

I am trying to do large scale (vector, matrix based) calculations (not only matrix operations, but other element-wise calculations, like square root, exponential, trigonometric, etc.) with GPU acceleration. I am looking for a good library with Pascal binding.
"Good" would mean, easy to switch between CPU and GPU, large operation set, active support, fast. I did a lot of work with TensorFlow (that can do non-AI stuff as well), but it is extremely complicated to build graphs, run sessions, etc. and the C binding (and the Pascal binding based on it) is not actively supported. I was recommended things like JAX (but as I see it only has Python binding), CuDA libraries (very low level) and ArrayFire (with C, Fortran binding that can be transformed into a Pascal binding, but I did not find anyone doing it).
(Whoever I talk to recommends changing to Python, but one does not easily leave a long-term love:-) )
Any better idea?

[Handoko]

Maybe this is worth checking out:
https://fastcode.sourceforge.net/

[jollytall]

The last entry seems to be from 2007. With the current pace of GPU developments, I have serious doubts that this one can be of use. Also, as I see the function set is rather limited.

[bobihot]

The last entry seems to be from 2007. With the current pace of GPU developments, I have serious doubts that this one can be of use. Also, as I see the function set is rather limited.

To make a project, to extend possibilities?
Colect a list of need functions and how to realise.

[Thaddy]

OpenCL will choose optimal available (either GPU or CPU)

[LV]

OpenCL will choose optimal available (either GPU or CPU)

Yes.
Parallel computing using OpenCL on GPUs and CPUs.
For example, projects using Delphi: https://github.com/LUXOPHIA/OpenCL,
or FPC with Lazarus:
https://github.com/joaopauloschuler/neural-api

P.S. This topic was also discussed on this forum, but quite a long time ago: https://forum.lazarus.freepascal.org/index.php?topic=8591.0.
P.P.S. The fpc 3.2.2 compiler has a cl unit. But I don't know how up-to-date it is.

[jollytall]

The best(?) I found so-far is ArrayFire. It is easy to use and does the basic stuff. However some simple things cupy can easily do (e.g. 1D convolution on a 2D or 3D array, but not on the first dimension) it can only do with a memory reallocation, what makes it slow.
OpenCL can do surely everything, but one need to program everything manually. Also on nvidia Cuda can be faster, but I did not find any good Pascal-Cuda interface with a good function library in between.
I am a bit disappointed (and it is not only Pascal, it is C and C++ as well), that there is no function rich library to call with GPU under it.
I did a lot of work on tensorflow and up to 2.18 it had a C library. It was not easy but worked (though a bit buggy). After 2.19 they do not provide a binary and building it is almost impossible (I could not).

[myisjwj]

First, make sure to debug it successfully using https://www.cnblogs.com/jwjss/p/18496010. Then, you can call it using OpenCL.