Forum > Third party
Pascal for TensorFlow
jollytall:
Phil,
I tried your code. It is a very good start for a beginner like me, so thanks for it.
In your MNIST.pas I think I found two errors:
- When calculating dz1, you have
--- Code: ---dz1 := dz2.ExecOp('MatMul', Parameters.w2.ExecOp('Transpose', TTensor.CreateInt32([1,0]).Temp).Temp).Temp *
h1 * (TTensor.CreateSingle(1).Temp - h1).Temp;
--- End code ---
but the first part
--- Code: ---dz2.ExecOp('MatMul', Parameters.w2.ExecOp('Transpose', TTensor.CreateInt32([1,0]).Temp).Temp).Temp * h1
--- End code ---
is already a TTensor, what is not freed. If the IterationCount is larger than in your example, it is a severe memory leak.
It has to be fixed as
--- Code: ---dz1 := (dz2.ExecOp('MatMul', Parameters.w2.ExecOp('Transpose', TTensor.CreateInt32([1,0]).Temp).Temp).Temp *
h1).Temp * (TTensor.CreateSingle(1).Temp - h1).Temp;
--- End code ---
- A bit smaller but similar memory leak is in the calculation of part 1
--- Code: ---part1 := LabelsT.ExecOp('Neg').Temp * predictions.ExecOp('Log');
--- End code ---
I t should be
--- Code: ---part1 := LabelsT.ExecOp('Neg').Temp * predictions.ExecOp('Log').Temp;
--- End code ---
and then no memory leak at all.
jollytall:
Can someone help me with a very simple question?
I see that if the T.ExecOp is used then T is passed on as the first parameter of the operation.
I needed some time, how the w2 := TTensor.CreateInt32([30, 10]).Temp.ExecOp('RandomUniform'); line actually first creates a Shape, and using it as a first parameter, generates a Tensor with random numbers, using tf.random.uniform. Looking at it, first I assumed it creates a 2 element Int32 vector and randomizes that. (Usual OO logic, that the method is working on the object and not using the object as an input to a sort-of class method.) Anyway now, I understand how it works.
I also see that ExecOp can take (max 3) additional TTensors as parameters (e.g. for matrix multiplication, etc.), what is useful when the operation needs extra Tensors.
What I do not see, how I can pass other parameters, not Tensors. E.g in Python tf.random.uniform has a format:
tf.random.uniform(
shape,
minval=0,
maxval=None,
dtype=tf.dtypes.float32,
seed=None,
name=None
)
The C interface has seed and seed2, but no min-max. I see that ExecOp declares seed and seed2 and dtype, so, as expected, builds on the C interface.
Does it mean that min-max is a Python extension and neither C, not Pascal has it?
Regarding the other C parameters (e.g. Seed in this case, but in other operations might be more interesting ones), can I specify them in an easy way with the TF unit, or directly calling TensorFlow unit?
jollytall:
FYI - Based on Phil's TensorFlow unit (the pascal wrapper around the C header files) I also made a new unit that can handle Tensors, Graphs, Nodes and Sessions both in the basic way (Graph-Nodes-Tensors-Sessions) and the Eager way, without using any object, class (and Phil's TF unit).
Hence (at least to me), the code is very easy to understand, use and develop further.
If you are interested, please send me a PM and I can send you a copy.
jollytall:
In the meantime I developed it a bit further and uploaded it to GitHub. Please check it at GitHub.com/tensorflowforpascal.
It has specific Exec<oper> functions for over 800 TF functions to run them directly without any Graph creation, or you can also use TGraph and TGraphExt where all the operations are available for building a Graph.
avra:
--- Quote from: jollytall on February 13, 2020, 05:28:44 pm ---Please check it at GitHub.com/tensorflowforpascal.
--- End quote ---
That link is not good. Is https://github.com/zsoltszakaly/tensorflowforpascal the good one?
Navigation
[0] Message Index
[#] Next page
[*] Previous page