Hello elxanders,
Here we go with a short example. In this example, the NN learns 3 logic operations XOR, AND and OR.
type TBackInputOutput = array[0..3] of array[0..2] of TNeuralFloat;
const outputs : TBackInputOutput =
(// XOR, AND, OR
(-0.1,-0.1,-0.1),
( 0.1,-0.1, 0.1),
( 0.1,-0.1, 0.1),
(-0.1, 0.1, 0.1)
);
const inputs : TBackInputOutput =
( // x1, x2, bias
(-0.9, -0.9, 1),
(-0.9, 0.9, 1),
( 0.9, -0.9, 1),
( 0.9, 0.9, 1)
);
procedure TForm1.BitBtn6Click(Sender: TObject);
var
NN: TNNet;
I: integer;
Cnt: integer;
pOutPut: TNNetVolume;
vInputs, vOutput: TBackInputOutput;
totalTimeSeconds, startTime, finishTime: double;
shouldPrint: boolean;
begin
shouldPrint := true;
vInputs := inputs;
vOutput := outputs;
pOutPut := TNNetVolume.Create(3,1,1,1);
NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(3) );
NN.AddLayer( TNNetLayerFullConnect.Create(3) );
NN.AddLayer( TNNetLayerFullConnect.Create(3) );
NN.AddLayer( TNNetLayerFullConnect.Create(3) );
NN.Randomize();
startTime := now();
for I := 1 to 3000 do
begin
if shouldPrint then WriteLn();
for Cnt := Low(inputs) to High(inputs) do
begin
NN.Compute(vInputs[cnt]);
NN.GetOutput(pOutPut);
NN.Backpropagate(vOutput[cnt]);
if shouldPrint then
WriteLn
(
I:7,'x',Cnt,' Output:',
pOutPut.Raw[0]:5:2,' ',
pOutPut.Raw[1]:5:2,' ',
pOutPut.Raw[2]:5:2,' - Training data:',
vOutput[cnt][0]:5:2,' ',
vOutput[cnt][1]:5:2,' ' ,
vOutput[cnt][2]:5:2,' '
);
end;
end;
finishTime := now();
totalTimeSeconds := (finishTime - startTime) * 24 * 60 * 60;
writeln('Total run time:', (totalTimeSeconds): 10: 5, ' seconds.');
NN.Free;
pOutPut.Free;
end;
You have 2 questions that affect performance:
* About creating and passing objects. For big datasets, I create all required objects before running the NN. There is an example here: testcnnalgo.lpr .
For simplicity, you can also use dynamic arrays with single elements.
* About batches: although I do not have batches and it could make the NN faster, some of my own benchmarks running against some well known APIs (won't give names for now) reveal that this implementation is up to 10x faster with CPU only (I mean, CPU implementation against CPU implementation). The OpenCL version is in the cards to come. Yes - you need to call Backpropagate each time.
The latest version of testcnnalgo.lpr shows an example with a decreasing learning rate. Yes, you can change learning rate and inertia at any time. I highly recommend running testcnnalgo.lpr program via command line to have a feeling.
There is a new implementation here:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/XorAndOrFeel free to share questions as they might help other users.