Recent

Author Topic: Conscious Artificial Intelligence - Project Update  (Read 60673 times)

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #75 on: September 17, 2020, 05:57:02 am »
 :) Hi Daringly :)
You'll need this folder to compile: https://github.com/joaopauloschuler/neural-api/tree/master/neural

Alternatively, in the case that you would like to try image classification without having to install anything, you can run via browser at:

https://colab.research.google.com/github/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierCPU.ipynb

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #76 on: September 17, 2020, 08:47:11 pm »
Thanks!

I do a lot of Sports actuarial analysis (probabilities for each match outcome), and this will hopefully let me do some much more complex analysis than what I am presently doing.

Blade

  • Full Member
  • ***
  • Posts: 177
Re: Conscious Artificial Intelligence - Project Update
« Reply #77 on: September 19, 2020, 07:26:33 pm »
Very nice.  Great project.

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #78 on: September 21, 2020, 08:44:13 pm »
Is there a set of documentation for these libraries?

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #79 on: September 22, 2020, 01:10:38 pm »
:) Hi daringly :)
Except for this read me (https://github.com/joaopauloschuler/neural-api), not much. Is there any particular part of the API that you would like me to detail further?



daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #80 on: September 22, 2020, 04:51:28 pm »
I'm not particularly good at Python/Keras, whereas I'm much more fluent in Pascal. I'd love a set of documents so that someone that never used Python or Keras could do things from the ground up.

Here's the meat of a keras model I wrote, and I have no idea how to implement this in your library:

model=tf.keras.Sequential()
model.add(layers.Dense(12, input_dim=14, activation='relu'))
model.add(layers.Dense(4, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics =['accuracy'])

print(model.summary)

monitor_val_acc=EarlyStopping(monitor='val_loss', patience=5)
model.fit(x,y, epochs=200, validation_data=(x_test, y_test), callbacks=[monitor_val_acc])

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #81 on: September 22, 2020, 04:54:55 pm »
How about an example where you have 2 inputs (x,y) and an output (z) which is the hypotenuse of a right-angle triangle?

So in our dataset, Z^2 = X^2 + Y^2.. use maybe 10 datapoints.

A simple start-to-finish on "solving" and improving on the solution to this problem could be extrapolated to all other sorts of problems.

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #82 on: September 24, 2020, 05:02:08 am »
 :) Hello daringly :)

Your Keras model translates to this:
Code: Pascal  [Select][+][-]
  1. var
  2.   Model: TNNet;
  3. begin
  4.   Model := TNNet.Create();
  5.   Model.AddLayer([
  6.     TNNetInput.Create(14),
  7.     TNNetFullConnectReLU.Create(12),
  8.     TNNetFullConnectReLU.Create(4),
  9.     TNNetFullConnect.Create(1) // This is hyperbolic tangent - not Sigmoid
  10.   ]);

print(model.summary) translates to:
Code: Pascal  [Select][+][-]
  1. Model.DebugStructure();

Above example has hyperbolic tangent instead of sigmoid.

For the fitting, you might follow this example:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/XorAndOr

The fitting method allows you to pass validation and test pairs also:
Code: Pascal  [Select][+][-]
  1. procedure Fit(pNN: TNNet;
  2.   pTrainingVolumes, pValidationVolumes, pTestVolumes: TNNetVolumePairList;
  3.   pBatchSize, Epochs: integer);

You won't need callbacks for monitoring. The fitting already saves the best solution for you and displays progress.

In the case that you really need callbacks, you can look at:
Code: Pascal  [Select][+][-]
  1.       property OnAfterStep: TNotifyEvent read FOnAfterStep write FOnAfterStep;
  2.       property OnAfterEpoch: TNotifyEvent read FOnAfterEpoch write FOnAfterEpoch;

CAI is SGD based (I don't have adam). I'm passionate towards SGD.

About Z^2 = X^2 + Y^2, I created a feature request at:
https://github.com/joaopauloschuler/neural-api/issues/33
« Last Edit: September 24, 2020, 05:11:18 am by schuler »

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #83 on: September 24, 2020, 02:14:49 pm »
Thanks! I'll fire this up this weekend.

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #84 on: September 29, 2020, 04:35:58 am »
@daringly

I coded the example you asked for:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/Hypotenuse/Hypotenuse.lpr

Training, validation and testing pairs are created with:
Code: Pascal  [Select][+][-]
  1.   function CreateHypotenusePairList(MaxCnt: integer): TNNetVolumePairList;
  2.   var
  3.     Cnt: integer;
  4.     LocalX, LocalY, Hypotenuse: TNeuralFloat;
  5.   begin
  6.     Result := TNNetVolumePairList.Create();
  7.     for Cnt := 1 to MaxCnt do
  8.     begin
  9.       LocalX := Random(100);
  10.       LocalY := Random(100);
  11.       Hypotenuse := sqrt(LocalX*LocalX + LocalY*LocalY);
  12.  
  13.       Result.Add(
  14.         TNNetVolumePair.Create(
  15.           TNNetVolume.Create([LocalX, LocalY]),
  16.           TNNetVolume.Create([Hypotenuse])
  17.         )
  18.       );
  19.     end;
  20.   end;
  21. ...
  22.     TrainingPairs := CreateHypotenusePairList(10000);
  23.     ValidationPairs := CreateHypotenusePairList(1000);
  24.     TestPairs := CreateHypotenusePairList(1000);

The neural network is created with 2 inputs and one output:
Code: Pascal  [Select][+][-]
  1. NN := TNNet.Create();
  2. NN.AddLayer( TNNetInput.Create(2) );
  3. NN.AddLayer( TNNetFullConnectReLU.Create(32) );
  4. NN.AddLayer( TNNetFullConnectReLU.Create(32) );
  5. NN.AddLayer( TNNetFullConnectReLU.Create(1) );

The fitting is called with:
Code: Pascal  [Select][+][-]
  1.   // Returns TRUE if difference is smaller than 0.1 .
  2.   function LocalFloatCompare(A, B: TNNetVolume; ThreadId: integer): boolean;
  3.   begin
  4.     Result := ( Abs(A.FData[0]-B.FData[0])<0.1 );
  5.   end;  
  6. ...
  7.     NFit := TNeuralFit.Create();
  8.     NFit.InitialLearningRate := 0.00001;
  9.     NFit.LearningRateDecay := 0;
  10.     NFit.L2Decay := 0;
  11.     NFit.InferHitFn := @LocalFloatCompare;
  12.     NFit.Fit(NN, TrainingPairs, ValidationPairs, TestPairs, {batchsize=}32, {epochs=}50);

A final test is done with:
Code: Pascal  [Select][+][-]
  1.     // tests the learning
  2.     for Cnt := 0 to 9 do
  3.     begin
  4.       NN.Compute(TestPairs[Cnt].I);
  5.       NN.GetOutput(pOutPut);
  6.       WriteLn
  7.       ( 'Inputs:',
  8.         TestPairs[Cnt].I.FData[0]:5:2,', ',
  9.         TestPairs[Cnt].I.FData[1]:5:2,' - ',
  10.         'Output:',
  11.         pOutPut.Raw[0]:5:2,' ',
  12.         ' Desired Output:',
  13.         TestPairs[Cnt].O.FData[0]:5:2
  14.       );
  15.     end;

These are results from my own run:
Code: Pascal  [Select][+][-]
  1. Inputs:19.00, 71.00 - Output:73.53  Desired Output:73.50
  2. Inputs:28.00,  6.00 - Output:28.66  Desired Output:28.64
  3. Inputs:34.00, 53.00 - Output:63.02  Desired Output:62.97
  4. Inputs:20.00, 35.00 - Output:40.25  Desired Output:40.31
  5. Inputs:53.00, 28.00 - Output:59.98  Desired Output:59.94
  6. Inputs:56.00, 12.00 - Output:57.31  Desired Output:57.27
  7. Inputs:77.00, 29.00 - Output:82.35  Desired Output:82.28
  8. Inputs:69.00, 31.00 - Output:75.65  Desired Output:75.64
  9. Inputs:24.00, 78.00 - Output:81.65  Desired Output:81.61
  10. Inputs:15.00, 22.00 - Output:26.66  Desired Output:26.63

 :) Wish everyone happy pascal coding  :)
« Last Edit: September 29, 2020, 12:20:51 pm by schuler »

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #85 on: September 30, 2020, 02:26:20 am »
When I compile the hypotenuse example, I get "FATAL: Cannot find MTPCPU used by neuralthread".

I did get the examples included in your package working.

$0.01 for your thoughts.

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #86 on: September 30, 2020, 10:11:54 am »
you'll need the package multithreadprocslaz .

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #87 on: October 02, 2020, 03:31:12 pm »
The example works, thanks!

I tried modifying it, and broke it.

I wanted to expand the example. With 3 inputs, I have the dimensions of a hexahedron. With 2 outputs, I'd calculate the hypotenuse of opposite corners, and the volume.

I changed the initial NN definition to:

NN.AddLayer( TNNetInput.Create(3) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(2) );     

The Create pair list function was modified:
for Cnt := 1 to MaxCnt do
    begin
      LocalX := Random(100);
      LocalY := Random(100);
      LocalZ := Random(100);
      Hypotenuse := sqrt(LocalX*LocalX + LocalY*LocalY + LocalZ * LocalZ);
      Volume := LocalX * LocalY * LocalZ;

      Result.Add(
        TNNetVolumePair.Create(
          TNNetVolume.Create([LocalX, LocalY, LocalZ]),
          TNNetVolume.Create([Hypotenuse, Volume])
        )
      );   

I adjusted the outputs, but that isn't causing the problems. All my outputs are now 0. What have I done?

« Last Edit: October 02, 2020, 03:45:27 pm by daringly »

daringly

  • Jr. Member
  • **
  • Posts: 73
Re: Conscious Artificial Intelligence - Project Update
« Reply #88 on: October 02, 2020, 07:57:18 pm »
What exactly does this statement do?

pOutPut := TNNetVolume.Create(1,1,1,1);

schuler

  • Full Member
  • ***
  • Posts: 223
Re: Conscious Artificial Intelligence - Project Update
« Reply #89 on: October 04, 2020, 07:05:51 am »
@daringly,
Very interesting!

Code: Pascal  [Select][+][-]
  1. pOutPut := TNNetVolume.Create({pSizeX=}1, {pSizeY=}1, {pDepth=}1, {FillValue=}1);
creates a 3D memory structure that can be accessed as 1D, 2D and 3D construct at the same time. Above example will create a 1x1x1 array filled with the value 1.

Given that your output now has size 2, you should try this:
Code: Pascal  [Select][+][-]
  1. pOutPut := TNNetVolume.Create({pSizeX=}2, {pSizeY=}1, {pDepth=}1, {FillValue=}1);

Given your use cases, I'm now considering adding non linear neural network layers so the network can learn expressions such as "sqr(A) + sqrt(A*B)" faster.

 

TinyPortal © 2005-2018