Conscious Artificial Intelligence - Project Update

[Dzandaa]

Dr Schuler  ,


Congratulation!!!

Great job.

B->

[schuler]

@metis and @Dzandaa,
Thank you!

@Thaddy,

Well, with a last name like that, you obviously felt the pressure to become a scholar.

You certainly have a point.

[schuler]

... the coding continues ...

The paper Searching for Activation Functions describes the search for a better activation function than ReLU. In their work, the authors found Swish to be the best replacement for ReLU. The downside of Swish is: it requires a lot of computation to calculate it. Later, the paper Searching for MobileNetV3 introduces the Hard Swish activation function. The Hard Swish gives similar results to Swish with a lot less computation.

I've just added to the source code the Hard Swish implementation and done new runs comparing ReLU, Swish and Hard Swish. Follow source codes and results.

Source codes are:

• ReLU: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifier.lpr.
• Swish: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierSwish.lpr.
• Hard Swish: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierHardSwish.lpr.

Results are (CIFAR-10 test classification accuracy):

• ReLU: 85.53% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifier20221206.csv.
• Swish: 86.55% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierSwish20221207.csv.
• Hard Swish: 86.82% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierHardSwish20221208.csv.

Looks pretty good to me!

Wish everyone happy pascal coding.

[Dzandaa]

Hi,
thank you for the update.

Have you tested the "mish" activation?
https://ai-scholar.tech/en/articles/treatise/mish-ai-374

It seems easy to implement.

B->

[schuler]

@Dzandaa,
It looks good.

I've just added a feature request to this: https://github.com/joaopauloschuler/neural-api/issues/106 .

[schuler]

DrDub from GitHub coded a simple web server for a neural network that receives an image and classifies it. I did plenty of updates on the top of his code. The code can be found at:

https://github.com/joaopauloschuler/neural-api/tree/master/examples/ResNet/server

[schuler]

  Hello
Just finished updating the autoencoder example: https://github.com/joaopauloschuler/neural-api/tree/master/examples/VisualAutoencoder .

This example uses the freshly tested (but coded long ago) TNNetUpsample layer for increasing resolution in the decoder side. TNNetUpsample converts channels (depth) into spatial data. For example, a 128x128x256 activation map will be converted to 256x256x64. This layer type does a similar job to Tensorflow's tf.nn.depth_to_space.

  Wish everyone happy pascal coding