Recent

Author Topic: Conscious Artificial Intelligence - Project Update  (Read 23610 times)

agorka

  • New Member
  • *
  • Posts: 25
Re: Conscious Artificial Intelligence - Project Update
« Reply #60 on: January 21, 2020, 11:01:09 am »
Hello!
I've got some news on my progress porting Keras model to your library!
I've finally trained it in Keras to 95% accuracy, tried to use it in Pascal, and faced this problem:

My model looks like
Code: Pascal  [Select][+][-]
  1.       TNNetInput.Create(512, 128, 1),
  2.       TNNetConvolutionReLU.Create(24, 7, 0, 1, 0).InitUniform(0),
  3.       TNNetConvolutionReLU.Create(24, 5, 0, 1, 0).InitUniform(0),
  4.       TNNetMaxPoolPortable.Create(2, 2),
  5.       TNNetConvolutionReLU.Create(24*2, 5, 0, 1, 0).InitUniform(0),
  6.       TNNetMaxPoolPortable.Create(2, 2),
  7.       TNNetConvolutionReLU.Create(24*4, 5, 0, 1, 0).InitUniform(0),
  8.       TNNetMaxPoolPortable.Create(2, 2),
  9.       TNNetConvolutionReLU.Create(24*4, 5, 0, 1, 0).InitUniform(0),
  10.       TNNetMaxPoolPortable.Create(2, 2),
  11.       TNNetFullConnectLinear.Create(NumClasses)
Seems that X and Y coordinates are swapped when loading Conv layer weights from strings saved in Keras.
I found that by looking to feature maps, Keras contained vertical lines while pascal contained horizontal and vise versa. I used numpy.swapaxes on weights for convolution layers and it did the trick, though images are still not 100% the same, maybe I'll also need to flip convolution filters to match with Keras.
The next problem was the dense layer.
In Keras I had a flatten layer before the dense, so weights of the dense layer were flat. I had to reshape them manually to 27x3x96, swap axes, flatten, and then save. This seems too cumbersome, cause I had to know the original image shape, and it can't be used in generic weight saving function...
Having non-square convolution could have solved my problem, so I could use 27x3 convolution instead of dense layer.
Maybe you have a better different idea how this problem could be solved in a more beautiful way?

Good news is that after all those tricks I've got very good prediction accuracy in Pascal! Thank you again for making this possible! =)
Oleg.

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #61 on: January 29, 2020, 05:09:59 pm »
@agorka,
Thank you for sharing good news! I'm curious to look at your code. Have you though about open sourcing it? Your code could help others.

Maybe, with time, we could polish both ends and make porting tasks easier.

Firewrath

  • New Member
  • *
  • Posts: 35
Re: Conscious Artificial Intelligence - Project Update
« Reply #62 on: January 29, 2020, 09:50:17 pm »
ok, so this is going to be a little OT from the current discussion going on, sorry. >.>

and first of all, I know pretty much nothing about AI other then article's I've read and such,
I'd like to play with it but I have an 11 y/o PC and almost all the tutorials are in python, so bleh. :P

But I was reading this guys blog:
http://blog.dlib.net/
and he posted something I found rather interesting that I havn't seen anywhere else, called a 'tiled pyramid'
(but mind, I don't follow AI news / updates a lot)

So I was wondering if it could be implemented in your work for training.
Post:
http://blog.dlib.net/2017/08/vehicle-detection-with-dlib-195_27.html
and Pic:
Shortened pic link

To me, it seems a really smart way to train an AI, but that's coming from someone who doesn't know how it all works.
So what do I know? ;)
but wanted to pass it along anyways. ^-^
Sorry. I currently don't have Internet Access. So my replies might take a week. -_-

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #63 on: March 12, 2020, 05:35:55 pm »
@Firewrath, I own you a reply. Will do eventually.

I would like to comment here a trick that you can do with this API or any other API when working with image classification: you can increase the input image size.

As per the following example, by increasing CIFAR-10 input image sizes from 32x32 to 48x48, you can gain up to 2% in classification accuracy.

You can change image sizes with:
Code: Pascal  [Select][+][-]
  1. ImgTrainingVolumes.ResizeImage(48, 48);
  2. ImgValidationVolumes.ResizeImage(48, 48);
  3. ImgTestVolumes.ResizeImage(48, 48);

Then, you need to change the input image size in your NN model:
Code: Pascal  [Select][+][-]
  1. NN.AddLayer([
  2. TNNetInput.Create(48, 48, 3),

There is a CIFAR-10 example showing this trick here:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierResize48.lpr

 :) On the power of native code, I wish everyone happy pascal coding!  :)
« Last Edit: March 12, 2020, 05:47:37 pm by schuler »

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #64 on: April 12, 2020, 02:30:46 am »
 :) hello :),

I would like to comment a new feature and new examples that have just been added:
  • An easy and fast way to load image datasets where each folder represents one class of images.
  • An image classification example that identifies plant diseases based on a photo taken from a leaf.
  • A jupyter notebook that trains a neural network coded in PASCAL for plant disease classification.
There are plenty of image datasets where each folder represents a class of images. To make the pascal coding simpler, a new procedure CreateVolumesFromImagesFromFolder has been added as exemplified below:
Code: Pascal  [Select][+][-]
  1.     // change ProportionToLoad to a smaller number if you don't have available 6GB of RAM.
  2.     ProportionToLoad := 1;
  3.     WriteLn('Loading ', Round(ProportionToLoad*100), '% of the Tiny ImageNet 200 dataset into memory.');
  4.     CreateVolumesFromImagesFromFolder
  5.     (
  6.       ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
  7.       {FolderName=}'tiny-imagenet-200/train', {pImageSubFolder=}'images',
  8.       {color_encoding=}0{RGB},
  9.       {TrainingProp=}0.9*ProportionToLoad,
  10.       {ValidationProp=}0.05*ProportionToLoad,
  11.       {TestProp=}0.05*ProportionToLoad
  12.     );

The above example loads the Tiny ImageNet 200 Dataset https://tiny-imagenet.herokuapp.com/ into RAM memory. 90% is loaded as training data while 5% is loaded for each validation and testing.

The next example shows how to load the dataset used for the paper Identification of Plant Leaf Diseases Using a 9-layer Deep Convolutional Neural Network https://data.mendeley.com/datasets/tywbtsjrjv/1:

Code: Pascal  [Select][+][-]
  1.     // change ProportionToLoad to a smaller number if you don't have available 32GB of RAM.
  2.     ProportionToLoad := 1;
  3.     WriteLn('Loading ', Round(ProportionToLoad*100), '% of the Plant leave disease dataset into memory.');
  4.     CreateVolumesFromImagesFromFolder
  5.     (
  6.       ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
  7.       {FolderName=}'plant', {pImageSubFolder=}'',
  8.       {color_encoding=}0{RGB},
  9.       {TrainingProp=}0.9*ProportionToLoad,
  10.       {ValidationProp=}0.05*ProportionToLoad,
  11.       {TestProp=}0.05*ProportionToLoad,
  12.       {NewSizeX=}128, {NewSizeY=}128
  13.     );

As you can see above, besides loading the dataset, images are being resized to 128x128. As I don't have 32GB of RAM in my development machine, I've tested the above code with https://vast.ai/ using this jupyter notebook: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.ipynb. You can find raw results at: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimplePlantLeafDisease/results .

As you can see on these raw results, the test classification (or the plant disease diagnostic) accuracy is 98.95% (this is pretty high - I wonder if we could do the same with corona virus).

CreateVolumesFromImagesFromFolder has parallel code so classes are loaded into memory in parallel. I consider this code tremendously fast and it outperforms code that I saw in other APIs. I should say thank you to FPC developers for coding FPImage and plenty of other super fast bits and pieces that allowed me to implement a fast CreateVolumesFromImagesFromFolder.

This is the neural network architecture used for classification:
Code: Pascal  [Select][+][-]
  1.     NN.AddLayer([
  2.       TNNetInput.Create(128, 128, 3),
  3.       TNNetConvolutionLinear.Create({Features=}64, {FeatureSize=}5, {Padding=}4, {Stride=}2),
  4.       TNNetMaxPool.Create(2),
  5.       TNNetMovingStdNormalization.Create(),
  6.       TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
  7.       TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
  8.       TNNetMaxPool.Create(2),
  9.       TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
  10.       TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
  11.       TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}2),
  12.       TNNetDropout.Create(0.5),
  13.       TNNetMaxPool.Create(2),
  14.       TNNetFullConnectLinear.Create(39),
  15.       TNNetSoftMax.Create()
  16.     ]);
The complete source code can be found at: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.pas

 :) I wish everyone happy pascal coding :)
« Last Edit: April 12, 2020, 08:02:37 am by schuler »

eljo

  • Sr. Member
  • ****
  • Posts: 408
Re: Conscious Artificial Intelligence - Project Update
« Reply #65 on: April 12, 2020, 07:07:56 am »
I just wanted to say I love your work. Thank you for your support.

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #66 on: April 14, 2020, 09:21:47 am »
@eljo,
Thank you for your kind words. They are motivating.

Created a full Tiny ImageNet 200 example: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleTinyImageNet

:) wish everyone happy pascal coding :)

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #67 on: May 03, 2020, 09:24:20 am »
 :) Hello :),
I have news about Increasing Image Resolution with Neural Networks.

I created a command line tool so everyone can now increase image resolution with no more than FPC (no external library are required) via command line:
Code: Pascal  [Select][+][-]
  1. #SuperResolution -i street.png -o street3.png
  2. Loading input file: street.png
  3. Input image size: 158x214x3
  4. Creating Neural Network...
  5. Resizing with tiles...
  6. Neural network file found at ../../../examples/SuperResolution : super-resolution-7-64-sep.nn
  7. Padding input image.
  8. Resizing with tiles to: 288x416x3
  9. Saving output file: street3.png

-i defines the input image file while -o defines the output image file.

You can get this command line tool from:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SuperResolution/README.md

:) wish everyone happy pascal coding :)

Alienigene

  • New member
  • *
  • Posts: 5
Re: Conscious Artificial Intelligence - Project Update
« Reply #68 on: May 23, 2020, 02:56:48 am »
Hello everyone, I don't know if this is the right place to ask, but I know this problem is rather frequent and it is related to Free Pascal/Lazarus IDE, so I figure here is a good place to ask.

The thing is very simple. My Operating System is Windows 10 v1909 x64. I have downloaded Conscious Artificial Intelligence (project on Sourceforge.net), the files in it are 64-bit. I have extracted the files, ran ArtificialArt.exe, pressed start and nothing happens. I noticed a console window beside the gui window, and it told me it couldn't find a bin file, told me to download cifar, and provided me the address. Simple enough, I downloaded and extracted the files to the folder of CAI.

Then I pressed start again, this time and all times after, an error message window pop up and says: Access violation. Press OK to ignore and risk data corruptio. Press Abort to kill the program. This always happens after loading the .bin files, Creating Neural Networks... and immediately after Creating generative. If I press OK, nothing will happen. If I press Abort, program doesn't end, console says Sending STOP request, but the program doesn't stop, requiring me to close the console to kill the program.

The same problem applies to GradientAscent.exe, it happens immediately after I load a neural network.
I noticed their icons are of the programs developed by Lazarus IDE, I searched the problem online and found it rather frequent, but no solutions, so I figured I would ask here.

I have installed FreePascal IDE 3.0.4 (x86), Lazarus IDE 2.0.6 (x86_64), and placed the executable path of FPC in system path Env.Var., I have no problem using Lazarus IDE itself so far, any ideas on what caused this and how to solve it?

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #69 on: May 24, 2020, 01:12:35 am »
Hello Alienigene,
Can you confirm please whether your processor has or hasn't AVX instruction set? In the case it hasn't, I can prepare a compiled version for you without the AVX instruction set.

Current binaries require AVX instruction set and they will provoke an exception at the time the NN is run with a processor that hasn't AVX.

In the case that you would like to try to compile by yourself, with Lazarus, go to Options/Project Options/Custom Options/Defines and then uncheck AVX. After this, go to Run/Build and then Run/Run.

Alienigene

  • New member
  • *
  • Posts: 5
Re: Conscious Artificial Intelligence - Project Update
« Reply #70 on: May 28, 2020, 10:35:37 am »
I am sorry that I have just seen it, been busy for a while, so I didn't reply instantly. My CPU is Intel Celeron CPU J1900 @ 1.99GHz, quad-core CPU, I looked at CPU-Z, it supports MMX, SSE1-4.2, EM64T and VT-x instruction sets, but no, no AVX, so I think that is the cause of the problem.
« Last Edit: May 28, 2020, 12:03:32 pm by Alienigene »

Alienigene

  • New member
  • *
  • Posts: 5
Re: Conscious Artificial Intelligence - Project Update
« Reply #71 on: May 29, 2020, 06:41:35 am »
I got a problem, big problem, I have downloaded the svncode trunk of cai from sourceforge, intended to compile all the stuff by myself, however there are not any project files(.lpi), there are lots of .pas files, some .inc files, some .lpr .lfm and .cl files, obviously multiple files are included in one project, but I honestly don't know which files are included in a project, and what file each project corresponds to, e.g. .exe .dll .lib .nn, I don't know the correspondences, so I can't compile anything...

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #72 on: May 31, 2020, 03:29:18 am »
Hi Alienigene, these projects are probably a good start (open with Project/Open Project):
  • examples/XorAndOr/XorAndOr.lpi
  • examples/SimpleImageClassifier/SimpleImageClassifier.lpi
Alternatively, if you have a gmail account, you can just open the following link, select "Runtime" and then "Run all":
https://colab.research.google.com/github/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierCPU.ipynb

:) Wish everyone happy pascal coding :)
« Last Edit: May 31, 2020, 08:06:15 pm by schuler »

schuler

  • Full Member
  • ***
  • Posts: 178
Re: Conscious Artificial Intelligence - Project Update
« Reply #73 on: July 30, 2020, 08:04:14 pm »
:) Hello - this is a quick update :)

These are the news from my end:

:) Wish everyone happy pascal coding. :)
« Last Edit: July 31, 2020, 03:11:27 am by schuler »

daringly

  • Jr. Member
  • **
  • Posts: 54
Re: Conscious Artificial Intelligence - Project Update
« Reply #74 on: September 16, 2020, 11:11:05 pm »
I'm trying to get your supersimple example to compile. I have your original uconvolutionneuralnetwork file. Do you have a new set of code that define units neuralnetwork, neuralvolume and nearalfit?

 

TinyPortal © 2005-2018