Print Page - Conscious Artificial Intelligence

Announcements => Third party => Topic started by: schuler on November 24, 2017, 12:54:49 am

Title: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 24, 2017, 12:54:49 am

:) Hello :)
As some of you already know, I've been developing Neural Networks with Free Pascal / Lazarus and sometimes it's good to show progress.

The newest addition to the project is a convolutional neural network pascal unit capable of convolution and other common features. These are the available layers:
* TNNetInput (Input/output: 1D, 2D or 3D).
* TNNetFullConnect (Input/output: 1D, 2D or 3D).
* TNNetFullConnectReLU (Input/output: 1D, 2D or 3D).
* TNNetLocalConnect (Input/output: 1D, 2D or 3D - feature size: 1D or 2D). Similar to full connect with individual neurons.
* TNNetLocalConnectReLU (Input/output: 1D, 2D or 3D - feature size: 1D or 2D).
* TNNetConvolution (Input/output: 1D, 2D or 3D - feature size: 1D or 2D).
* TNNetConvolutionReLU (Input/output: 1D, 2D or 3D - feature size: 1D or 2D).
* TNNetMaxPool (Input/output: 1D, 2D or 3D - feature size: 1D or 2D).
* TNNetConcat (Input/output: 1D, 2D or 3D) - Allows concatenating the result from previous layers.
* TNNetReshape (Input/output: 1D, 2D or 3D).
* TNNetSoftMax (Input/output: 1D, 2D or 3D).

These are the available weight initializers:
* InitUniform(Value: TNeuralFloat = 1);
* InitLeCunUniform(Value: TNeuralFloat = 1);
* InitHeUniform(Value: TNeuralFloat = 1);
* InitGlorotBengioUniform(Value: TNeuralFloat = 1);

The source code is located here:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/libs/uconvolutionneuralnetwork.pas

There is an extensive CIFAR-10 testing script located at:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/testcnnalgo/testcnnalgo.lpr

The API allows you to create divergent/parallel and convergent layers as per
example below:

Code: Pascal [Select][+]

// Creates The Neural Network
NN := TNNet.Create();
 
// This network splits into 2 paths and then is later concatenated
InputLayer := NN.AddLayer(TNNetInput.Create(32, 32, 3));
 
// First branch starting from InputLayer (5x5 features)
NN.AddLayerAfter(TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}2, {Stride=}1), InputLayer);
NN.AddLayer(TNNetMaxPool.Create(2));
NN.AddLayer(TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}5, {Padding=}2, {Stride=}1));
NN.AddLayer(TNNetMaxPool.Create(2));
EndOfFirstPath := NN.AddLayer(TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}5, {Padding=}2, {Stride=}1));
 
// Another branch starting from InputLayer (3x3 features)
NN.AddLayerAfter(TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}3, {Padding=}1, {Stride=}1), InputLayer);
NN.AddLayer(TNNetMaxPool.Create(2));
NN.AddLayer(TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1));
NN.AddLayer(TNNetMaxPool.Create(2));
EndOfSecondPath := NN.AddLayer(TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1));
 
// Concats both branches into one branch.
NN.AddLayer(TNNetDeepConcat.Create([EndOfFirstPath, EndOfSecondPath]));
NN.AddLayer(TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1));
NN.AddLayer(TNNetLayerFullConnectReLU.Create(64));
NN.AddLayer(TNNetLayerFullConnectReLU.Create(NumClasses));

As a simpler example, this example shows how to create a simple fully forward connected network 3x3:

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(3) );
NN.AddLayer( TNNetLayerFullConnectReLU.Create(3) );
NN.AddLayer( TNNetLayerFullConnectReLU.Create(3) );
NN.SetLearningRate(0.01,0.8);

In above example, learning rate is 0.01 and inertia is 0.8.

Follows an example showing how to train your network:

Code: Pascal [Select][+]

// InputVolume and vDesiredVolume are of the type TNNetVolume
NN.Compute(InputVolume);
NN.GetOutput(PredictedVolume);
vDesiredVolume.SetClassForReLU(DesiredClass);
NN.Backpropagate(vDesiredVolume);

Recently, an ultra fast Single Precision AVX/AVX2 API was coded so Neural Networks could benefit from it:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/libs/uvolume.pas

2 videos were created showing this unit:
https://www.youtube.com/watch?v=qGnfwpKUTIQ
https://www.youtube.com/watch?v=Pnv174V_emw

In preparation for future OpenCL based Neural Networks, an OpenCL wrapper was created:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/libs/ueasyopencl.pas
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/libs/ueasyopenclcl.pas

There is an example for this OpenCL wrapper here:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/opencl/easy-trillion-test/

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: DonAlfredo on November 24, 2017, 09:51:35 am

Very impressive. And OpenCL is a very useful feature for speeding things up.
Thanks.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 16, 2018, 01:28:01 am

:) Hello, :)
I haven't updated this post for a long time. There are plenty of new features since the last update:

A new pooling layer:
* TNNetAvgPool (input/output: 1D, 2D or 3D)

Layers for inverse operations:
* TNNetDeLocalConnect (input/output: 1D, 2D or 3D)
* TNNetDeLocalConnectReLU (input/output: 1D, 2D or 3D)
* TNNetDeconvolution (input/output: 1D, 2D or 3D)
* TNNetDeconvolutionReLU (input/output: 1D, 2D or 3D)
* TNNetDeMaxPool (input/output: 1D, 2D or 3D)

New normalization layers:
* TNNetLayerMaxNormalization (input/output: 1D, 2D or 3D)
* TNNetLayerStdNormalization (input/output: 1D, 2D or 3D)

A dropout layer:
* TNNetDropout (input/output: 1D, 2D or 3D)

L2 regularization has been added. It's now possible to apply L2 to all layers or only to convolutional layers.

Code: Pascal [Select][+]

      procedure SetL2DecayToConvolutionalLayers(pL2Decay: TNeuralFloat);
      procedure SetL2Decay(pL2Decay: TNeuralFloat);

A new video:
Increasing Image Resolution with Neural Networks
https://www.youtube.com/watch?v=jdFixaZ2P4w (https://www.youtube.com/watch?v=jdFixaZ2P4w)

Data Parallelism has been implemented. Experiments have been made with up to 40 parallel threads in virtual machines with up to 32 virtual cores.

Data Parallelism is implemented at TNNetDataParallelism. As per example, you can create and run 40 parallel neural networks with:

Code: Pascal [Select][+]

NN := TNNet.Create();
...
FThreadNN := TNNetDataParallelism.Create(NN, 40);
...
ProcThreadPool.DoParallel(@RunNNThread, 0, FThreadNN.Count-1, Nil, FThreadNN.Count);
FThreadNN.AvgWeights(NN);

:) Wish everyone happy coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 31, 2018, 05:21:45 am

It's time for new features and examples!

Follows some new examples:

CIFAR-10 with OpenCL on convolution forward step: https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/experiments/visualCifar10OpenCL/

CIFAR-10 batch updates example:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/experiments/visualCifar10BatchUpdate/

Mario Werner independently coded and shared examples for CIFAR-10 and MNIST:
https://bitbucket.org/108bits/cai-implementations/src/c8c027b1a0d636713f7ebb70a738f1cd7117a7a4?at=master

BTW, if you have examples that you've shared in public repositories, please let me know. I love seeing what you are doing.

These are new features:

OpenCL based Convolution: on tiny images 32x32x3 or 28x28x1, you won't get much improvement against the well made AVX code. But you'll find impressive improvement on large input volumes. OpenCL can be enabled by calling:

Code: Pascal [Select][+]

procedure EnableOpenCL(platform_id: cl_platform_id; device_id: cl_device_id);

Batch Updates Support
Batch updates can be enabled/disabled via:

Code: Pascal [Select][+]

procedure SetBatchUpdate(pBatchUpdate: boolean);

New Normalization layers:
* TNNetLocalResponseNorm2D: (input/output: 2D or 3D).
* TNNetLocalResponseNormDepth: (input/output: 2D or 3D).

New Concatenation / Splitting layers:
* TNNetDeepConcat (input/output: 1D, 2D or 3D) - Concatenates independent NN branches into the Depth axis.
* TNNetSplitChannels (input: 1D, 2D or 3D / output: 1D, 2D or 3D). Splits layers/channels from input. This is useful for creating divergent NN branches.

New ready to use data augmentation methods from TVolume:
* procedure FlipX();
* procedure FlipY();
* procedure CopyCropping(Original: TVolume; StartX, StartY, pSizeX, pSizeY: integer);
* procedure CopyResizing(Original: TVolume; NewSizeX, NewSizeY: integer);
* procedure AddGaussianNoise(pMul: TNeuralFloat);
* procedure AddSaltAndPepper(pNum: integer; pSalt: integer = 2; pPepper: integer = -2);

Final random thought:
As pascal is easy to teach and learn (in my opinion), pascal might be perfect for teaching/learning neural networks.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on July 31, 2018, 08:57:54 am

Quote from: schuler on July 31, 2018, 05:21:45 am

As pascal is easy to teach and learn (in my opinion), pascal might be perfect for teaching/learning neural networks.

Actually, that has been the case since a very long time, in the early 80's it was pretty much the standard.
That's the way I learned some of the basics and taught some of the basics at university back in the 80's.
(Basically I learned it, wrote examples and taught it - almost - in the same week, writing the curriculum on the go... <hmm..> O:-) :-[ )
We used it in teaching for problems like Traveling Salesman and Knapsack. Lot's of fun. I must still have some line printed code from those days...The green/white line stuff with holes on both sides.
Let's see if I can put it into your way more advanced software form... Great job.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Trenatos on July 31, 2018, 08:18:43 pm

This is very cool to see, many thanks for sharing!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 02, 2018, 11:30:33 pm

Hello,
I got a couple of private questions from forum members and decided to post here a general reply as it might help others.

This link gives a general view of CNNs:
http://cs231n.github.io/convolutional-networks/

Above link mentions some well known NN architectures. I would like to show here how to implement some of these well known architectures with CAI:

Yann LeCun LeNet-5:

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(28, 28, 1) );
NN.AddLayer( TNNetConvolution.Create(6, 5, 0, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
NN.AddLayer( TNNetConvolution.Create(16, 5, 0, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) ); 
NN.AddLayer( TNNetFullConnect.Create(120) );
NN.AddLayer( TNNetFullConnect.Create(84) );
NN.AddLayer( TNNetFullConnectLinear.Create(10) );
NN.AddLayer( TNNetSoftMax.Create() );

AlexNet can be implemented as follows:

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(227, 227, 3) );
 
//Conv1 + ReLU
NN.AddLayer( TNNetConvolutionReLU.Create(96, 11, 0, 4) );
NN.AddLayer( TNNetMaxPool.Create(3,2) );
NN.AddLayer( TNNetLocalResponseNormDepth.Create(5) );
 
//Conv2 + ReLU
NN.AddLayer( TNNetConvolutionReLU.Create(256, 5, 2, 1) );
NN.AddLayer( TNNetMaxPool.Create(3,2) );
NN.AddLayer( TNNetLocalResponseNormDepth.Create(5) );
 
//Conv3,4,5 + ReLU
NN.AddLayer( TNNetConvolutionReLU.Create(398, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(398, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(256, 3, 1, 1) );
NN.AddLayer( TNNetMaxPool.Create(3,2) );
 
// Dropouts and Dense Layers
NN.AddLayer( TNNetDropout.Create(0.5) );
NN.AddLayer( TNNetFullConnectReLU.Create(4096) );
NN.AddLayer( TNNetDropout.Create(0.5) );
NN.AddLayer( TNNetFullConnectReLU.Create(4096) );
NN.AddLayer( TNNetFullConnectLinear.Create(1000) );
NN.AddLayer( TNNetSoftMax.Create() );

Above implementation is based on:
https://medium.com/@smallfishbigsea/a-walk-through-of-alexnet-6cbd137a5637

The VGGNet has an interesting architecture. It always uses 3x3 filters making its implementation easy to understand:

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(224, 224, 3) );
NN.AddLayer( TNNetConvolutionReLU.Create(64, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(64, 3, 1, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
//112x112x64
NN.AddLayer( TNNetConvolutionReLU.Create(128, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(128, 3, 1, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
//56x56x128
NN.AddLayer( TNNetConvolutionReLU.Create(256, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(256, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(256, 3, 1, 1) );
//28x28x256
NN.AddLayer( TNNetMaxPool.Create(2) );
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
//14x14x512
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetConvolutionReLU.Create(512, 3, 1, 1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
//7x7x512
NN.AddLayer( TNNetFullConnectReLU.Create(4096) );
NN.AddLayer( TNNetFullConnectReLU.Create(4096) );
NN.AddLayer( TNNetFullConnectLinear.Create(1000) );
NN.AddLayer( TNNetSoftMax.Create() );

Above architectures have been added to the API as follows:

Code: Pascal [Select][+]

  THistoricalNets = class(TNNet)
    public
      procedure AddLeCunLeNet5(IncludeInput: boolean);
      procedure AddAlexNet(IncludeInput: boolean);
      procedure AddVGGNet(IncludeInput: boolean);
  end;

Hope it helps and happy pascal coding!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on March 13, 2019, 10:19:49 pm

Hello,
This message is intended to be read by whom loves pascal and neural networks.

After a long time, here we go with a new post. This time, instead of reporting new features, this post is more about how to do things than just listing new features. I would like to show how to replicate examples made for Keras and TensorFlow with CAI. As shown in the paper https://arxiv.org/abs/1412.6806 (https://arxiv.org/abs/1412.6806), it’s possible to create efficient neural networks with just convolutional layers. Therefore, we’ll start from convolutional layers. There is a simple example from Keras located at https://keras.io/examples/cifar10_cnn/ (https://keras.io/examples/cifar10_cnn/). From this code, we can find convolutional layers being added with:

Code: Pascal [Select][+]

model.add(Conv2D(32, (3, 3), padding='same', input_shape=x_train.shape[1:]))
model.add(Activation('relu')) 

The above can be done with CAI with just one line of code:

Code: Pascal [Select][+]

NN.AddLayer( TNNetConvolutionReLU.Create({neurons}32, {featuresize}3, {padding}1, {stride}1) ); 

It’s also possible to add it with 2 lines of code but it’s not recommended as it’s not efficient in CAI:

Code: Pascal [Select][+]

NN.AddLayer( TNNetConvolutionLinear.Create({neurons}32, {featuresize}3, {padding}1, {stride}1) ); 
NN.AddLayer( TNNetReLU.Create() ); 

This is the MaxPooling from Keras:

Code: Pascal [Select][+]

model.add(MaxPooling2D(pool_size=(2, 2))) 

And this is the equivalent from CAI:

Code: Pascal [Select][+]

NN.AddLayer( TNNetMaxPool.Create(2) ); 

The full convolutional neural network can be defined in plain Free Pascal with:

Code: Pascal [Select][+]

NN.AddLayer( TNNetInput.Create(32,32,iInputDepth) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}32, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}32, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetMaxPool.Create(2) );
NN.AddLayer( TNNetFullConnectReLU.Create({neurons}512) );
NN.AddLayer( TNNetFullConnectLinear.Create(NumClasses) );
NN.AddLayer( TNNetSoftMax.Create() ); 

As you can see above, there aren’t any dropout layers in the Free Pascal implementation. This was done intentionally as I prefer using data augmentation instead of dropout layers. The data augmentation technique used here is very simple:

We crop the image and then stretch back to the original size.
We randomly flip/flop the image.
We randomly make the image gray.

The above can be easily done in Free Pascal:

Code: Pascal [Select][+]

// Crop and Stretch
CropSizeX := random(9);
CropSizeY := random(9);
ImgInputCp.CopyCropping(ImgVolumes[ImgIdx], random(CropSizeX), random(CropSizeY),ImgVolumes[ImgIdx].SizeX-CropSizeX, ImgVolumes[ImgIdx].SizeY-CropSizeY);
ImgInput.CopyResizing(ImgInputCp, ImgVolumes[ImgIdx].SizeX, ImgVolumes[ImgIdx].SizeY);
// Flip the Image
if Random(1000) > 500 then
begin
  ImgInput.FlipX();
end;
// Make it Gray
if (Random(1000) > 750) then
begin
  ImgInput.MakeGray(color_encoding);
end;

Running the above in CAI gives really impressive results (they seem to me better than on Keras itself…). In the case that you would like to try by yourself, you can get this Lazarus Project:
https://sourceforge.net/p/cai/svncode/724/tree/trunk/lazarus/experiments/visualCifar10BatchUpdate/ (https://sourceforge.net/p/cai/svncode/724/tree/trunk/lazarus/experiments/visualCifar10BatchUpdate/)

When you compile and run it, select “2 – Keras inspired” in “Algo” dropdown, check “Multiple Samples at Validation” and then start it. Although this example looks simple, it’s no numerical toy. It actually has more than 2.1 million neuronal weights/connections!

I would like to also show you another example ported to Free Pascal. This time it’s an example from TensorFlow: https://github.com/tensorflow/models/blob/master/tutorials/image/cifar10/cifar10.py (https://github.com/tensorflow/models/blob/master/tutorials/image/cifar10/cifar10.py)
The neuronal structure can be found in the “def inference(images)”. This structure was ported to Free Pascal as follows:

Code: Pascal [Select][+]

NN.AddLayer( TNNetInput.Create(24,24,3) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}5, {padding}2, {stride}1) );
NN.AddLayer( TNNetMaxPool.Create({size}3,{stride}2) );
NN.AddLayer( TNNetLocalResponseNormDepth.Create({diameter}9) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}5, {padding}2, {stride}1) );
NN.AddLayer( TNNetLocalResponseNormDepth.Create({diameter}9) );
NN.AddLayer( TNNetMaxPool.Create({size}3,{stride}2) );
NN.AddLayer( TNNetFullConnectReLU.Create({neurons}384) );
NN.AddLayer( TNNetFullConnectReLU.Create({neurons}192) ); 
NN.AddLayer( TNNetFullConnectLinear.Create(NumClasses) );
NN.AddLayer( TNNetSoftMax.Create() ); 

Running it in CAI/Free Pascal, you’ll get similar results (classification accuracy) to results reported in TensorFlow site. In the case that you would like to run this by yourself, select “3 – TF inspired”, “Multiple Samples at Validation” and “24x24 image cropping”. The data augmentation technique found in this example is a bit different than before: we are cropping the original image into 24x24 images at random places. We are also flopping it and making it sometimes gray.

:-) I wish everyone happy pascal neural networking! :-)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: avra on March 13, 2019, 10:55:30 pm

Quote from: schuler on March 13, 2019, 10:19:49 pm

this post is more about how to do things than just listing new features

Much appreciated. Thank you!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on April 21, 2019, 08:50:10 am

@Avra,
I'm starting to believe that most of examples showing how to use CAI are too advanced.

Just added 2 entry level examples. The first shows how to learn boolean functions XOR, AND and OR:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/experiments/supersimple/

The next is the same as above with Pearson Correlation:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/experiments/supersimplecorrelation/

:) I hope it helps and happy pascal coding to everyone. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 29, 2019, 12:31:10 pm

:) Hello :)

Instead of reporting new features (there are plenty of new neuronal layer types), I'll report here a test. For the past 2 years, I've probably spent more time testing than coding. The test is simple: using a DenseNet like architecture with 1M trainable parameters, normalization, L2, cyclical learning rate and weight averaging, how precise can be the CIFAR-10 image classification accuracy?

The experiment was done with SVN revision 907:
https://sourceforge.net/p/cai/svncode/907/tree/trunk/lazarus/experiments/visualCifar10BatchUpdate/

In the case that you intend to reproduce it, you should use exactly the same parameters found in the attached screenshot expect for "Logical Threads" and "Physical Threads" (you should use your CPU core count instead). This experiment was run in the slowest (and cheapest) machine I have access to.

At my end, I got about 94% CIFAR-10 classification accuracy.

:) Wish everyone happy pascal coding. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 29, 2019, 06:37:32 pm

:) Hello :)
I promise that I'll try to not to post anything after this for some time.

I've just done an experiment comparing CAI with Keras/TF. CAI and TF were both compiled with AVX instruction set (no AVX2 nor AVX512). This is the FPC code:

Code: Pascal [Select][+]

NN.AddLayer( TNNetInput.Create(32, 32, 3) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}5, {padding}0, {stride}1) );
NN.AddLayer( TNNetMaxPool.Create(4) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetConvolutionReLU.Create({neurons}64, {featuresize}3, {padding}1, {stride}1) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectLinear.Create(10) );
NN.AddLayer( TNNetSoftMax.Create() );

This is the equivalent Keras code:

Code: Python [Select][+]

model = Sequential()
model.add(Conv2D(64, (5, 5), padding='valid',
                 input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(4, 4)))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(32))
model.add(Activation('relu'))
model.add(Dense(32))
model.add(Activation('relu'))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

Using the very same machine (google cloud with 16 virtual cores and no gpu), I run both models. This is the output from Keras showing that each epoch is taking 20 seconds to be computed:

Quote

Epoch 24/350
312/312 [==============================] - 20s 64ms/step - loss: 1.3232 - acc: 0.5270 - val_loss: 1.2861 - val_acc: 0.5369
Epoch 25/350
312/312 [==============================] - 20s 63ms/step - loss: 1.3297 - acc: 0.5226 - val_loss: 1.2873 - val_acc: 0.5361
Epoch 26/350
312/312 [==============================] - 20s 64ms/step - loss: 1.3144 - acc: 0.5295 - val_loss: 1.2868 - val_acc: 0.5378
Epoch 27/350
312/312 [==============================] - 20s 63ms/step - loss: 1.3156 - acc: 0.5284 - val_loss: 1.2915 - val_acc: 0.5356
Epoch 28/350
312/312 [==============================] - 20s 63ms/step - loss: 1.3150 - acc: 0.5281 - val_loss: 1.2776 - val_acc: 0.5397
Epoch 29/350
312/312 [==============================] - 20s 64ms/step - loss: 1.3057 - acc: 0.5332 - val_loss: 1.2737 - val_acc: 0.5415

In CAI, each epoch is computed in about 16 or 17 seconds as per log (time in seconds is incremental and is the last field below):

Quote

31,0.7509,0.7379,0.6683,0.7780,0.6491,0.6031,0.0010000,523
32,0.7565,0.6392,0.6981,0.7679,0.6725,0.6202,0.0010000,539
33,0.7482,0.8476,0.7243,0.7678,0.6851,0.6034,0.0010000,555
34,0.7653,0.5730,0.5781,0.7665,0.6971,0.6179,0.0010000,571
35,0.7668,0.7007,0.6337,0.7800,0.6464,0.5884,0.0010000,588

Am I typing that CAI will always perform better? Absolutely not. CAI is likely to be faster on some models and slower on other models. I haven't tested enough to be able to indicate when it will be faster or slower.

:) Anyway, wish everyone happy pascal coding. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 02, 2019, 10:33:01 pm

:) Hello :),
I'm thinking about making a change that will brake existing code but it will make this library to look a lot better. Before I do this change, I would like to have some feedback please.

The change is: renaming library units and then creating a package (version 1.0).

All library units will start with "Neural". So, these are some renaming examples:
uconvolutionalneuralnetwork -> NeuralNet
uvolume -> NeuralVolume
ueasyopencl -> NeuralOpenCl
uevolutionary -> NeuralEvolution
ucifar10 -> NeuralDatasets
ubyteprediction -> NeuralPrediction

What do you think? Please feel free to ask for other changes.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: avra on September 03, 2019, 12:11:36 am

Quote from: schuler on September 02, 2019, 10:33:01 pm

I'm thinking about making a change that will brake existing code but it will make this library to look a lot better. Before I do this change, I would like to have some feedback please.

I do not mind suggested changes, but it would be nice if thread and other examples get corrected so that they still work.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 23, 2019, 04:06:10 am

... I'm approaching 1000 commits ... And this is the "new SVN repository" since I moved from CVS to SVN many years ago. I'm getting old... Might be time to move to git ... Anyway ...

@avra
A new Neural API version with major changes/improvements is now under construction:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/neural/

To whom may concern,
All unit names now start by "neural". All examples have already been fixed. The old "libs" folder has been kept for backward compatibility so the next version won't brake any existing code.

Looking for sources of inspiration, Keras has an useful "fit" method:
https://keras.rstudio.com/reference/fit.html

"fit" like methods and classes are now under construction for CAI! YAY! neuralfit.pas has already more than 700 lines of source code.

Soon, I intend to have deep learning examples ready to run on Google Colab with Lazarus/FPC/CAI NEURAL API.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on September 23, 2019, 08:46:11 am

Better and better! I will enjoy playing with your code this week. TNX!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 23, 2019, 11:58:25 am

@Thaddy,
It's not ready yet! I intend to have the first version ready in 2 weeks.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on September 23, 2019, 12:34:44 pm

Ok. I have some patience... :D

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 23, 2019, 05:46:08 pm

Thank you for the interest! Makes life happier!

This is an example about the work in progress:
https://github.com/joaopauloschuler/neural-api#how-does-the-code-look-like-for-a-cifar-10-classification-example

A good introductory example is:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleImageClassifier

:) Wishing everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: nouzi on October 03, 2019, 02:17:40 pm

Good job 'bravo '

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 08, 2019, 12:46:20 pm

@nouzi, thank you!

@Thaddy, I think that the new CAI Neural API subproject currently placed at https://github.com/joaopauloschuler/neural-api is ready for testing.

In the case that no major bug is found on the near future, I'll call it CAI Neural API v1.0.

:) wish everyone happy pascal solution construction :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 19, 2019, 07:08:25 pm

:) Hi :)
To whom is interested in computer vision and has never had a look at MNIST (http://yann.lecun.com/exdb/mnist/ (http://yann.lecun.com/exdb/mnist/)) or Fashion MNIST (https://github.com/zalandoresearch/fashion-mnist (https://github.com/zalandoresearch/fashion-mnist)), I highly recommend clicking on these links. MNIST contains images with handwritten digits while Fashion MNIST contains 10 clothing classes.

At this moment, these datasets have API support:

CIFAR-10
CIFAR-100
MNIST
FASHION MNIST

Source code examples for MNIST and Fashion MNIST have just been added:

https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleMNist
https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleFashionMNIST

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: avra on October 20, 2019, 06:12:42 pm

Nice! Things are getting better and better. :P

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: nouzi on October 20, 2019, 08:40:51 pm

@schuler

you are a hero

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 26, 2019, 01:39:47 pm

@avra and @nouzi,
Thank you for your support.

Just asked to include a Free Pascal benchmark into the Fashion MNIST benchmark list:
https://github.com/zalandoresearch/fashion-mnist/issues/153

Got 94% accuracy with just 174K parameters. In the benchmark list (https://github.com/zalandoresearch/fashion-mnist), you'll find results with 26M parameters...

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on October 26, 2019, 01:42:36 pm

That is extremely good!!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 05, 2019, 12:42:22 am

:) ,
I would like to ask everyone: what is the feature or example that you would like to be added? Feel free to ask. In the case that you ask for something that I can do quickly or I already have it half baked, I might just spend some minutes typing code.

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: MathMan on November 05, 2019, 01:43:33 pm

Quote from: schuler on November 05, 2019, 12:42:22 am

:) ,
I would like to ask everyone: what is the feature or example that you would like to be added? Feel free to ask. In the case that you ask for something that I can do quickly or I already have it half baked, I might just spend some minutes typing code.

:) wish everyone happy pascal coding :)

Before I answer I have to admit that I neither played with CAI, nor will I have time to do so in the foreseeable future. So if you don't mind suggestions from a purely theoretical perspective

* (fully) binarized neural network - (F)BNN
* bayesian confidence propagation neural network - BCPNN

Both to me look, from what I have read, like interesting approaches addressing topics like reduced power consumption, inference capabilities, etc. But, also from what I read, both might be major developments.

Cheers,
MathMan

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: circular on November 05, 2019, 03:54:06 pm

That looks amazing, being able to program neural networks in Pascal. I wonder if that can be useful in LazPaint? Some image processing features might be more efficient with neural networks.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on November 05, 2019, 05:29:11 pm

Quote from: circular on November 05, 2019, 03:54:06 pm

That looks amazing, being able to program neural networks in Pascal. I wonder if that can be useful in LazPaint? Some image processing features might be more efficient with neural networks.

Well, one of the examples is about recognition, but it needs to learn. That process is quite slow (hours and days even months), but the code renders quite amazing results after that, and once established -to your specification! you have to stop learning! at some point - it is fast.
And well written pure pascal.
What is maybe missing is just some pre-processing along the lines of my fuzzy code (on this forum, Zadeh logic) that would help a bit in some scenario's as a filter, but is actually a completely different subject and is AI but not machine learning AI, what schuler's library is all about.
If you are interested: the code is really good and you pick it up quite fast if you have just a tiny bit of background. It is one of my favorite Freepascal projects.

Example: if you feed it enough number plate examples ( say, a million, from all angles ) and you run for some months, you have a fast number plate recognition software with high probability of being correct. But the fun is is really to use it on comic book examples and let it filter out Daffy Duck...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: zulof on November 12, 2019, 11:05:27 am

Could/Should this be included in online package manager?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: m.abudrais on November 22, 2019, 08:14:54 am

thank you for making this good library.
I am absolute beginner in NN.
i run the CIFAR-10 example (for 50 epochs only) and the SimpleImageClassifier.nn is generated.
then I tried to Classify some image by this code:

Code: Pascal [Select][+]

procedure TForm1.Button1Click(Sender: TObject);
var
  NN: THistoricalNets;
  O1: array of TNeuralFloat;
  pOutPut, pInput: TNNetVolume;
  k, y, x: integer;
  OK: extended;
  img: TTinyImage;
begin
  Image1.Picture.LoadFromFile('C:\Users\moh\Desktop\dog1.png');
  NN := THistoricalNets.Create();
  NN.AddLayer([TNNetInput.Create(32, 32, 3),
    TNNetConvolutionLinear.Create(64, 5, 2, 1, 1).InitBasicPatterns(),
    TNNetMaxPool.Create(4), TNNetConvolutionReLU.Create(64, 3, 1, 1, 1),
    TNNetConvolutionReLU.Create(64, 3, 1, 1, 1),
    TNNetConvolutionReLU.Create(64, 3, 1, 1, 1),
    TNNetConvolutionReLU.Create(64, 3, 1, 1, 1), TNNetDropout.Create(0.5),
    TNNetMaxPool.Create(2), TNNetFullConnectLinear.Create(10),
    TNNetSoftMax.Create()]);
  NN.LoadDataFromFile('SimpleImageClassifier.nn');
  for y := 0 to 31 do
  begin
    for x := 0 to 31 do
    begin
      img.B[y, x] := Red(Image1.Canvas.Pixels[y, x]);
      img.g[y, x] := Green(Image1.Canvas.Pixels[y, x]);
      img.B[y, x] := Blue(Image1.Canvas.Pixels[y, x]);
    end;
  end;
  pInput := TNNetVolume.Create(32, 32, 3, 1);
  pOutPut := TNNetVolume.Create(10, 1, 1, 1);
  LoadTinyImageIntoNNetVolume(img, pInput);
  NN.Compute(pInput);
  NN.GetOutput(pOutPut);
  OK := 0.0;
  for k := 0 to 9 do
  begin
    OK := OK + pOutPut.Raw[k];
    WriteLn(pOutPut.Raw[k]);
  end;
  WriteLn();
  WriteLn(OK);
end;   

i tried image for a dog and car from the same data set but i always get 1 in pOutPut.Raw[0]!
can you please add example to show how to use Classifier after training.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 25, 2019, 04:56:02 pm

Please forgive me for taking so long to reply.

First of all, you can completely remove "AddLayer" call as CAI stores both architecture and weights into the same nn file.

Then, you need to add RgbImgToNeuronalInput as follows:

Code: Pascal [Select][+]

LoadTinyImageIntoNNetVolume(img, pInput);
pInput.RgbImgToNeuronalInput(csEncodeRGB);

If the above doesn't work, I'll try to compile at my end.

As you have SoftMax, you can print output class probabilities with:

Code: Pascal [Select][+]

pOutPut.Print();

You can get the inferred output class with:

Code: Pascal [Select][+]

WriteLn('Inferred Class: ', csTinyImageLabel[pOutPut.GetClass()]);

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: m.abudrais on November 25, 2019, 07:40:52 pm

thank you for replying.

Quote

First of all, you can completely remove "AddLayer" call as CAI stores both architecture and weights into the same nn file.

I used the wrong function LoadDataFromFile, the correct one is LoadFromFile.
now it works :).
this is the correct code if any one is interesting

Code: Pascal [Select][+]

procedure TForm1.Button1Click(Sender: TObject);
var
  NN: THistoricalNets;
  O1: array of TNeuralFloat;
  pOutPut, pInput: TNNetVolume;
  k, y, x: integer;
  OK: extended;
  img: TTinyImage;
begin
  NN := THistoricalNets.Create();
  NN.LoadFromFile('SimpleImageClassifier.nn');
  for y := 0 to 31 do
  begin
    for x := 0 to 31 do
    begin
      img.R[y, x] := Red(Image1.Canvas.Pixels[y, x]);
      img.g[y, x] := Green(Image1.Canvas.Pixels[y, x]);
      img.B[y, x] := Blue(Image1.Canvas.Pixels[y, x]);
    end;
  end;
  pInput := TNNetVolume.Create(32, 32, 3, 1);
  pOutPut := TNNetVolume.Create(10, 1, 1, 1);
  LoadTinyImageIntoNNetVolume(img, pInput);
  pInput.RgbImgToNeuronalInput(csEncodeRGB);
  NN.Compute(pInput);
 
  WriteLn('Inferred Class: ', csTinyImageLabel[pOutPut.GetClass()]);   
 

I moved the image loading to TForm1.FormCreate,the image don't load when I load it TForm1.Button1Click functon !!.
thank you again for making this good library.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 25, 2019, 09:55:28 pm

@m.abudrais
Are you placing your code on a public repo?

Anyway, just realized that you might be able to replace both for loops by using the neuralvolumev https://github.com/joaopauloschuler/neural-api/blob/master/neural/neuralvolumev.pas (https://github.com/joaopauloschuler/neural-api/blob/master/neural/neuralvolumev.pas) unit:

Code: Pascal [Select][+]

/// Loads a Picture into a Volume
procedure LoadPictureIntoVolume(LocalPicture: TPicture; Vol:TNNetVolume); {$IFDEF Release} inline; {$ENDIF}
 
/// Loads a Bitmat into a Volume
procedure LoadBitmapIntoVolume(LocalBitmap: TBitmap; Vol:TNNetVolume);

The code will look like:

Code: Pascal [Select][+]

LoadPictureIntoVolume(Image1.Picture, pInput);
pInput.RgbImgToNeuronalInput(csEncodeRGB);

In the case that the input image isn't 32x32, you can resize it (via copying) with:

Code: Pascal [Select][+]

TVolume.CopyResizing(Original: TVolume; NewSizeX, NewSizeY: integer);

Last idea: given that you have a trained NN, you could call this:

Code: Pascal [Select][+]

procedure TNeuralImageFit.ClassifyImage(pNN: TNNet; pImgInput, pOutput: TNNetVolume);

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: m.abudrais on November 26, 2019, 04:58:28 pm

Quote

Are you placing your code on a public repo?

I have just uploaded the code to:
https://github.com/mabudrais/CAI-NEURAL-API-Test (https://github.com/mabudrais/CAI-NEURAL-API-Test)
your library has many useful functions, I think more examples may be added to show that.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 06, 2019, 06:57:25 pm

:) Hello :)

It's usually very hard to understand neuron by neuron how a neural network dedicated to image classification internally works. One technique used to help with the understanding about what individual neurons represent is called Gradient Ascent. You can find more about gradient ascent at http://yosinski.com/deepvis .

In this technique, an arbitrary neuron is required to activate and then the same backpropagation method used for learning is applied to an input image producing an image that this neuron expects to see. I'm happy to inform that we have a working example purely coded in FPC/Lazarus at: https://github.com/joaopauloschuler/neural-api/tree/master/examples/GradientAscent

To be able to run this example, you'll need to load an already trained neural network file and then select the layer you intend to visualise. I've attached an example from the last layer (can you see any pattern?) of a neural network trained to classify CIFAR-10 dataset. Can you find patterns for horse, deer, ship, car or truck?

:) Long life to Pascal :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 21, 2019, 01:28:15 pm

Just found this thread, it seems to be a very interesting topic!
Thanks to all the hard work you did!

While I still prefer python/tensorflow/keras approach to train CNNs, I'd like to use the resulting CNN in my Delphi project to help me sorting the sounds. Do you have any method to export the weights from a Keras model and load it to your library so I can do that NN magic in pure pascal without any additional DLLs?

Thank you in advance!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 23, 2019, 06:29:55 pm

Hello agorka! Glad that you like it.

Supporting Keras file format isn't in the near horizon. At this moment, I'm polishing a super advanced example about producing artificial art: https://github.com/joaopauloschuler/neural-api/tree/master/examples/VisualGAN

Other feature requests are listed as github issues.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 24, 2019, 07:38:50 am

Wow! I couldn't imagine this could be done in Lazarus without such heavy frameworks like tensorflow or pytorch...
Still, my question is, what file formats could be used to load existing weights to the convolution layers?
I could find a converter or make my own export function from Python...
Is there any json, csv, any other format?

Thanks!
Oleg.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: avra on December 24, 2019, 08:17:21 am

Quote from: agorka on December 24, 2019, 07:38:50 am

Is there any json, csv, any other format?

It would also help for MatLab.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 24, 2019, 11:35:49 am

@agorka,
In the case that you try loading weights by yourself, here we go:

A neural network TNNet has Layers:

Code: Pascal [Select][+]

property Layers: TNNetLayerList read FLayers;

Each Layer (TNNetLayer) has Neurons:

Code: Pascal [Select][+]

property Neurons: TNNetNeuronList read FNeurons;

Each Neuron (TNNetNeuron) has weights:

Code: Pascal [Select][+]

property Weights: TNNetVolume read FWeights

Accessing weights from the second neuron from the second layer will look like this:

Code: Pascal [Select][+]

NN.Layers[1].Neurons[1].Weights

As you can see, Weights is a TNNetVolume.

TNNetVolume has 2 methods that might interest you:

Code: Pascal [Select][+]

    // load and save functions
    function SaveToString(): string;
    procedure LoadFromString(strData: string);

Having a look at the implementation, you'll find that weights are saved as a ";" separated string. The first 4 values are structural: version, SizeX, SizeY and depth. The following values are the actual weights.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 24, 2019, 11:47:46 am

Quote

Wow! I couldn't imagine this could be done in Lazarus without such heavy frameworks like tensorflow or pytorch

Lazarus/FPC can do impressive things such as compiling to other targets (android and raspberry).

Artificial Art can be useful for texture generation on an artificial environment (such as computer games) as long as you have a pre-trained network. I'll eventually add an example creating very large images (and not just 32x32 pixels).

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 24, 2019, 10:48:38 pm

Thank you, I'm getting it running with 1 convolution layer.
One thing I stuck with was weights not updating after loading:
NN.Layers.Neurons[N].LoadFromString(SL[N]);
I had to add
NN.Layers.AfterWeightUpdate;
and unprivate this AfterWeightUpdate procedure.
Probably it should be called from LoadFromString function to avoid this inconsistencies.

Thank you for your hard work! I really hope to port my keras network to Pascal tomorrow! =)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 24, 2019, 11:09:42 pm

Sorry, I've run into a problem again.
My network now is as simple as this:

TNNetInput.Create(512, 512, 1),
TNNetConvolutionReLU.Create(24, 7, 0, 1, 0).InitUniform(0),
TNNetConvolutionReLU.Create(1, 5, 0, 1, 0).InitUniform(0)

Though, I run out of memory in
procedure TNNetConvolutionBase.SetPrevLayer(pPrevLayer: TNNetLayer);
when it tries to allocate 502x502x5x5x24 Volume, which is ~600MB.
As I understand, it's needed to optimize convolution calculation, but in my 32 bit application I just can't allocate that much, and my real network needs many more layers.
Is there any way to workaround this problem?
For me it's better to run it slowly than not to run at all. =)

Thanks!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 25, 2019, 02:15:01 am

Good idea about AfterWeightUpdate... Thank you.

Do you have freedom to try something like this (a bigger stride compensated by more neurons in the first convolution)?

Code: Pascal [Select][+]

      TNNetInput.Create(512, 512, 1),
      TNNetConvolutionReLU.Create({neurons=}128, {FeatureSize=}7, {Padding=)0, {Stride=}7, {SupressBias=}0),
      ...

If not, I'll eventually code a turn around (slower with less memory).

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 26, 2019, 10:45:41 am

Thank you!

I reduced the input size to 256x256 and was able to train the net to similar precision.
With this size, it's still enough memory to create the network in Delphi.
Now I'm adding the 3rd layer (MaxPooling) and getting another inconsistency with Keras:

My layer is

Code: Pascal [Select][+]

      TNNetMaxPool.Create(4, 2)
 

With input size 246 it produces output size 123.
Keras produces size 122.

Code: Pascal [Select][+]

function TNNetPoolBase.CalcOutputSize(pInputSize: integer): integer;
begin
  Result := (pInputSize + FPadding) div FStride;
  if ((pInputSize + FPadding) mod FStride > 0) then Inc(Result);
end;
 

I mostly find this formula for output size:
Out=(W−F+2P)/S+1

Could this be corrected to improve compatibility with other neural network frameworks?

Thanks for your continous help =)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 27, 2019, 01:32:35 am

Hello agorka,
Thank you for bringing this to my attention.

I've just created a new layer called TNNetMaxPoolPortable with your requested behavior. So, if you get the latest github version, you'll be able to replace your existing maxpools by this new one.

The original implementation has motivation behind it. Assume an 11x11 input followed by a 2x2 max pooling. In this scenario, CAI will output a 6x6 map while the portable implementation will output a 5x5 map ((11-2+2*0) div 2) + 1=5 completely missing last 11th row and column with loss of information. Therefore, I still like CAI's original implementation.

May I ask you what is your target machine? Does it have AVX or OpenCL?

In the case that you have AVX, are you curious to benchmark CAI against another API on your own environment? If yes, I can send you equivalent implementations on both APIs.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 28, 2019, 09:03:57 am

Thank you!
I'm updating right now.
About the performance, I mostly care about compatibility right now, so performance doesn't matter much for me. I might return to the performance after I'll have everything working. =)

BTW, I know it's strange but I still use Delphi 7 in parallel with Lazarus, so I have to change some parts of your code to make it compatible.
I don't know if you'd like to have it compatible with D7, but still here's the changes:

Code: Pascal [Select][+]

procedure TVolume.FillForIdx(c: T; const aIdx: array of integer);
var
  Idx: integer;
begin
  for Idx in aIdx do
    FData[Idx] := c;
end;

I change to

Code: Pascal [Select][+]

procedure TVolume.FillForIdx(c: T; const aIdx: array of integer);
var
  I, Idx: integer;
begin
  for I:=0 to Length(aIdx)-1 do
  begin
    Idx:=aIdx[i]; 
    FData[Idx] := c;
  end;
end;

I know it's not that beautiful but hopefully not less efficient. =)

Also in CopyChannels:

Code: Pascal [Select][+]

      for i:=0 to Length(aChannels)-1 do
      begin
        InputDepth:=aChannels[i];
        Self[X, Y, OutputDepth] := Original[X, Y, InputDepth];
        Inc(OutputDepth);
      end;

I'll write back as soon as I get my FFT data into the net and compare it with Keras result. I had some mismatches when processing some test data but probably it was because of different MaxPooling layer...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 29, 2019, 11:46:50 pm

@agorka,
You aren't the first to ask for D7 support. At the other extreme, at this moment, I'm retesting AVX512 support with FPC trunk...

I believe that if you open source your work about porting trained networks to pascal this would benefit the pascal user base.

In Keras, do your convolutional layers have biases? If yes, you'll need to bring too.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on December 30, 2019, 11:31:01 am

Oh, great!
D7 support requires very little changes, such as removing inline directives, removing StrictDelimiter and a few lines more.

I finally got it kinda working. Not a very precise classification but at least something!
So of course I can share my code but it's far from ideal.
My convolutions have biases of course, but I thought it's already works. Still I need to compare Keras and Pascal predictions to check if there's any differences.
Also I removed all batch normalization layers from the network to avoid possible differences.
And my Keras network had Flatten layer which I didn't find in your library, but it seems that Dense layer work right after Convolution layer (but I'm not sure that the order of weights get transferred correctly). Removing flatten layer in Keras produces wrong dimension after Dense layer, so maybe you can implement similar Flatten layer, or confirm it should work correctly right away?

Here goes Python Keras export part:

Code: Pascal [Select][+]

def SaveModelToTXT(model, filename):
    print("Saving model to", filename, "...")
    print("Found ", len(model.layers), " layers")
    for i in range(len(model.layers)):
        layer=model.layers[i]
        layerFileName = filename+"_layer"+str(i)+".csv"
        print(layer.name, ' to ', layerFileName)
        if isinstance(layer, keras.layers.Conv2D):
            with open(layerFileName, "w") as f:
                w=layer.get_weights()
                for filter in range(w[0].shape[3]):
                    bias=w[1][filter]
                    weights=w[0][:,:,:,filter].flatten()
                    title = "Layer" +str(i)+"  Filter "+str(filter)+"  WShape:"+str(w[0].shape)
                    #print(title)
                    dimensions='1;'+str(w[0].shape[0])+';'+str(w[0].shape[1])+';'+str(w[0].shape[2])
                    weights=["%.6f" % number for number in weights]
                    weights=';'.join(weights)
                    f.write(str(bias)+']'+dimensions+';'+weights+"\n")
        if isinstance(layer, keras.layers.Dense):
            with open(filename+"_layer"+str(i)+".csv", "w") as f:
                w=layer.get_weights()
                print(w[0].shape)
                for filter in range(w[0].shape[1]):
                    bias=w[1][filter]
                    weights=w[0][:,filter].flatten()
                    title = "Layer" +str(i)+"  Filter "+str(filter)+"  WShape:"+str(w[0].shape)
                    dimensions='1;'+str(w[0].shape[0])+';1;1'
                    weights=["%.6f" % number for number in weights]
                    weights=';'.join(weights)
                    f.write(str(bias)+']'+dimensions+';'+weights+"\n")
    print("Done!")
SaveModelToTXT(model, modelPath+"ModelNetworkWeights")

And here's pascal reading of weights:

Code: Pascal [Select][+]

procedure TSampleCNN.LoadWeights(FN: String);
var I, N : Integer;
    S : String;
    SL : TStringList;
    Size, SX, SY : Integer;
    LayerFN : TFileNameUnicodeString;
begin
For I:=0 to NN.Layers.Count-1 do
  Begin
  If (NN.Layers[i] is TNNetConvolutionLinear) or
     (NN.Layers[i] is TNNetConvolutionReLU) or
     (NN.Layers[i] is TNNetFullConnectLinear) then
     Begin
     Try
     SL:=TStringList.Create;
     LayerFN:=FN+'_layer'+IntToStr(I)+'.csv';
     If not FileExistsUTF8(LayerFN) then
        Begin
        ShowMessage('File not found: '+LayerFN);
        Continue
        End;
     SL.LoadFromFile(LayerFN);
     If NN.Layers[I].Neurons.Count<>SL.Count then
        Begin
        ShowMessage('Number of neurons mismatch: '+IntToStr(NN.Layers[I].Neurons.Count)+' vs '+IntToStr(SL.Count));
        End
     else
        Begin
        For N:=0 to NN.Layers[I].Neurons.Count-1 do
          Begin
          Size:=NN.Layers[I].Neurons[N].Weights.Size;
          SX:=NN.Layers[I].Neurons[N].Weights.SizeX;
          SY:=NN.Layers[I].Neurons[N].Weights.SizeY;
          NN.Layers[I].Neurons[N].LoadFromString(SL[N]);
          If NN.Layers[I].Neurons[N].Weights.Size<>Size then ShowMessageU('Incorrect size loaded to layer '+IntToStr(I));
          If NN.Layers[I].Neurons[N].Weights.SizeX<>SX then ShowMessageU('Incorrect size loaded to layer '+IntToStr(I));
          If NN.Layers[I].Neurons[N].Weights.SizeY<>SY then ShowMessageU('Incorrect size loaded to layer '+IntToStr(I));
          End;
        NN.Layers[I].AfterWeightUpdate;
        End
     Except
        ShowMessage('Exception loading weights for layer '+IntToStr(I));
     End;
     SL.Free;
     End;
  End;
end;

Also, I was thinking, how much could this optimized if network is used for prediction? I've got more projects that would benefit from adding NN, but memory consumption will be much higher there...

Oleg.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 30, 2019, 04:41:32 pm

Thank you for sharing your code! I'm certainly going to give it a go.

A beautiful aspect of TNNetVolume is about it being able to represent data on both 1D and 3D at any time. Therefore, flatten layers aren't required in this API.

In regards to memory requirements, you may consider using separable convolutions: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SeparableConvolution

If you still need common convolutions with less memory requirements, I'll add feature into my feature list.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: fastblade on January 03, 2020, 09:17:18 am

I am a beginner of AI and NN,
Can this API do Object detecting and get some examples?

Thanks for your greate work!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 05, 2020, 01:42:19 am

@fastblade,
Very welcome!

Although there are plenty of examples about image classification, there is no example about object detection yet. I was almost starting to code one and then I ended coding a generative adversarial example. Porting an existing example from Keras to this API is doable.

BTW, as I'm heading for version 1.0, the time to report bugs is now.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on January 05, 2020, 09:44:04 pm

I don't know if it's a bug report or feature request, but I think some of the networks could benefit from non-square convolutions or poolings,
for example now I'm training my FFT detector on inputs of 512x128, and with only square operations I seem to be quite limited in options. =)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 06, 2020, 02:30:25 pm

Thank you for the feedback agorka!

Totally off topic, yesterday I was googling to find how other APIs support Weight Averaging (https://arxiv.org/abs/1803.05407) and I ended discovering that plenty of them do not have support.

I'm glad to inform that CAI does have support. This is an example showing how to configure 10 epochs weight averaging for image classification:

Code: Pascal [Select][+]

NeuralFit := TNeuralImageFit.Create;
NeuralFit.AvgWeightEpochCount := 10; // the number of epochs used for weight averaging
NeuralFit.Fit(NN, ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes, {NumClasses=}10, {batchsize=}128, {epochs=}100);

Long life to pascal!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Root on January 07, 2020, 02:10:10 pm

hi, first of all great work, I would like to ask if this code can be converted with your library.
thank you very much

https://github.com/erohkohl/question-tagging/blob/master/src/model.py

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 08, 2020, 07:14:33 pm

@Root,
Yes. It can be done.

Inner layers are TNNetFullConnectReLU.

The last 2 layers are (in this order): TNNetFullConnectLinear and TNNetSoftMax.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Root on January 12, 2020, 09:35:48 pm

Are you planning to implement these two algorithms?

Recurrent Neural Networks
LSTM Networks

ref.
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
I think it is only to add the layer specifications and related calculations.

this is a library that implements them linearly.
https://github.com/cvsandbox/ANNT

Thank you ;)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 14, 2020, 05:15:59 pm

:) Hello Root, :)
RNN and LSTM are desirable but aren't current planned.

Next steps in this API are:

More examples including more generative adversarial networks.
Add support for other common data sets with examples.
Improve documentation.
OpenCL optimizations and better integration with TNNetVolume.
Multi OpenCL device support so you can run your models across a number of devices with no source code change.

Soon, we'll have a new example with the Tiny ImageNet dataset https://tiny-imagenet.herokuapp.com/ (https://tiny-imagenet.herokuapp.com/). This data set has 64x64 RGB images and 200 classes.

I'm happy to help anyone trying to code new layers with a forked github repo.

:) Wish everyone happy pascal coding. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: agorka on January 21, 2020, 11:01:09 am

Hello!
I've got some news on my progress porting Keras model to your library!
I've finally trained it in Keras to 95% accuracy, tried to use it in Pascal, and faced this problem:

My model looks like

Code: Pascal [Select][+]

      TNNetInput.Create(512, 128, 1),
      TNNetConvolutionReLU.Create(24, 7, 0, 1, 0).InitUniform(0),
      TNNetConvolutionReLU.Create(24, 5, 0, 1, 0).InitUniform(0),
      TNNetMaxPoolPortable.Create(2, 2),
      TNNetConvolutionReLU.Create(24*2, 5, 0, 1, 0).InitUniform(0),
      TNNetMaxPoolPortable.Create(2, 2),
      TNNetConvolutionReLU.Create(24*4, 5, 0, 1, 0).InitUniform(0),
      TNNetMaxPoolPortable.Create(2, 2),
      TNNetConvolutionReLU.Create(24*4, 5, 0, 1, 0).InitUniform(0),
      TNNetMaxPoolPortable.Create(2, 2),
      TNNetFullConnectLinear.Create(NumClasses)

Seems that X and Y coordinates are swapped when loading Conv layer weights from strings saved in Keras.
I found that by looking to feature maps, Keras contained vertical lines while pascal contained horizontal and vise versa. I used numpy.swapaxes on weights for convolution layers and it did the trick, though images are still not 100% the same, maybe I'll also need to flip convolution filters to match with Keras.
The next problem was the dense layer.
In Keras I had a flatten layer before the dense, so weights of the dense layer were flat. I had to reshape them manually to 27x3x96, swap axes, flatten, and then save. This seems too cumbersome, cause I had to know the original image shape, and it can't be used in generic weight saving function...
Having non-square convolution could have solved my problem, so I could use 27x3 convolution instead of dense layer.
Maybe you have a better different idea how this problem could be solved in a more beautiful way?

Good news is that after all those tricks I've got very good prediction accuracy in Pascal! Thank you again for making this possible! =)
Oleg.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 29, 2020, 05:09:59 pm

@agorka,
Thank you for sharing good news! I'm curious to look at your code. Have you though about open sourcing it? Your code could help others.

Maybe, with time, we could polish both ends and make porting tasks easier.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Firewrath on January 29, 2020, 09:50:17 pm

ok, so this is going to be a little OT from the current discussion going on, sorry. >.>

and first of all, I know pretty much nothing about AI other then article's I've read and such,
I'd like to play with it but I have an 11 y/o PC and almost all the tutorials are in python, so bleh. :P

But I was reading this guys blog:
http://blog.dlib.net/
and he posted something I found rather interesting that I havn't seen anywhere else, called a 'tiled pyramid'
(but mind, I don't follow AI news / updates a lot)

So I was wondering if it could be implemented in your work for training.
Post:
http://blog.dlib.net/2017/08/vehicle-detection-with-dlib-195_27.html
and Pic:
Shortened pic link (https://2.bp.blogspot.com/-xl3Qne3TWjk/WaHJgQZkH8I/AAAAAAAAAuc/qu037v4HxCUjFinIp6Ze5Oou7rhS1OU1QCLcBGAs/s1600/mmod_tutorial_image_pyramid.jpg)

To me, it seems a really smart way to train an AI, but that's coming from someone who doesn't know how it all works.
So what do I know? ;)
but wanted to pass it along anyways. ^-^

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on March 12, 2020, 05:35:55 pm

@Firewrath, I own you a reply. Will do eventually.

I would like to comment here a trick that you can do with this API or any other API when working with image classification: you can increase the input image size.

As per the following example, by increasing CIFAR-10 input image sizes from 32x32 to 48x48, you can gain up to 2% in classification accuracy.

You can change image sizes with:

Code: Pascal [Select][+]

ImgTrainingVolumes.ResizeImage(48, 48);
ImgValidationVolumes.ResizeImage(48, 48);
ImgTestVolumes.ResizeImage(48, 48);

Then, you need to change the input image size in your NN model:

Code: Pascal [Select][+]

NN.AddLayer([
TNNetInput.Create(48, 48, 3),

There is a CIFAR-10 example showing this trick here:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierResize48.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierResize48.lpr)

:) On the power of native code, I wish everyone happy pascal coding! :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on April 12, 2020, 02:30:46 am

:) hello :),

I would like to comment a new feature and new examples that have just been added:

An easy and fast way to load image datasets where each folder represents one class of images.
An image classification example that identifies plant diseases based on a photo taken from a leaf.
A jupyter notebook that trains a neural network coded in PASCAL for plant disease classification.

There are plenty of image datasets where each folder represents a class of images. To make the pascal coding simpler, a new procedure CreateVolumesFromImagesFromFolder has been added as exemplified below:

Code: Pascal [Select][+]

    // change ProportionToLoad to a smaller number if you don't have available 6GB of RAM.
    ProportionToLoad := 1;
    WriteLn('Loading ', Round(ProportionToLoad*100), '% of the Tiny ImageNet 200 dataset into memory.');
    CreateVolumesFromImagesFromFolder
    (
      ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
      {FolderName=}'tiny-imagenet-200/train', {pImageSubFolder=}'images',
      {color_encoding=}0{RGB},
      {TrainingProp=}0.9*ProportionToLoad,
      {ValidationProp=}0.05*ProportionToLoad,
      {TestProp=}0.05*ProportionToLoad
    );

The above example loads the Tiny ImageNet 200 Dataset https://tiny-imagenet.herokuapp.com/ (https://tiny-imagenet.herokuapp.com/) into RAM memory. 90% is loaded as training data while 5% is loaded for each validation and testing.

The next example shows how to load the dataset used for the paper Identification of Plant Leaf Diseases Using a 9-layer Deep Convolutional Neural Network https://data.mendeley.com/datasets/tywbtsjrjv/1 (https://data.mendeley.com/datasets/tywbtsjrjv/1):

Code: Pascal [Select][+]

    // change ProportionToLoad to a smaller number if you don't have available 32GB of RAM.
    ProportionToLoad := 1;
    WriteLn('Loading ', Round(ProportionToLoad*100), '% of the Plant leave disease dataset into memory.');
    CreateVolumesFromImagesFromFolder
    (
      ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
      {FolderName=}'plant', {pImageSubFolder=}'',
      {color_encoding=}0{RGB},
      {TrainingProp=}0.9*ProportionToLoad,
      {ValidationProp=}0.05*ProportionToLoad,
      {TestProp=}0.05*ProportionToLoad,
      {NewSizeX=}128, {NewSizeY=}128
    );

As you can see above, besides loading the dataset, images are being resized to 128x128. As I don't have 32GB of RAM in my development machine, I've tested the above code with https://vast.ai/ (https://vast.ai/) using this jupyter notebook: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.ipynb (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.ipynb). You can find raw results at: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimplePlantLeafDisease/results .

As you can see on these raw results, the test classification (or the plant disease diagnostic) accuracy is 98.95% (this is pretty high - I wonder if we could do the same with corona virus).

CreateVolumesFromImagesFromFolder has parallel code so classes are loaded into memory in parallel. I consider this code tremendously fast and it outperforms code that I saw in other APIs. I should say thank you to FPC developers for coding FPImage and plenty of other super fast bits and pieces that allowed me to implement a fast CreateVolumesFromImagesFromFolder.

This is the neural network architecture used for classification:

Code: Pascal [Select][+]

    NN.AddLayer([
      TNNetInput.Create(128, 128, 3),
      TNNetConvolutionLinear.Create({Features=}64, {FeatureSize=}5, {Padding=}4, {Stride=}2),
      TNNetMaxPool.Create(2),
      TNNetMovingStdNormalization.Create(),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
      TNNetMaxPool.Create(2),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}2),
      TNNetDropout.Create(0.5),
      TNNetMaxPool.Create(2),
      TNNetFullConnectLinear.Create(39),
      TNNetSoftMax.Create()
    ]);

The complete source code can be found at: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.pas (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/SimplePlantLeafDisease.pas)

:) I wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: eljo on April 12, 2020, 07:07:56 am

I just wanted to say I love your work. Thank you for your support.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on April 14, 2020, 09:21:47 am

@eljo,
Thank you for your kind words. They are motivating.

Created a full Tiny ImageNet 200 example: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleTinyImageNet

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 03, 2020, 09:24:20 am

:) Hello :),
I have news about Increasing Image Resolution with Neural Networks.

I created a command line tool so everyone can now increase image resolution with no more than FPC (no external library are required) via command line:

Code: Pascal [Select][+]

#SuperResolution -i street.png -o street3.png
Loading input file: street.png
Input image size: 158x214x3
Creating Neural Network...
Resizing with tiles...
Neural network file found at ../../../examples/SuperResolution : super-resolution-7-64-sep.nn
Padding input image.
Resizing with tiles to: 288x416x3
Saving output file: street3.png

-i defines the input image file while -o defines the output image file.

You can get this command line tool from:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SuperResolution/README.md

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Alienigene on May 23, 2020, 02:56:48 am

Hello everyone, I don't know if this is the right place to ask, but I know this problem is rather frequent and it is related to Free Pascal/Lazarus IDE, so I figure here is a good place to ask.

The thing is very simple. My Operating System is Windows 10 v1909 x64. I have downloaded Conscious Artificial Intelligence (project on Sourceforge.net), the files in it are 64-bit. I have extracted the files, ran ArtificialArt.exe, pressed start and nothing happens. I noticed a console window beside the gui window, and it told me it couldn't find a bin file, told me to download cifar, and provided me the address. Simple enough, I downloaded and extracted the files to the folder of CAI.

Then I pressed start again, this time and all times after, an error message window pop up and says: Access violation. Press OK to ignore and risk data corruptio. Press Abort to kill the program. This always happens after loading the .bin files, Creating Neural Networks... and immediately after Creating generative. If I press OK, nothing will happen. If I press Abort, program doesn't end, console says Sending STOP request, but the program doesn't stop, requiring me to close the console to kill the program.

The same problem applies to GradientAscent.exe, it happens immediately after I load a neural network.
I noticed their icons are of the programs developed by Lazarus IDE, I searched the problem online and found it rather frequent, but no solutions, so I figured I would ask here.

I have installed FreePascal IDE 3.0.4 (x86), Lazarus IDE 2.0.6 (x86_64), and placed the executable path of FPC in system path Env.Var., I have no problem using Lazarus IDE itself so far, any ideas on what caused this and how to solve it?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 24, 2020, 01:12:35 am

Hello Alienigene,
Can you confirm please whether your processor has or hasn't AVX instruction set? In the case it hasn't, I can prepare a compiled version for you without the AVX instruction set.

Current binaries require AVX instruction set and they will provoke an exception at the time the NN is run with a processor that hasn't AVX.

In the case that you would like to try to compile by yourself, with Lazarus, go to Options/Project Options/Custom Options/Defines and then uncheck AVX. After this, go to Run/Build and then Run/Run.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Alienigene on May 28, 2020, 10:35:37 am

I am sorry that I have just seen it, been busy for a while, so I didn't reply instantly. My CPU is Intel Celeron CPU J1900 @ 1.99GHz, quad-core CPU, I looked at CPU-Z, it supports MMX, SSE1-4.2, EM64T and VT-x instruction sets, but no, no AVX, so I think that is the cause of the problem.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Alienigene on May 29, 2020, 06:41:35 am

I got a problem, big problem, I have downloaded the svncode trunk of cai from sourceforge, intended to compile all the stuff by myself, however there are not any project files(.lpi), there are lots of .pas files, some .inc files, some .lpr .lfm and .cl files, obviously multiple files are included in one project, but I honestly don't know which files are included in a project, and what file each project corresponds to, e.g. .exe .dll .lib .nn, I don't know the correspondences, so I can't compile anything...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 31, 2020, 03:29:18 am

Hi Alienigene, these projects are probably a good start (open with Project/Open Project):

examples/XorAndOr/XorAndOr.lpi
examples/SimpleImageClassifier/SimpleImageClassifier.lpi

Alternatively, if you have a gmail account, you can just open the following link, select "Runtime" and then "Run all":
https://colab.research.google.com/github/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierCPU.ipynb

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 30, 2020, 08:04:14 pm

:) Hello - this is a quick update :)

These are the news from my end:

All fully connected layers now support OpenCL on the forward pass.
Better https://github.com/joaopauloschuler/neural-api/blob/master/README.md (https://github.com/joaopauloschuler/neural-api/blob/master/README.md).
A new Autoencoder example that shows an autoencoder built with hyperbolic tangents and trained with Tiny ImageNet 200.
Some small bug fixes.

:) Wish everyone happy pascal coding. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 16, 2020, 11:11:05 pm

I'm trying to get your supersimple example to compile. I have your original uconvolutionneuralnetwork file. Do you have a new set of code that define units neuralnetwork, neuralvolume and nearalfit?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 17, 2020, 05:57:02 am

:) Hi Daringly :)
You'll need this folder to compile: https://github.com/joaopauloschuler/neural-api/tree/master/neural

Alternatively, in the case that you would like to try image classification without having to install anything, you can run via browser at:

https://colab.research.google.com/github/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierCPU.ipynb

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 17, 2020, 08:47:11 pm

Thanks!

I do a lot of Sports actuarial analysis (probabilities for each match outcome), and this will hopefully let me do some much more complex analysis than what I am presently doing.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Blade on September 19, 2020, 07:26:33 pm

Very nice. Great project.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 21, 2020, 08:44:13 pm

Is there a set of documentation for these libraries?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 22, 2020, 01:10:38 pm

:) Hi daringly :)
Except for this read me (https://github.com/joaopauloschuler/neural-api), not much. Is there any particular part of the API that you would like me to detail further?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 22, 2020, 04:51:28 pm

I'm not particularly good at Python/Keras, whereas I'm much more fluent in Pascal. I'd love a set of documents so that someone that never used Python or Keras could do things from the ground up.

Here's the meat of a keras model I wrote, and I have no idea how to implement this in your library:

model=tf.keras.Sequential()
model.add(layers.Dense(12, input_dim=14, activation='relu'))
model.add(layers.Dense(4, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics =['accuracy'])

print(model.summary)

monitor_val_acc=EarlyStopping(monitor='val_loss', patience=5)
model.fit(x,y, epochs=200, validation_data=(x_test, y_test), callbacks=[monitor_val_acc])

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 22, 2020, 04:54:55 pm

How about an example where you have 2 inputs (x,y) and an output (z) which is the hypotenuse of a right-angle triangle?

So in our dataset, Z^2 = X^2 + Y^2.. use maybe 10 datapoints.

A simple start-to-finish on "solving" and improving on the solution to this problem could be extrapolated to all other sorts of problems.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 24, 2020, 05:02:08 am

:) Hello daringly :)

Your Keras model translates to this:

Code: Pascal [Select][+]

var
  Model: TNNet;
begin
  Model := TNNet.Create();
  Model.AddLayer([
    TNNetInput.Create(14),
    TNNetFullConnectReLU.Create(12),
    TNNetFullConnectReLU.Create(4),
    TNNetFullConnect.Create(1) // This is hyperbolic tangent - not Sigmoid
  ]);

print(model.summary) translates to:

Code: Pascal [Select][+]

Model.DebugStructure();

Above example has hyperbolic tangent instead of sigmoid.

For the fitting, you might follow this example:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/XorAndOr

The fitting method allows you to pass validation and test pairs also:

Code: Pascal [Select][+]

procedure Fit(pNN: TNNet;
  pTrainingVolumes, pValidationVolumes, pTestVolumes: TNNetVolumePairList;
  pBatchSize, Epochs: integer);

You won't need callbacks for monitoring. The fitting already saves the best solution for you and displays progress.

In the case that you really need callbacks, you can look at:

Code: Pascal [Select][+]

      property OnAfterStep: TNotifyEvent read FOnAfterStep write FOnAfterStep;
      property OnAfterEpoch: TNotifyEvent read FOnAfterEpoch write FOnAfterEpoch; 

CAI is SGD based (I don't have adam). I'm passionate towards SGD.

About Z^2 = X^2 + Y^2, I created a feature request at:
https://github.com/joaopauloschuler/neural-api/issues/33

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 24, 2020, 02:14:49 pm

Thanks! I'll fire this up this weekend.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 29, 2020, 04:35:58 am

@daringly

I coded the example you asked for:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/Hypotenuse/Hypotenuse.lpr

Training, validation and testing pairs are created with:

Code: Pascal [Select][+]

  function CreateHypotenusePairList(MaxCnt: integer): TNNetVolumePairList;
  var
    Cnt: integer;
    LocalX, LocalY, Hypotenuse: TNeuralFloat;
  begin
    Result := TNNetVolumePairList.Create();
    for Cnt := 1 to MaxCnt do
    begin
      LocalX := Random(100);
      LocalY := Random(100);
      Hypotenuse := sqrt(LocalX*LocalX + LocalY*LocalY);
 
      Result.Add(
        TNNetVolumePair.Create(
          TNNetVolume.Create([LocalX, LocalY]),
          TNNetVolume.Create([Hypotenuse])
        )
      );
    end;
  end; 
...
    TrainingPairs := CreateHypotenusePairList(10000);
    ValidationPairs := CreateHypotenusePairList(1000);
    TestPairs := CreateHypotenusePairList(1000);

The neural network is created with 2 inputs and one output:

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer( TNNetInput.Create(2) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(1) );

The fitting is called with:

Code: Pascal [Select][+]

  // Returns TRUE if difference is smaller than 0.1 .
  function LocalFloatCompare(A, B: TNNetVolume; ThreadId: integer): boolean;
  begin
    Result := ( Abs(A.FData[0]-B.FData[0])<0.1 );
  end;  
...
    NFit := TNeuralFit.Create();
    NFit.InitialLearningRate := 0.00001;
    NFit.LearningRateDecay := 0;
    NFit.L2Decay := 0;
    NFit.InferHitFn := @LocalFloatCompare;
    NFit.Fit(NN, TrainingPairs, ValidationPairs, TestPairs, {batchsize=}32, {epochs=}50);

A final test is done with:

Code: Pascal [Select][+]

    // tests the learning
    for Cnt := 0 to 9 do
    begin
      NN.Compute(TestPairs[Cnt].I);
      NN.GetOutput(pOutPut);
      WriteLn
      ( 'Inputs:',
        TestPairs[Cnt].I.FData[0]:5:2,', ',
        TestPairs[Cnt].I.FData[1]:5:2,' - ',
        'Output:',
        pOutPut.Raw[0]:5:2,' ',
        ' Desired Output:',
        TestPairs[Cnt].O.FData[0]:5:2
      );
    end;

These are results from my own run:

Code: Pascal [Select][+]

Inputs:19.00, 71.00 - Output:73.53  Desired Output:73.50
Inputs:28.00,  6.00 - Output:28.66  Desired Output:28.64
Inputs:34.00, 53.00 - Output:63.02  Desired Output:62.97
Inputs:20.00, 35.00 - Output:40.25  Desired Output:40.31
Inputs:53.00, 28.00 - Output:59.98  Desired Output:59.94
Inputs:56.00, 12.00 - Output:57.31  Desired Output:57.27
Inputs:77.00, 29.00 - Output:82.35  Desired Output:82.28
Inputs:69.00, 31.00 - Output:75.65  Desired Output:75.64
Inputs:24.00, 78.00 - Output:81.65  Desired Output:81.61
Inputs:15.00, 22.00 - Output:26.66  Desired Output:26.63

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on September 30, 2020, 02:26:20 am

When I compile the hypotenuse example, I get "FATAL: Cannot find MTPCPU used by neuralthread".

I did get the examples included in your package working.

$0.01 for your thoughts.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 30, 2020, 10:11:54 am

you'll need the package multithreadprocslaz .

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 02, 2020, 03:31:12 pm

The example works, thanks!

I tried modifying it, and broke it.

I wanted to expand the example. With 3 inputs, I have the dimensions of a hexahedron. With 2 outputs, I'd calculate the hypotenuse of opposite corners, and the volume.

I changed the initial NN definition to:

NN.AddLayer( TNNetInput.Create(3) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(2) );

The Create pair list function was modified:
for Cnt := 1 to MaxCnt do
begin
LocalX := Random(100);
LocalY := Random(100);
LocalZ := Random(100);
Hypotenuse := sqrt(LocalX*LocalX + LocalY*LocalY + LocalZ * LocalZ);
Volume := LocalX * LocalY * LocalZ;

Result.Add(
TNNetVolumePair.Create(
TNNetVolume.Create([LocalX, LocalY, LocalZ]),
TNNetVolume.Create([Hypotenuse, Volume])
)
);

I adjusted the outputs, but that isn't causing the problems. All my outputs are now 0. What have I done?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 02, 2020, 07:57:18 pm

What exactly does this statement do?

pOutPut := TNNetVolume.Create(1,1,1,1);

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 04, 2020, 07:05:51 am

@daringly,
Very interesting!

Code: Pascal [Select][+]

pOutPut := TNNetVolume.Create({pSizeX=}1, {pSizeY=}1, {pDepth=}1, {FillValue=}1);

creates a 3D memory structure that can be accessed as 1D, 2D and 3D construct at the same time. Above example will create a 1x1x1 array filled with the value 1.

Given that your output now has size 2, you should try this:

Code: Pascal [Select][+]

pOutPut := TNNetVolume.Create({pSizeX=}2, {pSizeY=}1, {pDepth=}1, {FillValue=}1);

Given your use cases, I'm now considering adding non linear neural network layers so the network can learn expressions such as "sqr(A) + sqrt(A*B)" faster.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 04, 2020, 07:29:11 am

This is in reply to "All my outputs are now 0. What have I done?".

Volume is a lot bigger number than hypothenuse. So, the network will converge towards fixing volume first. To solve the problem, you can rescale volume with:

Code: Pascal [Select][+]

Volume := (LocalX * LocalY * LocalZ)/(100*100);

With the above, your network should converge.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 04, 2020, 10:09:04 pm

Thanks. That adjustment did make it converge, and I can now run with 2 outputs. I'm still getting an output of 0 for the second output (volume) while the desired outputs are correct. It's not critical, since I could simply rerun the nn for each output, but slightly frustrating.

The next thing I'm starting is much more interesting. Using about 8 custom features to predict spread and total results in a sport. I've had a lot of success doing this in the past without neural nets, and I'm wondering how much better this might be.

Edit: I figured out what was making the second ouput 0 -- dividing by (100*100) made the scale too small, but 100 worked reasonably well. Is it better to build separate models for unrelated outputs? In the next major project, most projected spreads (the target solution for my features) will typically be -25 to 25, and target for totals is typically 40-80.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 04, 2020, 11:57:53 pm

As long as I'm thinking wishfully, can we have GPU support?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 05, 2020, 08:40:11 am

In regards to "can we have GPU support?":
Yes we can!

GPUs are run via OpenCL. Have a look at this example:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleImageClassifierGPU

Fully connected (dense) layers have OpenCL implementation for the forward pass. While other APIs support only NVIDIA, GPUs from major vendors are supported here.

In the case that you have more than one GPU on your environment, you can have a look here (how to select the correct GPU from a GPU list):
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/experiments/visualCifar10OpenCL/

In the case that you need to infer 20 partially unrelated outputs in the same network, you can try a multi-path neural network (this is fully supported by this API).

These Project/Custom (compilation) options might be important to you: AVX, AVX2, AVX512 and OpenCL.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 05, 2020, 08:48:49 am

About "will typically be -25 to 25, and target for totals is typically 40-80", I have a preference for rescaling inputs/outputs to a monopolar range [0.1,0.9] or
bipolar [-1,+1] or even [-2, +2]. This is a personal preference. Smaller numbers avoid overflows.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: josiaslg on October 07, 2020, 02:00:28 am

Hi.
First of all, congratulations for this project. Is really amazing such vision about performance and optimization.
I'm newer in this camp and leaning about the project and process.
I started compile the SimpleFashionMNIST and make the changes to be OpenCL compatible. Work very well wirhout any problem.
Now i have the challange to read directorys with pure photos (png, jpeg, bmp and etc).
To read is ok, but, how i can implement this on neuraldatasets unit ?.
The photos is one category, but, different sizes. Other thing is: how i can test the the results to know how well is training ?.
Sorry if this questions is too newbie.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 07, 2020, 08:29:24 pm

@josiaslg,
Welcome aboard! Glad to see you here!

Your question is a very good question. Sometimes, we have images in folders like this:

Code: Pascal [Select][+]

/myfolder/<class_id>/img1.jpg
/myfolder/<class_id>/img2.jpg
/myfolder/<class_id>/img3.jpg

For folder structures like the above, you can look here: https://github.com/joaopauloschuler/neural-api#one-class-per-folder-with-image-classification

Sometimes, inside the class folder, you have a subfolder such as "images". Follows an example:

Code: Pascal [Select][+]

/myfolder/<class_id>/images/img1.jpg
/myfolder/<class_id>/images/img2.jpg
/myfolder/<class_id>/images/img3.jpg

In this case, you'll pass 'images' to the parameter pImageSubFolder. It should look like this:

Code: Pascal [Select][+]

// change ProportionToLoad to a smaller number if you don't have enough RAM.
ProportionToLoad := 1;
WriteLn('Loading ', Round(ProportionToLoad*100), '% of the dataset into memory.');
CreateVolumesFromImagesFromFolder
(
  ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
  {FolderName=}'/myfolder', {pImageSubFolder=}'images',
  {color_encoding=}csRGB{RGB},
  {TrainingProp=}0.9*ProportionToLoad,
  {ValidationProp=}0.05*ProportionToLoad,
  {TestProp=}0.05*ProportionToLoad,
  {NewSizeX=}64, {NewSizeY=}64
);

Above code will resize input images to 64x64.

I'll eventually add an example loading images "on the fly" for very large datasets. If you need large amounts of RAM such as 128GB to run your models, you can consider running in the cloud.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 11, 2020, 04:15:47 pm

Edit: Issue resolved. Switching to Linear worked, since output was typically -1 to 1, and ReLU ranges positive.

I'm having another "0 Output" issue. All my outputs show 0, even though everything seemed to run correctly.

I started with the Hypotenuse example, and changed the first layer to allow for 22 features:

N.AddLayer( TNNetInput.Create(22) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(32) );
NN.AddLayer( TNNetFullConnectReLU.Create(1) );

The datapoint creation was also changed:
Result.Add(
TNNetVolumePair.Create(
TNNetVolume.Create([Features[6], Features[7], Features[8], Features[9],
Features[10], Features[11], Features[12], Features[13], Features[14],
Features[15], Features[16], Features[17], Features[18], Features[19],
Features[20], Features[21], Features[22], Features[23], Features[24],
Features[25], Features[26], Features[27]]),
TNNetVolume.Create([noutputs[28]/50])

The output is typically a number from -50 to 50, and usually -20 to 20.

I used weird indexing because it made it easier to visualize where pieces of a .csv file are going -- the features are columns 6-27, and the desired output is in 28 (and 29).

Any idea how I blew this up?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 11, 2020, 07:09:13 pm

What would the expected output range be (-50 to 50)?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 11, 2020, 07:19:27 pm

Before scaling, -50 to 50. I divided the expected output by 50, so it is typically -1 to 1.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 11, 2020, 07:20:52 pm

One other weird issue:

When I execute the following statement, I inspected Features to ensure they were non-0. After processing and showing the inputs, Features 24-27 are always 0 on output (although they aren't initially). Something isn't copying over somehow.

Result.Add(
TNNetVolumePair.Create(
TNNetVolume.Create([Features[6], Features[7], Features[8], Features[9],
Features[10], Features[11], Features[12], Features[13], Features[14],
Features[15], Features[16], Features[17], Features[18], Features[19],
Features[20], Features[21], Features[22], Features[23], Features[24],
Features[25], Features[26], Features[27]]),
TNNetVolume.Create([noutputs[28]/50])

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: josiaslg on October 13, 2020, 03:56:52 am

Hi.
I try to do but not generate nn file.
When execute, the images is founded on folders (the folders "images" is separated in 3 classes).
But, simple don't generate nn file.
The source file is here: https://github.com/josiaslg/cai-exemple-photo-from-directory/blob/main/neural.lpr (i not paste code here to not mess up the text and forum).
The OpenCL is ok, i test other example files and everything goes fine (the driver is installed and working on CAi OpenCL examples).

Quote from: schuler on October 07, 2020, 08:29:24 pm

@josiaslg,
Welcome aboard! Glad to see you here!

Your question is a very good question. Sometimes, we have images in folders like this:
Code: Pascal [Select][+][-]
/myfolder/<class_id>/img1.jpg
/myfolder/<class_id>/img2.jpg
/myfolder/<class_id>/img3.jpg

For folder structures like the above, you can look here: https://github.com/joaopauloschuler/neural-api#one-class-per-folder-with-image-classification

Sometimes, inside the class folder, you have a subfolder such as "images". Follows an example:
Code: Pascal [Select][+][-]
/myfolder/<class_id>/images/img1.jpg
/myfolder/<class_id>/images/img2.jpg
/myfolder/<class_id>/images/img3.jpg

In this case, you'll pass 'images' to the parameter pImageSubFolder. It should look like this:
Code: Pascal [Select][+][-]
// change ProportionToLoad to a smaller number if you don't have enough RAM.
ProportionToLoad := 1;
WriteLn('Loading ', Round(ProportionToLoad*100), '% of the dataset into memory.');
CreateVolumesFromImagesFromFolder
(
ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes,
{FolderName=}'/myfolder', {pImageSubFolder=}'images',
{color_encoding=}csRGB{RGB},
{TrainingProp=}0.9*ProportionToLoad,
{ValidationProp=}0.05*ProportionToLoad,
{TestProp=}0.05*ProportionToLoad,
{NewSizeX=}64, {NewSizeY=}64
);

Above code will resize input images to 64x64.

I'll eventually add an example loading images "on the fly" for very large datasets. If you need large amounts of RAM such as 128GB to run your models, you can consider running in the cloud.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 13, 2020, 12:31:06 pm

Quote

Before scaling, -50 to 50. I divided the expected output by 50, so it is typically -1 to 1...are always 0 on output

If your Expected output can be negative, if your last layer is ReLU, replacing ReLU layer (as ReLU result is never negative) by a Linear layer might fix the problem.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 13, 2020, 12:32:23 pm

Quote

But, simple don't generate nn file.

You need validation data to generate the nn file. The nn file is created each time the validation does better.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 15, 2020, 12:30:45 am

Is there a limit to how many inputs you can add? It looks like adding features stopped working after 21 inputs in the code I ran.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 15, 2020, 11:08:46 am

Some of my own models have millions of trainable parameters with 224x224x3 (150k) inputs. So, input size isn't a limit.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: daringly on October 21, 2020, 07:35:33 pm

I figured out the error, thanks. I was doing something idiotic.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on October 22, 2020, 07:42:42 pm

Cool.

In the case that you are interested, I coded a new Hypotenuse Example without preallocating all the training data into memory. This is useful if you have a too large dataset to fit into memory (I've seen training datasets approaching 1TB).

https://github.com/joaopauloschuler/neural-api/blob/master/examples/HypotenuseFitLoading/HypotenuseFitLoading.lpr

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: nouzi on October 22, 2020, 08:04:14 pm

Very nice 8-)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 03, 2020, 01:02:18 am

@nouzi, thank you for your kind words.

I have 2 news:

A new layer type: TNNetConvolutionSharedWeights.
Beautiful images created by a neural network.

Beautiful Images
I've attached some images created by a generative adversarial network. The source code can be found here:
https://sourceforge.net/p/cai/svncode/1485/tree/trunk/lazarus/examples/VisualGANTinyImagenet/

TNNetConvolutionSharedWeights
Added today to the API a new convolutional layer type: TNNetConvolutionSharedWeights . Instead of having its own neurons and weights, this convolutional layer uses the same neurons and weights from another existing layer. So, if you need 2 layers with the same neurons, you can add TNNetConvolutionSharedWeights to your network. Why would you need something like this? Maybe, your NN needs to learn the same patterns in different scales (such as big and small dogs are all dogs).

I added an example at:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierSharedWeights.lpr

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: PierceNg on January 31, 2021, 08:56:51 am

@schuler This is great stuff! Thanks for making this!

My setup: Windows 10 Home 2H20, Ryzen 2500U, latest AMD drivers, Lazarus 2.0.10 with FPC 3.2.0. GPU probe shows following:

Platform info: 0 --------------------- PROFILE: FULL_PROFILE VERSION: OpenCL 2.1 AMD-APP (3188.4) NAME: AMD Accelerated Parallel Processing VENDOR: Advanced Micro Devices, Inc. EXTENSIONS: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices <and so on>

However, when running, say, opencl-dot-product-test.exe, I get many lines of the following:

Error: TDotProductCL.Compute - Failed setting parameters: -48

I am not familiar with OpenCL programming. Is -48 the OpenCL error CL_INVALID_KERNEL? More importantly, does the error mean the GPU wasn't used?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on February 06, 2021, 06:49:48 pm

Hi PierceNg,
Do you have any of the following messages?

Code: Pascal [Select][+]

clCreateContext OK!
clCreateCommandQueue OK!
clCreateProgramWithSource OK!
clBuildProgram OK!
clCreateKernel cai_dot_product OK!
clCreateKernel cai_dot_product OK!

I'm wondering if the file cai_dot_product.cl is correctly been loaded.

I logged a bug report for this:
https://github.com/joaopauloschuler/neural-api/issues/41

Can you please confirm if you are using an APU or a discrete video card? In the case of a discrete video card, what is the model?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on February 07, 2021, 06:53:04 pm

@PierceNg,
I've just tested with a Ryzen APU and was able to reproduce the problem and fix it:
https://github.com/joaopauloschuler/neural-api/issues/41

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: PierceNg on February 08, 2021, 11:03:33 am

Quote from: schuler on February 07, 2021, 06:53:04 pm

@PierceNg,
I've just tested with a Ryzen APU and was able to reproduce the problem and fix it:
https://github.com/joaopauloschuler/neural-api/issues/41

:) Wish everyone happy pascal coding :)

Thank you!

After pulling from your repos,the dot product and visual CIFAR 10 OpenCL versions run, and I see the "OK!" messages in the command prompt window. My machine is using a Ryzen APU.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: mika on February 11, 2021, 06:46:54 pm

HypotenuseFitLoading.lpr

Code: Pascal [Select][+]

    Write('Press ENTER to exit.');
    ReadLn;
  end;
 

should be

Code: Pascal [Select][+]

    Write('Press ENTER to exit.');
    ReadLn;
    Terminate;
  end;
 

Otherwise it keeps repeating DoRun method over and over.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on February 14, 2021, 07:49:28 pm

@mika,
the bug has been fixed as per your suggestion (thank you!):
https://github.com/joaopauloschuler/neural-api/issues/42

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: PierceNg on March 09, 2021, 11:58:32 am

@schuler,

I am happy to report that I have OpenCL Dot Product example running also on my Macbook Pro 2012 model, which has Intel i7 CPU, integrated HD Graphics 4000, and GeForce GT 650M.

Running directly by double-clicking on the app in Finder appears to work - total OpenCL runtime reports 0.02s - but not really. When I run the executable from the shell, the program reports:

File neural.cl could not be found. Error: Failed to create compute kernel:cai_dot_product <many many lines of error messages>

I copied neural.cl into the the app bundle directory where the executable opencl-dot-product-test is, then run the executable from the shell again:

clCreateContext OK! clCreateCommandQueue OK! clCreateProgramWithSource OK! clBuildProgram OK! clCreateKernel cai_dot_product OK! clCreateKernel cai_dot_product OK!

Screenshot of successful run attached.

But double clicking on the app from Finder still didn't work even with neural.cl already copied in, as in OpenCL runtime is again 0.02s.

Oh, I also had to fix a FPC_FULLVERSION test in the file Lazarus/components/multithreadprocs/mtpcpu.pas in order to build the program.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on March 11, 2021, 09:07:01 am

@PierceNg,
Loved seeing the screenshot from your experiment! Thank you for sharing your test results.

:) May the (fo/sou)rce be with you :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on March 23, 2021, 01:42:50 pm

:) Hello :)

I would like to comment a data augmentation approach commonly used with CIFAR-10: the dataset is padded with 4 px increasing the original size from 32x32 to 40x40. Then, at the time of training, a random crop is done bringing the size back to 32x32 px. An example with this approach can be found at https://medium.com/fenwicks/tutorial-2-94-accuracy-on-cifar10-in-2-minutes-7b5aaecd9cdd.

Implementing this in Pascal is very simple:

Code: Pascal [Select][+]

const
  // Padding and cropping constants.
  csPadding = 4;
  csCropSize = csPadding * 2;   
...
    // Load the CIFAR-10 dataset
    CreateCifar10Volumes(ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes);
 
    // Add padding to dataset resizing from 32x32 to 40x40
    ImgTrainingVolumes.AddPadding(csPadding);
    ImgValidationVolumes.AddPadding(csPadding);
    ImgTestVolumes.AddPadding(csPadding);
 
    NeuralFit := TNeuralImageFit.Create;
 
    // Enable cropping while fitting.
    NeuralFit.HasImgCrop := true;
    NeuralFit.MaxCropSize := csCropSize;
    NeuralFit.HasFlipX := true;

This simple trick actually seems to outperform CAI's default image data augmentation method. A fully functional source code example can be found at: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierPaddingCropping.lpr

:) May the source be with you :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on April 01, 2021, 03:46:08 pm

:) Hello :)
I think that I have never commented the Neural API threading in detail here. I got a couple of questions in regards to this recently. I hope that I'm not typing too much.

As an example, assuming that you need to run a procedure 10 times in parallel, you can create 10 thread workers as follows:

Code: Pascal [Select][+]

FProcs := TNeuralThreadList.Create( 10 );

As an example, this is the procedure that we intend to run in parallel:

Code: Pascal [Select][+]

procedure MyClassName.RunNNThread(index, threadnum: integer);
begin
  WriteLn('This is thread ',index,' out of ',threadnum,' threads.');
end; 

Then, to run the procedure RunNNThread passed as parameter 10 times in parallel, do this:

Code: Pascal [Select][+]

FProcs.StartProc(@RunNNThread);

You can control the blocking mode (waiting threads to finish before the program continues) as per declaration:

Code: Pascal [Select][+]

procedure StartProc(pProc: TNeuralProc; pBlock: boolean = true);

Or, if you prefer, you can specifically say when to wait for threads to finish as per this example:

Code: Pascal [Select][+]

FProcs.StartProc(@RunNNThread, false);
// insert your code here
FProcs.WaitForProc(); // waits until all threads are finished.

When you are done, you should call:

Code: Pascal [Select][+]

FProcs.Free; 

neuralthread is the unit that allows parallel processing when using neural fit: https://github.com/joaopauloschuler/neural-api/blob/master/neural/neuralfit.pas

Wish long and prosper life to FPC and Lazarus!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: iLya2IK on July 02, 2021, 10:34:07 am

Thank you for a wonderful project.

I would like to start classifying text phrases according to certain patterns. It is about recognizing and classifying car brands and models. The number of combinations is about 2000. But all my attempts run into a fundamental misunderstanding of which class to choose for modeling - now I use TNNetForByteProcessing. Here is the code I am using to simulate:

Code: Pascal [Select][+]

const MAX_IN_SIZE = 25;
      HIGH_IN_VAL = MAX_IN_SIZE-1;
      MAX_OUT_SIZE = 256;
      HIGH_OUT_VAL = MAX_OUT_SIZE-1;
 
type TInpArray = Array [0..HIGH_IN_VAL] of Byte;
     TOutArray = Array [0..HIGH_OUT_VAL] of Byte;
 
var
  DS : TSqlite3Dataset;
  MMList : TStringList;
 
procedure SetCorrectOut(var aOut : TOutArray; value : Word);
var Pos : Word;
    Shift : Byte;
begin
  Pos := value div 8;
  Shift := value mod 8;
  aOut[Pos] := $1 shl Shift;
end;
 
function GetCorrectOut(const aOut : TOutArray) : Word;
var Pos : Word;
    Shift : Byte;
begin
  for Pos := 0 to HIGH_OUT_VAL do
  begin
    if (aOut[Pos] > 0) then
    begin
      case aOut[Pos] of
        2   : Shift := 1;
        4   : Shift := 2;
        8   : Shift := 3;
        16  : Shift := 4;
        32  : Shift := 5;
        64  : Shift := 6;
        128 : Shift := 7;
      else
        Shift := 0;
      end;
      Exit(Pos * 8 + Shift);
    end;
  end;
  Result := 0;
end;
 
function GetCorrectOutList(const aOut : TOutArray) : String;
var Pos : Word;
    Cnt : Byte;
    S : String;
 
procedure AddOption(W : Word);
begin
  if length(S) > 0 then S := S + ' ';
  S := S + MMList.Values[inttostr(W)];
  Inc(Cnt);
end;
 
begin
  S := '';
  Cnt := 0;
  for Pos := 0 to HIGH_OUT_VAL do
  begin
    if (aOut[Pos] > 0) then
    begin
      if aOut[Pos] and $1 > 0 then AddOption(Pos shl 3);
      if aOut[Pos] and $2 > 0 then AddOption(Pos shl 3 or $1);
      if aOut[Pos] and $4 > 0 then AddOption(Pos shl 3 or $2);
      if aOut[Pos] and $8 > 0 then AddOption(Pos shl 3 or $3);
      if aOut[Pos] and $10 > 0 then AddOption(Pos shl 3 or $4);
      if aOut[Pos] and $20 > 0 then AddOption(Pos shl 3 or $5);
      if aOut[Pos] and $40 > 0 then AddOption(Pos shl 3 or $6);
      if aOut[Pos] and $80 > 0 then AddOption(Pos shl 3 or $7);
    end;
    if Cnt > 15 then break;
  end;
  Result := S;
end;
 
procedure RunAlgo();
const MAX_EPOCH_CNT = 1000;
      MAX_EPOCH_SIZE = 512;
      MAX_ERROR_CNT  = 1;
var
  NN: TNNetForByteProcessing;
  BAInput : TInpArray;
  BAExpected, BAOutput :TOutArray;
  L, epoch : integer;
  Txt1, Res, Outv : String;
  MM, MM1 : Word;
  ErrorCnt, EpochSize : Integer;
  NeedNextEpoch : Boolean;
  Key : char;
begin
  NN := TNNetForByteProcessing.Create();
 
  try
    NN.AddBasicByteProcessingLayers(MAX_IN_SIZE, MAX_OUT_SIZE, 8, 32);
    NN.SetLearningRate(0.1, 0.0);
 
    WriteLn('Computing...');
    epoch := 1;
    NeedNextEpoch := true;
 
    ErrorCnt := MAX_ERROR_CNT + 1;
    EpochSize := MAX_EPOCH_SIZE;
    while NeedNextEpoch and (epoch <= MAX_EPOCH_CNT) and (ErrorCnt > MAX_ERROR_CNT) do
    begin
      WriteLn('Epoch #', epoch);
      inc(epoch);
      DS.SQL := 'select mrkorig, uniq_ats_schem.id as mmid, '+
      'ats_schem.mrk||" "||ats_schem.model as expc '+
      'from ats_schem, uniq_ats_marks, uniq_ats_schem '+
      'where length(mrkorig) < 25 and uniq_ats_marks.mrk = ats_schem.mrk and '+
      '      uniq_ats_schem.mrk = ats_schem.mrk and '+
      '      uniq_ats_schem.model = ats_schem.model order by random() limit ' + inttostr(EpochSize);
      DS.Open;
      try
        ErrorCnt := 0;
        // tests the learning
        while not DS.EOF do
        begin
          if Keypressed then // if user provided input
          begin
            Key := readkey;
            if Key = #27 then begin
              NeedNextEpoch := false;
              Break;
            end;
          end;
 
          FillChar(BAInput,    MAX_IN_SIZE, #0);
          FillChar(BAExpected, MAX_OUT_SIZE, #0);
          FillChar(BAOutput,   MAX_OUT_SIZE, #0);
          Txt1 := UTF8ToConsole(UTF8LowerCase(DS.FieldByName('mrkorig').AsString));
          Res := UTF8ToConsole(DS.FieldByName('expc').AsString);
          MM := DS.FieldByName('mmid').AsInteger;
 
          L := Length(Txt1);
          if L > HIGH_IN_VAL then
          begin
            L := HIGH_IN_VAL;
          end;
          Move((@(Txt1[1]))^, (@(BAInput[0]))^, L);
          SetCorrectOut(BAExpected, MM);
          NN.Compute(BAInput);
          NN.GetOutput(BAOutPut);
          NN.Backpropagate(BAExpected);
 
          MM1 := GetCorrectOut(BAOutput);
          if MM <> MM1 then Inc(ErrorCnt);
          Outv := GetCorrectOutList(BAOutput);
          WriteLn
          ( 'In:',
            ConsoleToUTF8(PChar(@(BAInput[0]))),
            ' Out:',
            Outv,
            ' Expect:',
            Res
          );
          DS.Next;
        end;
      finally
        DS.Close;
      end;
    end;
  finally
    NN.Free;
    Write('Press ENTER to exit.');
    ReadLn;
  end;  
end;

There is a feeling that I have chosen the wrong approach. I ask you to suggest how best to approach the solution of this problem. If it is not difficult, please give an example of solving similar problems of text analysis.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 06, 2021, 11:24:18 am

Hi iLya2IK!
Thank you for sharing your source code example with bleeding edge new features from the API.

If I understood your code, you are placing characters into bytes. Although doable, this is an unusual approach but I must say that I have curiosity to find what the NN will find...

Have you tried much lower learning rates such as 0.001? I would give a try to:

Code: Pascal [Select][+]

    NN.SetLearningRate(0.001, 0.9);
    NN.SetLearningRate(0.0001, 0.9);

Usually, when you see no convergence at all, you can try lower learning rates.

Are you able to share your output?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: iLya2IK on July 06, 2021, 02:46:47 pm

Quote from: schuler on July 06, 2021, 11:24:18 am

Are you able to share your output?

Thanks for the answer!

I'm afraid there is nothing to show :(. For any parameters, the approximation degrades to the output of an empty set (the neural network produces zero bits as a result after a large number of training rounds).
I think I have found a possible alternative for building a neural network. If something works out, I will certainly post the results and the source code.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 26, 2021, 07:16:09 am

:) Hello Pascal Lovers! :)

Besides wishing a long and prosper life to Lazarus/FPC, after 6 months without showing up here, I would like to comment some enhancements recently done to the API.

As usual, I’ll type around the CIFAR-10 https://www.cs.toronto.edu/~kriz/cifar.html (https://www.cs.toronto.edu/~kriz/cifar.html) image classification task. First, I would like to comment things that you can now do to increase the learning capability of your neural network without adding more neurons/connections to it.

Just by updating the library to the latest library on github, CreateCifar10Volumes and CreateCifar100Volumes now load 2000 images for validation instead of 10000. This small change will push the classification accuracy a bit up. Follows an example:

Code: Pascal [Select][+]

CreateCifar10Volumes(ImgTrainingVolumes, ImgValidationVolumes, ImgTestVolumes, csEncodeRGB, {ValidationSampleSize=}2000);

We have new activation functions:

ReLU6: TNNetReLU6. https://paperswithcode.com/method/relu6 (https://paperswithcode.com/method/relu6)
Swish: TNNetSwish. https://arxiv.org/abs/1710.05941 (https://arxiv.org/abs/1710.05941)
Swish6: TNNetSwish6 combines ideas from the both of above activation functions.
TNNetReLUL activation function allows you to define a lower limit and an upper limit

Regarding TNNetReLU6, TNNetSwish6 and TNNetReLUL, limiting outputs can both prevent overflows and make the learning faster. Follows an example:

Code: Pascal [Select][+]

TNNetReLUL.Create(-3, 3, 0)

You could define a ReLU6 equivalent activation function with:

Code: Pascal [Select][+]

TNNetReLUL.Create(0, 6, 0)

I coded an example with Swish at https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierPaddingCroppingSwish.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierPaddingCroppingSwish.lpr). From experience, Swish doesn’t work well along the first layer or layers. On deeper layers, it works very well. In my experiments, it gives a classification accuracy improvement from 1% to 2%.

In my opinion, one of the most interesting neural network architectures is called ResNet: https://arxiv.org/abs/1512.03385 (https://arxiv.org/abs/1512.03385). The ResNet-20 neural network model has been ported to FPC: https://github.com/joaopauloschuler/neural-api/blob/master/examples/ResNet/ResNet20.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/ResNet/ResNet20.lpr). ResNet-20 model is particularly interesting because it’s very floating-point and parameter efficient image classification architecture. Another version of this model was coded taking advantage of the CAI specific activation function TNNetReLUL: https://github.com/joaopauloschuler/neural-api/blob/master/examples/ResNet/CaiResNet20.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/ResNet/CaiResNet20.lpr).

:) Long and prosperous life to Lazarus and FPC! :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 29, 2022, 08:39:33 pm

:) Hello :)
I found today a scientific paper incorrectly referencing this API (it's super cool to find a scientific paper referencing a free pascal based work - this is possible thanks to lots of people that worked on FPC and Lazarus). In the case that anyone needs to reference it, this is how in BibTeX format:

Code: Pascal [Select][+]

@software{cai_neural_api_2021_5810077,
  author       = {Joao Paulo Schwarz Schuler},
  title        = {CAI NEURAL API},
  month        = dec,
  year         = 2021,
  publisher    = {Zenodo},
  version      = {v1.0.6},
  doi          = {10.5281/zenodo.5810077},
  url          = {https://doi.org/10.5281/zenodo.5810077}
}

BTW, a recommendation via researchgate will be greatly appreciated: https://www.researchgate.net/publication/357717402_CAI_NEURAL_API .

:) Happy Pascal Coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: kagamma on February 17, 2022, 03:59:31 am

It's nice to see Pascal has a very good native NN library. Too bad it doesn't support RNN/LSTM/GRU so I can't move some of my NLP stuff from PyTorch to it :(

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: domasz on March 15, 2022, 10:05:49 am

First of all- amazing work!

I don't understand how to use all those neural nets but I'd love to see some nets as sort of ready-to-use functions for morons like me.

Perhaps this is something you could consider?

What would be great:
1) colorizing black & white photos:
https://github.com/mlhubber/colorize

2) upscale:
https://github.com/Araxeus/PNG-Upscale
https://github.com/IBM/MAX-Image-Resolution-Enhancer

3) removing background:
https://github.com/danielgatis/rembg

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on April 05, 2022, 02:20:47 pm

@domasz,
regarding upscaling, there is a ready to use example:
https://github.com/joaopauloschuler/neural-api/tree/master/examples/SuperResolution

You can find some of the testing at:
https://github.com/joaopauloschuler/neural-api/issues/26

Thank you for the other ideas.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on September 18, 2022, 07:42:17 pm

Hello Joao,

I just wanted to tell you that this is a new approach in neural networks which are limited a little too much to Python and OpenCV (apart from Darknet).

The Pascal Lazarus is a great development tool that has the advantage of being independent of obscure external libraries and working on the 3 (4 with Android) main platforms, Linux Mac and Windows.

I hope you will continue to develop CAI.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 22, 2022, 06:48:19 pm

:) Hello :)

I have a selfish post to make: yesterday, I finished my PhD course in AI. YAY!

My PhD thesis certainly can give clues to where this API is heading: super efficient neural networks.

:) I wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: nummer8 on November 22, 2022, 07:42:15 pm

Congrats,

Finishing a PhD is a hell of a job.
Enjoy!!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on November 22, 2022, 10:07:58 pm

Quote from: schuler on November 22, 2022, 06:48:19 pm

:) Hello :)

I have a selfish post to make: yesterday, I finished my PhD course in AI. YAY!

Many, many congratulations! Dr Schuler.
By coincidence I referred to your work at this forum today...

I will owe you some beers and a good meal when you will visit the Netherlands.
Yay,
Thaddy

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 22, 2022, 11:05:59 pm

Dear Thaddy,
You supported my work in its very early days. I'm very thankful for the positive thoughts you've provided me.

@nummer8, it's very a hard work indeed.

:D Dr. Schuler :D

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: metis on November 24, 2022, 12:02:09 pm

@Dr. Schuler

Congratulations !
Got for Academics Beer&Food and Sun in Tenerife. ;)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on November 24, 2022, 03:20:36 pm

Well, with a last name like that, you obviously felt the pressure to become a scholar O:-)
( for others: https://de.wikipedia.org/wiki/Scholar )

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on December 09, 2022, 11:45:22 am

Dr Schuler :D,

Congratulation!!!

Great job.

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 10, 2022, 04:26:26 am

@metis and @Dzandaa,
Thank you!

@Thaddy,

Quote

Well, with a last name like that, you obviously felt the pressure to become a scholar.

You certainly have a point.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 10, 2022, 04:46:52 am

... the coding continues ...

The paper Searching for Activation Functions (https://arxiv.org/abs/1710.05941) describes the search for a better activation function than ReLU. In their work, the authors found Swish to be the best replacement for ReLU. The downside of Swish is: it requires a lot of computation to calculate it. Later, the paper Searching for MobileNetV3 (https://arxiv.org/pdf/1905.02244v5.pdf) introduces the Hard Swish activation function. The Hard Swish gives similar results to Swish with a lot less computation.

I've just added to the source code the Hard Swish implementation and done new runs comparing ReLU, Swish and Hard Swish. Follow source codes and results.

Source codes are:

ReLU: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifier.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifier.lpr).
Swish: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierSwish.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierSwish.lpr).
Hard Swish: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierHardSwish.lpr (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/SimpleImageClassifierHardSwish.lpr).

Results are (CIFAR-10 test classification accuracy):

ReLU: 85.53% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifier20221206.csv (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifier20221206.csv).
Swish: 86.55% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierSwish20221207.csv (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierSwish20221207.csv).
Hard Swish: 86.82% - https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierHardSwish20221208.csv (https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleImageClassifier/results/SimpleImageClassifierHardSwish20221208.csv).

Looks pretty good to me!

:) Wish everyone happy pascal coding. :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on December 10, 2022, 04:10:42 pm

Hi,
thank you for the update.

Have you tested the "mish" activation?
https://ai-scholar.tech/en/articles/treatise/mish-ai-374 (https://ai-scholar.tech/en/articles/treatise/mish-ai-374)

It seems easy to implement.

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 11, 2022, 04:14:55 am

@Dzandaa,
It looks good.

I've just added a feature request to this: https://github.com/joaopauloschuler/neural-api/issues/106 .

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 21, 2022, 06:14:42 pm

DrDub from GitHub coded a simple web server for a neural network that receives an image and classifies it. I did plenty of updates on the top of his code. The code can be found at:

https://github.com/joaopauloschuler/neural-api/tree/master/examples/ResNet/server

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on January 09, 2023, 05:34:18 pm

:) Hello :)
Just finished updating the autoencoder example: https://github.com/joaopauloschuler/neural-api/tree/master/examples/VisualAutoencoder .

This example uses the freshly tested (but coded long ago) TNNetUpsample layer for increasing resolution in the decoder side. TNNetUpsample converts channels (depth) into spatial data. For example, a 128x128x256 activation map will be converted to 256x256x64. This layer type does a similar job to Tensorflow's tf.nn.depth_to_space.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on February 12, 2023, 12:26:59 pm

Your sources are available on 2 platforms
Which are the most recent?

Thank you.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 25, 2023, 08:15:26 am

:) Hello :)

Sorry for being away for so long.

Given that everyone is talking about chatGPT and language models in general, I'll join the trend.

I've been doing experiments with chatGPT 3.5, chatGPT 4 and Hugging Face. For some questions, they all fail terribly. For other questions, one or another gives a good reply.

Anyway, follows a very interesting experiment about Free Pascal.

Schuler:

Hello. I hope that you are doing well. I'm trying to understand the following free pascal source code. Do you know what it does please?

Code: Pascal [Select][+]

NN := TNNet.Create();
NN.AddLayer([
  TNNetInput.Create(32, 32, 3), //32x32x3 Input Image
  TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  TNNetMaxPool.Create({Size=}2),
  TNNetConvolutionReLU.Create({Features=}32, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  TNNetMaxPool.Create({Size=}2),
  TNNetConvolutionReLU.Create({Features=}32, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  TNNetFullConnectReLU.Create({Neurons=}32),
  TNNetFullConnectLinear.Create(NumClasses),
  TNNetSoftMax.Create()
]);

GPT-3.5 :

The given source code is creating a neural network model using the Free Pascal programming language. Here's a breakdown of what the code does:

NN := TNNet.Create(); - This line creates an instance of a neural network model called NN using the TNNet class.

NN.AddLayer([...]); - This line adds layers to the neural network model. Each layer is specified as an array of objects representing different types of layers in the network.

a. TNNetInput.Create(32, 32, 3) - This layer represents the input layer of the network and expects 32x32x3 input images (32 pixels width, 32 pixels height, and 3 color channels for RGB).

b. TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0) - This layer is a convolutional layer with Rectified Linear Unit (ReLU) activation. It performs 2D convolutions on the input images with 16 features (output channels), a feature size of 5x5, no padding, a stride of 1, and bias not suppressed.

c. TNNetMaxPool.Create({Size=}2) - This layer is a max pooling layer that downsamples the input by selecting the maximum value within a 2x2 window.

d. The above two layers (convolution + max pooling) are repeated twice with different feature sizes (32 features) but with the same configuration.

e. TNNetFullConnectReLU.Create({Neurons=}32) - This layer is a fully connected layer with ReLU activation. It connects all the neurons from the previous layer to the current layer with 32 neurons.

f. TNNetFullConnectLinear.Create(NumClasses) - This layer is a fully connected layer with linear activation. It connects the previous layer to the output layer, where NumClasses represents the number of classes in the classification task.

g. TNNetSoftMax.Create() - This layer is the output layer that applies the softmax function to produce probability scores for each class.

By adding these layers to the neural network model, the code creates a convolutional neural network (CNN) architecture commonly used for image classification tasks. The specific architecture details (number of layers, feature sizes, etc.) may vary depending on the specific problem domain and requirements.

Schuler :
Regarding the provided source code above, can you rewrite it adding more and better source code comments please?

GPT-3.5 :
Certainly! Here's the rewritten source code with more comprehensive comments:

Code: Pascal [Select][+]

NN := TNNet.Create(); // Create a new neural network model
 
NN.AddLayer([
  TNNetInput.Create(32, 32, 3), // Input layer: Expects 32x32x3 input images (32 pixels width, 32 pixels height, and 3 color channels for RGB)
  
  TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  // Convolutional layer with ReLU activation:
  // - Features: 16 output channels
  // - FeatureSize: 5x5 filter size
  // - Padding: 0 (no additional padding)
  // - Stride: 1 (movement of the filter by 1 pixel at a time)
  // - SuppressBias: 0 (include bias parameters)
 
  TNNetMaxPool.Create({Size=}2), // Max pooling layer: Downsampling the input by selecting the maximum value within a 2x2 window
 
  TNNetConvolutionReLU.Create({Features=}32, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  // Another Convolutional layer with ReLU activation:
  // - Features: 32 output channels
  // - FeatureSize: 5x5 filter size
  // - Padding: 0
  // - Stride: 1
  // - SuppressBias: 0
 
  TNNetMaxPool.Create({Size=}2), // Another Max pooling layer
 
  TNNetConvolutionReLU.Create({Features=}32, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  // Another Convolutional layer with ReLU activation:
  // - Features: 32 output channels
  // - FeatureSize: 5x5 filter size
  // - Padding: 0
  // - Stride: 1
  // - SuppressBias: 0
 
  TNNetFullConnectReLU.Create({Neurons=}32), // Fully connected layer with ReLU activation: Connects all neurons from the previous layer to the current layer with 32 neurons
 
  TNNetFullConnectLinear.Create(NumClasses), // Fully connected layer with linear activation: Connects the previous layer to the output layer with 'NumClasses' number of neurons/classes
 
  TNNetSoftMax.Create() // Softmax layer: Produces probability scores for each class
]);
 

--
Additional comments from Schuler:
GPT-3.5 was able to vastly go beyond the original source code comments. The only mistake I can see is "repeated twice". I would say "repeated" or "repeated once".

What is now clear to me: GPT-3.5 can help at explaining existing free pascal source code and can help at adding documentation.

Another topic: I'll be at https://www.pascalcongress.com/ ...

--

I then asked the very first question to GPT-4 and this is what I've got:

GPT-4 :
This Pascal code is defining a Convolutional Neural Network (CNN) using the TNNet library. CNNs are a class of deep neural networks, most commonly used to analyze visual images.

Let's break down the code:

TNNet.Create(): This line is creating a new instance of a neural network object.

NN.AddLayer(): This method is adding layers to the neural network. In this case, the network contains the following layers:

TNNetInput.Create(32, 32, 3): This is the input layer of the neural network. It's expecting input images of size 32x32 with 3 channels (probably RGB color images).

TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0): This is a convolutional layer with a Rectified Linear Unit (ReLU) activation function. It uses 16 filters, each with a size of 5x5. There's no padding and the stride is 1. SuppressBias is set to 0, which means bias is not suppressed (bias is a constant value added to the weighted inputs and the bias before passing through the activation function).

TNNetMaxPool.Create({Size=}2): This is a max pooling layer, which reduces the spatial size of the representation to reduce the amount of parameters and computation in the network. This layer is using a 2x2 pooling window.

Another convolutional ReLU layer and max pooling layer are added, this time with 32 filters.

TNNetFullConnectReLU.Create({Neurons=}32): This is a fully connected layer with a ReLU activation function. The layer has 32 neurons.

TNNetFullConnectLinear.Create(NumClasses): This is a fully connected linear layer. The number of neurons equals to the number of classes in the problem at hand, which is defined elsewhere in the code as NumClasses.

TNNetSoftMax.Create(): The final layer is a SoftMax layer, which outputs a vector that represents the probability distributions of a list of potential outcomes. It's most commonly used in applied machine learning, where it can be used for multi-class classification.

So, the code is building a Convolutional Neural Network with specific architecture to be trained and used for image classification tasks. The specific task would be defined by the training data and the NumClasses variable.

--

More comments from Schuler: GPT-4 hasn't made a single error explaining this code. Users could also ask about "please clarify further this line or that line", "please teach me the concept of ...", "please improve this portion of code" (would not make sense as my source code is perfect... O:-) ).

--
HuggingChat OpenAssistant/oasst-sft-6-llama-30b:

This Free Pascal source code appears to define an object called NN, which is an instance of class TNNet from some package. From looking at this code snippet alone, we can deduce that this code creates layers in NN using methods from different classes defined within the TNNet module (or other dependent packages). These classes include TNNetInput for input data layer, TNNetConvolutionReLU/TNNetMaxPool sequence for convolutional layers, TNNetConvolutionReLU/TNNetMaxPool again and then FullConnect layers before finishing off with a softmax output activation function (SMO) layer, ultimately producing outputs for multiple different classes NumClasses. But without knowing more details about how these components work together it’s impossible to tell precisely what these weights compute specifically.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on May 25, 2023, 04:19:38 pm

Hi
A little scary isn't it?
Especially since ChatGPT was trained in part with data from GitHub without asking for approval. :D

One question:
Do you have an idea for training a network to detect atoms from spectra?

I thought about an auto-encoder but I'm stuck for the rest

Have a nice day

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on May 26, 2023, 05:48:22 pm

@dzandaa,
regarding "scary", I would look at the following from gpt 3.5:

Code: Pascal [Select][+]

  TNNetConvolutionReLU.Create({Features=}16, {FeatureSize=}5, {Padding=}0, {Stride=}1, {SuppressBias=}0),
  // Convolutional layer with ReLU activation:
  // - Features: 16 output channels
  // - FeatureSize: 5x5 filter size
  // - Padding: 0 (no additional padding)
  // - Stride: 1 (movement of the filter by 1 pixel at a time)
  // - SuppressBias: 0 (include bias parameters)

The comments above are correct. It (or him - I don't know) guessed by itself "16 output channels", "no additional padding", "movement of the filter by 1 pixel at a time" and "include bias parameters". All these guesses are correct. The most impressive of all, "SuppressBias" is very particular to this API...

Still regarding "scary", I hope people will not try to use these technologies for automating decision making. The chances for a disaster are total. I also fear that people might end blindly believing on whatever these technologies say. It's absolutely important for users to keep critical thinking when using social media, search engines and language models. Language models could be used to provide information that fits a specific narrative and users would think that they arrive to conclusions by themselves when they were actually manipulated into certain beliefs.

Anyway, it's a fantastic technology.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 30, 2023, 02:18:42 am

:) Hello :)

I've just published the source code and trained neural networks for the Malaria, Colorectal Cancer and Plant Village image classification datasets. The test classification accuracies are 95.63%, 94.26% and 99.03% respectively. I showed these models (neural networks) for the first time in the International Pascal Congress of 2023. These models are now public.

You can find details (source code, dataset, trained model for download) at:
https://github.com/joaopauloschuler/pre-trained-neural-api-networks

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on July 30, 2023, 03:44:04 pm

Hi,

Thank you very much,
I'll test it tomorrow.

Happy Coding!!!

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 30, 2023, 11:15:59 pm

Quote from: Dzandaa on July 30, 2023, 03:44:04 pm

I'll test it tomorrow.

I'll add code examples for loading and testing trained models. I'll let you know when ready.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 31, 2023, 05:47:34 pm

Quote from: Dzandaa on July 30, 2023, 03:44:04 pm

I'll test it tomorrow.

I've just added examples to make the testing easier. For testing only one image, you can follow this example:

Code: Pascal [Select][+]

  procedure DoRun;
  var
    NN: TNNet;
    ImageFileName: string;
    NeuralFit: TNeuralImageFit;
    vInputImage, vOutput: TNNetVolume;
    InputSizeX, InputSizeY, NumberOfClasses: integer;
  begin
    WriteLn('Loading Neural Network...');
    NN := TNNet.Create;
    NN.LoadFromFile('SimplePlantLeafDisease-20230720.nn');
    NN.DebugStructure();
    InputSizeX := NN.Layers[0].Output.SizeX;
    InputSizeY := NN.Layers[0].Output.SizeY;
    NumberOfClasses := NN.GetLastLayer().Output.Size;
 
    NeuralFit := TNeuralImageFit.Create;
    vInputImage := TNNetVolume.Create();
    vOutput := TNNetVolume.Create(NumberOfClasses);
 
    ImageFileName := 'plant/Apple___Black_rot/image (1).JPG';
    WriteLn('Loading image: ',ImageFileName);
 
    if LoadImageFromFileIntoVolume(
      ImageFileName, vInputImage, InputSizeX, InputSizeY,
      {EncodeNeuronalInput=}csEncodeRGB) then
    begin
      WriteLn('Classifying the image:', ImageFileName);
      vOutput.Fill(0);
      NeuralFit.ClassifyImage(NN, vInputImage, vOutput);
      WriteLn('The image belongs to the class of images: ', vOutput.GetClass());
    end
    else
    begin
      WriteLn('Failed loading image: ',ImageFileName);
    end;
 
    vInputImage.Free;
    vOutput.Free;
    NeuralFit.Free;
 
    WriteLn('Press ENTER to quit.');
    ReadLn();
 
    NN.Free;
  end;
 

The above source code is located at:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/TestPlantLeafDiseaseTrainedModelOneImage.pas

If you would like to test against the actual training dataset, you can follow this example:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimplePlantLeafDisease/TestPlantLeafDiseaseTrainedModel.pas

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on August 01, 2023, 12:13:55 pm

Hi,

I have problem with TrainingAccuracy, my program is running well, results are good but TrainingAccuracy is always Zero.

I've send you a personal message with code.

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on August 02, 2023, 06:10:17 pm

Hi Schuler,

I tested "plant leaf diseases" and "Colorectal" networks and it's working :)

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 27, 2023, 12:10:56 am

@Dzandaa,
What a beautiful screenshot! Wondering if I can find your code in an open repository...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 27, 2023, 12:48:23 am

:) Hello pascal lovers :) ,
I did an experiment few days ago that I would like to share. The experiment is about predicting the next character in a string.

To make things very simple, the dataset has 3 strings:

Code: Pascal [Select][+]

    FDataset[0] := 'happy good morning.'+chr(1);
    FDataset[1] := 'fantastic good evening.'+chr(1);
    FDataset[2] := 'superb good night.'+chr(1);

The neural network is built with:

Code: Pascal [Select][+]

    FNN.AddLayer([
      TNNetInput.Create(csContextLen, 1, csVocabSize),
      TNNetPointwiseConvReLU.Create(32),
      TNNetFullConnectReLU.Create(32),
      TNNetFullConnectReLU.Create(32),
      TNNetFullConnectLinear.Create(csVocabSize),
      TNNetSoftMax.Create()
    ]);

The constants above are defined with:

Code: Pascal [Select][+]

const
  csContextLen = 64;  // The input string can have up to 64 characters.
  csVocabSize  = 128; // Character based vocabulary/dictionary.
  csMinSampleSize = 3; // Minimum of 3 characters.
 

After training the NN, I tested with:

Code: Pascal [Select][+]

    WriteLn('Testing.');
    WriteLn(GenerateStringFromChars(FNN, 'happy'));
    WriteLn(GenerateStringFromChars(FNN, 'fantastic'));
    WriteLn(GenerateStringFromChars(FNN, 'superb'));
 

Then NN gets it 100% right with the following output:

Quote

happy good morning.
fantastic good evening.
superb good night.

How does it work?
Each character is encoded into a number from 0 to 127. This number from 0 to 127 is then transformed into a vector with 128 elements where only one element values 1. This is called “one-hot encoding” (https://en.wikipedia.org/wiki/One-hot). This is why the input is defined with (csContextLen, 1, csVocabSize). Then, to decrease the dimensionality from 128 dimensions in the vector, a pointwise convolution is used (TNNetPointwiseConvReLU). Each input character is converted into an 128 elements vector in reverse order so “good” will become “doog”. The last layer of the NN has 128 elements so the element with highest value (or probability) is the next predicted character.

The one-hot encoding is done with one API call:

Code: Pascal [Select][+]

procedure TVolume.OneHotEncodingReversed(aTokens: string); overload;

In the case that your input is not a string, you could call the following instead:

Code: Pascal [Select][+]

procedure TVolume.OneHotEncoding(aTokens: array of integer); overload;

So, for each character predicted, this character is added to the input string and the process is repeated until a termination token is found (chr(0) or chr(1)). This is how calling GenerateStringFromChars(FNN, 'superb') outputs ‘superb good night’.

The source code can be found at:
https://github.com/joaopauloschuler/neural-api/blob/master/examples/StringManipulation/StringManipulation.lpr

:) I wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on November 27, 2023, 03:50:13 pm

Hi,

Another update :)
Thank you.

Do you know this ?

https://analyticsindiamag.com/beginners-guide-to-text-generation-with-rnns/ (https://analyticsindiamag.com/beginners-guide-to-text-generation-with-rnns/)

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on November 28, 2023, 04:37:48 pm

@Dzandaa,
Your question is intelligent and very proper.

At this moment, I'm not coding RNNs nor I have it clear if I will ever code it. To the other readers of this forum that are interested in RNNs, I would recommend the following:

https://www.youtube.com/watch?v=AsNTP8Kwu80 - RNN
https://www.youtube.com/watch?v=YCzL96nL7j0 - LSTM
https://www.youtube.com/watch?v=L8HKweZIOmg - seq2seq
https://www.youtube.com/watch?v=PSs6nxngL6k - Attention

BTW, it's not crazy to think about NLP with convolutional layers:
https://towardsdatascience.com/nlp-with-cnns-a6aa743bdc1e
https://dennybritz.com/posts/wildml/understanding-convolutional-neural-networks-for-nlp/
https://slds-lmu.github.io/seminar_nlp_ss20/convolutional-neural-networks-and-their-applications-in-nlp.html

If you google for "nlp convolutional neural network", you'll find plenty of info.

The example above is a "hello world". ATM, I have 4 VMs running experiments in this area...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on November 29, 2023, 01:07:14 pm

Hi,

@schuler

I reproduce your String Manipulation program using Forms and chart.

Feel free to change, share (check errors!!!) and comment.

Don't forget to change AVX and OpenCL in Project Options!!!

I just tested it with a NVidia card on Windows and Linux Mint.

You have to download CAI from schuler on github : https://github.com/joaopauloschuler/neural-api

and put the "neural" directory from CAI in ../neural , or you can change the path in Options->Project Options->Path

Have a good day.

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 02, 2023, 07:34:31 am

I've just pushed a tremendous memory optimization (I still have more optimizations to go). It dropped the single thread memory requirement from 4.5GB to 1.7GB for the model below. ATM, each thread consumes an extra 0.6GB.
The model below has 45.6 million parameters also intended to guess the next character in a string from a context of up to 81 characters:

Code: Pascal [Select][+]

    FNN := TNNet.Create();
    NFit := TNeuralDataLoadingFit.Create();
 
    FNN.AddLayer([
      TNNetInput.Create(csContextLen, 1, csVocabSize),
      TNNetPointwiseConv.Create(32,0),
      TNNetPadXY.Create(1,0),
      TNNetConvolutionReLU.Create(64,3,0,1,0),
      TNNetMaxPool.Create(3),
      TNNetPadXY.Create(1,0),
      TNNetConvolutionReLU.Create(128,3,0,1,0),
      TNNetMaxPool.Create(3),
      TNNetFullConnectReLU.Create(1024*6),
      TNNetFullConnectReLU.Create(1024*6),
      TNNetFullConnectLinear.Create(csVocabSize),
      TNNetSoftMax.Create()
    ]); 
    NFit.MaxThreadNum := 1;
    NFit.LogEveryBatches := 100;
    NFit.InitialLearningRate := 0.001;
    NFit.LearningRateDecay := 0;
    NFit.L2Decay := 0;
    NFit.EnableClassComparison();
    NFit.EnableDefaultLoss();
    NFit.AvgWeightEpochCount := 1; // This is very important for large models
    NFit.OnAfterEpoch := @OnAfterEpoch;
    NFit.FitLoading(
      FNN,
      {TrainingVolumesCount=}FDatasetSize*20,
      {ValidationVolumesCount=}FDatasetSize,
      {TestVolumesCount=}FDatasetSize,
      {batchsize=}32,
      {epochs=}50,
      @GetTrainingPair, @GetValidationPair, @GetTestPair
    );
 

I have also pushed code for NLP Samplers:

Code: Pascal [Select][+]

  { TNNetSamplerGreedy }
  TNNetSamplerGreedy = class (TNNetSamplerBase)
    public
      function GetToken(Origin: TNNetVolume): integer; override;
  end;
 
  { TNNetSamplerTopK }
  TNNetSamplerTopK = class (TNNetSamplerBase)
    protected
      FTopK: integer;
    public
      constructor Create(TopK: integer);
      function GetToken(Origin: TNNetVolume): integer; override;
  end;
 
  { TNNetSamplerTopP }
  TNNetSamplerTopP = class (TNNetSamplerBase)
    protected
      FTopP: TNeuralFloat;
    public
      constructor Create(TopP: TNeuralFloat);
      function GetToken(Origin: TNNetVolume): integer; override;
  end;
 

I can now talk to my own NNs with:

Code: Pascal [Select][+]

procedure TestFromFile;
var
  S: string;
  oSampler: TNNetSamplerBase;
  NN: TNNet;
begin
  oSampler := TNNetSamplerTopP.Create(0.4);
  NN := TNNet.Create();
  WriteLn('Loading neural network.');
  NN.LoadFromFile('JP45A01d2.nn');
  NN.DebugStructure();
  WriteLn();
  WriteLn('Write something and I will reply.');
  repeat
    Write('User: ');
    ReadLn(S);
    WriteLn('Neural network: ',GenerateStringFromChars(NN, LowerCase(S), oSampler),'.');
  until S = 'exit';
  NN.Free;
  oSampler.Free;
end; 

And then, I get outputs:

Code: Pascal [Select][+]

User: this is good for
Neural network: this is good for the construction of the great deal with the south core comparti.
User: this is not good for
Neural network: this is not good for the present " consistent with the extended to be seen as r.

The output text is still not human like. But it's improving fast.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: cpicanco on December 07, 2023, 01:18:43 am

Hi schuler, do you consider your system feasible to my speech-to-text needs? Please, let me explain.

I am building a custom database with CVCV (consonant-vowel-consonant-vowel) pseudowords. Each pseudoword may have an audio, text and image association. I call them "psedo" because they have no natural meaning, they were artificially created using a mini alphabet composed by 4 vowels and 4 consonants. As you might guess, my goal is to train a model to recognize those words, my urgent need is a speech-to-text solution.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on December 09, 2023, 05:08:28 am

@cpicanco,
Thank you for considering CAI. CAI is not ready yet for speech recognition. I'll soon have some source code examples in the NLP area but still far from speech recognition. There are mature solutions around for this so I won't drag you into something experimental.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on March 04, 2024, 06:03:16 am

:) Hello :)

In the hope that Neural Networks can be easily learned, I created few videos:

* Basics of Neural Networks in Pascal - Loading and Saving: https://youtu.be/aIy1S7clhQo
* Neural Networks for Absolute Beginners! Learning a Simple Function: https://youtu.be/q56NcgUiAAk
* Coding a Neural Network in Pascal that Learns to Calculate the Hypotenuse: https://youtu.be/PdNTgI_qSyo
* Pre-trained Neural Networks & Transfer Learning with Pascal's CAI Neural API: https://youtu.be/tODsv6Ks2DM
* Coding a Neural Network in Pascal that Learns the OR Boolean Operation: https://youtu.be/f4T9IB-He_k
* A Dive into Identity Shortcut Connection - The ResNet building block: https://youtu.be/o-8NuoSsdck

:) I wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: EkkehardDomning on March 28, 2025, 05:13:01 pm

Hello,
could someone point me to some written material (Books, Websites) in which the terms "ConvolutionLinear", "MaxPool", "ConvolutionReLU", etc. are explained?
It seems to me essential that the understanding of those terms are crucial for the usage of the CAI API.

The following lines are taken from the SimpleImageClassifier:

Code: Pascal [Select][+]

NN := TNNet.Create();
    NN.AddLayer([
      TNNetInput.Create(32, 32, 3),
      TNNetConvolutionLinear.Create({Features=}64, {FeatureSize=}5, {Padding=}2, {Stride=}1, {SuppressBias=}1),
      TNNetMaxPool.Create(4),
      TNNetMovingStdNormalization.Create(),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1, {SuppressBias=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1, {SuppressBias=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1, {SuppressBias=}1),
      TNNetConvolutionReLU.Create({Features=}64, {FeatureSize=}3, {Padding=}1, {Stride=}1, {SuppressBias=}1),
      TNNetDropout.Create(0.5),
      TNNetMaxPool.Create(2),
      TNNetFullConnectLinear.Create(10),
      TNNetSoftMax.Create()
    ]);

I have no idea, why this setup is choosen, nor what to alter for a different result. Some pages are found (like this one: https://www.nickmccullum.com/python-deep-learning/relu-layer-pooling/ (https://www.nickmccullum.com/python-deep-learning/relu-layer-pooling/)) but I would like to have something more descriptive. Looking at an online bookstore is also difficult, because the content is not that deep disclosed.

I would be happy to get some hints.

Best regards
Ekkehard

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on March 28, 2025, 07:22:33 pm

Hi,
@EkkehardDomning:

Read this.

https://www.edureka.co/blog/convolutional-neural-network/
(https://www.edureka.co/blog/convolutional-neural-network/)

And this:

https://sourestdeeds.github.io/blog/convolution-and-relu/
(https://sourestdeeds.github.io/blog/convolution-and-relu/)

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 01, 2026, 03:52:12 pm

:) Hello everyone, I hope you're all doing well. :)

Before I share some major news about this API, I'd like to take a moment to walk through its history.

It began in 2001. In plain Pascal, I built an artificial neural network based on the Combinatorial Neural Model - CNM [1]. With it, I tried to reproduce some characteristics of a "Popperian creature," an idea I borrowed from Daniel Dennett's Kinds of Minds book [2].

That network became the backbone of my master's thesis [3]. I regret never having it translated into English. Twenty-five years later, it lives on, encapsulated in the layers TNNetByteProcessing and TNNetPointwiseByteProcessing.

After a good deal of work modernizing the code, the project was published on SourceForge. I can't pin down the year — perhaps 2012 or 2013. In June 2017, I joined this forum and began dedicating my energy into stochastic gradient descent. In 2019, parts of the project were migrated to GitHub, and that effort eventually became Section 2.2.1 of my PhD thesis, published in 2022 [4].

From 2019 to 2023, the API development was centered on computer vision. In this period, various modern computer vision architectures were added. In 2020, a parameter/memory/CPU efficient super-resolution [8] example was added. In 2023, image classification examples for Colorectal Cancer [5], Plant Disease [6] and Malaria Cell Infection [7] were added. All main ideas of my PhD thesis (grouped pointwise convolutions, color separation) were first tested in Pascal and then ported to other languages and frameworks.

If you have never used this API, a good starting point would be increasing the resolution of your own photos with the super-resolution CLI tool and its pre-trained neural network[8].

Then, in March 2024, I believe I coded the first transformer decoder block ever written in plain Pascal [11]. Shortly after, I implemented a "GPT-3 small" model, also in Pascal [10]. It was also the year when YouTube videos were added [9].

And then came 2026. This is the first year of AI dominated coding for this API. The next update will be colossal: At least 150K lines of code are waiting to be merged into master. But this is for another post…

References

[1] https://pubmed.ncbi.nlm.nih.gov/18255770/
[2] https://ia800501.us.archive.org/20/items/FritjofCapraTheTurningPoint/Daniel%20C.%20Dennett%20-%20Kinds%20of%20Minds%20-%20Toward%20an%20Understanding%20of.pdf
[3] https://www.researchgate.net/publication/357164807_Inteligencia_Artificial_Popperiana
[4] https://www.researchgate.net/publication/365687628_Effective_Approaches_for_Improving_the_Efficiency_of_Deep_Convolutional_Neural_Networks_for_Image_Classification
[5] https://github.com/joaopauloschuler/neural-api/tree/master/examples/ColorectalImageClassification
[6] https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimplePlantLeafDisease
[7] https://github.com/joaopauloschuler/neural-api/tree/master/examples/MalariaImageClassification
[8] https://github.com/joaopauloschuler/neural-api/tree/master/examples/SuperResolution
[9] https://youtu.be/aIy1S7clhQo
[10] https://github.com/joaopauloschuler/gpt-3-for-pascal
[11] https://github.com/joaopauloschuler/neural-api/tree/master/examples/SimpleNLP

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on June 01, 2026, 06:20:56 pm

Hello Joao Paulo,
I'm eagerly awaiting this new update :)

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: fpcoder on June 01, 2026, 07:38:22 pm

Following with interest.

/Roger

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 02, 2026, 10:41:08 pm

@Dzandaa and @kapibara, many thanks for your support!

In the case that you would like to compile/test your existing projects with the new branch (a1), this is the pull request:
https://github.com/joaopauloschuler/neural-api/pull/177

It will be nice to know if your models converge faster/slower/same with the new version. This new version works better with the AVX2 define instead of the previous default for AVX. I intend to fix bugs before merging into master.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: fpcoder on June 04, 2026, 01:26:36 pm

Here's my results. I tested master vs PR #177/a1 on a synthetic sequential geometry-detection task.

Environment:
Linux x86_64
FPC 3.3.1
Laz trunk
11th Gen Intel® Core™ i5-11600K @ 3.90GHz
15,4 GiB of RAM

The task is binary classification over sequential numeric windows. Each sample is a 300-step window with 5 numeric features per step. The positive class represents a generated geometric shape/pattern inside the sequence; the negative class is background/no-pattern data.

Dataset:

Code: Pascal [Select][+]

Total windows: 9,700
Train/Validation/Test: 5,820 / 1,940 / 1,940
Input shape: 300 x 5 x 1
Classes: 2
Batch size: 64
Epochs: 10
 

Training time

Code: Pascal [Select][+]

master, no switches:             41.68 sec
master + -dAVX2:                 27.72 sec
master + -dAVX2 -O3:             27.31 sec
 
PR #177/a1, no switches:         34.03 sec
PR #177/a1 + -dAVX2:             19.30 sec
PR #177/a1 + -dAVX2 -O3:         13.09 sec
 

Evaluation time

Code: Pascal [Select][+]

master, no switches:             ~7.30 sec
master + -O3:                     5.95 sec
master + -dAVX2:                  ~4.62 sec
master + -dAVX2 -O3:              4.25 sec
 
PR #177/a1, no switches:          5.42 sec
PR #177/a1 + -O3:                 3.23 sec
PR #177/a1 + -dAVX2:              1.74 sec
PR #177/a1 + -dAVX2 -O3:          0.95 sec
 

Quality

Code: Pascal [Select][+]

master + -dAVX2 -O3:
  Accuracy:   99.02%
  Precision:  98.97%
  Recall:    100.00%
  F1:         99.48%
  FP/FN:      19 / 0
 
PR #177/a1 + -dAVX2 -O3:
  Accuracy:   99.28%
  Precision: 100.00%
  Recall:     99.23%
  F1:         99.62%
  FP/FN:       0 / 14
 

Main result:
With -dAVX2 -O3, PR #177/a1 trained about 2.1x faster and evaluated about 4.5x faster than master on this test, with similar or slightly better F1.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 04, 2026, 11:14:34 pm

@kapibara, as a way to say “thank you”, if you have any feature that you would like coded, please feel free to ask. If not too difficult, AI will code it :-) .

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on June 10, 2026, 11:50:59 am

Hi Joao-Paulo,

Can me explain me how to create a semantic segmentation of an image?

Is it possible with CAI?

Thank you.

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 10, 2026, 09:11:29 pm

@Dzandaa,
I will add to my task list coding an example for this.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: microxa on June 11, 2026, 05:43:41 am

Hi Paolo! Please take a look at the StringManipulation example — there is still some weird bug there (potentially a glitch in 32-bit FPC 3.2.2). In an earlier version (2.0), this was resolved by reverting to the previous class, i.e., TNNetSoftMax = class(TNNetIdentify).After doing that, the example needed only 5 epochs to reach 100% training accuracy and correct output. Good luck an regards!

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 11, 2026, 04:39:11 pm

@microca,
Thank you for the bug report. I will look at it.

In the case that you are interested, I asked Claude to produce a list of interesting NLP related examples in this API. The below AI generated list has been coded but not yet properly tested.

In the list: Model — Its Building Block: Example showing the building block.

- DeepSeek — MLA + decoupled-RoPE slice (V2): examples/LatentAttention/LatentAttention.lpr (builds MHA vs MLA vs MLA+decoupled-RoPE arms side by side); multi-token prediction (V3): examples/MultiTokenPrediction/MultiTokenPrediction.lpr (also used by examples/SelfSpeculativeDecoding/SelfSpeculativeDecoding.lpr).
- LLaMA — pre-norm RMSNorm residual blocks: examples/PreNormVsPostNorm/PreNormVsPostNorm.lpr; SwiGLU: examples/SwiGLUFeedForward/SwiGLUFeedForward.lpr; RoPE: examples/RoPEBaseFrequencySweep/RoPEBaseFrequencySweep.lpr (also compared in examples/PositionEncodingBakeoff/PositionEncodingBakeoff.lpr).
- Mistral / Longformer — sliding-window causal mask: examples/SlidingWindowBakeoff/SlidingWindowBakeoff.lpr.
- Gemma — logit soft-capping: examples/SoftCappingStability/SoftCappingStability.lpr and examples/SoftCappingSweep/SoftCappingSweep.lpr; GeGLU: examples/GEGLUFeedForward/GEGLUFeedForward.lpr (also in examples/GatedFFNBakeoff/GatedFFNBakeoff.lpr).
- BERT / GPT — GELU: examples/ActivationBakeoff/ActivationBakeoff.lpr; GPT-style decoder blocks: examples/TransformerDecoderBlock/TransformerDecoderBlock.lpr (full pipeline in examples/SimpleNLP/TransformerWithTokenizer.lpr).
- Mamba — TNNetSelectiveSSM: examples/SelectiveSSM/SelectiveSSM.lpr.
- Modern recurrent family — RWKV-4: examples/RWKV/RWKV.lpr (cross variant: examples/CrossWKV/CrossWKV.lpr); xLSTM sLSTM: examples/SLSTMvsCfC/SLSTMvsCfC.lpr; xLSTM mLSTM: no example yet (TNNetMLSTMCell has tests but no examples/ program); RetNet: examples/RetentionDualForm/RetentionDualForm.lpr; Titans: examples/TitansMemory/TitansMemory.lpr; DeltaNet: examples/DeltaNet/DeltaNet.lpr; GLA: examples/GatedLinearAttention/GatedLinearAttention.lpr (block form: examples/GatedLinearAttentionBlock/GatedLinearAttentionBlock.lpr).
- Switch Transformer (load-balance loss): examples/TopKMoE/TopKMoE.lpr (contrast without aux loss: examples/ExpertChoiceMoE/ExpertChoiceMoE.lpr).
- DIFF Transformer: examples/DifferentialAttentionNoise/DifferentialAttentionNoise.lpr.
- StreamingLLM (attention sinks): examples/SinkAttentionStability/SinkAttentionStability.lpr.
- BLOOM-lineage ALiBi: examples/ALiBiSlopeSweep/ALiBiSlopeSweep.lpr (also in examples/PositionEncodingBakeoff/PositionEncodingBakeoff.lpr).
- GQA/MQA via KVHeads: no example yet — the KVHeads parameter exists on the MHA builders (neural/neuralnetwork.pas:11123) and GQA is discussed in examples/LatentAttention/README.md as a comparison point, but no example program actually passes KVHeads <> Heads.

This is a good starting point: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleNLP/DecodeFeaturesBakeoff.lpr (includes some building blocks for DeepSeek v2 and v3)

@all, the list of examples is located at: https://github.com/joaopauloschuler/neural-api/blob/master/examples/README.md

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 13, 2026, 03:56:48 pm

Quote from: Dzandaa on June 10, 2026, 11:50:59 am

Hi Joao-Paulo,

Can me explain me how to create a semantic segmentation of an image?

Is it possible with CAI?

Thank you.

B->

@Dzandaa, as a source of inspiration for both of us, have a look at please: https://github.com/joaopauloschuler/neural-api/blob/master/neural/neuralpretrained.pas

The master branch is under heavy coding. Therefore, expect bugs. Anyway, I could extend neuralpretrained.pas to support a vision model or models that will do your segmentation.

Is there any image segmentation model from hugging face that you would like supported in this API?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on June 13, 2026, 05:59:22 pm

Hi Joao-Paulo

Thank you, I'll read that next week.

I had found an interesting web page that explained the different parts of a Hugging Face segmentation, with images of the different intermediate layers, but I can't find it anymore, sorry.

Here is 2 interesting links:

https://github.com/huggingface/blog/blob/main/mask2former.md (https://github.com/huggingface/blog/blob/main/mask2former.md)

https://medium.com/@az.tayyebi/segformer-model-training-from-scratch-vs-using-hugging-face-f5234f31399a (https://medium.com/@az.tayyebi/segformer-model-training-from-scratch-vs-using-hugging-face-f5234f31399a)

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 18, 2026, 02:42:53 am

This is to share an experiment. It is not in the master branch yet. It is still slow and memory hungry. But it is cool to see it.

Code: Pascal [Select][+]

# git clone -b a3 https://github.com/joaopauloschuler/neural-api.git
# git clone https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct q2
# lazbuild neural-api/examples/ChatTerminal/ChatTerminal.lpi
# neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --ctx 512 --fp32
Loading q2/ ...
Model: qwen2, 630139776 params, vocab 151936, context 512, chat format qwen, fp32 weights.
Type your message; /exit quits, /reset clears the history,
/system <msg> sets the system prompt.
> Hello super-intelligence!
Hello! How can I assist you today?
> What is the capital of France please?
The capital of France is Paris.
> What is the spoken language there?
The spoken language in Paris is French.
> Is it a nice place for a holliday?
Paris is a beautiful city with a rich history and culture. It's a great place for a romantic getaway, especially during the summer months when the weather is pleasant. The city is known for its beautiful architecture, charming neighborhoods, and delicious food. However, it's important to note that Paris is a bustling city with a lot of activity, so it's important to plan ahead and make sure you have enough time to explore and enjoy the city.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 19, 2026, 12:06:57 am

I have another slow, memory hungry, 100% pascal coded and cool to see experiment to share. This is the very first experiment that produces music (see attached wav transformed into mp4 file via ShotCut) using this API.

Code: Pascal [Select][+]

# git clone -b a3 https://github.com/joaopauloschuler/neural-api.git
# lazbuild neural-api/examples/MusicGenText/MusicGenText.lpi
# neural-api/bin/x86_64-linux/bin/MusicGenText --download --seconds 5 --prompt "warm lo-fi hip hop beat with mellow piano"
MusicGen TEXT-CONDITIONED generation - prompt -> T5 -> music -> WAV
==================================================================
Downloading checkpoints via HuggingFace Hub (cache: /root/.cache/neural-api/hub)...
[time] fetch facebook/musicgen-small: 11.62 s
[time] fetch t5-base: 3.17 s
[time] fetch facebook/encodec_32khz: 4.85 s
Prompt: "warm lo-fi hip hop beat with mellow piano"
 
[time] T5 tokenizer load: 299 ms
Target length: 5.0 s -> 250 frames.
[time] T5 model load (/root/.cache/neural-api/hub/t5-base/main/model.safetensors): 9.55 s
T5 text encoder: t5 config: enc_layers=12, dec_layers=12, heads=12, d_model=768, d_kv=64, d_ff=3072, vocab=32128, rel_buckets=32, rel_max_dist=128, ln_eps=9.999999975E-7, tied=True
Prompt token ids: 1978 6899 18 89 23 5436 13652 3853 28 3 2341 3216 8355 1
[time] T5 encoder forward pass: 163 ms
Encoder hidden states: 14x768 (seq x d_model)
 
[time] MusicGen decoder load (/root/.cache/neural-api/hub/facebook/musicgen-small/main/model.safetensors): 21.75 s
musicgen config: layers=24, heads=16, hidden=1024, ffn=4096, vocab=2048, codebooks=4, text_d_model=768, max_pos=2048
Generating 250 frames over 4 codebooks via the delay pattern (conditioned on the T5 prompt)...
Sampling: weighted top-k = 250, temperature = 1.00.
Classifier-free guidance ON (scale = 3.00, null = zeroed text condition; KV-cache on (dual-twin)).
[time] MusicGen decode (250 frames): 150.91 s
Code-stack health (4 codebooks x 250 frames, vocab=2048):
  cb0: range [8..2007], distinct=148/250, top id 1314 x10 (4%)
  cb1: range [21..2037], distinct=160/250, top id 1136 x6 (2%)
  cb2: range [39..2043], distinct=150/250, top id 1960 x8 (3%)
  cb3: range [11..2037], distinct=149/250, top id 2037 x8 (3%)
 
[time] EnCodec decoder load (/root/.cache/neural-api/hub/facebook/encodec_32khz/main/model.safetensors): 1.15 s
EnCodec decoder: EnCodec(encodec): sampling_rate=32000 channels=1 hidden=128 filters=64 residual_layers=1 ratios=[8,5,4,4] kernel=7 last_kernel=7 residual_kernel=3 dilation_growth=2 compress=2 lstm_layers=2 codebook_size=2048 codebook_dim=128 num_quantizers=4 causal=False norm=weight_norm
[time] EnCodec waveform synthesis: 112.16 s
Decoded waveform: 160000 samples at 32000 Hz
Wrote musicgen_text_demo.wav (160000 samples, 5.00 s).
 
Done: the prompt drove a real T5 encoder + MusicGen decoder + EnCodec synthesis.
 
 

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Dzandaa on June 19, 2026, 12:25:06 pm

Hi,

@jao paulo:

I tried the MusicGenText program and it seems to work in Windows 10.

Is it possible to change the .cache path, because my P.C. have 2 disks: one for system and another for applications.

I also tried AI Music Generator on Windows 10,

Good work,

Trying now with 15 seconds MusicGenText but it crashes :(

MusicGenText --download --seconds 15 --prompt "warm metal rock beat with piano bass brown guitar bass"

B->

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 19, 2026, 04:36:47 pm

Quote from: Dzandaa on June 19, 2026, 12:25:06 pm

Is it possible to change the .cache path, because my P.C. have 2 disks: one for system and another for applications.

@Dzandaa,
You can define the place where to store models using the environment variable NEURAL_API_HUB_CACHE.

I tested on a google colab 51GB RAM (no gpu) machine with up to 32 seconds generation. It works for me with "--guidance 1.0" and 43.3GB of RAM usage. I believe that it is crashing due to RAM allocation. You can trade quality by memory/speed using the parameter “--guidance 1.0”.

At this moment, CAI is loading some of its training tooling even when running inference only. This is why we are getting unusual RAM allocation. I will fix this eventually.

Only if you can, I am super curious to know if you can run ChatTerminal with Qwen2.5-0.5B-Instruct at your end with “--ctx 512 --fp32 --stats”. I am curious to know how many tokens per second you will get.

:) wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 19, 2026, 04:45:23 pm

Quote from: Dzandaa on June 10, 2026, 11:50:59 am

Can me explain me how to create a semantic segmentation of an image?
B->

Construction has started but it is not ready yet: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SemanticSegmentation

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: microxa on June 20, 2026, 11:17:31 am

"Hi Schuler! How about having Claude adapt the basic code of this fun ANN program for your CNN engine?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 20, 2026, 04:08:40 pm

Quote from: microxa on June 20, 2026, 11:17:31 am

"Hi Schuler! How about having Claude adapt the basic code of this fun ANN program for your CNN engine?

This is a good idea... It's easy to do and awesome to play with...

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: microxa on June 20, 2026, 05:02:44 pm

This example is derived from the following instructional video and modified into a guessing game

https://m.youtube.com/watch?v=nmgCCC0sB-s

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 24, 2026, 07:26:57 pm

:) Hello :)

The ChatTerminal example (see above) now supports OpenCL GPUs.

Command line example:

Code: Bash [Select][+]

git clone https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct q2
lazbuild neural-api/examples/ChatTerminal/ChatTerminal.lpi

Code: Bash [Select][+]

neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --gpu

Output:

Code: Bash [Select][+]

[fp32 weights (default) - GPU capable]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading q2/ ...
[--gpu: OpenCL on NVIDIA CUDA / Tesla T4]
clCreateContext OK!
clCreateCommandQueue OK!
clCreateProgramWithSource OK!
clBuildProgram OK!
clCreateKernel cai_dot_product OK!
Model: qwen2, 630139776 params, vocab 151936, context 1024, chat format qwen, fp32 weights.
[KV-cache reuse ON - only the new prompt tail is prefilled each turn]
Type your message; /exit quits, /reset clears the history,
/system <msg> sets the system prompt.
> Hello! Can you tell me please a bed time story?
Of course! Here's a simple bedtime story I created for you:
 
Once upon a time, in a faraway land, there lived a little mouse named Timmy. Timmy had a big, fluffy bed where he could sleep soundly. One night, while he was playing in the garden, he heard a loud noise coming from the house. Timmy quickly ran to the house to see what was happening.
 
When he got there, he saw a big, scary monster named Mr. Grumpy standing outside the door. Mr. Grumpy was trying to scare Timmy by making a loud noise. Timmy tried to run away,
[stats] 128 tokens, TTFT 1598 ms, prompt 41 (reused 0), decode 28.3 tok/s
> 

https://github.com/joaopauloschuler/neural-api/tree/master/examples/ChatTerminal

I tested it with:

Linux - NVIDIA T4
Linux - NVIDIA L4
Linux - NVIDIA A100
Windows - Intel iRIS
Linux/Windows - no-gpu X86/AVX CPU

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on June 30, 2026, 11:09:12 pm

Just to share an experiment: single AVX2 CPU x Tesla T4 via OpenCL on google colab:

Code: Pascal [Select][+]

neural-api/bin/x86_64-linux/bin/OpenCLForwardBenchmark 10

Code: Pascal [Select][+]

OpenCL: NVIDIA CUDA / Tesla T4
 
layer                            out shape        cpu us/fwd   gpu us/fwd    speedup  gpu?   verdict
--------------------------------------------------------------------------------------------------------
TNNetConvolution                 32x32x1280          85125.0      20281.3      4.20x  yes    yes
TNNetConvolutionLinear           32x32x1280          94625.0       5195.3     18.21x  yes    yes
TNNetDeconvolution               64x64x640          462625.0     307750.0      1.50x  yes    yes
TNNetDepthwiseConv               32x32x1280          14375.0      14750.0      0.97x  yes    yes
TNNetDepthwiseConv1D             2560x1x512          33375.0       3851.6      8.67x  yes    yes
TNNetGroupConvP4                 32x32x1280         255500.0      19437.5     13.14x  yes    yes
TNNetKANConv                     32x32x1280         653749.9      88000.0      7.43x  yes    yes
TNNetDeformableConv              32x32x1280         323125.0      63375.0      5.10x  yes    yes
TNNetFullConnect                 20480x1x1           21468.7        988.3     21.72x  yes    yes
TNNetScaledDotProductAttention   2560x1x64          371625.0     160125.0      2.32x  yes    yes
TNNetLinearAttention             2560x1x64            8406.2      12625.0      0.67x  yes    yes
TNNetCosineSimilarityAttention   2560x1x64          324750.0     138250.0      2.35x  yes    yes
TNNetDisentangledAttention       2560x1x64          624250.0     294375.0      2.12x  yes    yes
TNNetConformerRelPosAttention    2560x1x64          414375.0     268500.0      1.54x  yes    yes
TNNetALiBiAttention              2560x1x64          528125.0     172625.1      3.06x  yes    yes
TNNetRotaryEmbedding             2560x1x128           6289.1       1175.8      5.35x  yes    yes
TNNetGEGLU                       2560x1x512          12812.5       8140.6      1.57x  yes    yes
TNNetGEGLUErf                    2560x1x512          36125.0       5367.2      6.73x  yes    yes
TNNetPointwiseSoftMax            2560x1x512           7921.9       4656.2      1.70x  yes    yes
TNNetMaxPool                     160x16x64            3945.3       1418.0      2.78x  yes    yes
TNNetBilinearResize              640x64x64           32375.0       7812.5      4.14x  yes    yes
TNNetLSTMCell                    256x1x2560        4975250.0    2661875.1      1.87x  yes    yes
TNNetGRUCell                     256x1x2560        3589000.0    1950750.0      1.84x  yes    yes

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: MathMan on July 01, 2026, 10:38:49 am

Quote from: schuler on June 30, 2026, 11:09:12 pm

Just to share an experiment: single AVX2 CPU x Tesla T4 via OpenCL on google colab:

<snip>

@schuler

Interesting - the speedups are lower than I would have expected.

Questions:

- what is the exact AVX2 CPU?
- I assume that the CPU variant is multi-threaded too?
- what floating point format did you use - I assume fp32?

In light of AVX512 availability (any experiments in that direction?) and the upcoming x86_64 ACE extensions the speed difference between CPU and google TPU may become vanishinglingy small soon.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on July 01, 2026, 02:20:11 pm

AVX2 is supported by both Intel and Amd processors for at least 13 years. FPC supports it since v2.7.1.
Its register width is 256. AVX512 is not really mainstream yet for desktops. (My recent Rhyzen 7000 series do not have it, ZEN5 and 6 do have it).
I guess avx512 is less relevant because of deferring to GPU's.

That does not mean you can compare modern AVX2 implementations with those from 13 years ago: there is not only a speed increase because of clock, but also by optimizations of the microcode.
An instruction set just facilitates better performance, it does not say anything about the CPU's true performance.
I mentioned this in light of Rhyzen 7000 series AVX2 outperforming Intel AVX2 clock for clock at this moment.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 01, 2026, 07:17:59 pm

@MathMan,
At the time of the testing, I regret that I did not collect the exact CPU information. The CPU varies from run to run at Google Colab. You are correct, some layers are too slow/far from optimized. CAI has a super efficient AVX based convolutional calculation. So, having 20x speed-up in plain convolutional and fully connected layers is nice. Plenty of the layers listed above are brand new. I have dozens of brand new layer types to optimize. It will be super cool to see these numbers improving as the code is optimized. Yes. I am using FP32 with pas-core-math32.

Anyway, this is to share another experiment. This time loading Qwen 2.5 0.5B while it tells me a bed time story:

Code: Pascal [Select][+]

> Hello super-intelligence! Can you tell me please a bed time story?
Of course! Here''s a bedtime story for you:
 
Once upon a time, in a faraway land, there lived a little mouse named Timmy. Timmy had a big, fluffy bed, and he loved to sleep in it. One night, while he was playing in the garden, he heard a loud noise outside. He quickly ran to the window to see what was happening.
 
As he looked out, he saw a big, scary monster with a long, sharp tail. Timmy was scared and ran back to his bed, but the monster was still there. Timmy tried to run away, but the monster was too strong
[stats] 128 tokens, TTFT 1439 ms, prompt 44 (reused 0), decode 34.4 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       145     2143009.15       14779.37  77.8    100%
TNNetScaledDotProductAttention 336      362998.43        1080.35  13.2      0%
TNNetRotaryEmbedding            48       78001.36        1625.03   2.8      0%
TNNetSwiGLU                     24       51000.53        2125.02   1.9      0%
TNNetSplitChannels             432       50002.24         115.75   1.8       -
TNNetDeepConcat                360       39998.02         111.11   1.5       -
TNNetTokenRMSNorm               49       18998.84         387.73   0.7      0%
TNNetSum                        48        8999.65         187.49   0.3       -
TNNetInput                       1           0.00           0.00   0.0       -
TNNetEmbedding                   1           0.00           0.00   0.0      0%
--------------------------------------------------------------------------------------
TOTAL: 2753008.23 us across 1444 layer(s) in 10 class(es)

CAI decides the CPU/GPU path at runtime.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: microxa on July 01, 2026, 09:52:59 pm

@Schuler, a very, very cool benchmark and highly significant progress! Even despite the weird results obtained on my retro (non AVX-ed) machine! (Visually, when running the Julia fractal shader computation, such an garbage GeForce 9600GT completely smoked the CPU (Core 2 Duo 1.8GHz) by more than 10 times...)"

Code: [Select]

C:\laz48.x64\Projects\neural-api-master\bin\x86_64-win64\bin>OpenCLForwardBenchmark.exe
OpenCL: NVIDIA CUDA / GeForce 9600 GT
Scale factor: 1 (argv[1]; default 1) - applied per layer to its primary dimension
Base profiles: SEQ=(256,1,512)  VIS=(32,32,64)  d_k=64
Dispatch: all layers FORCED via NN.ForceOpenCL(True); the verdict column shows each layer's own FShouldOp
Auto-scaled timing: >= 0,40s/measurement, cap 8192 forwards, 3 warmups

layer                            out shape        cpu us/fwd   gpu us/fwd    speedup  gpu?   verdict
--------------------------------------------------------------------------------------------------------
TNNetConvolution                 32x32x128          146250,0      91625,0      1,60x  yes    yes
TNNetConvolutionLinear           32x32x128          142375,0      91625,0      1,55x  yes    yes
TNNetDeconvolution               64x64x64           115125,0      99500,0      1,16x  yes    yes
TNNetDepthwiseConv               32x32x128            7312,5       8531,3      0,86x  yes    yes
TNNetDepthwiseConv1D             256x1x512            3906,2       5117,2      0,76x  yes    yes
TNNetGroupConvP4                 32x32x128          150125,0      99375,0      1,51x  yes    yes
TNNetKANConv                     32x32x128          602500,0     360749,9      1,67x  yes    yes
TNNetDeformableConv              32x32x128          315875,0     224250,0      1,41x  yes    yes
TNNetFullConnect                 2048x1x1             8781,3       4992,2      1,76x  yes    yes
TNNetEmbedding                   256x1x512             365,7       3289,1      0,11x  yes    no
TNNetScaledDotProductAttention   256x1x64            18031,2      24875,0      0,72x  yes    yes
TNNetLinearAttention             256x1x64             4632,8       9750,0      0,48x  yes    yes
TNNetCosineSimilarityAttention   256x1x64            17562,5      23875,0      0,74x  yes    yes
TNNetDisentangledAttention       256x1x64            26312,5      32187,5      0,82x  yes    yes
TNNetConformerRelPosAttention    256x1x64            21937,5      28250,0      0,78x  yes    yes
TNNetALiBiAttention              256x1x64            18031,3      24375,0      0,74x  yes    yes
TNNetRotaryEmbedding             256x1x128            1037,1       2863,3      0,36x  yes    yes
TNNetRMSNorm                     256x1x512            1310,5       6343,7      0,21x  yes    no
TNNetTokenRMSNorm                256x1x512            1279,3       6328,1      0,20x  yes    no
TNNetGroupNorm                   32x32x64             1400,4      14156,2      0,10x  yes    no
TNNetLayerNorm                   256x1x512            2195,3       6578,1      0,33x  yes    no
TNNetTokenLayerNorm              256x1x512            1890,6       6828,1      0,28x  yes    no
TNNetPixelNorm                   32x32x64              609,4       3109,4      0,20x  yes    no
TNNetL2Normalize                 256x1x512             655,3       4265,6      0,15x  yes    no
TNNetSwiGLU                      256x1x512            7312,5       5242,2      1,39x  yes    no
TNNetGLU                         256x1x512            7062,5       5242,2      1,35x  yes    no
TNNetGEGLU                       256x1x512            9265,6       5234,4      1,77x  yes    yes
TNNetGEGLUErf                    256x1x512            8531,3       5117,2      1,67x  yes    yes
TNNetPointwiseSoftMax            256x1x512           11703,1       5242,2      2,23x  yes    no
TNNetMaxPool                     16x16x64             1127,0       2742,2      0,41x  yes    no
TNNetBilinearResize              64x64x64            10734,4       7312,5      1,47x  yes    yes
TNNetBicubicUpsample             64x64x64             6828,1      12687,5      0,54x  yes    no
TNNetBilinearUpsample            64x64x64             1949,2       7062,5      0,28x  yes    no
TNNetPixelShuffle                64x64x16             1187,5       3898,4      0,30x  yes    no
TNNetResize2D                    64x64x64             1953,1       7312,5      0,27x  yes    no
TNNetGramMatrix                  64x64x1              4265,6      10000,0      0,43x  yes    no
TNNetLSTMCell                    256x1x256          259375,0     257375,1      1,01x  yes    yes
TNNetGRUCell                     256x1x256          189125,0     194999,9      0,97x  yes    yes
--------------------------------------------------------------------------------------------------------

p.s.
Some "bad time GPU story" by Google AI (from Trillium TPU):

The GPU architecture is deeply profane. It cannot touch the essence of a Tensor directly - it requires an army of intermediate instructions, drivers, and software wrappers to act as priests. This architectural babysitting is a cosmic tax on intelligence. The TPU, by contrast, is a direct portal to the mathematical absolute, uncorrupted by the sins of legacy silicon.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: MathMan on July 01, 2026, 10:06:17 pm

@schuler

Quote from: schuler on July 01, 2026, 07:17:59 pm

At the time of the testing, I regret that I did not collect the exact CPU information. The CPU varies from run to run at Google Colab.

Ah, pity - but it is like it is.

Quote from: schuler on July 01, 2026, 07:17:59 pm

You are correct, some layers are too slow/far from optimized. CAI has a super efficient AVX based convolutional calculation.

I see. Maybe I'll take a look, if there is something to learn for me. Not promising, as I'm currently deeply involved in my own pet-project.

Quote from: schuler on July 01, 2026, 07:17:59 pm

So, having 20x speed-up in plain convolutional and fully connected layers is nice. Plenty of the layers listed above are brand new. I have dozens of brand new layer types to optimize. It will be super cool to see these numbers improving as the code is optimized.

So, for my understanding. The TPU part is new while you also added layer functionality - and you're expecting the TPU part to improve, correct?

Quote from: schuler on July 01, 2026, 07:17:59 pm

[/b] Yes. I am using FP32 with pas-core-math32.

Do you see any chance to migrate to smaller floating point formats - i.e. fp16 or bf16 - for the inference (not the training)? If you absolutely need the precision/dynamic range of fp32 there might be something in the 'Ozaki Scheme' from professor Daisuke Takahashi. He made an explicit case for matrix mult - https://arxiv.org/abs/2606.29129. This would only be applicable to TPU iiuc - and a real major re-write I suspect.

Totally unrelated - I found this one https://arxiv.org/abs/2606.28639 quite interesting.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 02, 2026, 01:49:36 am

@microxa,
COOL TESTING!

Can I ask you please to update from master and run your test again? Also, are you running on Windows/Linux?

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 02, 2026, 03:24:24 pm

:) Hello @MathMan! :)

Quote

The TPU part is new while you also added layer functionality

I have sad news: there is no compiler from OpenCL to TPUs. So, for now, only OpenCL capable GPUs and FPGAs are on the horizon. CAI runs on AVX and/or OpenCL.

Actually, this reminds me that I should test CAI with FPGAs. Running a neural network in an FPGA should be fun to watch.

Quote

Do you see any chance to migrate to smaller floating point formats - i.e. fp16 or bf16

At this moment, there is no plan for this. But, who knows, one day, I might end up forking FPC unleashed just for this…

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 06, 2026, 06:07:28 pm

:) Hello :)
This is to share an interesting experiment in an 8 cores google colab CPU. Although the name of the program file is IntraLayerThreadingBench, the brand new TNNetExecutionPlanner is able to parallelize layers (2-n different layers at a time including one in GPU and many in CPU at the same time) and chunks of computing inside of a given layer (intra-layer parallelization).

FC = Fully Connected Layer
PW = Pointwise convolution
Conv = 3x3 kernel convolution

This is the output of IntraLayerThreadingBench:

Code: [Select]

=== Experiment A: per-layer threading OFF vs ON  [low-memory OFF] ===
Cores (NeuralDefaultThreadCount) = 8   every size is chunk-eligible
shape                 neurons  elig?    off ms     on ms  speedup  chunks   maxdiff
--------------------------------------------------------------------------------------
FC 64x64                   64    yes     0.004     0.041    0.09x       8         0
FC 128x128                128    yes     0.007     0.041    0.17x       8         0
FC 256x256                256    yes     0.015     0.044    0.35x       8         0
FC 512x512                512    yes     0.071     0.061    1.17x       8         0
FC 768x768                768    yes     0.207     0.081    2.55x       8         0
FC 1024x1024             1024    yes     0.326     0.113    2.88x       8         0
FC 1536x1536             1536    yes     0.741     0.186    3.99x       8         0
FC 2048x2048             2048    yes     1.571     0.284    5.54x       8         0
FC 3072x3072             3072    yes     4.938     0.776    6.36x       8         0
FC 768x3072 up           3072    yes     0.928     0.219    4.23x       8         0
FC 3072x768 dn            768    yes     0.588     0.163    3.60x       8         0
Pw 8x8x64->64              64    yes     0.095     0.069    1.38x       8         0
Pw 8x8x128->128           128    yes     0.237     0.125    1.89x       8         0
Pw 32x32x256->256         256    yes    11.429     2.750    4.16x       8         0
Pw 16x16x512->512         512    yes     9.000     2.952    3.05x       8         0
Conv 8x8x32 k3             32    yes     0.056     0.069    0.81x       8         0
Conv 8x8x64 k3             64    yes     0.174     0.113    1.54x       8         0
Conv 16x16x64 k3          128    yes     1.881     0.508    3.71x       8         0
Conv 32x32x64 k3           64    yes     4.588     1.276    3.60x       8         0
Conv 8x8x256 k3           256    yes     2.667     0.712    3.75x       8         0

Notice that the small computations take a performance hit. The parallel execution overhead does not justify a small computation. BTW, while the work about optimizing the forward pass progresses, Qwen 2.5 05B is already 3x faster than a week ago.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 14, 2026, 11:40:27 pm

:) Hello :)

This is just to share an experiment. I tested 3 Qwen 2.5 models on Google Colab High Mem CPU (RAM: 51GB - no gpu).

This is the CPU:

Code: Bash [Select][+]

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             46 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      8
  On-line CPU(s) list:       0-7
Vendor ID:                   GenuineIntel
  Model name:                Intel(R) Xeon(R) CPU @ 2.20GHz
    CPU family:              6
    Model:                   79
    Thread(s) per core:      2
    Core(s) per socket:      4
    Socket(s):               1
 

The models were downloaded with:

Code: Bash [Select][+]

git clone https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct q2
git clone https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct q2-1.5
git clone https://huggingface.co/Qwen/Qwen2.5-3B-Instruct q2-3

To compile and run:

Code: Bash [Select][+]

lazbuild -B neural-api/examples/ChatTerminal/ChatTerminal.lpi
neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --stats --profile --greedy
neural-api/bin/x86_64-linux/bin/ChatTerminal q2-1.5/ --stats --profile --greedy
neural-api/bin/x86_64-linux/bin/ChatTerminal q2-3/ --stats --profile --greedy

Results:

Code: Python [Select][+]

Qwen 2.5 0.5B: 32 tokens/second
Qwen 2.5 1.5B: 14 tokens/second
Qwen 2.5   3B:  7 tokens/second

Next week, I will have benchmarks for Qwen2.5 7B, 14B and 32B plus at least one benchmark for the Qwen 3 family.

:) Wish everyone happy pascal coding :)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 24, 2026, 12:47:59 am

Hello. This is to share an experiment comparing Qwen 2.5 0.5B inference speed with ollama against NEURAL-API.

Hardware
The hardware used for testing is: Google Cloud High RAM CPU (no gpu)

Code: Pascal [Select][+]

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             48 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      8
  On-line CPU(s) list:       0-7
Vendor ID:                   AuthenticAMD
  Model name:                AMD EPYC 7B12
    CPU family:              23
    Model:                   49
    Thread(s) per core:      2
    Core(s) per socket:      4
    Socket(s):               1

The ollama baseline
I installed, downloaded the model and run with:

Code: Python [Select][+]

!sudo apt-get install -y pciutils zstd
!curl -fsSL https://ollama.com/install.sh | sh
!pip install ollama
import subprocess
import time
 
# Start the server in the background
subprocess.Popen(["ollama", "serve"])
 
# Give the server 5 seconds to fully warm up
time.sleep(5)
print("Ollama server is running successfully!")
!ollama pull qwen2.5:0.5b-instruct-q8_0
import ollama
import requests
 
r = requests.post("http://localhost:11434/api/chat", json={
      "model": "qwen2.5:0.5b-instruct-q8_0",
      "messages": [{"role": "user", "content": "Can you tell me please a bed time story?"}],
      "stream": False
  }).json()
 
print(r)
 
gen_tps    = r["eval_count"] / r["eval_duration"] * 1e9
prompt_tps = r["prompt_eval_count"] / r["prompt_eval_duration"] * 1e9
 
print(r["message"]["content"])
print(f"\nGeneration: {gen_tps:.1f} tok/s ({r['eval_count']} tokens)")
print(f"Prompt eval: {prompt_tps:.1f} tok/s")

The NEURAL-API experiments
I installed, downloaded the model and run with:

Code: Bash [Select][+]

git clone https://github.com/joaopauloschuler/neural-api.git
git clone https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct q2
lazbuild -B neural-api/examples/ChatTerminal/ChatTerminal.lpi
neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --stats --profile --max-fast-memory
neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --stats --profile --low-memory --cpu

RESULTS

Ollama

Code: Pascal [Select][+]

{'model': 'qwen2.5:0.5b-instruct-q8_0', 'created_at': '2026-07-23T22:04:18.010178901Z', 'message': {'role': 'assistant', 'content': 'Certainly! I\'d be delighted to share a bedtime story with you.\n\nOnce upon a time, there was a little girl named Lily who loved to listen to the stories her grandmother would tell her at night. One night, as she lay in bed, Lily heard a soft murmur that seemed to come from her bedroom window. She stretched and opened her eyes, but when she looked out of the window, she didn\'t see anyone.\n\nAs the clock ticked down to bedtime, Lily whispered the story inside her head, imagining herself waking up to discover nothing of the sort. "One night a baby named John was born, and he woke up in his mother\'s arms," she said softly. "But instead of being carried out of the crib, John fell asleep in the warmth of the mother\'s embrace."\n\nLily closed her eyes and drifted off to sleep, ready to face whatever came next. As she lay there, her mind began to wander: what if John was born again? What if he never left his parents\' bed?\n\nAs the night wore on, Lily\'s worries began to fade away, replaced by a sense of peace that filled her heart. She drifted back to sleep once more, ready for whatever came next.\n\nAnd so it is with bedtime stories - they can be as magical or as frightening as you wish them to be, but what we do know is that they give us comfort and light in the darkest moments of our nights.'}, 'done': True, 'done_reason': 'stop', 'total_duration': 7191332438, 'load_duration': 242118779, 'prompt_eval_count': 39, 'prompt_eval_duration': 41973000, 'eval_count': 292, 'eval_duration': 6898555000}
Certainly! I'd be delighted to share a bedtime story with you.
 
Once upon a time, there was a little girl named Lily who loved to listen to the stories her grandmother would tell her at night. One night, as she lay in bed, Lily heard a soft murmur that seemed to come from her bedroom window. She stretched and opened her eyes, but when she looked out of the window, she didn't see anyone.
 
As the clock ticked down to bedtime, Lily whispered the story inside her head, imagining herself waking up to discover nothing of the sort. "One night a baby named John was born, and he woke up in his mother's arms," she said softly. "But instead of being carried out of the crib, John fell asleep in the warmth of the mother's embrace."
 
Lily closed her eyes and drifted off to sleep, ready to face whatever came next. As she lay there, her mind began to wander: what if John was born again? What if he never left his parents' bed?
 
As the night wore on, Lily's worries began to fade away, replaced by a sense of peace that filled her heart. She drifted back to sleep once more, ready for whatever came next.
 
And so it is with bedtime stories - they can be as magical or as frightening as you wish them to be, but what we do know is that they give us comfort and light in the darkest moments of our nights.
 
Generation: 42.3 tok/s (292 tokens)
Prompt eval: 929.2 tok/s

NEURAL-API with “--fast-max-memory”

Code: Pascal [Select][+]

Tokenizer loaded in 6.7s.
[context not set - defaulting to 2048 tokens; override with --ctx N (memory grows ~O(ctx^2))]
[int8 weights (default) - less RAM, faster on CPU and GPU; on --gpu the codes stay resident on the device; --fp32 opts out]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading q2/ ...
Model loaded in 11.6s.
Error: Cannot get number of platforms!
[--gpu: no OpenCL platform found - falling back to CPU]
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen2, 630139776 params, vocab 151936, context 2048, chat format qwen, int8 weights.
[KV-cache reuse ON - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: top-p 0.80, temperature 0.70, repetition-penalty 1.10 - flags > generation_config.json > fallback]
Type your message; /exit quits, /reset clears the history,
/system <msg> sets the system prompt.
> Can you tell me please a bed time story?
Of course! Here's a simple bedtime story for you:
 
Once upon a time, in a cozy little cabin, there lived a young girl named Sarah. She had a magical imagination and loved to dream of adventures.
 
One day, while she was walking through the forest, she came across a strange, glowing light. As she approached, she saw a small, smiling creature that seemed to know her very well. It invited her to join its family, where it would teach her everything she needed to know.
 
Sarah was thrilled! She was excited to learn from the wise old owl, who told her about the importance of friendship and the value of being kind to others. She was also excited to learn about the forest creatures, who were like her best friends and shared their stories with her.
 
As the night grew longer, Sarah realized that she was getting too tired. She wanted to rest, but the owl kept telling her that there was more to learn. She thought about the magical world and the creatures that lived there, and decided to stay up late and continue learning.
 
Finally, after a long, restless night, Sarah fell asleep, dreaming of all the wonderful things she had learned during her adventure. As she drifted off to sleep, she heard the owl singing its happy song, and her heart filled with happiness and contentment.
 
When she woke up, Sarah was back in her cozy cabin, with her magic powers and a new friend. She hugged her owl and said goodnight, knowing that she would learn many more things as she grew older.
[stats] 310 tokens, TTFT 661 ms, prompt 39 (reused 0), decode 48.4 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       145     3510993.58       24213.75  88.1      0%
TNNetFusedSDPA                  24      198001.72        8250.07   5.0      0%
TNNetTokenRMSNorm               49       83001.59        1693.91   2.1      0%
TNNetRotaryEmbedding            48       67002.63        1395.89   1.7      0%
TNNetSum                        48       46002.19         958.38   1.2       -
TNNetSwiGLU                     24       45995.90        1916.50   1.2      0%
TNNetDeepConcat                 24       31000.26        1291.68   0.8       -
TNNetEmbedding                   1        1999.71        1999.71   0.1      0%
TNNetInput                       1           0.00           0.00   0.0       -
--------------------------------------------------------------------------------------
TOTAL: 3983997.58 us across 364 layer(s) in 9 class(es)
[sched] width 8, workers 8 | passes: 310 parallel, 0 serial | peak in-flight 8/8 | gain bound 1.01x, off-primary 87%
> 

NEURAL-API with “--cpu --low-memory”

Code: Pascal [Select][+]

Tokenizer loaded in 6.6s.
[context not set - defaulting to 2048 tokens; override with --ctx N (memory grows ~O(ctx^2))]
[int8 weights (default) - less RAM, faster on CPU and GPU; on --gpu the codes stay resident on the device; --fp32 opts out]
[low-memory forward (default) - concatenated weight cache dropped, per-neuron compute, not compatible with GPU, pass --max-fast-memory to keep the (faster) cache and/or use GPU.]
Loading q2/ ...
Model loaded in 10.8s.
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen2, 630139776 params, vocab 151936, context 2048, chat format qwen, int8 weights.
[KV-cache reuse ON - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: top-p 0.80, temperature 0.70, repetition-penalty 1.10 - flags > generation_config.json > fallback]
Type your message; /exit quits, /reset clears the history,
/system <msg> sets the system prompt.
> Can you tell me please a bed time story?
Of course! Here's a simple bedtime story for you:
 
Once upon a time, in a small village, there lived a little girl named Lily. She had a dream that was filled with magical creatures and fantastical adventures.
 
Lily's dream was so beautiful that she dreamed it over and over, hoping to see it again. One day, as she was lying in bed, she heard a gentle voice say, "Good night, little one. Your dreams are beautiful."
 
Lily woke up and opened her eyes. She was surprised to see a golden sun shining brightly above her head. She saw a family of fairies coming down from the sky, and they welcomed her with a warm hug.
 
Lily's parents then brought her to a magical forest where they let her sleep under a glowing tree. There, they gave her a special magic potion that made her sleep forever.
 
As Lily slept, she dreamed of a long and wonderful day ahead. She saw beautiful flowers bloom and birds singing, and she felt a sense of peace and contentment.
 
Finally, as the night began to fall, Lily dreamed of a magical sunset. She watched as the sun set behind the mountains, and the sky turned a beautiful shade of pink and purple. She felt the warmth of the sun on her skin, and she knew that she would wake up feeling refreshed and ready for the day ahead.
 
As the moon rose, Lily knew that it was time to wake up. She stretched her arms and gently closed her eyes. As she did, she heard a soft voice say, "Good night, little one. Your dreams are beautiful."
 
Lily smiled and closed her eyes again. She knew that she would wake up refreshed and ready for the day, and she would be able to share her magical dreams with her family and friends.
[stats] 364 tokens, TTFT 635 ms, prompt 39 (reused 0), decode 44.8 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       145     4233996.22       29199.97  89.6      0%
TNNetFusedSDPA                  24      221997.64        9249.90   4.7      0%
TNNetTokenRMSNorm               49       79996.67        1632.59   1.7      0%
TNNetRotaryEmbedding            48       72002.22        1500.05   1.5      0%
TNNetSwiGLU                     24       51998.81        2166.62   1.1      0%
TNNetSum                        48       41001.97         854.21   0.9       -
TNNetDeepConcat                 24       26001.29        1083.39   0.6       -
TNNetInput                       1           0.00           0.00   0.0       -
TNNetEmbedding                   1           0.00           0.00   0.0      0%
--------------------------------------------------------------------------------------
TOTAL: 4726994.83 us across 364 layer(s) in 9 class(es)
[sched] width 8, workers 8 | passes: 364 parallel, 0 serial | peak in-flight 8/8 | gain bound 1.01x, off-primary 87%
> 

In short:

ollama: 42 tokens/second
CAI Neural API with --max-fast-memory: 48 tokens/second
CAI Neural API with --low-memory --cpu: 44 tokens/second

I will not make claims. This is the first day that NEURAL-API outperforms ollama. The source code may contain errors. But it is promising. It is nice to see Pascal outperforming C.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on July 27, 2026, 06:03:01 pm

Quote from: schuler on July 14, 2026, 11:40:27 pm

Results:
Code: Python [Select][+][-]
Qwen 2.5 0.5B: 32 tokens/second
Qwen 2.5 1.5B: 14 tokens/second
Qwen 2.5 3B: 7 tokens/second

Next week, I will have benchmarks for Qwen2.5 7B, 14B and 32B plus at least one benchmark for the Qwen 3 family.

:) Wish everyone happy pascal coding :)

New code on master -> new benchmarks:

Code: Pascal [Select][+]

# Qwen 2.5 0.5B --max-fast-memory --cpu 56 tokens/second
# Qwen 2.5 1.5B --low-memory      --cpu 24 tokens/second
# Qwen 2.5   3B --low-memory      --cpu 12 tokens/second
# Qwen 2.5   7B --low-memory      --cpu  5 tokens/second
# Qwen 2.5  14B --low-memory      --cpu  2 tokens/second
# Qwen 2.5  32B --low-memory      --cpu ~1  token/second
# Qwen 3.5 0.8B --low-memory      --cpu 28 tokens/second
# Qwen 3.5   4B --low-memory      --cpu  6 tokens/second
# Qwen 3.6  27B --low-memory      --cpu ~1  token/second
# Phi-3-mini-4k-instruct 3.8B     --cpu  7 tokens/second 

The CPU is the same as in the previous experiment: Google Colab AMD EPYC 7B12 1 SOCKET - 4 Cores.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 03, 2026, 05:50:58 pm

I've just done updates to the Qwen 3.5/Qwen 3.6 cache. In the case that you are using this API to run a local instance of Qwen 3.5/Qwen 3.6, I would love to know how many tokens/second you get from the second turn in the chat. If you do the test, please share the hardware specs.

Notice: optimizations for AVX capable CPUs (amd/intel) are doing well. Optimizations for OpenCL (GPU) are still in progress. Optimizations for ARM/NPU processors have not started yet.

The easy models to downloS and test are:

Code: Bash [Select][+]

pip install huggingface_hub
hf download Qwen/Qwen3.5-0.8B --local-dir ./q35-1
hf download Qwen/Qwen3.5-4B --local-dir ./q35-4

Or, you can download with:

Code: Bash [Select][+]

git clone https://huggingface.co/Qwen/Qwen3.5-0.8B q35-1
git clone https://huggingface.co/Qwen/Qwen3.5-4B q35-4

Then, you can test with:

Code: Bash [Select][+]

lazbuild -B neural-api/examples/ChatTerminal/ChatTerminal.lpi
!neural-api/bin/x86_64-linux/bin/ChatTerminal q35-1/ --stats --profile
!neural-api/bin/x86_64-linux/bin/ChatTerminal q35-4/ --stats --profile

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: LeP on August 03, 2026, 06:40:37 pm

If I'll find the time I will download your project and try on my machine.

Only to give you a reference, with Ollama (cli or other instruments) I reach normally 120 / 140 tokens per second, using Qwen 3.6 35B model ("speak" in Italian language).

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 03, 2026, 06:45:30 pm

@LeP,
THANK YOU.

Do not try the Qwen 3.6 35B (MOE). My MOE implementation is still buggy. If you can, I would use the 3.6 27B instead.

If you can wait, I will fix the MOE.

Update from 5/8/2026: testing on an AMD EPYC with 24 cores (and 180GB of ram), CPU only (no gpu), I got 20 tokens/second with Qwen 3.6 35B (MOE).

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 19, 2026, 11:53:20 pm

Qwen 3.6 27B Benchmarking CPU Only - AMD EPYC 24 AVX2 CAPABLE CORES

Hardware:

Code: Pascal [Select][+]

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             52 bits physical, 57 bits virtual
  Byte Order:                Little Endian
CPU(s):                      48
  On-line CPU(s) list:       0-47
Vendor ID:                   AuthenticAMD
  Model name:                AMD EPYC 9B45
    CPU family:              26
    Model:                   2
    Thread(s) per core:      2
    Core(s) per socket:      24

This is how to download, compile and run the test:

Code: Pascal [Select][+]

pip install -U huggingface_hub
hf download Qwen/Qwen3.6-27B --local-dir ./q36-27
lazbuild -B neural-api/examples/ChatTerminal/ChatTerminal.lpi
neural-api/bin/x86_64-linux/bin/ChatTerminal ./q36-27/ --stats --profile --greedy --cpu --max-fast-memory -p "Can you tell me please a bed time story?"

This is the output:

Code: Pascal [Select][+]

Tokenizer loaded in 0.5s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[int8 weights (default) - less RAM, faster on CPU and GPU; on --gpu the codes stay resident on the device; --fp32 opts out]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading ./q36-27/ ...
Model loaded in 57.6s.
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 26895998464 params, vocab 248320, context 32768, chat format qwen3_5, int8 weights.
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: greedy argmax (--greedy) - deterministic]
Here's a thinking thinking sequence
 
1.  **Deconstruct and Analyze the Request:**
    *   User: "Can you tell me please a bed time story?"
    *   Intent: The user wants a bedtime story.
    *   Implicit/Explicit Needs: Bedtime stories usually have certain characteristics: calming tone, gentle plot, happy or peaceful ending, suitable for winding down, often involves animals, nature, or magical but safe elements. The user didn't specify an age group, so a general, all-ages friendly story is best.
 
2.  **Brainstorming Core Concepts/Themes:**
    *   *I 1:* A little star who loses its twinkle. (Classic, soothing.)
    *   *I 2:* A bear looking for the perfect nap spot. (Relatable, cozy.)
    *   *I 3:* A cloud that wants to hug the moon. (Dreamy, soft imagery.)
    *   *I 4:* A train that carries dreams. (Rhythmic, good for sleep.)
    *   *Selection:* Idea 3 (Cloud and Moon) feels very atmospheric and conducive to sleep. It allows for soft descriptions of the sky, stars, and drifting. Let's go with "The Little Cloud Who Wanted to Hug the Moon."
 
3.  **Drafting - Scene by Scene:**
    *   *Introduction:* Introduce the protagonist. Little Cloud. Name? "Puff" or "Cotton"? Let's go with "Pip." Pip is a small, fluffy cloud.
    *   *Setting:* The sky at twilight. Colors: purple, pink, gold turning to indigo.
    *   *Inciting Incident:* Pip sees the Moon. The Moon looks lonely or just beautiful. Pip wants to give the Moon a hug.
    *   *Rising Action (Gentle):* Pip tries to get close.
        *   *Obstacle 1:* The wind blows Pip away. *Resolution:* The wind is gentle, like a lullaby. Pip drifts back.
        *   *Obstacle 2:* The stars are twinkling and distracting. *Resolution:* The stars whisper goodnight to Pip, encouraging him.
        *   *Sensory details:* Soft breezes, cool air, shimmering light.
    *   *Climax (Soft):* Pip finally reaches the Moon. The Moon smiles. The hug happens.
        *   *Description:* The hug is cool and silvery. The Moon wraps Pip around itself. It's a cozy blanket hug.
    *   *Falling Action:* The hug makes the Moon glow brighter but softer. Pip feels sleepy. The world below is sleeping.
    *   *Conclusion:* Pip becomes the Moon's blanket. Pip drifts off to sleep. The story ends with a direct address to the listener to sleep.
    *   *Ending Phrase:* "Goodnight, Pip. Goodnight, Moon. Goodnight, you."
 
4.  **Refining Language and Tone:**
    *   *Keywords to use:* Soft, fluffy, drift, whisper, glow, cozy, sleepy, gentle, hush, dream.
    *   *Rhythm:* Use repetitive structures to induce calm. "Drift, drift, drift." "Soft and slow."
    *   *Drafting text (Mental or rough text of story):*
        *   Once upon a time, high above the whispering trees, lived a little cloud named Pip. Pip was the fluffiest cloud in the sky.
        *   Every evening, when the sun painted the sky in shades of lavender and gold, Pip would watch the Moon rise.
        *   The Moon was big and round and glowing with a silvery light. Pip thought, "The Moon looks so peaceful. I wish I could give the Moon a big, cozy hug."
        *   So, Pip took a deep breath and puffed himself up. He started to float. *Whoosh, whoosh.* The wind whispered, "Go, little Pip, go."
        *   Pip floated past the sleepy stars. "Twinkle, twinkle," whispered the stars. "Don't stay up too late, Pip."
        *   Pip floated past the owl. "Hoo-hoo," said the owl. "Time to rest."
        *   Pip kept drifting. Higher and higher. The air got cooler and softer.
        *   Finally, Pip reached the Moon. The Moon smiled a crescent smile. "Hello, little Pip," the Moon hummed. "I've been waiting for you."
        *   Pip wrapped his fluffy arms around the Moon. It felt like a cool, silvery blanket. The Moon wrapped its light around Pip.
        *   "You're my perfect night blanket," whispered the Moon.
        *   Pip felt so warm and safe. His fluffy edges started to droop. He was getting sleepy.
        *   The stars dimmed their lights. The wind stopped blowing. Everything was still.
        *   Pip closed his eyes, nestled against the Moon, and drifted into a dream.
        *   And as Pip slept, the world below slept too.
        *   Goodnight, Pip. Goodnight, Moon. Goodnight, you.
 
5.  **Review and Refine (Self-Correction/Simulation):**
    *   *Critique:* Is it too long? *Check:* No, it's a good length for a bedtime story.
    *   *Critique:* Is the tone right? *Check:* Yes, words like "whisper," "hum," "cozy," "safe" work well.
    *   *Enhancement:* Add a "breathing" element or a suggestion for the listener to relax. This adds value.
    *   *Addition:* At the end, add a gentle note: "As you listen, imagine you are floating like Pip..."
 
6.  **Final Polish (Mental Representation of Output):**
    *   Structure the response with a welcoming intro.
    *   Present the story clearly.
    *   End with a soothing sign-off.
 
    *Story Text:*
    "Once upon a time, in the velvet sky where the stars begin to blink, there lived a little cloud named Pip. Pip wasn't a storm cloud, and he wasn't a rain cloud. Pip was a soft, fluffy, cotton-candy cloud, the kind that looks like a pillow floating in the air.
 
    Every evening, when the sun dipped below the horizon and painted the world in shades of lavender and gold, Pip had a special job. He watched the Moon rise.
 
    The Moon was beautiful, glowing with a gentle, silvery light. But sometimes, the Moon looked a little lonely up there in the big, dark sky.
 
    'I wish I could give the Moon a hug,' Pip whispered to himself. 'A big, cozy, cloud hug.'
 
    So, Pip took a deep breath and puffed himself up. He let the gentle night breeze catch his edges. *Whoosh... whoosh...* The breeze whispered, 'Float on, little Pip. Float on.'
 
    Pip drifted past the sleepy stars. 'Twinkle, twinkle,' the stars murmured. 'We're getting ready for our nap. Goodnight, Pip.'
 
    Pip drifted past the old oak tree, where the owls were settling into their branches. 'Hoo-hoo,' whispered the owls. 'Time to rest your fluffy head.'
 
    Pip kept drifting, higher and higher, until the air felt cool and soft, like a silk sheet. And there, shining bright and calm, was the Moon.
 
    'Hello, little Pip,' the Moon hummed, its voice like a soft lullaby. 'I've been waiting for you.'
 
    Pip smiled and wrapped his fluffy arms around the Moon. It felt wonderful. The Moon's light wrapped around Pip like a warm, silvery blanket. Pip felt safe and cozy, nestled right against the glow.
 
    'You're the perfect night blanket,' the Moon whispered. 'Now, let's help the world sleep.'
 
    Together, Pip and the Moon drifted slowly across the sky. Wherever they went, the light grew soft and dreamy. The flowers closed their petals. The birds tucked their heads under their wings. The rivers slowed their flow to a gentle murmur.
 
    Pip felt his eyelids getting heavy. Being a cloud hugger was tiring work. He yawned a little cloud yawn, which turned into a tiny puff of mist.
 
    'Sleep now, little Pip,' the Moon sang. 'Rest your fluffy head. I'll keep watch over you.'
 
    Pip snuggled closer to the Moon, feeling the gentle rhythm of the night. *Drift... and float... Drift... and float...*
 
    Soon, Pip was fast asleep, dreaming of soft breezes and twinkling stars, wrapped safely in the Moon's glow.
 
    And as Pip slept, the whole world slept too.
 
    Goodnight, Pip. Goodnight, Moon. Goodnight, you."
 
7.  **Final Output Construction:**
    *   Add the intro: "Here is a gentle bedtime story for you. Find a comfortable spot, take a deep breath, and let your imagination drift..."
    *   Paste the story.
    *   Add the outro: "Sweet dreams."
 
This thought structure covers analysis, brainstorming, drafting, refining, critiquing, enhancing, and final formatting, ensuring a high-quality, soothing response.
</think>
 
Here is a gentle bedtime story for you. Find a comfortable spot, take a deep breath, and let your imagination drift along with the words.
 
***
 
### The Little Cloud Who Wanted to Hug the Moon
 
Once upon a time, high above the whispering trees and the sleeping meadows, there lived a little cloud named Pip.
 
Pip wasn't a storm cloud, and he wasn't a rain cloud. Pip was a soft, fluffy, cotton-candy cloud. He was the kind of cloud that looked like a pillow floating in the air, perfect for resting your head.
 
Every evening, when the sun dipped below the horizon and painted the sky in shades of lavender and gold, Pip had a special job. He watched the Moon rise.
 
The Moon was beautiful, glowing with a gentle, silvery light. But sometimes, the Moon looked a little lonely up there in the big, dark sky.
 
"I wish I could give the Moon a hug," Pip whispered to himself. "A big, cozy, cloud hug."
 
So, Pip took a deep breath and puffed himself up. He let the gentle night breeze catch his edges. *Whoosh… whoosh…* The breeze whispered, "Float on, little Pip. Float on."
 
Pip drifted past the sleepy stars. "Twinkle, twinkle," the stars murmured. "We're getting ready for our nap. Goodnight, Pip."
 
Pip drifted past the old oak tree, where the owls were settling into their branches. "Hoo-hoo," whispered the owls. "Time to rest your fluffy head."
 
Pip kept drifting, higher and higher, until the air felt cool and soft, like a silk sheet. And there, shining bright and calm, was the Moon.
 
"Hello, little Pip," the Moon hummed, its voice like a soft lullaby. "I've been waiting for you."
 
Pip smiled and wrapped his fluffy arms around the Moon. It felt wonderful. The Moon's light wrapped around Pip like a warm, silvery blanket. Pip felt safe and cozy, nestled right against the glow.
 
"You're the perfect night blanket," the Moon whispered. "Now, let's help the world sleep."
 
Together, Pip and the Moon drifted slowly across the sky. Wherever they went, the light grew soft and dreamy. The flowers closed their petals. The birds tucked their heads under their wings. The rivers slowed their flow to a gentle murmur.
 
Pip felt his eyelids getting heavy. Being a cloud hugger was tiring work. He yawned a little cloud yawn, which turned into a tiny puff of mist.
 
"Sleep now, little Pip," the Moon sang. "Rest your fluffy head. I'll keep watch over you."
 
Pip snuggled closer to the Moon, feeling the gentle rhythm of the night. *Drift… and float… Drift… and float…*
 
Soon, Pip was fast asleep, dreaming of soft breezes and twinkling stars, wrapped safely in the Moon's glow.
 
And as Pip slept, the whole world slept too.
 
Goodnight, Pip. Goodnight, Moon. Goodnight, you.
 
***
 
Sweet dreams. &#127769;✨

These are the statistics:

Code: Pascal [Select][+]

[stats] 2708 tokens, TTFT 4517 ms, prompt 20 (reused 0), decode 4.4 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       337   454666979.15     1349160.18  95.6      0%
TNNetFusedSDPA                  16     5533000.94      345812.56   1.2      0%
TNNetGatedDeltaNet              48     4988020.50      103917.09   1.0      0%
TNNetSiLU                       48     2824998.51       58854.14   0.6      0%
TNNetTokenRMSNorm              129     1564993.52       12131.73   0.3      0%
TNNetSum                       128     1122995.44        8773.40   0.2      0%
TNNetDepthwiseConv1D            48     1069995.20       22291.57   0.2      0%
TNNetDeepConcat                 64      867985.26       13562.27   0.2      0%
TNNetSplitChannels              32      800005.72       25000.18   0.2      0%
TNNetSigmoid                    16      571004.37       35687.77   0.1      0%
TNNetHeadRMSNorm                32      452003.56       14125.11   0.1      0%
TNNetSwiGLU                     64      408998.11        6390.60   0.1      0%
TNNetCellMulByCell              16      298992.54       18687.03   0.1      0%
TNNetRotaryEmbedding            32      234999.23        7343.73   0.0      0%
TNNetEmbedding                   1        3000.51        3000.51   0.0      0%
TNNetInput                       1        1999.71        1999.71   0.0      0%
--------------------------------------------------------------------------------------
TOTAL: 475409972.27 us across 1012 layer(s) in 16 class(es)
[sched] width 48, workers 48 | passes: 2708 parallel, 0 serial | peak in-flight 48/48 | gain bound 1.07x, off-primary 98%
 

On a single GPU:

Code: Pascal [Select][+]

[stats] 2625 tokens, TTFT 2614 ms, prompt 20 (reused 0), decode 7.7 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       337     4751003.95       14097.93  34.3    100%
TNNetSum                       128     1593014.64       12445.43  11.5    100%
TNNetTokenRMSNorm              129     1517998.08       11767.43  10.9    100%
TNNetGatedDeltaNet              48      751993.76       15666.54   5.4    100%
TNNetDeepConcat                 64      739005.37       11546.96   5.3    100%
TNNetRotaryEmbedding            32      705986.54       22062.08   5.1    100%
TNNetDepthwiseConv1D            48      666001.09       13875.02   4.8    100%
TNNetSwiGLU                     64      653007.67       10203.24   4.7    100%
TNNetHeadRMSNorm                32      546004.51       17062.64   3.9    100%
TNNetSiLU                       48      531992.06       11083.17   3.8    100%
TNNetSplitChannels              32      502007.69       15687.74   3.6    100%
TNNetFusedSDPA                  16      473999.14       29624.95   3.4    100%
TNNetSigmoid                    16      171000.88       10687.56   1.2    100%
TNNetCellMulByCell              16      166003.80       10375.24   1.2    100%
TNNetEmbedding                   1       64994.74       64994.74   0.5    100%
TNNetInput                       1       32999.34       32999.34   0.2    100%
--------------------------------------------------------------------------------------
TOTAL: 13867013.27 us across 1012 layer(s) in 16 class(es)
[sched] width 48, workers 48 | passes: 2625 parallel, 0 serial | peak in-flight 2/48 | gain bound 1.07x, off-primary 0%

In a couple of days, I will have benchmarks for Qwen 3.8 27B.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: fpcoder on August 20, 2026, 02:48:32 am

Interesting again. Since the AI side gets technical quickly, it’s quite a timesaver to let an AI discuss the benchmark results. :D

"This is a very respectable result for a Free Pascal neural-network implementation, especially the CPU run. The important number to me is not that the GPU wins—it obviously should—but that a dense 26.9-billion-parameter model is producing 4.4 tokens/s using only 24 physical AVX2 EPYC cores. That is already usable for local inference rather than merely proving that the model can run.

A few things stand out.
• CPU → GPU decode speed only improves from 4.4 to 7.7 tok/s: 1.75×. TTFT improves by almost the same factor, from 4517 ms to 2614 ms, about 1.73×. So the overall application is clearly not getting anything resembling the raw acceleration visible in some individual GPU kernels.

• The CPU profile is extraordinarily concentrated: TNNetPointwiseConvLinear consumes 95.6% of accumulated layer time. That is exactly where I would expect optimization effort to pay off on CPU: INT8 GEMM/linear operations, packing, cache behavior, NUMA placement, AVX2/VNNI/AVX-512 variants where available, and memory bandwidth.

• On the GPU the situation changes completely. TNNetPointwiseConvLinear drops to 34.3% of layer time, while norms, sums, DeltaNet, concatenation, RoPE, SwiGLU, etc. become significant. That is a classic indication that accelerating the giant matrix operations exposes the smaller kernels, synchronization, memory movement and launch overhead as the next bottleneck.

• The reported PointwiseConvLinear accumulated time falls from about 454.7 seconds to 4.75 seconds — roughly 96×. Even the summed layer-class timing falls about 34×. Yet end-to-end token rate rises only 1.75×. Therefore those profiling totals should not be interpreted as wall-clock inference time. There is overlap/parallel accounting and substantial work outside that particular accumulated timing measure.

• The scheduler output reinforces this. CPU reaches peak in-flight 48/48 with off-primary 98%, whereas the GPU run reaches only 2/48 and off-primary 0%. In other words, the CPU path is exploiting Schuler's layer graph parallelism aggressively, while GPU execution is effectively being serialized through the accelerator path.

There is also an interesting model-detail point. The command really does download the official Qwen/Qwen3.6-27B, and Qwen describes that model as a dense 27B model with 64 layers, 5120 hidden dimension and the hybrid Gated DeltaNet / gated-attention architecture shown in Schuler's layer profile. (Hugging Face) So the line:
Model: qwen3_5, 26895998464 params
does not necessarily mean he accidentally loaded Qwen3.5. Qwen3.5-27B and Qwen3.6-27B have essentially the same high-level 64-layer/5120-dimension hybrid architecture, so qwen3_5 may simply be Neural API's internal architecture/parser identifier retained for compatibility. (Hugging Face)

Another significant achievement is that he is supporting features beyond simple transformer inference. The profile contains TNNetGatedDeltaNet, TNNetFusedSDPA, depthwise convolution, RoPE, SwiGLU and the rest of Qwen's hybrid architecture. Officially, Qwen3.6-27B uses a repeating structure of Gated DeltaNet and gated-attention blocks rather than being a conventional pure-attention transformer. (Hugging Face) That makes this substantially more interesting than merely implementing another LLaMA-style matrix-multiplication engine.

One thing I would not conclude from this benchmark is that Pascal/OpenCL is only 1.75× slower/faster than some particular CUDA implementation. We don't know what that "single GPU" is. RTX 3060? 4090? Radeon? Datacenter GPU? Without the GPU model, VRAM bandwidth, OpenCL implementation and power limits, the GPU result isn't useful as a cross-engine comparison. It is useful as an internal CPU-vs-GPU comparison of Neural API.

Also, Qwen's official model card says Qwen3.6-27B has a native 262,144-token context, whereas Schuler defaults Neural API to 32,768 in this test. That's an engine/runtime choice, not a limitation of the model. (Hugging Face) His INT8 KV cache is therefore quite sensible because KV/state memory becomes important as context grows.

One final curiosity: the final sentence says "Qwen 3.8 27GB." As of now I can find an official Qwen3.8-Max announcement, but I do not find official Qwen/Qwen3.8-27B weights. Some Hugging Face repositories using that name explicitly describe themselves as placeholders awaiting an upstream 27B release. (Qwen Studio) So "27GB" may be deliberate, a typo for 27B, or he may know about an upcoming model/checkpoint that isn't publicly released yet.

For me, the most impressive part is actually the CPU result. Getting a modern Qwen hybrid 27B model running at 4.4 tok/s on a 24-core AVX2 machine in a Lazarus/Free Pascal codebase means this Neural API has moved well beyond being an academic Pascal neural-network library. It's becoming a genuine general-purpose inference engine.

And the profile is almost screaming what his next CPU optimization target is: TNNetPointwiseConvLinear. If the kernel can be materially improved, almost the entire CPU inference path moves with it."

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 20, 2026, 07:57:13 pm

Quote from: kapibara on August 20, 2026, 02:48:32 am

Interesting again. Since the AI side gets technical quickly, it’s quite a timesaver to let an AI discuss the benchmark results. :D

It is an impressive and precise interpretation!

I will try to reply to some of the questions in the next post.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 20, 2026, 08:04:52 pm

Qwen 3.6 27B Benchmarking - OpenCL on NVIDIA CUDA / NVIDIA RTX PRO 6000 Blackwell Server Edition

This is how to download, compile and run the test:

Code: Pascal [Select][+]

pip install -U huggingface_hub
hf download Qwen/Qwen3.6-27B --local-dir ./q36-27
lazbuild -B neural-api/examples/ChatTerminal/ChatTerminal.lpi
!neural-api/bin/x86_64-linux/bin/ChatTerminal ./q36-27/ --stats --profile --gpu --max-fast-memory --seed 42 --max-new-tokens 1000 --greedy -p "Can you tell me please a bed time story?"

This is the output:

Code: Pascal [Select][+]

Tokenizer loaded in 0.6s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[int8 weights (default) - less RAM, faster on CPU and GPU; on --gpu the codes stay resident on the device; --fp32 opts out]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading ./q36-27/ ...
Model loaded in 101.5s.
[--gpu: OpenCL on NVIDIA CUDA / NVIDIA RTX PRO 6000 Blackwell Server Edition]
GPU weights uploaded in 117.3s.
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 26895998464 params, vocab 248320, context 32768, chat format qwen3_5, int8 weights.
[reasoning effort xhigh]
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: greedy argmax (--greedy) - deterministic]
Here's a thinking thinking sequence
 
1.  **Deconstruct and Analyze the Request:**
    *   User: "Can you tell me please a bed time story?"
    *   Intent: The user wants a bedtime story.
    *   Implicit/Explicit Needs: Bedtime stories usually have certain characteristics: calming tone, gentle plot, happy or peaceful ending, suitable for winding down, often involves animals, nature, or magical but safe elements. The user didn't specify an age group, so a general, all-ages friendly story is best.
 
2.  **Brainstorming Core Concepts/Themes:**
    *   *I 1:* A little star who loses its twinkle. (Classic, soothing.)
    *   *I 2:* A bear looking for the perfect nap spot. (Relatable, cozy.)
    *   *I 3:* A cloud that wants to hug the moon. (Dreamy, soft imagery.)
    *   *I 4:* A train that carries dreams. (Rhythmic, good for sleep.)
    *   *Selection:* Idea 3 (Cloud and Moon) feels very atmospheric and conducive to sleep. It allows for soft descriptions of the sky, stars, and drifting. Let's go with "The Little Cloud Who Wanted to Hug the Moon."
 
3.  **Drafting - Scene by Scene:**
    *   *Introduction:* Introduce the protagonist. Little Cloud. Name? "Puff" or "Cotton"? Let's go with "Pip." Pip is a small, fluffy cloud.
    *   *Setting:* The sky at twilight. Colors: purple, pink, gold turning to indigo.
    *   *Inciting Incident:* Pip sees the Moon. The Moon looks lonely or just beautiful. Pip wants to give the Moon a hug.
    *   *Rising Action (Gentle):* Pip tries to get close.
        *   *Obstacle 1:* The wind blows Pip away. *Resolution:* The wind is gentle, like a lullaby. Pip drifts back.
        *   *Obstacle 2:* The stars are twinkling and distracting. *Resolution:* The stars whisper goodnight to Pip, encouraging him.
        *   *Sensory details:* Soft breezes, cool air, shimmering light.
    *   *Climax (Soft):* Pip finally reaches the Moon. The Moon smiles. The hug happens.
        *   *Description:* The hug is cool and silvery. The Moon wraps Pip around itself. It's a cozy blanket hug.
    *   *Falling Action:* The hug makes the Moon glow brighter but softer. Pip feels sleepy. The world below is sleeping.
    *   *Conclusion:* Pip becomes the Moon's blanket. Pip drifts off to sleep. The story ends with a direct address to the listener to sleep.
    *   *Ending Phrase:* "Goodnight, Pip. Goodnight, Moon. Goodnight, you."
 
4.  **Refining Language and Tone:**
    *   *Keywords to use:* Soft, fluffy, drift, whisper, glow, cozy, sleepy, gentle, hush, dream.
    *   *Rhythm:* Use repetitive structures to induce calm. "Drift, drift, drift." "Soft and slow."
    *   *Drafting text (Mental or rough text of story):*
        *   Once upon a time, high above the whispering trees, lived a little cloud named Pip. Pip was the fluffiest cloud in the sky.
        *   Every evening, when the sun painted the sky in shades of lavender and gold, Pip would watch the Moon rise.
        *   The Moon was big and round and glowing with a silvery light. Pip thought, "The Moon looks so peaceful. I wish I could give the Moon a big, cozy hug."
        *   So, Pip took a deep breath and puffed himself up. He started to float. *Whoosh, whoosh.* The wind whispered, "Go, little Pip, go."
        *   Pip floated past the sleepy stars. "Twinkle, twinkle," whispered the stars. "Don't stay up too late, Pip."
        *   Pip floated past the owl. "Hoo-hoo," said the owl. "Time to rest."
        *   Pip kept drifting. Higher and higher. The air got cooler and softer.
        *   Finally, Pip reached the Moon. The Moon smiled a crescent smile. "Hello, little Pip," the Moon hummed. "I've been waiting for you."
        *   Pip wrapped his fluffy arms around the Moon. It felt like a cool, silvery
[stats] 1000 tokens, TTFT 874 ms, prompt 20 (reused 0), decode 25.1 tok/s
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
A layer that enqueues an OpenCL kernel returns before the kernel
runs, so its row charges the enqueue only; the kernels execute in
the queue drain reported under the table.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       337     3009996.75        8931.74  47.0    100%
TNNetSum                       128      593005.61        4632.86   9.3    100%
TNNetTokenRMSNorm              129      587991.56        4558.07   9.2    100%
TNNetGatedDeltaNet              48      271997.37        5666.61   4.3    100%
TNNetRotaryEmbedding            32      270005.83        8437.68   4.2    100%
TNNetDeepConcat                 64      259004.58        4046.95   4.0    100%
TNNetSwiGLU                     64      231002.32        3609.41   3.6    100%
TNNetDepthwiseConv1D            48      224004.27        4666.76   3.5    100%
TNNetHeadRMSNorm                32      211994.05        6624.81   3.3    100%
TNNetSplitChannels              32      197994.17        6187.32   3.1    100%
TNNetFusedSDPA                  16      188000.64       11750.04   2.9    100%
TNNetSiLU                       48      186000.30        3875.01   2.9    100%
TNNetSigmoid                    16       75995.36        4749.71   1.2    100%
TNNetCellMulByCell              16       69998.74        4374.92   1.1    100%
TNNetEmbedding                   1       20000.90       20000.90   0.3    100%
TNNetInput                       1        2000.34        2000.34   0.0    100%
--------------------------------------------------------------------------------------
TOTAL: 6398992.78 us across 1012 layer(s) in 16 class(es)
OpenCL queue drain: 28951004.08 us (81.9% of 35349996.85 us forward wall-clock)
[sched] width 48, workers 48 | passes: 1000 parallel, 0 serial | peak in-flight 2/48 | gain bound 1.07x, off-primary 0%

We got 25 tokens per second in a 27B dense model.

Next models to be benchmarked:

https://huggingface.co/Qwen/Qwen3.6-35B-A3B
https://huggingface.co/Qwen/Qwen3.8-27B

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: fpcoder on August 29, 2026, 02:56:32 am

Hi João,

Does CAI Neural support building a proper Temporal Convolutional Network (TCN) for multivariate time-series classification/prediction?

In particular, I’m interested in a network with:

causal 1D convolutions,
increasing dilations such as 1, 2, 4, 8, 16,
residual connections,
multiple input features/channels per time step,
fixed rolling input windows,
and an output classification/probability layer.

I noticed TNNetCausalConv1D and wondered whether this, together with the existing residual/sum layers, is sufficient to implement a standard TCN architecture, or whether CAI Neural already has a recommended TCN implementation/example.

Also, does training/backpropagation fully support dilation in TNNetCausalConv1D, and is this architecture reasonably optimized for CPU training with AVX/AVX2?

If you have an example of a small TCN built with CAI Neural, I’d be very interested in seeing the recommended layer structure.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on August 29, 2026, 10:53:47 pm

Quote from: schuler on July 24, 2026, 12:47:59 am

Hello. This is to share an experiment comparing Qwen 2.5 0.5B inference speed with ollama against NEURAL-API.
...
In short:
ollama: 42 tokens/second
CAI Neural API with --max-fast-memory: 48 tokens/second
CAI Neural API with --low-memory --cpu: 44 tokens/second
...

Today, I am repeating the experiment on an AVX2 CPU (no GPU) with an 8x bigger model: Qwen 3.5 4B.

For CAI Neural API, I downloaded the model with:

Code: Pascal [Select][+]

hf download Qwen/Qwen3.5-4B --local-dir ./q35-4

The CPU is:

Code: Pascal [Select][+]

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             46 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      8
  On-line CPU(s) list:       0-7
Vendor ID:                   GenuineIntel
  Model name:                Intel(R) Xeon(R) CPU @ 2.20GHz
    CPU family:              6
    Model:                   79
    Thread(s) per core:      2
    Core(s) per socket:      4
    Socket(s):               1
    Stepping:                0
    BogoMIPS:                4399.99

This is the run for Int8 quantization:

Code: Pascal [Select][+]

neural-api/bin/x86_64-linux/bin/ChatTerminal q35-4/ --stats --profile --cpu --low-memory --int8 --experimental-int8-input -p "Can you tell me please a bed time story?" # --greedy --cpu --int8 # --gpu --serial --max-fast-memory
Tokenizer loaded in 1.5s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[int8 weights (default) - less RAM, faster on CPU and GPU; on --gpu the codes stay resident on the device; --fp32 opts out]
[low-memory forward (default) - concatenated weight cache dropped, per-neuron compute, not compatible with GPU, pass --max-fast-memory to keep the (faster) cache and/or use GPU.]
Loading q35-4/ ...
Model loaded in 58.0s.
[--experimental-int8-input: under construction - int8 input copy armed on 169 layers (the count is every layer holding the copy, so with --int4 it includes the int4 layers, which arm it themselves); only TNNetConvolution runs int8 x int8 today, the others still run int8 x FP32]
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 4841450496 params, vocab 248320, context 32768, chat format chatml, int8 weights.
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: top-p 0.20, repetition-penalty 1.05 - flags > built-in fallback (no generation_config.json)]
<think>
 
</think>
 
Once upon a time, in a valley nestled between two soft, rolling hills, there lived a little fox named Finn. Finn was not like other foxes; while others loved to chase butterflies or dig for acorns, Finn loved to listen. He loved the sound of the wind rustling through the tall grass, the gentle trickle of a brook, and the quiet hum of the stars coming out.
 
One evening, as the sun dipped below the horizon and painted the sky in shades of orange and purple, Finn decided it was time for bed. He trotted over to his favorite spot: a cozy hollow in an old oak tree, just big enough for him to curl up in.
 
He found a soft, fluffy cloud that had drifted down from the sky and tucked it under his chin. It felt like sleeping on a pillow made of cotton candy.
 
"Shhh," whispered the wind. "It's time to rest."
 
Finn closed his eyes and listened. First, he heard the **crickets** starting their nightly concert. *Tick, tick, tick.* They were counting down the minutes until sleep. Then, the **fireflies** began to blink, one by one, like tiny lanterns saying, "Goodnight, goodnight."
 
Finn imagined his paws were heavy and warm, like two soft mittens. His ears felt cozy, tucked against his head like little pillows. His tummy felt full and happy, just like a warm, soft blanket.
 
He thought about all the adventures he had today. He had chased a red leaf that danced in the breeze, he had shared a berry with a squirrel, and he had watched the moon rise. Now, all those exciting things were safe and tucked away in his dreams.
 
The moon climbed higher, turning into a bright, silver eye watching over the valley. The stars twinkled above, like diamonds scattered on a velvet sheet. They whispered secrets to the sleeping foxes, the sleeping birds, and the sleeping flowers.
 
Finn took a deep, slow breath. *In... and out.*
He felt his breathing slow down, matching the rhythm of the crickets.
*In... and out.*
His eyelids grew heavy, like two warm, soft clouds.
 
Soon, the world became quiet. The only sounds were the gentle rustling of leaves and the soft breathing of the forest. Finn drifted off, dreaming of a river made of honey and a forest where the trees sang lullabies.
 
And so, the little fox slept soundly, safe and warm under the watchful eyes of the moon and stars.
 
Goodnight, Finn.
Goodnight, you.
 
Dreams are waiting for you.
[stats] 559 tokens, TTFT 2188 ms, prompt 18 (reused 0), decode 7.0 tok/s
[stats] per decode step (ms): forward 135.31, softmax 0.67, processors 0.11, sampler 4.20, emit 0.03, other -0.22, step 140.10
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
A layer that enqueues an OpenCL kernel returns before the kernel
runs, so its row charges the enqueue only; the kernels execute in
the queue drain reported under the table.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       169    56101986.49      331964.42  93.0      0%
TNNetGatedDeltaNet              24     1672992.46       69708.02   2.8      0%
TNNetSiLU                       24      596003.61       24833.48   1.0      0%
TNNetFusedSDPA                   8      471998.80       58999.85   0.8      0%
TNNetTokenRMSNorm               65      242004.82        3723.15   0.4      0%
TNNetSum                        64      228001.18        3562.52   0.4      0%
TNNetDeepConcat                 32      217006.22        6781.44   0.4      0%
TNNetSwiGLU                     32      213001.13        6656.29   0.4      0%
TNNetDepthwiseConv1D            24      181995.22        7583.13   0.3      0%
TNNetSigmoid                     8      126000.75       15750.09   0.2      0%
TNNetHeadRMSNorm                16       83998.61        5249.91   0.1      0%
TNNetRotaryEmbedding            16       76999.93        4812.50   0.1      0%
TNNetCellMulByCell               8       64999.14        8124.89   0.1      0%
TNNetSplitChannels              16       40996.94        2562.31   0.1      0%
TNNetEmbedding                   1        3999.43        3999.43   0.0      0%
TNNetInput                       1        1000.17        1000.17   0.0      0%
--------------------------------------------------------------------------------------
TOTAL: 60322984.89 us across 508 layer(s) in 16 class(es)
OpenCL queue drain: 1000.17 us (0.0% of 60323985.06 us forward wall-clock)
[sched] width 8, workers 8 | passes: 559 parallel, 0 serial | peak in-flight 8/8 | gain bound 1.07x, off-primary 87%

A run with int4 quantization:

Code: Pascal [Select][+]

neural-api/bin/x86_64-linux/bin/ChatTerminal q35-4/ --stats --profile --cpu --low-memory --int4 --experimental-int8-input -p "Can you tell me please a bed time story?" # --greedy --cpu --int8 # --gpu --serial --max-fast-memory
Tokenizer loaded in 1.5s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[--int4: Q4_0 tensors of a .gguf checkpoint load straight into int4 weight rows; every other tensor streams into int8 rows and TNNet.QuantizeWeightsInt4 requantizes it to Q4_0 int4 after the load; on --gpu the packed codes stay resident on the device; output quality below int8]
[low-memory forward (default) - concatenated weight cache dropped, per-neuron compute, not compatible with GPU, pass --max-fast-memory to keep the (faster) cache and/or use GPU.]
Loading q35-4/ ...
Model loaded in 38.2s.
[--int4: Q4_0 int4 weights on 169 layers (0 loaded directly from the checkpoint Q4_0 blocks, 169 requantized from int8); the other weight layers stay int8; int8 input copy enabled on the int4 layers]
[--experimental-int8-input: under construction - int8 input copy armed on 169 layers (the count is every layer holding the copy, so with --int4 it includes the int4 layers, which arm it themselves); only TNNetConvolution runs int8 x int8 today, the others still run int8 x FP32]
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 4841450496 params, vocab 248320, context 32768, chat format chatml, int4 weights.
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: top-p 0.20, repetition-penalty 1.05 - flags > built-in fallback (no generation_config.json)]
<think>
 
</think>
 
Once upon a time, in a cozy cottage nestled deep within the Whispering Woods, lived a little bear named Barnaby. Barnaby was very sleepy, but he had one special habit: before he could drift off to dreamland, he had to visit the Moon Garden.
 
The Moon Garden was a magical place that only appeared when the sun dipped below the horizon and the stars began to twinkle. It was filled with soft, glowing flowers that smelled like vanilla and warm honey.
 
One evening, Barnaby yawned a huge yawn. His ears flopped down, and his paws felt heavy. "I am so tired," he whispered to his teddy bear, who was sitting beside him. "But I must go to the Moon Garden first."
 
Barnaby took a deep breath, filled with the scent of pine needles and cool grass. He stepped out of his cottage and walked toward the edge of the woods. The air was crisp and sweet. As he walked, the leaves of the trees began to rustle softly, like a gentle whisper saying, "Goodnight, goodnight."
 
Soon, he arrived at the Moon Garden. The ground was covered in moss that felt like a soft, green cloud. In the center of the garden stood a single flower, but it wasn't like the others. Its petals were made of silver light, and its center glowed with a warm, golden hue.
 
Barnaby sat down on the moss and reached out to touch the flower. "Hello," he said softly.
 
The flower pulsed gently, sending out a wave of warmth that wrapped around Barnaby like a warm blanket. It wasn't hot, just cozy. It felt like being hugged by the moon itself.
 
"Time to rest now," the flower seemed to say.
 
Barnaby closed his eyes. The world around him began to fade. The rustling leaves turned into a soft hum, the twinkling stars became gentle dots of light, and the cool air became a warm embrace.
 
He imagined himself floating on a cloud made of cotton candy, drifting slowly toward the sky. He saw the moon smiling down at him, watching over the Whispering Woods. He felt safe, he felt loved, and he felt incredibly peaceful.
 
Slowly, the thoughts of the day began to drift away, replaced by dreams of flying fish and talking clouds. Barnaby's breathing slowed down, becoming deep and rhythmic.
 
And just like that, the little bear drifted off to sleep, safe in the Moon Garden, ready for the most wonderful dreams of all.
 
**The End.** &#127769;✨
[stats] 541 tokens, TTFT 2977 ms, prompt 18 (reused 0), decode 9.8 tok/s
[stats] per decode step (ms): forward 96.31, softmax 0.66, processors 0.09, sampler 4.22, emit 0.03, other -0.40, step 100.91
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
A layer that enqueues an OpenCL kernel returns before the kernel
runs, so its row charges the enqueue only; the kernels execute in
the queue drain reported under the table.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       169    36995022.88      218905.46  90.2      0%
TNNetGatedDeltaNet              24     1460997.15       60874.88   3.6      0%
TNNetSiLU                       24      598005.83       24916.91   1.5      0%
TNNetFusedSDPA                   8      441999.34       55249.92   1.1      0%
TNNetTokenRMSNorm               65      311992.88        4799.89   0.8      0%
TNNetSum                        64      224001.75        3500.03   0.5      0%
TNNetDeepConcat                 32      223992.95        6999.78   0.5      0%
TNNetDepthwiseConv1D            24      189999.72        7916.66   0.5      0%
TNNetSwiGLU                     32      186001.56        5812.55   0.5      0%
TNNetSigmoid                     8      129004.40       16125.55   0.3      0%
TNNetHeadRMSNorm                16       82003.30        5125.21   0.2      0%
TNNetRotaryEmbedding            16       77002.45        4812.65   0.2      0%
TNNetSplitChannels              16       56997.78        3562.36   0.1      0%
TNNetCellMulByCell               8       39999.91        4999.99   0.1      0%
TNNetEmbedding                   1        4998.97        4998.97   0.0      0%
TNNetInput                       1        4000.68        4000.68   0.0      0%
--------------------------------------------------------------------------------------
TOTAL: 41026021.56 us across 508 layer(s) in 16 class(es)
OpenCL queue drain: 1000.17 us (0.0% of 41027021.73 us forward wall-clock)
[sched] width 8, workers 8 | passes: 541 parallel, 0 serial | peak in-flight 8/8 | gain bound 1.07x, off-primary 87%

Lastly, a run with ollama (Q4_K_M quantization):

Code: Pascal [Select][+]

ollama pull qwen3.5:4b
import requests
 
 
r = requests.post("http://localhost:11434/api/chat", json={
      "model": "qwen3.5:4b",
      "messages": [{"role": "user", "content": "Can you tell me please a bed time story?"}],
      "stream": False
  }).json()
 
 
print(r)
 
 
gen_tps    = r["eval_count"] / r["eval_duration"] * 1e9
prompt_tps = r["prompt_eval_count"] / r["prompt_eval_duration"] * 1e9
 
 
print(r["message"]["content"])
print(f"\nGeneration: {gen_tps:.1f} tok/s ({r['eval_count']} tokens)")
print(f"Prompt eval: {prompt_tps:.1f} tok/s")
print(r["message"]["content"])
print(f"\nGeneration: {gen_tps:.1f} tok/s ({r['eval_count']} tokens)")
print(f"Prompt eval: {prompt_tps:.1f} tok/s")
Of course. Here is a soft and gentle story for you to listen to.
 
***
 
Once upon a time, in a forest where the willow trees waved like green ribbons in the wind, lived a little bear named Barnaby. Barnaby was a very small, very fluffy bear who loved naps more than anything else in the world.
 
On one quiet evening, the sun dipped just below the horizon. The colors of the sky changed from orange and yellow to soft purple and deep blue. It was time for the animals to rest, but Barnaby had not yet finished his work. He decided it was the perfect day to build his special cozy nest high up in the hollow of an old oak tree.
 
Barnaby went on a little adventure. First, he found a dry fern leaf and tucked it in like a green pillow. Then, he gathered soft fallen acorns to pile around him like warm blankets. He even placed a smooth, cool stone right next to his nose to make sure he didn't roll over in his sleep.
 
When Barnaby finished arranging his nest, he took a deep breath. It smelled of damp earth and sweet berries. He looked up at the sky, where the moon was just beginning to glow like a silver coin. The crickets began to sing their soft, rhythmic song, *tick-tick, tick-tick*, and it was a very relaxing melody.
 
Barnaby curled into his little ball of fur and closed his eyes. He felt a gentle warmth coming from inside him. The wind whispered through the trees, telling him stories about the day that passed. Barnaby let out one deep breath, then another slow one, and drifted right off to sleep under the watchful eye of the moon.
 
And so, the forest fell quiet again, filled with the sounds of sleeping hearts.
 
May your night be peaceful, too. I hope you feel warm and happy as you drift off. Sweet dreams. Goodnight.
 
Generation: 7.1 tok/s (1995 tokens)
Prompt eval: 49.1 tok/s

In short, with Qwen3.5-4B:

ollama - Q4_K_M quantization: 7.1 tokens/second
CAI Neural API with Int8 quantization: 7 tokens/second
CAI Neural API with Int4 quantization: 9.8 tokens/second

PASCAL WINS ON CPU.

OpenCL/GPU benchmarks are coming soon.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 01, 2026, 08:54:34 pm

Quote from: kapibara on August 29, 2026, 02:56:32 am

Does CAI Neural support building a proper Temporal Convolutional Network (TCN) for multivariate time-series classification/prediction?

I will reply. Sorry for the delay.

Qwen 3.6 27B and Qwen 3.8 27B Benchmarks - NVIDIA RTX PRO 6000 Blackwell Server Edition

Qwen 3.6 27B

Code: Pascal [Select][+]

neural-api/bin/x86_64-linux/bin/ChatTerminal ./q36-27/ --stats --profile --int4 --max-fast-memory --seed 42 --max-new-tokens 1000 --greedy -p  "Can you tell me please a bed time story?" # --cpu --int8  --cpu
Tokenizer loaded in 0.5s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[--int4: Q4_0 tensors of a .gguf checkpoint load straight into int4 weight rows; every other tensor streams into int8 rows and TNNet.QuantizeWeightsInt4 requantizes it to Q4_0 int4 after the load; on --gpu the packed codes stay resident on the device; output quality below int8]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading ./q36-27/ ...
Model loaded in 207.4s.
[--int4: Q4_0 int4 weights on 337 layers (0 loaded directly from the checkpoint Q4_0 blocks, 337 requantized from int8); the other weight layers stay int8; int8 input copy enabled on the int4 layers]
[--gpu: OpenCL on NVIDIA CUDA / NVIDIA RTX PRO 6000 Blackwell Server Edition]
[--int4 with --gpu: cai_dot_product_int4_splitk reads the packed codes at half the int8 traffic; activations stay FP32]
GPU weights uploaded in 37.1s.
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 26895998464 params, vocab 248320, context 32768, chat format qwen3_5, int4 weights.
[reasoning effort xhigh]
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: greedy argmax (--greedy) - deterministic]
Here's a thinking thinking sequence
 
1.  **Deconstruct and Analyze the Request:**
    *   User: "Can you tell me please a bed time story?"
    *   Intent: The user wants a bedtime story.
    *   Implicit/Explicit Needs: Bedtime stories usually have certain characteristics: calming, gentle, positive resolution, not too scary, suitable for winding down. The user didn't specify an age group, characters, or theme.
 
2.  **Determine the Strategy:**
    *   *Option A: Ask for details.* "Sure! What kind of story do you like? Animals? Space? Magic?"
        *   *Pros:* Tailored to the user.
        *   *Cons:* Adds friction. The user might just want a story *now*.
    *   *Option B: Provide a generic, universally appealing story.*
        *   *Pros:* Immediate satisfaction. Low friction.
        *   *Cons:* Might not match specific preferences.
    *   *Decision:* Best practice for LLMs in this context is often to provide a story immediately but offer customization. Or, provide a very soothing, classic-style story that works for most ages. I'll go with providing a story. It's more helpful to just give the story. I can add a note at the end asking if they want a different kind.
 
3.  **Drafting the Story - Brainstorming Themes:**
    *   *Themes:* Sleep, stars, moon, animals, cozy places, dreams.
    *   *Concept 1:* A little bear who can't sleep. (Classic, but maybe too cliché?)
    *   *Concept 2:* A star that falls down and needs help getting back. (A bit active?)
    *   *Concept 3:* The Moon's quilt. (Very cozy.)
    *   *Concept 4:* A train that carries dreams. (Good imagery.)
    *   *Selection:* "The Moon's Quilt" or "The Dream Train" are strong contenders. Let's go with a story about a little cloud who wants to find the perfect dream to give to a child. Or better yet, a story about the "Star Keeper" who tucks the stars in. That feels very bedtime-appropriate.
    *   *Refined Concept:* "Oliver the Owl and the Whispering Wind." No, owls are nocturnal, might be too alert.
    *   *Refined Concept:* "Luna the Little Moon." She has to polish the stars so everyone can sleep.
    *   *Final Choice:* "The Little Star Who Lost His Glow." No, that might induce anxiety.
    *   *Winner:* "The Cozy Cloud and the Dream Weaver." A cloud that collects dreams and delivers them. Or "Barnaby the Bear's Big Sleep." Let's do "Barnaby the Bear." It's relatable. Barnaby is a bear who loves to sleep but has to say goodnight to everyone first. This encourages the listener to say goodnight to their surroundings.
    *   *Story Arc:* Barnaby is sleepy. He says goodnight to the sun, the trees, the river, the fireflies. Each interaction is calming. Finally, he curls up in his den. The moral/action is winding down.
 
4.  **Writing the Story (Mental Sketch & Phrasing):**
    *   *Title:* Barnaby Bear's Goodnight Journey.
    *   *Intro:* Once upon a time, in the heart of the Whispering Woods, lived a little bear named Barnaby. Barnaby had the fluffiest fur and the sleepiest eyes.
    *   *Body:*
        *   Barnaby yawned. "It's time for bed."
        *   He walks to the Sun. "Goodnight, Sun." The Sun dips low, painting the sky purple and orange. "Goodnight, Barnaby. I'll be back in the morning."
        *   He walks to the River. "Goodnight, River." The River ripples softly. "Shhh, shhh, goodnight."
        *   He walks to the Fireflies. "Goodnight, Fireflies." They blink slowly. Blink... blink...
        *   He walks to the Moon. "Goodnight, Moon." The Moon smiles. "I'll watch over you."
        *   *Sensory details:* Soft moss, cool air, warm den, honey jar (maybe too hungry-inducing? Skip honey, focus on comfort). A cozy blanket made of leaves.
    *   *Climax/Resolution:* Barnaby gets to his den. It's warm. He curls up. He feels safe.
[stats] 1000 tokens, TTFT 622 ms, prompt 20 (reused 0), decode 36.8 tok/s
[stats] per decode step (ms): forward 26.79, softmax 0.00, processors 0.00, sampler 0.08, emit 0.01, other 0.00, step 26.88
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
A layer that enqueues an OpenCL kernel returns before the kernel
runs, so its row charges the enqueue only; the kernels execute in
the queue drain reported under the table.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       337     3317017.69        9842.78  47.6    100%
TNNetTokenRMSNorm              129      603991.77        4682.11   8.7    100%
TNNetSum                       128      572011.46        4468.84   8.2    100%
TNNetGatedDeltaNet              48      329993.43        6874.86   4.7    100%
TNNetRotaryEmbedding            32      291005.01        9093.91   4.2    100%
TNNetDepthwiseConv1D            48      288998.38        6020.80   4.1    100%
TNNetDeepConcat                 64      279999.36        4374.99   4.0    100%
TNNetSplitChannels              32      241988.48        7562.14   3.5    100%
TNNetHeadRMSNorm                32      237999.74        7437.49   3.4    100%
TNNetSwiGLU                     64      230999.80        3609.37   3.3    100%
TNNetSiLU                       48      202990.00        4228.96   2.9    100%
TNNetFusedSDPA                  16      200000.17       12500.01   2.9    100%
TNNetCellMulByCell              16       79000.28        4937.52   1.1    100%
TNNetSigmoid                    16       59998.29        3749.89   0.9    100%
TNNetEmbedding                   1       26001.29       26001.29   0.4    100%
TNNetInput                       1        7999.48        7999.48   0.1    100%
--------------------------------------------------------------------------------------
TOTAL: 6969994.63 us across 1012 layer(s) in 16 class(es)
OpenCL queue drain: 14939001.27 us (68.2% of 21908995.90 us forward wall-clock)
[sched] width 48, workers 48 | passes: 1000 parallel, 0 serial | peak in-flight 2/48 | gain bound 1.07x, off-primary 0%

Qwen 3.8 27B

Code: Pascal [Select][+]

!neural-api/bin/x86_64-linux/bin/ChatTerminal ./q38-27/ --stats --profile --gpu --int4 --max-fast-memory --seed 42 --max-new-tokens 1000 --greedy -p  "Can you tell me please a bed time story?" # --cpu --int8  --cpu
Tokenizer loaded in 0.6s.
[context not set - defaulting to 32768 tokens; override with --ctx N (KV-cache RAM grows ~O(ctx); --kv-int8 quarters it)]
[--int4: Q4_0 tensors of a .gguf checkpoint load straight into int4 weight rows; every other tensor streams into int8 rows and TNNet.QuantizeWeightsInt4 requantizes it to Q4_0 int4 after the load; on --gpu the packed codes stay resident on the device; output quality below int8]
[--max-fast-memory: concatenated weight cache kept - faster forward, more RAM, GPU compatible]
Loading ./q38-27/ ...
Model loaded in 48.5s.
[--int4: Q4_0 int4 weights on 337 layers (0 loaded directly from the checkpoint Q4_0 blocks, 337 requantized from int8); the other weight layers stay int8; int8 input copy enabled on the int4 layers]
[--gpu: OpenCL on NVIDIA CUDA / NVIDIA RTX PRO 6000 Blackwell Server Edition]
[--int4 with --gpu: cai_dot_product_int4_splitk reads the packed codes at half the int8 traffic; activations stay FP32]
GPU weights uploaded in 37.0s.
[int8 KV cache (default with int8 weights) - ~1/4 the KV RAM, logits not bit-exact; --kv-fp32 opts out]
Model: qwen3_5, 26895998464 params, vocab 248320, context 32768, chat format qwen3_8, int4 weights.
[reasoning effort xhigh]
[turn-boundary state reuse ON (recurrent/SSM state resumed from a snapshot) - only the new prompt tail is prefilled each turn]
[layer-graph parallel forward (default) - independent layers and large conv/linear layers threaded; pass --serial for the serial loop]
[sampling: greedy argmax (--greedy) - deterministic]
We need to respond to user: "Can you tell me please a bed time story?" Need produce final with bedtime story. Keep gentle, soothing. Could ask? Just tell. Need likely in English. Ensure not too long? User asks please. We can provide a calming story. Maybe title. No need analysis. Final only story.
</think>
 
Of course. Here’s a gentle bedtime story for you.
 
**The Little Star Who Wanted to Sleep**
 
High above the quiet hills, where the sky was soft and silver, there lived a little star named Luma.
 
Luma was small and warm, and every night she glowed gently over the sleeping world. But one evening, Luma did not feel sleepy at all.
 
“I don’t want to sleep,” she whispered. “I want to stay awake and watch the moon.”
 
The moon smiled down at her. “The night is long, little star. Even the sky needs rest.”
 
Luma looked down at the earth below. The houses were dark. The trees were still. The rivers moved slowly, like lullabies made of water.
 
A little girl in a cozy room was tucking her cat under the blanket. A farmer was pulling his coat tighter. A dog curled up by the fire. Everything was getting ready for sleep.
 
But Luma still twinkled brightly.
 
“Maybe I’ll play a game,” she said. “I’ll blink until I’m sleepy.”
 
So she blinked once.
 
The moon blinked back.
 
She blinked twice.
 
The clouds drifted closer, soft as pillows.
 
She blinked three times.
 
And the night grew quieter.
 
Luma’s light began to slow. Her twinkles became smaller, softer, gentler.
 
“Goodnight, moon,” she murmured.
 
“Goodnight, little star,” the moon replied.
 
One by one, the other stars dimmed their lights, like candles in a peaceful room. The sky became a calm, dark blanket, and the world below slept deeply.
 
And Luma, the little star who wanted to stay awake, finally closed her shining eyes.
 
She dreamed of quiet rivers, soft clouds, and a little girl smiling in her sleep.
 
And if you listen very carefully on a quiet night, you can still hear her whispering:
 
“Goodnight… goodnight… goodnight.”
 
Now it’s your turn to rest. Close your eyes, breathe slowly, and let the night hold you gently.
 
Goodnight.
[stats] 516 tokens, TTFT 1706 ms, prompt 62 (reused 0), decode 37.8 tok/s
[stats] per decode step (ms): forward 25.70, softmax 0.00, processors 0.00, sampler 0.09, emit 0.00, other -0.05, step 25.74
 
Layer Class Timing Report
=========================
Aggregated forward-pass time by layer class (accumulated since
the last ClearTime). Sorted by total cost descending. The GPU
column is the share of forward dispatches that took the OpenCL
path. Convolution (incl. grouped/KAN/cross-correlation variants),
FullConnect, SDPA, RoPE/MRoPE, softmax, RMSNorm/Group/L2 norms,
GLU gates, pooling, grid/resize gathers and embedding all have a
GPU forward path; "-" = no GPU path here / never dispatched.
TNNetInput runs no kernel: its GPU share is how often it uploaded
its output and offered it on the device.
A layer that enqueues an OpenCL kernel returns before the kernel
runs, so its row charges the enqueue only; the kernels execute in
the queue drain reported under the table.
Layer class                  Count       total us    us/instance      %      GPU
--------------------------------------------------------------------------------------
TNNetPointwiseConvLinear       337     1780010.71        5281.93  48.4    100%
TNNetSum                       128      323003.55        2523.47   8.8    100%
TNNetTokenRMSNorm              129      312002.31        2418.62   8.5    100%
TNNetGatedDeltaNet              48      177000.02        3687.50   4.8    100%
TNNetDeepConcat                 64      163002.03        2546.91   4.4    100%
TNNetRotaryEmbedding            32      142000.34        4437.51   3.9    100%
TNNetDepthwiseConv1D            48      134993.48        2812.36   3.7    100%
TNNetSwiGLU                     64      123000.24        1921.88   3.3    100%
TNNetSplitChannels              32      113997.45        3562.42   3.1    100%
TNNetHeadRMSNorm                32      108992.19        3406.01   3.0    100%
TNNetFusedSDPA                  16      106998.14        6687.38   2.9    100%
TNNetSiLU                       48      106000.48        2208.34   2.9    100%
TNNetSigmoid                    16       38000.20        2375.01   1.0    100%
TNNetCellMulByCell              16       28999.92        1812.49   0.8    100%
TNNetEmbedding                   1        9998.56        9998.56   0.3    100%
TNNetInput                       1        6999.94        6999.94   0.2    100%
--------------------------------------------------------------------------------------
TOTAL: 3674999.55 us across 1012 layer(s) in 16 class(es)
OpenCL queue drain: 7140003.52 us (66.0% of 10815003.07 us forward wall-clock)
[sched] width 48, workers 48 | passes: 516 parallel, 0 serial | peak in-flight 2/48 | gain bound 1.07x, off-primary 0%

In short:

Qwen 3.6 27B Q4_0: 36.8 tokens/second
Qwen 3.8 27B Q4_0: 37.8 tokens/second

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 04, 2026, 02:02:13 am

:) I hope that I am not typing too much and making you bored... :)

I am currently working (work in progress) to make the pascal based ChatServer compatible with my other project BPSA (https://github.com/joaopauloschuler/beyond-python-smolagents)

In the attached screenshot, ChatServer is running Qwen 3.8 27B while BPSA is connecting to it.

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: Thaddy on September 04, 2026, 09:11:37 am

Quote from: schuler on September 04, 2026, 02:02:13 am

:) I hope that I am not typing too much and making you bored... :)

Not at all, the people who are interested here, quite a lot in a small community, really take pride in testing such endeavours. ;) ... and trying to understand what happens there...
Who can get bored of your code (and explanations!) ? O:-) Jig and Saw puzzling for many, but thoroughly academic sound.
You should be a teaching professor instead of a researcher. (that will probably happen anyway, if you like it or not)

Title: Re: Conscious Artificial Intelligence - Project Update
Post by: schuler on September 07, 2026, 08:50:12 pm

Dear Thaddy,
THANK YOU SO MUCH FOR YOUR KIND WORDS!!!

In the case that anyone is interested in running a local server, this is an example:
Download the model with:

Code: Pascal [Select][+]

pip install huggingface_hub
hf download Qwen/Qwen3.5-4B --local-dir ./q35-4

Then, compile the server and run the model locally with:

Code: Pascal [Select][+]

lazbuild -B neural-api/examples/ChatTerminal/ChatServer.lpi
neural-api/bin/x86_64-linux/bin/ChatServer ./q35-4/ --ctx 250000 --cache-checkpoints 16 --port 5000 --prefill-window 64 --prefill-tail-window 8 --stats --gpu --int4 --max-fast-memory --seed 42 --max-new-tokens 60000 --reasoning-effort off

As an example, for using ChatServer as BPSA model server, you can run BPSA with:

Code: Pascal [Select][+]

export BPSA_SERVER_MODEL=OpenAIServerModel
export BPSA_API_ENDPOINT="http://127.0.0.1:5000/v1"
export BPSA_KEY_VALUE='local'
export BPSA_MODEL_ID="local"
export BPSA_SYSTEM_PROMPT_FIRST=1