Conscious Artificial Intelligence - Project Update

[kapibara]

Following with interest.

/Roger

[schuler]

@Dzandaa and @kapibara, many thanks for your support!

In the case that you would like to compile/test your existing projects with the new branch (a1), this is the pull request:
https://github.com/joaopauloschuler/neural-api/pull/177

It will be nice to know if your models converge faster/slower/same with the new version. This new version works better with the AVX2 define instead of the previous default for AVX. I intend to fix bugs before merging into master.

Wish everyone happy pascal coding

[kapibara]

Here's my results. I tested master vs PR #177/a1 on a synthetic sequential geometry-detection task.

Environment:
  Linux x86_64
  FPC 3.3.1
  Laz trunk
  11th Gen Intel® Core™ i5-11600K @ 3.90GHz
  15,4 GiB of RAM

The task is binary classification over sequential numeric windows. Each sample is a 300-step window with 5 numeric features per step. The positive class represents a generated geometric shape/pattern inside the sequence; the negative class is background/no-pattern data.

Dataset:

Code: Pascal  [Select][+][-]

1. Total windows: 9,700
2. Train/Validation/Test: 5,820 / 1,940 / 1,940
3. Input shape: 300 x 5 x 1
4. Classes: 2
5. Batch size: 64
6. Epochs: 10
7.  

Training time

Code: Pascal  [Select][+][-]

1. master, no switches:             41.68 sec
2. master + -dAVX2:                 27.72 sec
3. master + -dAVX2 -O3:             27.31 sec
4.  
5. PR #177/a1, no switches:         34.03 sec
6. PR #177/a1 + -dAVX2:             19.30 sec
7. PR #177/a1 + -dAVX2 -O3:         13.09 sec
8.  

Evaluation time

Code: Pascal  [Select][+][-]

1. master, no switches:             ~7.30 sec
2. master + -O3:                     5.95 sec
3. master + -dAVX2:                  ~4.62 sec
4. master + -dAVX2 -O3:              4.25 sec
5.  
6. PR #177/a1, no switches:          5.42 sec
7. PR #177/a1 + -O3:                 3.23 sec
8. PR #177/a1 + -dAVX2:              1.74 sec
9. PR #177/a1 + -dAVX2 -O3:          0.95 sec
10.  

Quality

Code: Pascal  [Select][+][-]

1. master + -dAVX2 -O3:
2.   Accuracy:   99.02%
3.   Precision:  98.97%
4.   Recall:    100.00%
5.   F1:         99.48%
6.   FP/FN:      19 / 0
7.  
8. PR #177/a1 + -dAVX2 -O3:
9.   Accuracy:   99.28%
10.   Precision: 100.00%
11.   Recall:     99.23%
12.   F1:         99.62%
13.   FP/FN:       0 / 14
14.  

Main result:
With -dAVX2 -O3, PR #177/a1 trained about 2.1x faster and evaluated about 4.5x faster than master on this test, with similar or slightly better F1.

[schuler]

@kapibara, as a way to say “thank you”, if you have any feature that you would like coded, please feel free to ask. If not too difficult, AI will code it :-) .

[Dzandaa]

Hi Joao-Paulo,

Can me explain me how to create a semantic segmentation of an image?

Is it possible with CAI?

Thank you.

B->

[schuler]

@Dzandaa,
I will add to my task list coding an example for this.

[microxa]

Hi Paolo! Please take a look at the StringManipulation example — there is still some weird bug there (potentially a glitch in 32-bit FPC 3.2.2). In an earlier version (2.0), this was resolved by reverting to the previous class, i.e., TNNetSoftMax = class(TNNetIdentify).After doing that, the example needed only 5 epochs to reach 100% training accuracy and correct output. Good luck an regards!

[schuler]

@microca,
Thank you for the bug report. I will look at it.

In the case that you are interested, I asked Claude to produce a list of interesting NLP related examples in this API. The below AI generated list has been coded but not yet properly tested.

In the list: Model — Its Building Block: Example showing the building block.

  - DeepSeek — MLA + decoupled-RoPE slice (V2): examples/LatentAttention/LatentAttention.lpr (builds MHA vs MLA vs MLA+decoupled-RoPE arms side by side); multi-token prediction (V3): examples/MultiTokenPrediction/MultiTokenPrediction.lpr (also used by examples/SelfSpeculativeDecoding/SelfSpeculativeDecoding.lpr).
  - LLaMA — pre-norm RMSNorm residual blocks: examples/PreNormVsPostNorm/PreNormVsPostNorm.lpr; SwiGLU: examples/SwiGLUFeedForward/SwiGLUFeedForward.lpr; RoPE: examples/RoPEBaseFrequencySweep/RoPEBaseFrequencySweep.lpr (also compared in examples/PositionEncodingBakeoff/PositionEncodingBakeoff.lpr).
  - Mistral / Longformer — sliding-window causal mask: examples/SlidingWindowBakeoff/SlidingWindowBakeoff.lpr.
  - Gemma — logit soft-capping: examples/SoftCappingStability/SoftCappingStability.lpr and examples/SoftCappingSweep/SoftCappingSweep.lpr; GeGLU: examples/GEGLUFeedForward/GEGLUFeedForward.lpr (also in examples/GatedFFNBakeoff/GatedFFNBakeoff.lpr).
  - BERT / GPT — GELU: examples/ActivationBakeoff/ActivationBakeoff.lpr; GPT-style decoder blocks: examples/TransformerDecoderBlock/TransformerDecoderBlock.lpr (full pipeline in examples/SimpleNLP/TransformerWithTokenizer.lpr).
  - Mamba — TNNetSelectiveSSM: examples/SelectiveSSM/SelectiveSSM.lpr.
  - Modern recurrent family — RWKV-4: examples/RWKV/RWKV.lpr (cross variant: examples/CrossWKV/CrossWKV.lpr); xLSTM sLSTM: examples/SLSTMvsCfC/SLSTMvsCfC.lpr; xLSTM mLSTM: no example yet (TNNetMLSTMCell has tests but no examples/ program); RetNet: examples/RetentionDualForm/RetentionDualForm.lpr; Titans: examples/TitansMemory/TitansMemory.lpr; DeltaNet: examples/DeltaNet/DeltaNet.lpr; GLA: examples/GatedLinearAttention/GatedLinearAttention.lpr (block form: examples/GatedLinearAttentionBlock/GatedLinearAttentionBlock.lpr).
  - Switch Transformer (load-balance loss): examples/TopKMoE/TopKMoE.lpr (contrast without aux loss: examples/ExpertChoiceMoE/ExpertChoiceMoE.lpr).
  - DIFF Transformer: examples/DifferentialAttentionNoise/DifferentialAttentionNoise.lpr.
  - StreamingLLM (attention sinks): examples/SinkAttentionStability/SinkAttentionStability.lpr.
  - BLOOM-lineage ALiBi: examples/ALiBiSlopeSweep/ALiBiSlopeSweep.lpr (also in examples/PositionEncodingBakeoff/PositionEncodingBakeoff.lpr).
  - GQA/MQA via KVHeads: no example yet — the KVHeads parameter exists on the MHA builders (neural/neuralnetwork.pas:11123) and GQA is discussed in examples/LatentAttention/README.md as a comparison point, but no example program actually passes KVHeads <> Heads.

This is a good starting point: https://github.com/joaopauloschuler/neural-api/blob/master/examples/SimpleNLP/DecodeFeaturesBakeoff.lpr (includes some building blocks for DeepSeek v2 and v3)

@all, the list of examples is located at: https://github.com/joaopauloschuler/neural-api/blob/master/examples/README.md

[schuler]

Hi Joao-Paulo,

Can me explain me how to create a semantic segmentation of an image?

Is it possible with CAI?

Thank you.

B->

@Dzandaa, as a source of inspiration for both of us, have a look at please: https://github.com/joaopauloschuler/neural-api/blob/master/neural/neuralpretrained.pas

The master branch is under heavy coding. Therefore, expect bugs. Anyway, I could extend neuralpretrained.pas to support a vision model or models that will do your segmentation.

Is there any image segmentation model from hugging face that you would like supported in this API?

[Dzandaa]

Hi Joao-Paulo

Thank you, I'll read that next week.

I had found an interesting web page that explained the different parts of a Hugging Face segmentation, with images of the different intermediate layers, but I can't find it anymore, sorry.

Here is 2 interesting links:

https://github.com/huggingface/blog/blob/main/mask2former.md

https://medium.com/@az.tayyebi/segformer-model-training-from-scratch-vs-using-hugging-face-f5234f31399a


B->

[schuler]

This is to share an experiment. It is not in the master branch yet. It is still slow and memory hungry. But it is cool to see it.

Code: Pascal  [Select][+][-]

1. # git clone -b a3 https://github.com/joaopauloschuler/neural-api.git
2. # git clone https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct q2
3. # lazbuild neural-api/examples/ChatTerminal/ChatTerminal.lpi
4. # neural-api/bin/x86_64-linux/bin/ChatTerminal q2/ --ctx 512 --fp32
5. Loading q2/ ...
6. Model: qwen2, 630139776 params, vocab 151936, context 512, chat format qwen, fp32 weights.
7. Type your message; /exit quits, /reset clears the history,
8. /system <msg> sets the system prompt.
9. > Hello super-intelligence!
10. Hello! How can I assist you today?
11. > What is the capital of France please?
12. The capital of France is Paris.
13. > What is the spoken language there?
14. The spoken language in Paris is French.
15. > Is it a nice place for a holliday?
16. Paris is a beautiful city with a rich history and culture. It's a great place for a romantic getaway, especially during the summer months when the weather is pleasant. The city is known for its beautiful architecture, charming neighborhoods, and delicious food. However, it's important to note that Paris is a bustling city with a lot of activity, so it's important to plan ahead and make sure you have enough time to explore and enjoy the city.

[schuler]

I have another slow, memory hungry, 100% pascal coded and cool to see experiment to share. This is the very first experiment that produces music (see attached wav transformed into mp4 file via ShotCut) using this API.

Code: Pascal  [Select][+][-]

1. # git clone -b a3 https://github.com/joaopauloschuler/neural-api.git
2. # lazbuild neural-api/examples/MusicGenText/MusicGenText.lpi
3. # neural-api/bin/x86_64-linux/bin/MusicGenText --download --seconds 5 --prompt "warm lo-fi hip hop beat with mellow piano"
4. MusicGen TEXT-CONDITIONED generation - prompt -> T5 -> music -> WAV
5. ==================================================================
6. Downloading checkpoints via HuggingFace Hub (cache: /root/.cache/neural-api/hub)...
7. [time] fetch facebook/musicgen-small: 11.62 s
8. [time] fetch t5-base: 3.17 s
9. [time] fetch facebook/encodec_32khz: 4.85 s
10. Prompt: "warm lo-fi hip hop beat with mellow piano"
11.  
12. [time] T5 tokenizer load: 299 ms
13. Target length: 5.0 s -> 250 frames.
14. [time] T5 model load (/root/.cache/neural-api/hub/t5-base/main/model.safetensors): 9.55 s
15. T5 text encoder: t5 config: enc_layers=12, dec_layers=12, heads=12, d_model=768, d_kv=64, d_ff=3072, vocab=32128, rel_buckets=32, rel_max_dist=128, ln_eps=9.999999975E-7, tied=True
16. Prompt token ids: 1978 6899 18 89 23 5436 13652 3853 28 3 2341 3216 8355 1
17. [time] T5 encoder forward pass: 163 ms
18. Encoder hidden states: 14x768 (seq x d_model)
19.  
20. [time] MusicGen decoder load (/root/.cache/neural-api/hub/facebook/musicgen-small/main/model.safetensors): 21.75 s
21. musicgen config: layers=24, heads=16, hidden=1024, ffn=4096, vocab=2048, codebooks=4, text_d_model=768, max_pos=2048
22. Generating 250 frames over 4 codebooks via the delay pattern (conditioned on the T5 prompt)...
23. Sampling: weighted top-k = 250, temperature = 1.00.
24. Classifier-free guidance ON (scale = 3.00, null = zeroed text condition; KV-cache on (dual-twin)).
25. [time] MusicGen decode (250 frames): 150.91 s
26. Code-stack health (4 codebooks x 250 frames, vocab=2048):
27.   cb0: range [8..2007], distinct=148/250, top id 1314 x10 (4%)
28.   cb1: range [21..2037], distinct=160/250, top id 1136 x6 (2%)
29.   cb2: range [39..2043], distinct=150/250, top id 1960 x8 (3%)
30.   cb3: range [11..2037], distinct=149/250, top id 2037 x8 (3%)
31.  
32. [time] EnCodec decoder load (/root/.cache/neural-api/hub/facebook/encodec_32khz/main/model.safetensors): 1.15 s
33. EnCodec decoder: EnCodec(encodec): sampling_rate=32000 channels=1 hidden=128 filters=64 residual_layers=1 ratios=[8,5,4,4] kernel=7 last_kernel=7 residual_kernel=3 dilation_growth=2 compress=2 lstm_layers=2 codebook_size=2048 codebook_dim=128 num_quantizers=4 causal=False norm=weight_norm
34. [time] EnCodec waveform synthesis: 112.16 s
35. Decoded waveform: 160000 samples at 32000 Hz
36. Wrote musicgen_text_demo.wav (160000 samples, 5.00 s).
37.  
38. Done: the prompt drove a real T5 encoder + MusicGen decoder + EnCodec synthesis.
39.  
40.  

[Dzandaa]

Hi,

@jao paulo:

I tried the MusicGenText program and it seems to work in Windows 10.

Is it possible to change the .cache path, because my P.C. have 2 disks: one for system and another for applications.

I also tried AI Music Generator on Windows 10,

Good work,

Trying now with 15 seconds MusicGenText but it crashes

MusicGenText --download --seconds 15 --prompt "warm  metal rock beat with  piano bass brown guitar bass"

B->

[schuler]

Is it possible to change the .cache path, because my P.C. have 2 disks: one for system and another for applications.

@Dzandaa,
You can define the place where to store models using the environment variable NEURAL_API_HUB_CACHE.

I tested on a google colab 51GB RAM (no gpu) machine with up to 32 seconds generation. It works for me with "--guidance 1.0" and 43.3GB of RAM usage. I believe that it is crashing due to RAM allocation. You can trade quality by memory/speed using the parameter “--guidance 1.0”.

At this moment, CAI is loading some of its training tooling even when running inference only. This is why we are getting unusual RAM allocation. I will fix this eventually.

Only if you can, I am super curious to know if you can run ChatTerminal with Qwen2.5-0.5B-Instruct at your end with “--ctx 512 --fp32 --stats”. I am curious to know how many tokens per second you will get.

wish everyone happy pascal coding

[schuler]

Can me explain me how to create a semantic segmentation of an image?
B->

Construction has started but it is not ready yet: https://github.com/joaopauloschuler/neural-api/tree/master/examples/SemanticSegmentation