### Bookstore

 Computer Math and Games in Pascal (preview) Lazarus Handbook

### Author Topic: Conscious Artificial Intelligence - Project Update  (Read 37986 times)

#### iLya2IK

• New Member
• Posts: 22
##### Re: Conscious Artificial Intelligence - Project Update
« Reply #120 on: July 02, 2021, 10:34:07 am »
Thank you for a wonderful project.

I would like to start classifying text phrases according to certain patterns. It is about recognizing and classifying car brands and models. The number of combinations is about 2000. But all my attempts run into a fundamental misunderstanding of which class to choose for modeling - now I use TNNetForByteProcessing. Here is the code I am using to simulate:
Code: Pascal  [Select][+][-]
1. const MAX_IN_SIZE = 25;
2.       HIGH_IN_VAL = MAX_IN_SIZE-1;
3.       MAX_OUT_SIZE = 256;
4.       HIGH_OUT_VAL = MAX_OUT_SIZE-1;
5.
6. type TInpArray = Array [0..HIGH_IN_VAL] of Byte;
7.      TOutArray = Array [0..HIGH_OUT_VAL] of Byte;
8.
9. var
10.   DS : TSqlite3Dataset;
11.   MMList : TStringList;
12.
13. procedure SetCorrectOut(var aOut : TOutArray; value : Word);
14. var Pos : Word;
15.     Shift : Byte;
16. begin
17.   Pos := value div 8;
18.   Shift := value mod 8;
19.   aOut[Pos] := \$1 shl Shift;
20. end;
21.
22. function GetCorrectOut(const aOut : TOutArray) : Word;
23. var Pos : Word;
24.     Shift : Byte;
25. begin
26.   for Pos := 0 to HIGH_OUT_VAL do
27.   begin
28.     if (aOut[Pos] > 0) then
29.     begin
30.       case aOut[Pos] of
31.         2   : Shift := 1;
32.         4   : Shift := 2;
33.         8   : Shift := 3;
34.         16  : Shift := 4;
35.         32  : Shift := 5;
36.         64  : Shift := 6;
37.         128 : Shift := 7;
38.       else
39.         Shift := 0;
40.       end;
41.       Exit(Pos * 8 + Shift);
42.     end;
43.   end;
44.   Result := 0;
45. end;
46.
47. function GetCorrectOutList(const aOut : TOutArray) : String;
48. var Pos : Word;
49.     Cnt : Byte;
50.     S : String;
51.
53. begin
54.   if length(S) > 0 then S := S + ' ';
55.   S := S + MMList.Values[inttostr(W)];
56.   Inc(Cnt);
57. end;
58.
59. begin
60.   S := '';
61.   Cnt := 0;
62.   for Pos := 0 to HIGH_OUT_VAL do
63.   begin
64.     if (aOut[Pos] > 0) then
65.     begin
66.       if aOut[Pos] and \$1 > 0 then AddOption(Pos shl 3);
67.       if aOut[Pos] and \$2 > 0 then AddOption(Pos shl 3 or \$1);
68.       if aOut[Pos] and \$4 > 0 then AddOption(Pos shl 3 or \$2);
69.       if aOut[Pos] and \$8 > 0 then AddOption(Pos shl 3 or \$3);
70.       if aOut[Pos] and \$10 > 0 then AddOption(Pos shl 3 or \$4);
71.       if aOut[Pos] and \$20 > 0 then AddOption(Pos shl 3 or \$5);
72.       if aOut[Pos] and \$40 > 0 then AddOption(Pos shl 3 or \$6);
73.       if aOut[Pos] and \$80 > 0 then AddOption(Pos shl 3 or \$7);
74.     end;
75.     if Cnt > 15 then break;
76.   end;
77.   Result := S;
78. end;
79.
80. procedure RunAlgo();
81. const MAX_EPOCH_CNT = 1000;
82.       MAX_EPOCH_SIZE = 512;
83.       MAX_ERROR_CNT  = 1;
84. var
85.   NN: TNNetForByteProcessing;
86.   BAInput : TInpArray;
87.   BAExpected, BAOutput :TOutArray;
88.   L, epoch : integer;
89.   Txt1, Res, Outv : String;
90.   MM, MM1 : Word;
91.   ErrorCnt, EpochSize : Integer;
92.   NeedNextEpoch : Boolean;
93.   Key : char;
94. begin
95.   NN := TNNetForByteProcessing.Create();
96.
97.   try
99.     NN.SetLearningRate(0.1, 0.0);
100.
101.     WriteLn('Computing...');
102.     epoch := 1;
103.     NeedNextEpoch := true;
104.
105.     ErrorCnt := MAX_ERROR_CNT + 1;
106.     EpochSize := MAX_EPOCH_SIZE;
107.     while NeedNextEpoch and (epoch <= MAX_EPOCH_CNT) and (ErrorCnt > MAX_ERROR_CNT) do
108.     begin
109.       WriteLn('Epoch #', epoch);
110.       inc(epoch);
111.       DS.SQL := 'select mrkorig, uniq_ats_schem.id as mmid, '+
112.       'ats_schem.mrk||" "||ats_schem.model as expc '+
113.       'from ats_schem, uniq_ats_marks, uniq_ats_schem '+
114.       'where length(mrkorig) < 25 and uniq_ats_marks.mrk = ats_schem.mrk and '+
115.       '      uniq_ats_schem.mrk = ats_schem.mrk and '+
116.       '      uniq_ats_schem.model = ats_schem.model order by random() limit ' + inttostr(EpochSize);
117.       DS.Open;
118.       try
119.         ErrorCnt := 0;
120.         // tests the learning
121.         while not DS.EOF do
122.         begin
123.           if Keypressed then // if user provided input
124.           begin
126.             if Key = #27 then begin
127.               NeedNextEpoch := false;
128.               Break;
129.             end;
130.           end;
131.
132.           FillChar(BAInput,    MAX_IN_SIZE, #0);
133.           FillChar(BAExpected, MAX_OUT_SIZE, #0);
134.           FillChar(BAOutput,   MAX_OUT_SIZE, #0);
135.           Txt1 := UTF8ToConsole(UTF8LowerCase(DS.FieldByName('mrkorig').AsString));
136.           Res := UTF8ToConsole(DS.FieldByName('expc').AsString);
137.           MM := DS.FieldByName('mmid').AsInteger;
138.
139.           L := Length(Txt1);
140.           if L > HIGH_IN_VAL then
141.           begin
142.             L := HIGH_IN_VAL;
143.           end;
144.           Move((@(Txt1[1]))^, (@(BAInput[0]))^, L);
145.           SetCorrectOut(BAExpected, MM);
146.           NN.Compute(BAInput);
147.           NN.GetOutput(BAOutPut);
148.           NN.Backpropagate(BAExpected);
149.
150.           MM1 := GetCorrectOut(BAOutput);
151.           if MM <> MM1 then Inc(ErrorCnt);
152.           Outv := GetCorrectOutList(BAOutput);
153.           WriteLn
154.           ( 'In:',
155.             ConsoleToUTF8(PChar(@(BAInput[0]))),
156.             ' Out:',
157.             Outv,
158.             ' Expect:',
159.             Res
160.           );
161.           DS.Next;
162.         end;
163.       finally
164.         DS.Close;
165.       end;
166.     end;
167.   finally
168.     NN.Free;
169.     Write('Press ENTER to exit.');
171.   end;
172. end;

There is a feeling that I have chosen the wrong approach. I ask you to suggest how best to approach the solution of this problem. If it is not difficult, please give an example of solving similar problems of text analysis.

#### schuler

• Full Member
• Posts: 194
##### Re: Conscious Artificial Intelligence - Project Update
« Reply #121 on: July 06, 2021, 11:24:18 am »
Hi iLya2IK!
Thank you for sharing your source code example with bleeding edge new features from the API.

If I understood your code, you are placing characters into bytes. Although doable, this is an unusual approach but I must say that I have curiosity to find what the NN will find...

Have you tried much lower learning rates such as 0.001? I would give a try to:
Code: Pascal  [Select][+][-]
1.     NN.SetLearningRate(0.001, 0.9);
2.     NN.SetLearningRate(0.0001, 0.9);

Usually, when you see no convergence at all, you can try lower learning rates.

Are you able to share your output?

#### iLya2IK

• New Member
• Posts: 22
##### Re: Conscious Artificial Intelligence - Project Update
« Reply #122 on: July 06, 2021, 02:46:47 pm »
Are you able to share your output?