Recent

Author Topic: Conscious Artificial Intelligence - Project Update  (Read 38832 times)

iLya2IK

  • New Member
  • *
  • Posts: 22
Re: Conscious Artificial Intelligence - Project Update
« Reply #120 on: July 02, 2021, 10:34:07 am »
Thank you for a wonderful project.

I would like to start classifying text phrases according to certain patterns. It is about recognizing and classifying car brands and models. The number of combinations is about 2000. But all my attempts run into a fundamental misunderstanding of which class to choose for modeling - now I use TNNetForByteProcessing. Here is the code I am using to simulate:
Code: Pascal  [Select][+][-]
  1. const MAX_IN_SIZE = 25;
  2.       HIGH_IN_VAL = MAX_IN_SIZE-1;
  3.       MAX_OUT_SIZE = 256;
  4.       HIGH_OUT_VAL = MAX_OUT_SIZE-1;
  5.  
  6. type TInpArray = Array [0..HIGH_IN_VAL] of Byte;
  7.      TOutArray = Array [0..HIGH_OUT_VAL] of Byte;
  8.  
  9. var
  10.   DS : TSqlite3Dataset;
  11.   MMList : TStringList;
  12.  
  13. procedure SetCorrectOut(var aOut : TOutArray; value : Word);
  14. var Pos : Word;
  15.     Shift : Byte;
  16. begin
  17.   Pos := value div 8;
  18.   Shift := value mod 8;
  19.   aOut[Pos] := $1 shl Shift;
  20. end;
  21.  
  22. function GetCorrectOut(const aOut : TOutArray) : Word;
  23. var Pos : Word;
  24.     Shift : Byte;
  25. begin
  26.   for Pos := 0 to HIGH_OUT_VAL do
  27.   begin
  28.     if (aOut[Pos] > 0) then
  29.     begin
  30.       case aOut[Pos] of
  31.         2   : Shift := 1;
  32.         4   : Shift := 2;
  33.         8   : Shift := 3;
  34.         16  : Shift := 4;
  35.         32  : Shift := 5;
  36.         64  : Shift := 6;
  37.         128 : Shift := 7;
  38.       else
  39.         Shift := 0;
  40.       end;
  41.       Exit(Pos * 8 + Shift);
  42.     end;
  43.   end;
  44.   Result := 0;
  45. end;
  46.  
  47. function GetCorrectOutList(const aOut : TOutArray) : String;
  48. var Pos : Word;
  49.     Cnt : Byte;
  50.     S : String;
  51.  
  52. procedure AddOption(W : Word);
  53. begin
  54.   if length(S) > 0 then S := S + ' ';
  55.   S := S + MMList.Values[inttostr(W)];
  56.   Inc(Cnt);
  57. end;
  58.  
  59. begin
  60.   S := '';
  61.   Cnt := 0;
  62.   for Pos := 0 to HIGH_OUT_VAL do
  63.   begin
  64.     if (aOut[Pos] > 0) then
  65.     begin
  66.       if aOut[Pos] and $1 > 0 then AddOption(Pos shl 3);
  67.       if aOut[Pos] and $2 > 0 then AddOption(Pos shl 3 or $1);
  68.       if aOut[Pos] and $4 > 0 then AddOption(Pos shl 3 or $2);
  69.       if aOut[Pos] and $8 > 0 then AddOption(Pos shl 3 or $3);
  70.       if aOut[Pos] and $10 > 0 then AddOption(Pos shl 3 or $4);
  71.       if aOut[Pos] and $20 > 0 then AddOption(Pos shl 3 or $5);
  72.       if aOut[Pos] and $40 > 0 then AddOption(Pos shl 3 or $6);
  73.       if aOut[Pos] and $80 > 0 then AddOption(Pos shl 3 or $7);
  74.     end;
  75.     if Cnt > 15 then break;
  76.   end;
  77.   Result := S;
  78. end;
  79.  
  80. procedure RunAlgo();
  81. const MAX_EPOCH_CNT = 1000;
  82.       MAX_EPOCH_SIZE = 512;
  83.       MAX_ERROR_CNT  = 1;
  84. var
  85.   NN: TNNetForByteProcessing;
  86.   BAInput : TInpArray;
  87.   BAExpected, BAOutput :TOutArray;
  88.   L, epoch : integer;
  89.   Txt1, Res, Outv : String;
  90.   MM, MM1 : Word;
  91.   ErrorCnt, EpochSize : Integer;
  92.   NeedNextEpoch : Boolean;
  93.   Key : char;
  94. begin
  95.   NN := TNNetForByteProcessing.Create();
  96.  
  97.   try
  98.     NN.AddBasicByteProcessingLayers(MAX_IN_SIZE, MAX_OUT_SIZE, 8, 32);
  99.     NN.SetLearningRate(0.1, 0.0);
  100.  
  101.     WriteLn('Computing...');
  102.     epoch := 1;
  103.     NeedNextEpoch := true;
  104.  
  105.     ErrorCnt := MAX_ERROR_CNT + 1;
  106.     EpochSize := MAX_EPOCH_SIZE;
  107.     while NeedNextEpoch and (epoch <= MAX_EPOCH_CNT) and (ErrorCnt > MAX_ERROR_CNT) do
  108.     begin
  109.       WriteLn('Epoch #', epoch);
  110.       inc(epoch);
  111.       DS.SQL := 'select mrkorig, uniq_ats_schem.id as mmid, '+
  112.       'ats_schem.mrk||" "||ats_schem.model as expc '+
  113.       'from ats_schem, uniq_ats_marks, uniq_ats_schem '+
  114.       'where length(mrkorig) < 25 and uniq_ats_marks.mrk = ats_schem.mrk and '+
  115.       '      uniq_ats_schem.mrk = ats_schem.mrk and '+
  116.       '      uniq_ats_schem.model = ats_schem.model order by random() limit ' + inttostr(EpochSize);
  117.       DS.Open;
  118.       try
  119.         ErrorCnt := 0;
  120.         // tests the learning
  121.         while not DS.EOF do
  122.         begin
  123.           if Keypressed then // if user provided input
  124.           begin
  125.             Key := readkey;
  126.             if Key = #27 then begin
  127.               NeedNextEpoch := false;
  128.               Break;
  129.             end;
  130.           end;
  131.  
  132.           FillChar(BAInput,    MAX_IN_SIZE, #0);
  133.           FillChar(BAExpected, MAX_OUT_SIZE, #0);
  134.           FillChar(BAOutput,   MAX_OUT_SIZE, #0);
  135.           Txt1 := UTF8ToConsole(UTF8LowerCase(DS.FieldByName('mrkorig').AsString));
  136.           Res := UTF8ToConsole(DS.FieldByName('expc').AsString);
  137.           MM := DS.FieldByName('mmid').AsInteger;
  138.  
  139.           L := Length(Txt1);
  140.           if L > HIGH_IN_VAL then
  141.           begin
  142.             L := HIGH_IN_VAL;
  143.           end;
  144.           Move((@(Txt1[1]))^, (@(BAInput[0]))^, L);
  145.           SetCorrectOut(BAExpected, MM);
  146.           NN.Compute(BAInput);
  147.           NN.GetOutput(BAOutPut);
  148.           NN.Backpropagate(BAExpected);
  149.  
  150.           MM1 := GetCorrectOut(BAOutput);
  151.           if MM <> MM1 then Inc(ErrorCnt);
  152.           Outv := GetCorrectOutList(BAOutput);
  153.           WriteLn
  154.           ( 'In:',
  155.             ConsoleToUTF8(PChar(@(BAInput[0]))),
  156.             ' Out:',
  157.             Outv,
  158.             ' Expect:',
  159.             Res
  160.           );
  161.           DS.Next;
  162.         end;
  163.       finally
  164.         DS.Close;
  165.       end;
  166.     end;
  167.   finally
  168.     NN.Free;
  169.     Write('Press ENTER to exit.');
  170.     ReadLn;
  171.   end;  
  172. end;

There is a feeling that I have chosen the wrong approach. I ask you to suggest how best to approach the solution of this problem. If it is not difficult, please give an example of solving similar problems of text analysis.

schuler

  • Full Member
  • ***
  • Posts: 194
Re: Conscious Artificial Intelligence - Project Update
« Reply #121 on: July 06, 2021, 11:24:18 am »
Hi iLya2IK!
Thank you for sharing your source code example with bleeding edge new features from the API.

If I understood your code, you are placing characters into bytes. Although doable, this is an unusual approach but I must say that I have curiosity to find what the NN will find...

Have you tried much lower learning rates such as 0.001? I would give a try to:
Code: Pascal  [Select][+][-]
  1.     NN.SetLearningRate(0.001, 0.9);
  2.     NN.SetLearningRate(0.0001, 0.9);

Usually, when you see no convergence at all, you can try lower learning rates.

Are you able to share your output?

iLya2IK

  • New Member
  • *
  • Posts: 22
Re: Conscious Artificial Intelligence - Project Update
« Reply #122 on: July 06, 2021, 02:46:47 pm »
Are you able to share your output?
Thanks for the answer!

I'm afraid there is nothing to show  :(. For any parameters, the approximation degrades to the output of an empty set (the neural network produces zero bits as a result after a large number of training rounds).
I think I have found a possible alternative for building a neural network. If something works out, I will certainly post the results and the source code.

 

TinyPortal © 2005-2018