Recent

Author Topic: Hopeless Debugging  (Read 5843 times)

BLL

  • Full Member
  • ***
  • Posts: 231
Re: Hopeless Debugging
« Reply #15 on: March 09, 2018, 11:33:36 am »
Hi
Thanks for the reply. I tried in vain to use pascal serial routines. sdposerial was completely unreliable, often seizing up ports, which is why I went to calling C progs instead. With added debug switches, my access violation error has become a range check error, so some progress.

Brian

BLL

  • Full Member
  • ***
  • Posts: 231
Re: Hopeless Debugging
« Reply #16 on: March 09, 2018, 11:53:55 am »
Jamie, concerning TProcess. As I understand it, ShellExecute is for MS Windows only - I am using linux jessie on a RaspberryPi. What do you mean by "shell shortcuts"?

Thanks

Brian

taumetric

  • Newbie
  • Posts: 1
Re: Hopeless Debugging
« Reply #17 on: March 10, 2018, 11:47:17 pm »
Hi,
I am sure you checked these but just to be complete.
It sounds like overwriting of memory as that would cause USB to fail (messes up data structure) and the changing of that string variable DataList.Strings[12](and others) all of which would be on the heap somewhere.
Be careful of the Range Check Error as it may be in your code but not related.
Do you have any pointer operations? If so it could be a data dependent (as in info from the PIC via USB) operation that is causing a pointer to be set incorrectly especially if mixing arrays with strings or short strings and passing pointers.  Data dependent operations could be why it runs for a while before faulting.
Also, do you have any threading? If so check all that any shared data has critical sections.  Bad or missing critical sections causes race conditions and overwrites.


engkin

  • Hero Member
  • *****
  • Posts: 2128
Re: Hopeless Debugging
« Reply #18 on: March 11, 2018, 01:35:53 am »
Hi Brian, I just read some of your posts and I came across this notice in your code:
Code: Pascal  [Select]
  1.     //spurious 1st char sometimes appears. If so, get rid of it!

You seem to expect wrong data sometimes (noise?) and you use Val and ErrCode to check that you have a valid number, as you had explained.

I just want to let you know that if you expect a number like 9100 (four digits) or 97100 (five digits) and instead you received 9E100 (five characters). Pass it to Val and see what happens.

You also mentioned you are sure the problem is related to doElecState. Try to enclose it in a Try Except block as in:
Code: Pascal  [Select]
  1. procedure TMainForm.doElecState;
  2. var
  3. ...
  4. begin
  5.   try
  6.     your original code here
  7.   except
  8.   end;
  9. end;
and see if your problem disappears.

Consider using a checksum in the data coming from the PIC.
« Last Edit: March 11, 2018, 02:29:56 am by engkin »

josh

  • Hero Member
  • *****
  • Posts: 651
Re: Hopeless Debugging
« Reply #19 on: March 11, 2018, 01:46:08 am »
Hi,

Just curious if getting range error, it could be checking for string location [1] of s null string; also could be the data your checking in the routine has been altered during the running of the routine.

I have done some mods, I have not tested it obviously I cant; so hopefully you will get the idea, When main function is called; it immediately creates a working copy of the Data; not sure what structure your using so you will need to assign and create the working data type ( Then free the working datatype when exiting);

It will not compile as is, but should be easy enough to adapt, hopefully its over kill, but should help in tracking down the issue; you could put breaks in the get_valid_data functions to se what data is coming in.

Code: Pascal  [Select]
  1. Function TMainForm.doElecState:Boolean; //
  2.  
  3. var V, I, P, Today, DAverage, solarData,anyString: String;
  4. current, voltage, power, excess, used_today, daily_average, cost, tmp, solarI: Real;
  5. tmp1, j, Chg:integer;
  6. LBV: real;
  7. gud: boolean;
  8.  
  9. working_Data:TStringList;// DataList Type;
  10.  
  11. Function Check_Value_Is_Valid_Integer(Var AString:String; Var AValue:Integer):Boolean;
  12. var VErrorCode;
  13. begin
  14.   result:=true;
  15.   If AString='' then exit(False);
  16.   if (Ord(AString[1]) < 48) or (Ord(AString[1]) > 57) then Delete(AString, 1, 1);
  17.   If AString='' then exit(False);
  18.   Val(AString, AValue, VErrorCode);
  19.   If VErrorCode<>0 then exit(False);
  20.   // You can check for valid range if you like ie If ((AValue<-100) or (AValue>200)) then exit(false);
  21. end;
  22.  
  23. Function Check_Value_Is_Valid_Real(Var AString:String; Var AValue:Real):Boolean;
  24. var VErrorCode;
  25. begin
  26.   result:=true;
  27.   If AString='' then exit(False);
  28.   if (Ord(AString[1]) < 48) or (Ord(AString[1]) > 57) then Delete(AString, 1, 1);
  29.   If AString='' then exit(False);
  30.   Val(AString, AValue, VErrorCode);
  31.   If VErrorCode<>0 then exit(False);
  32.   // You can check for valid range if you like ie If ((AValue<-100.00) or (AValue>200.00)) then exit(false);
  33. end;
  34.  
  35. begin
  36.  Result:=False;
  37.  Working_Data.TStringList.Create;
  38.  Working_Data:=DataList;// Make a copy of DataList immediately; incase it gets changed mid run of routine.
  39.  if(Working_Data.Count <> 16) then exit(False); // Exit function with reult set as false
  40.  if Assigned(ElecForm) then
  41.  begin
  42.    v:=Working_Data.Strings[0];
  43.    If Check_Value_Is_Valid_Real(v,tmp) Then
  44.    begin
  45.      voltage := 2 * (tmp - 200);
  46.      ElecForm.VoltBar.Progress := round(voltage);
  47.      ElecForm.VdigLabel.Caption := V + 'V';
  48.      ElecForm.VdigLabel.Repaint;
  49.    End
  50.    else
  51.    begin
  52.      Working_Data.Free;
  53.      Exit(False);
  54.    end;
  55.  
  56.    i:=Working_Data.Strings[1];
  57.    If Check_Value_Is_Valid_Real(i,current) Then
  58.    begin
  59.      ElecForm.IdigLabel.Caption := I + 'A';
  60.      ElecForm.IdigLabel.Repaint;
  61.       //Set appropriate range for display
  62.      if current > 10.0 then
  63.      begin
  64.        ElecForm.IunitsLabel.Caption := HAScale;
  65.        ElecForm.CurrentBar.Progress := round(5 * current);
  66.      end
  67.      else
  68.      begin
  69.        ElecForm.IunitsLabel.Caption := LAScale;
  70.        ElecForm.CurrentBar.Progress := round(10 * current);
  71.      end;
  72.    End
  73.    else
  74.    begin
  75.      Working_Data.Free;
  76.      Exit(False);
  77.    end;
  78.  
  79.    P := Working_Data.Strings[2];
  80.    If Check_Value_Is_Valid_Real(p,power) Then
  81.    begin
  82.      if power > 1000 then
  83.      begin
  84.        ElecForm.PUnitsLabel.Caption := HPScale;
  85.        ElecForm.Label14.Caption := 'Power (kW):';
  86.        ElecForm.PowerBar.Progress := round(power/50);
  87.        ElecForm.PdigLabel.Caption := FloatToStrF(power/1000, ffFixed,2,2) + 'kW';
  88.      end
  89.      else
  90.      begin
  91.        ElecForm.PUnitsLabel.Caption := LPScale;
  92.        ElecForm.Label14.Caption := 'Power (W):';
  93.        ElecForm.PowerBar.Progress := round(power / 10);
  94.        ElecForm.PdigLabel.Caption := P + 'W';
  95.      end;
  96.    end
  97.    else
  98.    begin
  99.      Working_Data.Free;
  100.      Exit(False);
  101.    end;
  102.  
  103.    //add values to StatusList
  104.    { StatusList.Strings[13] :=  ls + 'V = ' + ElecForm.VdigLabel.Caption + ', I = '
  105.           + ElecForm.IdigLabel.Caption + ', P = ' + ElecForm.PdigLabel.Caption + le; }
  106.    //Used today
  107.    Today := Working_Data.Strings[10];
  108.    If Check_Value_Is_Valid_Real(today,used_today) Then
  109.    begin
  110.      if(used_today >= 0) and (used_today < 100) then
  111.      begin
  112.        ElecForm.TodayBar.Progress := round(used_today * 10);
  113.        ElecForm.TdigLabel.Caption := Today + 'kWh';
  114.        StatusList.Strings[14] := ls + 'Used today = ' + ElecForm.TdigLabel.Caption + le;
  115.        ElecForm.TdigLabel.Repaint;
  116.      end;
  117.      if(used_today > 6.0) then ElecForm.TodayBar.ForeColor  := clRed
  118.      else ElecForm.TodayBar.ForeColor := clLime;
  119.    End;
  120.  
  121.    //daily average
  122.    DAverage := Working_Data.Strings[9];
  123.    If Check_Value_Is_Valid_Real(DAverage,daily_average) Then
  124.    Begin
  125.      if(daily_average < 20) then
  126.      begin
  127.        ElecForm.DAvg.Progress := round(daily_average * 5);
  128.        ElecForm.DAdigLabel.Caption := DAverage + 'kWh';
  129.        //StatusList.Strings[15] := ls + 'Daily Avg = ' + ElecForm.DAdigLabel.Caption + le;
  130.      end;
  131.      if(daily_average > 6) then ElecForm.DAvg.ForeColor := clRed
  132.      else ElecForm.DAvg.ForeColor := clLime;
  133.    end;
  134.  
  135.    //Excess units
  136.    AnyString:= Working_Data.Strings[11];
  137.    If Check_Value_Is_Valid_Real(AnyString,excess) Then
  138.    Begin
  139.      if(excess < 0) then excess := 0;
  140.      ElecForm.ExcessUnits.Caption := FloatToStrF(excess, ffFixed,3,1)+'kWh';
  141.    end;
  142.  
  143.    //Currency unit
  144.    if not Assigned(setupform) then exit;
  145.    if curSymbol = '' then
  146.    begin
  147.      AnyString:= setupData.Strings[3];
  148.      If Check_Value_Is_Valid_Integer(AnyString,tmp1) Then
  149.      Begin
  150.        if tmp1 = 1 then curSymbol := '€' else curSymbol := '£';
  151.      end
  152.      else
  153.      begin
  154.        Working_Data.Free;
  155.        Exit(False);
  156.      end;
  157.    end;
  158.  
  159.    //Cost
  160.    AnyString:= Working_Data.Strings[12];
  161.    If Check_Value_Is_Valid_Real(AnyString,cost) Then
  162.    begin
  163.      if cost < 0 then cost := 0;
  164.      ElecForm.Cost.Caption := AnyString;
  165.    end
  166.    else
  167.    begin
  168.      Working_Data.Free;
  169.      Exit(False);
  170.    end;
  171.  
  172.      //end;
  173.     //StatusList.Strings[16] := '';
  174.    { if CurSymbol = '£' then
  175.      StatusList.Strings[16] := ls + 'Cost = ' + '£'
  176.          + DataList.Strings[12] + le
  177.     else
  178.      StatusList.Strings[16] := ls + 'Cost = ' + DataList.Strings[12]
  179.         + '€' + le; }
  180.    ElecForm.Cost.Caption := '';
  181.    if CurSymbol = '€' then ElecForm.Cost.Caption := 'Cost = ' + DataList.Strings[12] + CurSymbol
  182.    else ElecForm.Cost.Caption := 'Cost = ' + CurSymbol + DataList.Strings[12];
  183.  
  184.  
  185.      //process LBV, ChgI
  186.    for j := 0 to 5 do
  187.    begin
  188.      gud := false;
  189.        //read SOLAR battery voltage value
  190.      solarData := srData('SOLAR', '12548'); //0x3104
  191.        //get rid of CR at end of string
  192.      if Length(solarData) > 1 then SetLength(solarData, Length(solarData) - 1);
  193.      If Check_Value_Is_Valid_Real(solarData,LBV) Then
  194.      begin
  195.        gud := true;
  196.        break;
  197.      end;
  198.    end;
  199.  
  200.      //final check in case all 5 read attempts failed
  201.    if gud = false then
  202.    begin
  203.      Working_Data.Free;
  204.      Exit(False);
  205.    end
  206.    else
  207.    begin
  208.      LBV := LBV/100; //mV to V
  209.      ElecForm.LBVValue.Caption := FloatToStrF(LBV, ffFixed,3,2) + 'V';
  210.         {StatusList.Strings[18] := ls + 'Pri Bat = '
  211.             + ElecForm.LBVValue.Caption + ' Sec = ' + le;}
  212.          //offset as scale starts at 8V
  213.      LBV := LBV - 8;
  214.      if LBV < 0 then LBV := 0;
  215.      ElecForm.LBVBar.Progress := round(LBV * 10);
  216.    end;
  217.  
  218.      //Battery charge percentage
  219.    for j := 0 to 5 do
  220.    begin
  221.      gud := false;
  222.      solarData := srData('SOLAR', '12570'); //0x311A
  223.      if Length(solarData) > 1 then SetLength(solarData, Length(solarData) - 1);
  224.      If Check_Value_Is_Valid_Real(solarData,Chg) Then
  225.      begin
  226.        gud := true;
  227.        break;
  228.      end;
  229.    end;
  230.  
  231.    if gud = false then exit
  232.    else
  233.    begin
  234.      if Chg > 25 then ElecForm.BatState.ForeColor := clLime
  235.      else ElecForm.BatState.ForeColor := clRed;
  236.      ElecForm.BatState.Progress := Chg;
  237.    end;
  238.  
  239.      //LBI
  240.    AnyString:= Working_Data.Strings[6];
  241.    If Check_Value_Is_Valid_Real(AnyString,LBI) Then
  242.    begin
  243.      LBI := LBI - 499;
  244.      if LBI < 0 then LBI := 0;
  245.      ElecForm.ChgI.Progress := round (LBI/5.843);
  246.      ElecForm.LBIValue.Caption := FloatToStrF(LBI/10.5, ffFixed,3,2) + 'A';
  247.    end
  248.    else
  249.    begin
  250.      Working_Data.Free;
  251.      Exit(False);
  252.    end;
  253.  
  254.      //solar charge current
  255.    for j := 0 to 5 do
  256.    begin
  257.      gud := false;
  258.      solarData := srData('SOLAR', '12549'); //0x3105
  259.        //get rid of CR at end of string, only if string isn't empty
  260.         //or an Access violation occurs
  261.      if Length(solarData) > 1 then
  262.      SetLength(solarData, Length(solarData) - 1);
  263.      If Check_Value_Is_Valid_Real(solarData,solarI) Then
  264.      begin
  265.        gud := true;
  266.        break;
  267.      end;
  268.    end;
  269.  
  270.    if gud = false then
  271.    begin
  272.      Working_Data.Free;
  273.      Exit(False);
  274.    end
  275.    else
  276.    begin
  277.      //mA to A
  278.      if solarI < 0 then solarI := 0;
  279.      solarI := solarI/100;
  280.      ElecForm.SolarIval.Caption  := FloatToStrF(solarI, ffFixed,3,2) + 'A';
  281.      ElecForm.SolarBar.Progress := 2 * round(solarI);
  282.    end;
  283.     {StatusList.Strings[19] := ls + 'Load = ' + ElecForm.LBIValue.Caption
  284.         + ', Mchg = ???A, ' + 'Schg = ' + ElecForm.SolarIval.Caption + le;}
  285.    Result:=True;
  286.  end;
  287. end;
  288.  
« Last Edit: March 11, 2018, 01:51:44 pm by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

Thaddy

  • Hero Member
  • *****
  • Posts: 7182
Re: Hopeless Debugging
« Reply #20 on: March 11, 2018, 10:38:25 am »
@BLL
If now you know have a range check error, you are in luck because range checks are really easy to debug and can also be caught at compile time if you define the expected range instead of an integer type.
Can you show us the line where that happens (compile with -glh and {$R+} and the types used on that line?
My first entry in the wiki about defensive programming addresses that issue more or less. http://wiki.freepascal.org/Defensive_programming_techniques

(Next week there will be much more, but the range check code and text stays the same because it has been scrutinized by our peers)
« Last Edit: March 11, 2018, 10:45:15 am by Thaddy »
inline variables like in D10.3 are a bit like Brexit: if you are given the wrong information it sounds like a good idea. Every kid loves candy, but it makes you fat and your teeth will disappear.

BLL

  • Full Member
  • ***
  • Posts: 231
Re: Hopeless Debugging
« Reply #21 on: March 11, 2018, 03:55:43 pm »
Hi all,
Thanks for the help. Great minds must think alike. I am now sure that the problem is a bad string. To this end, I have written a function not unlike one of the replies. It takes the string and the function either returns a real, if the string is valid, -1 for an empty string and any non numeric characters are deleted. I wrote it as a little test program and gave i all sorts of numbers, characters, spaces etc and it performed fine. I have this morning replaced all the Val routines in doElecState with calls to this test function. The program has so far been running for a few hours with no error dialogue.

An interesting spinoff of this is that some of the data comes via Modbus/RS485 from my solar charger. Previously, it took up to 6 read requests to get back valid data. Now it works perfectly each and every tiime. There is a problem with the PIC, which I have not been able to track down, despite many hours of trying: My program sends a character to the PIC. An interrupt is generated whenever characters arrive at the PIC and the PIC then sends back the string of data. Problem is that sometimes it returns nothing or only part of the data. I have tried various baud rates to no avail, so until I can track that problem down, I have to check the received data. I guess that a checksum might well be the way to go. Thanks Josh & Engkin for your ideas and code which I will go and study.

I really do appreciate the help I am getting - thank you all very much.

Brian

josh

  • Hero Member
  • *****
  • Posts: 651
Re: Hopeless Debugging
« Reply #22 on: March 11, 2018, 07:57:58 pm »
Hi

Just knocked up a quick project that show an alternative method, whether its usefull I do not know.

i have created a record that has all the values and wether they are valid.

You populate an string array, call one routine, it then validates each one, removes any non numerical characters first, and popuilate the record.

You can then check ie if Current_Data.Voltage.Valid then voltage_guage.progress:=Current_data.voltage.value;

It may be of use. Just an idea.

Attached project
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

avra

  • Hero Member
  • *****
  • Posts: 1377
    • Additional info
Re: Hopeless Debugging
« Reply #23 on: March 12, 2018, 09:45:00 am »
I tried in vain to use pascal serial routines. sdposerial was completely unreliable, often seizing up ports, which is why I went to calling C progs instead.
I am using synaser for years with various loggers, custom made AVR and ARM devices, with MODBUS and other protocols over real and USB COM ports. You should really try it instead of external C progs, because it is rock solid. You could also take a look at some threads and follow wiki links mentioned there:
https://forum.lazarus.freepascal.org/index.php/topic,33423.msg266332.html#msg266332
https://forum.lazarus.freepascal.org/index.php/topic,36885.msg246260.html#msg246260
https://forum.lazarus.freepascal.org/index.php/topic,36523.msg245030.html#msg245030
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Thaddy

  • Hero Member
  • *****
  • Posts: 7182
Re: Hopeless Debugging
« Reply #24 on: March 12, 2018, 10:32:12 am »
@Avra
@BLL
That's good advice, Avra. I would recommend the same. It is rock solid. (It is also maintained, but doesn't need frequent updates so people think synapse/synaser is not maintained)

Anyway: last status was range error: that is easy to debug.
inline variables like in D10.3 are a bit like Brexit: if you are given the wrong information it sounds like a good idea. Every kid loves candy, but it makes you fat and your teeth will disappear.

BLL

  • Full Member
  • ***
  • Posts: 231
Re: Hopeless Debugging
« Reply #25 on: March 12, 2018, 10:50:00 am »
Hi
Thanks again for the replies. I tried synaser and it was hopeless - perhaps it doesn't like the RasPi? Have you used it on a Raspi3 with jessie?
I have now got some of the program running from within lazarus, which has already thrown up some errors. However, one page of my program displays weather data from a weather station. There is a command line utility called te923con which reads data from the weather station and displays it to screen as a colon delimited string. Problem is that it requires sudo or it returns a configuration error and a result of -1. Now, as I am running my program as ordinary user pi, I don't get the weather data. Is there a way of calling te923con from within my program but as an elevated user and if so, how please? If this problem can be overcome then I can debug the whole program from within lazarus.

Brian

avra

  • Hero Member
  • *****
  • Posts: 1377
    • Additional info
Re: Hopeless Debugging
« Reply #26 on: March 12, 2018, 11:30:40 am »
I tried synaser and it was hopeless - perhaps it doesn't like the RasPi?
You didn't follow my links. First link directly mentions RPi, second link mentions it indirectly. So yes, synapse runs fine on RPi. I am sorry to hear that you have such a different experience. If you follow the links you will find simple examples, wiki articles, most common mistakes and educational discussion. Use it.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Thaddy

  • Hero Member
  • *****
  • Posts: 7182
Re: Hopeless Debugging
« Reply #27 on: March 12, 2018, 11:48:03 am »
@BLL
I answer from a Raspberry Pi (can be zero, 1,2 or 3) and any examples I gave for Synapse were written on a Raspberry Pi 3, if necessary only checked against Windows 10......
If you still have problems provide code to reproduce it. I can check it and put you on the right track. Basically the RPi's are issue-free with Synapse/Synaser..
« Last Edit: March 12, 2018, 11:52:43 am by Thaddy »
inline variables like in D10.3 are a bit like Brexit: if you are given the wrong information it sounds like a good idea. Every kid loves candy, but it makes you fat and your teeth will disappear.

Bram71

  • New member
  • *
  • Posts: 14
Re: Hopeless Debugging
« Reply #28 on: March 12, 2018, 12:13:02 pm »
In order for a "normal" user to use the serial port under Linux you have to add the user to the group "dailout".

Like the command below:

sudo adduser rpi dialout

After that you no longer need to run your program with sudo

BLL

  • Full Member
  • ***
  • Posts: 231
Re: Hopeless Debugging
« Reply #29 on: March 12, 2018, 12:49:06 pm »
When I do sudo adduser pi dialout, it says that pi is already a member!

Brian