Run shell command and capture output in a TSringGrid

[Aruna]

Hello, if I want to run a shell command in Linux and capture the output and display it in a TSTrinGrid how would one proceed, please? Sadly this time when I asked chatGPT it did hallucinate about non-existent properties..  I have attached a screenshot. For example, if I want to capture the output of the following command how would I go about implementing this?

Code: Bash  [Select][+][-]

1. lsblk -S

And a slightly more tricky one below:

Code: Bash  [Select][+][-]

1. find . -name '*.?*' -type f | rev | cut -d. -f1 | rev  |  sort | uniq --count | sort -n

If you run this in the Lazarus Directory/Folder it will give you a total file type and count as shown below: (Try running it under the Linux source folder)

Code: Bash  [Select][+][-]

1. aruna@debian:~/lazarus3/lazarus$ time find . -name '*.?*' -type f | rev | cut -d. -f1 | rev  |  sort | uniq --count | sort -n
2.       1 ahk
3.       1 c
4.       1 db3
5.       1 dci
6.       1 desktop
7.       1 Dockerfile
8.       1 fbk
9.       1 gitattributes
10.       1 hfp
11.       1 hhc
12.       1 hhk
13.       1 hpr
14.       1 java
15.       1 kof
16.       1 kwd
17.       1 lpg
18.       1 manifest
19.       1 msg
20.       1 odp
21.       1 ods
22.       1 PNG
23.       1 RES
24.       1 rtf
25.       1 spec
26.       1 uml
27.       1 xls
28.       1 xsl
29.       1 yml
30.       2 dcr
31.       2 JPG
32.       2 mbf
33.       2 odg
34.       2 old
35.       2 patch
36.       2 pro
37.       2 RUS
38.       2 sql
39.       2 zip
40.       3 json
41.       3 lst
42.       3 odt
43.       3 overrides
44.       3 pl
45.       3 properties
46.       4 cfg
47.       4 Debian
48.       4 dfm
49.       4 iss
50.       4 pdf
51.       4 uni
52.       4 wav
53.       5 icns
54.       5 lps
55.       5 mdx
56.       5 or
57.       5 ttf
58.       5 xct
59.       6 plist
60.       6 TXT
61.       7 1
62.       7 sample
63.       8 css
64.       8 md
65.       8 template
66.       9 ini
67.       9 isl
68.      10 chm
69.      13 dbf
70.      13 ppm
71.      13 rc
72.      14 jpg
73.      15 lrs
74.      16 lrj
75.      17 gitignore
76.      24 html
77.      28 cur
78.      38 lrf
79.      39 svg
80.      54 xpm
81.      57 rsj
82.      73 bat
83.      75 dpr
84.      80 pot
85.      83 fpc
86.      85 sh
87.     101 bmp
88.     106 ico
89.     149 compiled
90.     150 lpl
91.     151 lpk
92.     205 ex-meta
93.     226 res
94.     297 txt
95.     315 xml
96.     334 gif
97.     383 lpr
98.     415 lpi
99.     619 cpp
100.     623 inc
101.     656 pp
102.     836 h
103.     882 po
104.    1273 lfm
105.    1647 ppu
106.    1655 o
107.    2751 pas
108.    6492 png
109.  
110. real    0m1.958s
111. user    0m0.122s
112. sys     0m0.139s
113.  

So, how do I get this into a StringGrid please?

[TRon]

A couple of remarks:
For testing purposes you can use runcommand which returns a string that contains the output of the command. Use the non-deprecated version.

If you require piping and all that then TProcess is not going to do that for you. It is a shell feature, so use your preferred shell as exename instead (which also mandates how the  arguments should be passed, see also your shell man page)

Once you have received the resulting output from running the command you can process that outputstring, for example by splitting the contents of that string by using f.e. a TStringList or use one of the string helper routines.

Once you got all data separated the way you want to then you can process that and add the data to the individual cells of the grid. see also wiki https://wiki.freepascal.org/Grids_Reference_Page

[TRon]

Ok, seems like you could use a bit of help. Keep in mind that I cheated a little.

Code: Pascal  [Select][+][-]

1. program test;
2.  
3. {$mode objfpc}{$h+}
4.  
5. uses
6.   classes, sysutils, process, fpjson, jsonparser;
7.  
8. // lsblk -S
9. procedure One;
10. var
11.   outstr: string;
12.   JSN  : TJSONData;
13.   Device     : TJSONObject;
14.   DeviceEnum : TJSONEnum;
15.   Devices    : TJSONData;
16.   idx  : integer;
17. begin
18.   if RunCommand('lsblk', ['-S', '-J'], outstr) then
19.   begin
20.     writeln('outstr:');
21.     writeln(outstr);
22.  
23.     JSN := GetJSON(outstr);
24.  
25.     Devices := JSN.FindPath('blockdevices');
26.     for DeviceEnum in Devices do
27.     begin
28.       Device := TJSONObject(DeviceEnum.Value);
29.  
30.       for idx := 0 to Device.Count-1 do
31.       begin
32.         writeln('fieldname  = ', Device.names[idx]);
33.         writeln('fieldvalue = ', Device[Device.names[idx]].AsString);
34.         // insert code to copy data to cell
35.       end;
36.     end;
37.     JSN.Free;
38.   end;
39. end;
40.  
41.  
42. // find . -name ''*.?*'' -type f | rev | cut -d. -f1 | rev  |  sort | uniq --count | sort -n
43. procedure two;
44. var
45.   outstr: string;
46.   lines: TStringList;
47.   line: string;
48.   col1: string;
49.   col2: string;
50. begin
51.   if RunCommand('bash', ['-c', 'find . -name ''*.?*'' -type f | rev | cut -d. -f1 | rev  |  sort | uniq --count | sort -n'], outstr) then
52.   begin
53.     writeln('outstr:');
54.     writeln(outstr);
55.     lines := TStringList.Create;
56.     lines.text := outstr;
57.     for line in lines do
58.     begin
59.       col1 := line.SubString(1, 7).Trim;
60.       col2 := line.SubString(8).Trim;
61.       writeln('col1 = ', col1, '   col2 = ', col2);
62.       // insert code to copy data to cell
63.     end;
64.     lines.Free;
65.   end;
66. end;
67.  
68. begin
69.   One;
70.   Two;
71. end.
72.  

Use that what works best for you.

[Aruna]

My apologies for the delayed response TRon, I appreciate your patience. We went to Brockville to see the tunnel (you should visit this place it is amazing) and we stopped at Thousand Islands to go on a boat cruise. So my excuse is that I was not home  . And at 64 years my vision is not what it used to be when it comes to typing on cell phone screen keyboards. I very much prefer my old desktop at home. I live in Toronto btw.. so that was a 337km ride to get there. Took us almost 4 hours just to get there. This is irrelevant to the forum but I am posting a picture of the Brockville Tunnel. You have to see it to believe it. If folks feel this is not relevant to pascal I will be happy to take the pictures down.

A couple of remarks:
For testing purposes you can use runcommand which returns a string that contains the output of the command. Use the non-deprecated version.

Thank you.

If you require piping and all that then TProcess is not going to do that for you. It is a shell feature, so use your preferred shell as exename instead (which also mandates how the  arguments should be passed, see also your shell man page)

Yes I want the piping.

Once you have received the resulting output from running the command you can process that outputstring, for example by splitting the contents of that string by using f.e. a TStringList or use one of the string helper routines.

Understood and thank you again. How easy (or difficult) is loading a TStringGrid from a TStringList?

Once you got all data separated the way you want to then you can process that and add the data to the individual cells of the grid. see also wiki https://wiki.freepascal.org/Grids_Reference_Page

I will go through and see if I can understand how to go about doing this. Thank you once again for all the help and support.

[Aruna]

Ok, seems like you could use a bit of help. Keep in mind that I cheated a little.

Umm.. I would not say that is cheating but finding a way to get the job done? I had no idea of the '-J'  flag in lsblk. I tried it and almost fell off the chair when I saw the output. NIce!

Code: Bash  [Select][+][-]

1. aruna@debian:~/lazarus3/lazarus$ lsblk -J
2. {
3.    "blockdevices": [
4.       {"name":"sda", "maj:min":"8:0", "rm":false, "size":"232.9G", "ro":false, "type":"disk", "mountpoint":null,
5.          "children": [
6.             {"name":"sda1", "maj:min":"8:1", "rm":false, "size":"100M", "ro":false, "type":"part", "mountpoint":null},
7.             {"name":"sda2", "maj:min":"8:2", "rm":false, "size":"212.2G", "ro":false, "type":"part", "mountpoint":null},
8.             {"name":"sda3", "maj:min":"8:3", "rm":false, "size":"1K", "ro":false, "type":"part", "mountpoint":null},
9.             {"name":"sda5", "maj:min":"8:5", "rm":false, "size":"19.7G", "ro":false, "type":"part", "mountpoint":null},
10.             {"name":"sda6", "maj:min":"8:6", "rm":false, "size":"903M", "ro":false, "type":"part", "mountpoint":null}
11.          ]
12.       },
13.       {"name":"sdb", "maj:min":"8:16", "rm":false, "size":"931.5G", "ro":false, "type":"disk", "mountpoint":null,
14.          "children": [
15.             {"name":"sdb1", "maj:min":"8:17", "rm":false, "size":"250G", "ro":false, "type":"part", "mountpoint":null},
16.             {"name":"sdb2", "maj:min":"8:18", "rm":false, "size":"250G", "ro":false, "type":"part", "mountpoint":null},
17.             {"name":"sdb3", "maj:min":"8:19", "rm":false, "size":"16.8G", "ro":false, "type":"part", "mountpoint":null},
18.             {"name":"sdb4", "maj:min":"8:20", "rm":false, "size":"1K", "ro":false, "type":"part", "mountpoint":null},
19.             {"name":"sdb5", "maj:min":"8:21", "rm":false, "size":"204G", "ro":false, "type":"part", "mountpoint":null},
20.             {"name":"sdb6", "maj:min":"8:22", "rm":false, "size":"196.9G", "ro":false, "type":"part", "mountpoint":null},
21.             {"name":"sdb7", "maj:min":"8:23", "rm":false, "size":"6.2G", "ro":false, "type":"part", "mountpoint":null},
22.             {"name":"sdb8", "maj:min":"8:24", "rm":false, "size":"7.7G", "ro":false, "type":"part", "mountpoint":null}
23.          ]
24.       },
25.       {"name":"sdc", "maj:min":"8:32", "rm":false, "size":"465.8G", "ro":false, "type":"disk", "mountpoint":null,
26.          "children": [
27.             {"name":"sdc1", "maj:min":"8:33", "rm":false, "size":"1K", "ro":false, "type":"part", "mountpoint":null},
28.             {"name":"sdc2", "maj:min":"8:34", "rm":false, "size":"232.4G", "ro":false, "type":"part", "mountpoint":"/media/aruna/linux-next"},
29.             {"name":"sdc3", "maj:min":"8:35", "rm":false, "size":"232.4G", "ro":false, "type":"part", "mountpoint":"/"},
30.             {"name":"sdc5", "maj:min":"8:37", "rm":false, "size":"928M", "ro":false, "type":"part", "mountpoint":"[SWAP]"}
31.          ]
32.       },
33.       {"name":"sr0", "maj:min":"11:0", "rm":true, "size":"1024M", "ro":false, "type":"rom", "mountpoint":null}
34.    ]
35. }
36.  

Use that what works best for you.

Give me a little time to test your code then will get back to you. Many Thanks as always for your time and guidance.

[TRon]

My apologies for the delayed response TRon, I appreciate your patience.

No problem at all. Live usually carries on.

We went to Brockville to see the tunnel (you should visit this place it is amazing) and we stopped at Thousand Islands to go on a boat cruise. So my excuse is that I was not home 

That is time much better spend than come up with some boring code. I hope you enjoyed because that looks amazing.

almost fell off the chair when I saw the output. NIce!

Oh, your output seems a bit more Convoluted than what it did for me (simple life, single partitions  ).

My example code did not account for that so you might want to take that into consideration when trying my (simple) example. Processing JSON correctly can sometimes be a PITA.

[Aruna]

We went to Brockville to see the tunnel (you should visit this place it is amazing) and we stopped at Thousand Islands to go on a boat cruise. So my excuse is that I was not home 

That is time much better spend than come up with some boring code. I hope you enjoyed because that looks amazing.

Thank you.Yes we had a great time and yes it was a very pleasant surprise.

Oh, your output seems a bit more Convoluted than what it did for me (simple life, single partitions  ).

Ah well there is a story behind all those partitions. I will tell you someday  Remember I said I have multiple kernels on this system? I am attaching three screenshots that should be self-explanatory to someone with your experience.

My example code did not account for that so you might want to take that into consideration when trying my (simple) example. Processing JSON correctly can sometimes be a PITA.

Your code worked fine until I asked it to use only the -J flag then it had trouble. But I have managed to put together some code that does what I want. It does run the command and load the TStringGrid. But now we have a different problem. The delimiter I used is space ' ' and there are multiple spaces in the output. Please look at the bottom of the attached file lsblk_load_to_StringGrid.png and you will see what I mean. Any pointers on how to work around this mess is most welcome. The code is below:

Code: Pascal  [Select][+][-]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.     Classes, SysUtils, Forms, Controls, Grids, StdCtrls, Process,Dialogs;
9.  
10. type
11.  
12.   { TForm1 }
13.  
14.   TForm1 = class(TForm)
15.     Button1: TButton;
16.     StringGrid1: TStringGrid;
17.     procedure Button1Click(Sender: TObject);
18. //    procedure LoadLSBLKToGrid();
19.   private
20.  
21.   public
22.  
23.   end;
24.  
25. var
26.   Form1: TForm1;
27.  
28. implementation
29.  
30. {$R *.lfm}
31.  
32. { TForm1 }
33.  
34. procedure LoadLSBLKToGrid;
35. var
36.   Process: TProcess;
37.   Output: TStringList;
38.   i: Integer;
39.   j:Integer;
40.   Line: string;
41.   Fields: TStringList;
42.   HeaderFields: TStringList;
43. begin
44.   // Create and configure the TProcess
45.   Process := TProcess.Create(nil);
46.   Output := TStringList.Create;
47.   Fields := TStringList.Create;
48.   HeaderFields := TStringList.Create;
49.   try
50.     Process.Executable := 'lsblk';
51.     Process.Parameters.Add('-S');
52.     Process.Options := [poUsePipes, poStderrToOutPut];
53.     Process.Execute;
54.  
55.     // Read the output
56.     Output.LoadFromStream(Process.Output);
57.  
58.     // Ensure we have at least one line (header)
59.     if Output.Count < 1 then
60.     begin
61.       ShowMessage('No output from lsblk command');
62.       Exit;
63.     end;
64.  
65.     // Get the header line and determine the number of columns
66.     HeaderFields.Delimiter := ' ';
67.     HeaderFields.StrictDelimiter := True;
68.     HeaderFields.DelimitedText := Output[0];
69.  
70.     // Setup TStringGrid
71.     Form1.StringGrid1.ColCount := HeaderFields.Count;
72.     Form1.StringGrid1.RowCount := Output.Count; // Number of rows
73.  
74.     writeln('row.count :'+inttostr(Headerfields.Count));
75.     writeln('row.count :'+inttostr(Output.Count));
76.  
77.  
78.     // Set headers for the TStringGrid
79.     for i := 0 to HeaderFields.Count - 1 do
80.       Form1.StringGrid1.Cells[i, 0] := HeaderFields[i];
81.  
82.     // Parse the output and fill the TStringGrid
83.     for i := 1 to Output.Count - 1 do
84.     begin
85.       Line := Output[i];
86.       Fields.Clear;
87.       Fields.Delimiter := ' ';
88.       Fields.StrictDelimiter := True;
89.       Fields.DelimitedText := Line;
90.  
91.       // Fill the grid
92.       if True then   //Fields.Count = HeaderFields.Count then
93.       begin
94.         for j := 0 to Fields.Count - 1 do
95.           Form1.StringGrid1.Cells[j, i] := Fields[j];
96.       end;
97.     end;
98.   finally
99.     Process.Free;
100.     Output.Free;
101.     Fields.Free;
102.     HeaderFields.Free;
103.   end;
104. end;
105.  
106.  
107.  
108. procedure TForm1.Button1Click(Sender: TObject);
109. begin
110.   LoadLSBLKToGrid;
111. end;
112.  
113. end.
114.  

It is not allowing me to upload seems there is a limit of 500kb so am gonna post again with the relevant screenshots. Apologies for this mess.

[Aruna]

First screenshot is 'some' of the kernels on this system ( not all ) and second screen shot is teh Lazarus IDE giving me a hard time with loading the TStringGrid with teh output of lsblk- S.

[TRon]

Ah well there is a story behind all those partitions. I will tell you someday  Remember I said I have multiple kernels on this system? I am attaching three screenshots that should be self-explanatory to someone with your experience.

* TRon pretending to have all wisdom and experience in the world

But I do get the gist.

Your code worked fine until I asked it to use only the -J flag then it had trouble. But I have managed to put together some code that does what I want. It does run the command and load the TStringGrid. But now we have a different problem. The delimiter I used is space ' ' and there are multiple spaces in the output. Please look at the bottom of the attached file lsblk_load_to_StringGrid.png and you will see what I mean. Any pointers on how to work around this mess is most welcome. The code is below:

Well, eh ... the reason I cheated with JSON is the fact that it is nearly impossible to parse the output from lsblk -S correctly. JSON conveniently encapsulates all (field) output between quotes so that it is possible to correctly determine which spaces actually belong to the field and which ones don't.

Besides that, lsblk also allows for additional columns/fields or a custom list of columns/fields to be shown (in any order and column size you want) so processing the column-widths in a hard-coded manner is also not really an option.

So, it might be possible using stringlists to get this to work with f.e. for your particular setup but making the code work in a universal manner would be much harder to do.

Please give that some (additional) thought before investing time for any solution/approach.

It is not allowing me to upload seems there is a limit of 500kb so am gonna post again with the relevant screenshots. Apologies for this mess.

No worries. Things are indeed a bit limited (and for good reasons). Usually I scale down and reduce the colors of pictures before attaching or you can use an external image hosting service.

[Aruna]

Ah well there is a story behind all those partitions. I will tell you someday  Remember I said I have multiple kernels on this system? I am attaching three screenshots that should be self-explanatory to someone with your experience.

* TRon pretending to have all wisdom and experience in the world

But I do get the gist.

Well way back sometime in 2009 I finally got tired of the blue screen(s) of death in Windows. I was running a 'pirated' copy but that was no justifiable reason to keep trashing my system. When I started to lose my data I called it quits and looked around for alternative options.

Ubuntu Karmic had just been released and I decided to go with it. I had no idea what Linux was or what it could do but it was FREE!

My system back then had a single hard disk and after installing Ubuntu Karmic everything stabilized and I was happy. Then came the learning curve. With multiple crashes due to my experimenting. Over the years I plugged in a second hard disk and everything was great.

Then one day I had updated my system to Debian and while learning and trying things on re-boot I had no grub. I was lost and had no idea how to proceed or resolve it. So I ran to the computer shop and they had a 1TB on sale so purchased that. My system has two HD bays I did not have space for a third one. I bought a usb dock station plugged in the 1TB and installed Debian onto it. On grub-update it found all the other operating systems I had and pulled them all in. On reboot, I was a happy cookie I had all my old systems back. Now if I pull the 1TB the system falls back to the old grub configuration. Which is good.

But I have some files/programs/apps that I have over the years soft linked to the 1TB. So if I try to access them when the 1TB is puled things get messy. So after considerable consideration, it was decided to leave things the way they are. It works, why mess with it? But now I am stuck with those partitions and anyone who feels they can fix things is most welcome to try 

Your code worked fine until I asked it to use only the -J flag then it had trouble. But I have managed to put together some code that does what I want. It does run the command and load the TStringGrid. But now we have a different problem. The delimiter I used is space ' ' and there are multiple spaces in the output. Please look at the bottom of the attached file lsblk_load_to_StringGrid.png and you will see what I mean. Any pointers on how to work around this mess is most welcome. The code is below:

Well, eh ... the reason I cheated with JSON is the fact that it is nearly impossible to parse the output from lsblk -S correctly. JSON conveniently encapsulates all (field) output between quotes so that it is possible to correctly determine which spaces actually belong to the field and which ones don't.

You know I did wonder why you had decided to go with JSON? I had to learn the hard way that it is near impossible to parse lsbk output without jumping through hoops and loops. Too much darn work. I should have listened to you the first time. But then I learned stuff along the way so it is all good.

Besides that, lsblk also allows for additional columns/fields or a custom list of columns/fields to be shown (in any order and column size you want) so processing the column-widths in a hard-coded manner is also not really an option.

I do try to never hard code anything.

So, it might be possible using stringlists to get this to work with f.e. for your particular setup but making the code work in a universal manner would be much harder to do.

I am not going to try anymore it is just too time consuming. Maybe one day I will come back to this 

Please give that some (additional) thought before investing time for any solution/approach.

Fully understood and will comply 100%.

It is not allowing me to upload seems there is a limit of 500kb so am gonna post again with the relevant screenshots. Apologies for this mess.

No worries. Things are indeed a bit limited (and for good reasons). Usually I scale down and reduce the colors of pictures before attaching or you can use an external image hosting service.

Ah, of course I will keep that in mind for any future picture uploads. Thank you.

[TRon]

Well way back sometime in 2009 I finally got tired of the blue screen(s) of death in Windows.
....

Sorry for cutting your story short but thank you very much for the very entertaining and recognizable write-up. It brings back memories and a lot of recognition.

You know I did wonder why you had decided to go with JSON? I had to learn the hard way that it is near impossible to parse lsbk output without jumping through hoops and loops. Too much darn work. I should have listened to you the first time. But then I learned stuff along the way so it is all good.

Well, I did not literally wrote it down but you are clearly smart to pick up on the hints

You could use hard coded string lengths when the column widths are big enough to be able to contain every possible answer (I just do not know for sure what that number (w/sh)ould be).

But I agree to not go that route because if there is any chance of being able to retrieve a better structured output then I also prefer to use that instead.