Recent

Author Topic: FindAllFiles not working on Ubuntu Folders  (Read 6469 times)

KarenT

  • Full Member
  • ***
  • Posts: 120
FindAllFiles not working on Ubuntu Folders
« on: July 22, 2017, 03:37:03 pm »
Hello,

Just when I thought FindAllFiles was a joy to use, I tried to gather files in a Linux-Hidden Folder. It returns zero files.

/home/karen/.config

Is there a workaround for "dot" files and folders?

Not sure if I need first to do something to make them visible to Lazarus/FPC, or whether the "dot" is irritating enough to FindAllFiles for it to ignore my request.

{edit}
It is definitely the "dot" causing an issue.

I found a Peter Below Folder scan function and it returns the Files, but no folders within, but it also struggles with file-names with no Extension.

Code: Pascal  [Select][+][-]
  1. function ScanDrive(aPath, aMask: String; aList: TStringList): Boolean;
  2.  
  3.   function ScanDirectory(var Path: String ): Boolean;
  4.     Var
  5.       SRec: TSearchRec;
  6.       pathlen: Integer;
  7.       res: Integer;
  8.     Begin
  9.       pathlen:= Length(path);
  10.       { first pass, files }
  11.       res := FindFirst( path+aMask, faAnyfile, SRec );
  12.       If res = 0 Then
  13.       try
  14.         while res = 0 do
  15.         begin
  16.           aList.Add( path + SRec.Name );
  17.           res := FindNext(SRec);
  18.           Application.ProcessMessages;
  19.         end;
  20.       finally
  21.         FindClose(SRec)
  22.       end;
  23.       Result := not (FileScanAborted or Application.Terminated);
  24.       If not Result Then Exit;
  25.  
  26.       {second pass, directories}
  27.       res := FindFirst( path+'*.*', faDirectory, SRec );
  28.       If res = 0 Then
  29.       try
  30.         While (res = 0) and Result Do Begin
  31.           If ((Srec.Attr and faDirectory) = faDirectory) and
  32.              (Srec.name <> '.') And
  33.              (Srec.name <> '..')
  34.            Then Begin
  35.             path := path + SRec.name + '\';
  36.             Result := ScanDirectory( path );
  37.             SetLength( path, pathlen );
  38.           End;
  39.           res := FindNext(SRec);
  40.         End;
  41.       finally
  42.         FindClose(SRec)
  43.       end;
  44.     End;
  45. begin
  46.   FileScanAborted:=False;
  47.   Screen.Cursor:=crHourglass;
  48.   try
  49.     Result:=ScanDirectory(aPath);
  50.   finally
  51.     Screen.Cursor:=crDefault
  52.   end;
  53. end;
  54.  
« Last Edit: July 22, 2017, 03:57:15 pm by KarenT »

Xor-el

  • Sr. Member
  • ****
  • Posts: 404
Re: FindAllFiles not working on Ubuntu Folders
« Reply #1 on: July 22, 2017, 04:14:25 pm »
although I have not used this personally, Ted Smith (Gizmo) uses this unit in his awesome hashing program (quickhash) and it seems to work well.

Gizmo

  • Hero Member
  • *****
  • Posts: 831
Re: FindAllFiles not working on Ubuntu Folders
« Reply #2 on: July 22, 2017, 05:20:34 pm »
You have to modify the unit slighty. I created an "enhanced" version of it (with the help of Engkin and Bart) which is part of QuickHash (https://github.com/tedsmith/quickhash/blob/master/FindAllFilesEnhanced.pas)

and then you can use it as follows for Linux to find hidden data (as seen here https://github.com/tedsmith/quickhash/blob/master/unit2.pas#L1745) :

Code: [Select]
TotalFilesToExamine := FindAllFilesEx(Foldername, '*', False, True);

Thanks Xor-el for your kind words :-) Base64 decoding is coming soon, and implemented into the latest master commit.
« Last Edit: July 22, 2017, 05:24:06 pm by Gizmo »

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #3 on: July 22, 2017, 05:54:32 pm »
You have to modify the unit slighty. I created an "enhanced" version of it (with the help of Engkin and Bart) which is part of QuickHash (https://github.com/tedsmith/quickhash/blob/master/FindAllFilesEnhanced.pas)

Thank you Xor-el and Gizmo. I added in FindAllFilesEx and it works better, but, not sure yet what is missing as my test list for .config shows 160 Files+Folders, but using Nautilus and Properties for .config it shows 282 items.

Looks like 122 Folders and/or Files have gone missing.

I thought I would jump back here in case I am doing something wrong before starting to list and count everything. I tried

  slFiles:=FindAllFilesEx('/home/karen/.config/','*',True,True);            <-- returned 160
and
  slFiles:=FindAllFilesEx('/home/karen/.config/','*.*',True,True);        <-- retrned 134

Code: Pascal  [Select][+][-]
  1. implementation
  2.  
  3. {$R *.lfm}
  4.  
  5. { TForm1 }
  6. uses
  7.     LazUTF8Classes, FindAllFilesEnhanced;
  8. var
  9.   slFiles : TStringListUTF8;
  10.  
  11. procedure TForm1.Button1Click(Sender: TObject);
  12. begin
  13.   slFiles:=TStringListUTF8.Create;
  14.   slFiles:=FindAllFilesEx('/home/karen/.config/','*',True,True);
  15.   lb1.Items.Assign(slFiles);
  16.   slFiles.Free;
  17.   lbl1.Caption:=IntToStr(lb1.Items.Count);
  18. end;
  19.  

« Last Edit: July 22, 2017, 05:56:43 pm by KarenT »

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4458
  • I like bugs.
Re: FindAllFiles not working on Ubuntu Folders
« Reply #4 on: July 22, 2017, 05:57:43 pm »
You have to modify the unit slighty. I created an "enhanced" version of it (with the help of Engkin and Bart) which is part of QuickHash (https://github.com/tedsmith/quickhash/blob/master/FindAllFilesEnhanced.pas)
Your function has parameter IncludeHiddenDirs but it is not used. How can it work?
Code: Pascal  [Select][+][-]
  1. function FindAllFilesEx(const SearchPath: string; SearchMask: string;
  2.   SearchSubDirs: boolean; IncludeHiddenDirs: boolean): TStringListUTF8;
  3. var
  4.   Searcher: TListFileSearcher;
  5. begin
  6.   Result   := TStringListUTF8.Create;
  7.   Searcher := TListFileSearcher.Create(Result);
  8.   Searcher.DirectoryAttribute := Searcher.DirectoryAttribute or faHidden;
  9.   try
  10.     Searcher.Search(SearchPath, SearchMask, SearchSubDirs);
  11.   finally
  12.     Searcher.Free;
  13.   end;
  14. end;
The FindAllFiles found in LazUtils / FileUtil has parameter "DirAttr: Word" which supports all attributes including faHidden.
It should work without copying and modifying the code. ... or maybe I don't fully understand the issue.

BTW, you can now use a Delphi compatible TStringList instead of TStringListUTF8 without worries.
« Last Edit: July 22, 2017, 06:03:22 pm by JuhaManninen »
Mostly Lazarus trunk and FPC 3.2 on Manjaro Linux 64-bit.

wp

  • Hero Member
  • *****
  • Posts: 11830
Re: FindAllFiles not working on Ubuntu Folders
« Reply #5 on: July 22, 2017, 05:58:37 pm »
You have to modify the unit slighty. I created an "enhanced" version of it (with the help of Engkin and Bart) which is part of QuickHash (https://github.com/tedsmith/quickhash/blob/master/FindAllFilesEnhanced.pas)

Code: [Select]
TotalFilesToExamine := FindAllFilesEx(Foldername, '*', False, True);

I copied this from the specified links:
Code: Pascal  [Select][+][-]
  1. function FindAllFilesEx(const SearchPath: string; SearchMask: string; SearchSubDirs: boolean; IncludeHiddenDirs: boolean): TStringListUTF8;  
  2. var
  3.   Searcher: TListFileSearcher;
  4. begin
  5.   Result := TStringListUTF8.Create;
  6.   Searcher := TListFileSearcher.Create(Result);
  7.   Searcher.DirectoryAttribute := Searcher.DirectoryAttribute or faHidden;
  8.   try
  9.     Searcher.Search(SearchPath, SearchMask, SearchSubDirs);
  10.   finally
  11.     Searcher.Free;
  12.   end;
  13. end;

Mabe I am missing something, but the parameter IncludeHiddenDirs does not seem to be used anywhere. The Searcher.Attribute is set as if IncludeHiddenDirs were true. So, either remove the IncludeHiddenDirs as a parameter or apply the "or faHidden" only in case of "if IncludeHiddenDirs"

Lazarus' FindAllFiles now has an optional DirAttr parameter as well. Therefore FindAllFilesEx can be emulated by or-ing the default DirAttr faDirectory with faHidden.

[EDIT]
Ah, Juha was faster...
« Last Edit: July 22, 2017, 06:00:50 pm by wp »

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #6 on: July 22, 2017, 06:07:47 pm »
Me too. Just coming back to say, the "True" as the last Parameter makes no difference if it is "False."

wp

  • Hero Member
  • *****
  • Posts: 11830
Re: FindAllFiles not working on Ubuntu Folders
« Reply #7 on: July 22, 2017, 06:11:46 pm »
Looks like 122 Folders and/or Files have gone missing.
Folders and files are probably missing because you set the parameter SearchSubDirs to false.

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #8 on: July 22, 2017, 06:14:37 pm »
Folders and files are probably missing because you set the parameter SearchSubDirs to false.

Thanks but I had both Boolean set to True all the time.
« Last Edit: July 22, 2017, 06:54:11 pm by JuhaManninen »

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #9 on: July 22, 2017, 06:14:52 pm »
BTW, you can now use a Delphi compatible TStringList instead of TStringListUTF8 without worries.

:), thanks, I was using the Delphi style, but only tried the UTF8 version just in case and just pasted my latest code here. But, good to know.

Still cannot get it to work for more than 160 but will give the Laz-version another try with faHidden.

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #10 on: July 22, 2017, 06:24:19 pm »
The Help says "faSysFile" is for Linux but I can't get it to work any better with the Laz-FindAllFiles

  FindAllFiles(slFiles,'/home/karen/.config/','*.*',True,faAnyFile or faSysFile or faHidden);
or
  FindAllFiles(slFiles,'/home/karen/.config/','*.*',True,faAnyFile or faSysFile);
or
  FindAllFiles(slFiles,'/home/karen/.config/','*.*',True,faAnyFile or faHidden);

Am I going wrong with that? It returns only 134 items.

[EDIT]
Ooops, it looks like 160, might be correct for FindAllFilesEx, I just realized that Nautilus is probably counting Folders too. DUH!! Not sure there are 122 Folders, but will play around in a smaller .folder.

« Last Edit: July 22, 2017, 06:33:49 pm by KarenT »

wp

  • Hero Member
  • *****
  • Posts: 11830
Re: FindAllFiles not working on Ubuntu Folders
« Reply #11 on: July 22, 2017, 06:35:00 pm »
Your file mask is '*.*', and this means, in my eyes, that there must be a period in the filename. Try a simple '*'.

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #12 on: July 22, 2017, 11:38:11 pm »
Thank you, it is all working. I have no idea why Nautilus counted 122 extra things, but there are only 37 Folders in the entire .config Tree. But I counted all the files and 160 is the correct number.

Please note that the laz-FindAllFiles does **not** return the correct number of files no matter how I set the Attributes, but the version modified by Gizmo, "FindAllFilesEx(..." **does** return all the files.

Than you Gizmo.

wp

  • Hero Member
  • *****
  • Posts: 11830
Re: FindAllFiles not working on Ubuntu Folders
« Reply #13 on: July 23, 2017, 12:45:07 am »
Please note that the laz-FindAllFiles does **not** return the correct number of files no matter how I set the Attributes, but the version modified by Gizmo, "FindAllFilesEx(..." **does** return all the files.
I don't believe this because Gizmo's version is practically identical with the Lazarus version if DirAttr is applied correctly. Did you check the search mask? In Gizmos's code you had used '*' while your post for the Laz version showed a '*.*'. If the search mask contains a period (.) then files without a period are not counted.
« Last Edit: July 23, 2017, 12:58:27 am by wp »

KarenT

  • Full Member
  • ***
  • Posts: 120
Re: FindAllFiles not working on Ubuntu Folders
« Reply #14 on: July 23, 2017, 09:28:42 pm »
I don't believe this because Gizmo's version is practically identical with the Lazarus version

Seeing is believing.

AND, "practically," is not actually. :)

 

TinyPortal © 2005-2018