directory search for files containing a search string

[tjpren]

Hello,

Last week I was very happy that my program to find files on a computer worked.  I do a directory search with file mask.

This week, I'm trying to expand this by looking for a string in each file (basically like the windows explorer search function).

I get an error in runtime with the FileContents.LoadFromFile(Path + Rec.Name).

During compile, I get a warning that FileContents does not seem to be initialised.

Any thoughts?

Code: [Select]

procedure TfrMain.FileSearch(const PathName, FileName : string; const InDir : boolean);
var Rec  : TSearchRec;
    Path : string;
    FileContents: TStringlist;
    Texttofind: string;

begin
Path := IncludeTrailingBackslash(PathName);
Texttofind :='abc';

if FindFirst(Path + FileName, faAnyFile - faDirectory, Rec) = 0 then
 try
   repeat
   FileContents.LoadFromFile(Path + Rec.Name);
   if Pos(Texttofind, FileContents.Text) <> 0 then
     ListBox1.Items.Add(Path + Rec.Name);
   until FindNext(Rec) <> 0;
 finally
   FindClose(Rec);
 end;

If not InDir then Exit;

if FindFirst(Path + '*.*', faDirectory, Rec) = 0 then
 try
   repeat
    if ((Rec.Attr and faDirectory) <> 0)  and (Rec.Name<>'.') and (Rec.Name<>'..') then
     FileSearch(Path + Rec.Name, FileName, True);
   until FindNext(Rec) <> 0;
 finally
   FindClose(Rec);
 end;
end; //procedure FileSearch   

[Bart]

You must first create an instance of FileContents:

Code: [Select]

begin
  ...
  FileContents = TStringList.Create;
  try
    //Do stuff with FileContents
    ...
  Finally
    FileContents.Free;
  end;
end;

Bart

[tjpren]

Bart,

Many thanks for the help.  Adding .create allowed FileContents to be initialised.

My search works pretty well.  If I give it a string, and tell it to search texts, generally it works OK, except when it can't open files.

Typically it throws an error "unable to open file C:\windows\SchedLgu.txt"

I think this is where I need to look at some error handling.

Is there anyway to have it ignore the error and continue?  Or if it is unable to open, to ignore?

Regards

[Bart]

Generally you can do something like this:

Code: [Select]

begin
  ...
  FileContents := TStringList.Create;
  try //try..finally block
    try //try..except block
      FileContents.LoadFromFile(SomeFileName);
      //process contents of the file
    except
      on E: EStreamError do  //Something went wrong during loading
      begin
        //show an error message perhaps: ShowMessage('Error: ' + E.Message);
      end;
    end;//try..except
  finally
    //putting this in try..finally ensures it will always be freed
    FileContents.Free;
  end;//try..finally
  ...
end;

As a side note: Loading text files into a StringList is OK, but when they are very big, it gets real slow, because it want to read the entire contents into memory.
So, if in the en your program must be able to search in files > 1 Mb for instance, this you might need another approach.

Bart

[tjpren]

Thanks Bart,

That helped - it's nearly useable.  I didn't know about the Except.  At least now when it encounters the locked file, or file that it can't read, it doesn't crash.

However, what seems to happen, is that the directory it is searching through finishes, and then the program stops.  It doesn't resume scanning all other directories.

I seem to need something that would just ignore files that it can't open.

Any thoughts?


 

Code: [Select]

procedure TfrMain.TxtSearch(const PathName, FileName : string; const InDir : boolean);
var
    Rec  : TSearchRec;
    Path : string;
    FileContents: TStringlist;
    Texttofind: string;
begin
Application.ProcessMessages;
Path := IncludeTrailingBackslash(PathName);
Texttofind :=edit3.text; //enter text to find
FileContents := TStringList.Create;
if FindFirst(Path + FileName, faAnyFile - faDirectory, Rec) = 0 then
 Try
   Try
    repeat
    FileContents.LoadFromFile(Path + Rec.Name);  //Load contents of file
    if Pos(Texttofind, FileContents.Text) <> 0 then //Look inside file for string
    ListBox1.Items.Add(Path + Rec.Name);
    until FindNext(Rec) <> 0;
   Except  //Error Handling
     on E: EStreamError do
     begin
     ShowMessage('Error: ' + E.Message);
     end;
   end;
 finally
   FindClose(Rec);
 end;

If not InDir then Exit;

if FindFirst(Path + '*.*', faDirectory, Rec) = 0 then
 try
   repeat
    if ((Rec.Attr and faDirectory) <> 0)  and (Rec.Name<>'.') and (Rec.Name<>'..') then
     TxtSearch(Path + Rec.Name, FileName, True);
   until FindNext(Rec) <> 0;
 finally
   FindClose(Rec);
 end;
end; //procedure FileSearch   

[theo]

Code: Text  [Select][+][-]

1.    Try
2.     repeat
3.     FileContents.LoadFromFile(Path + Rec.Name);  //Load contents of file
4.     if Pos(Texttofind, FileContents.Text) <> 0 then //Look inside file for string
5.     ListBox1.Items.Add(Path + Rec.Name);
6.     until FindNext(Rec) <> 0;
7.    Except  //Error Handling

Think again. How can this resume, and where?  Your loop is inside try...except.

[Bart]

There are several general solutions available for recursively iterating through directories.

Typically they are constructs like

Code: [Select]

procedure FindFiles(Path: String);
var
  SR: TSearchRec;
begin
  if FindFirst(Path, faAnyFile, SR) = 0 then
  repeat
    begin
      if ((SR.Attr and faDirectory) = faDirectory) then
      begin //it is a directory
        if (SR.Name <> '.') and (SR.Name <> '..') then FindFiles(Path+SR.Name)
      end
      else
      begin   //it is not a directory
        If not AllWentWellWhenIProcessedTheFile(Path+SR.Name) then HandleTheError();
      end;
    end;
  until FindNext(SR) <> 0;
  FindClose(SR);
end;

For better readability I would suggest you make the routine that handles the file contents a separate function or procedure.
B.t.w. I am missing a call to FileContents.Free in your code.

The code above uses a recursive algorithm (the procedure calls itself).
It can be re-written to not do that.

See my EnumDirs unit to see how this can be done.
This may be a bit over-complicated for your purpose, but is re-usable code.
Supply EnumDirs with a path, a maks and a callback method and you're good to go.

Bart

[tjpren]

Bart,

Thanks for the info - you've taken my breath away with your file (which I downloaded).  It appears to be very comprehensive.

I can see it will take me awhile to try and understand it, so I'll be off air  while I try and apply it.

Again thanks for the help, and I'll try and come back and let you know how I am going.

Regards

[Bart]

For using this unit you need the ExtMasks unit as well.

Bart

[theo]

There ist also TFileSearcher in FileUtil: http://lazarus-ccr.sourceforge.net/docs/lcl/fileutil/tfilesearcher.search.html

[tjpren]

Bart,

I've looked over your downloads, but unfortunatley i've got no idea where to start - I'm pretty novice at programming.

I've loaded it up in the Lazarus IDE, but when I run it, I can't see what is supposed to happen.

Are you able to give me a few hints.

I've also decide to look at my project, and create a separate procedure for the text comparison.  I'll do the file search in one procedure, and call the text comparison after each file matching the mask is found.

I think this is also what Theo was referring to.

Regards

[Bart]

Here's a snippet from my backupprogram:

Code: [Select]

type
  TBackupForm = class(TForm)
  ...
    DirEnumerator: TDirEnumerator;
    //Callback routine for EnumDir
    procedure HandleFiles(const FileName: string; const SR: TSearchRec; Out ErrorCode: LongInt);
  ...
    SearchOptions: TfsOptions;
  ...
  end;//TBackupForm

...

procedure TBackupFor.DoBackup;
begin
  DirEnumerator := TDirEnumerator.Create;
  DirEnumerator.FileHandler := @HandleFiles;
  SearchOptions := [soProcessDirs, soProcessMessages, soMaskIsCaseSensitive];
  {$IFDEF MSWINDOWS}
  SearchOptions := SearchOptions + [soMaskIsWindowsStyle];
  {$ENDIF}
  if Recursive then SearchOptions := SearchOptions + [soRecursive];
  DirEnumerator.Path := SourceRoot;
  DirEnumerator.MaskList := '*'; //Will do for Linux and Windows
  DirEnumerator.ExclMaskList := ExcludeFilter;
  DirEnumerator.Attribute := Attr;
  DirEnumerator.Options := SearchOptions;
  Try
    DirEnumerator.EnumDir;
  Except
    on E: EEnumDirError do
    begin
      //Handle errors
    end;
  end;
  ....
end;

DirEnumerator.EnumDir will find all files, recursive if specified in options, and pass them to the method specified in DirEnumerator.FileHandler (in this case TBackupForm.HandleFiles).
Here the actual work is done, and error are reported back to EnumDir via ErrorCode out paramter.

Bart

[tjpren]

Bart,

Thanks for your post - I'll have a go at using your code shortly.

Whilst waiting for your availability, I started to write a new procedure for my original code - actually a function. I've encountered my first hurdle. I've never actually written a function from the start - just modified someone elses.

From the body of my file search, when I find a file that ends in .txt, i want to jump to the function, so I say:
   SearchTxt(Path, Rec);

I'm going to pass the path and file as parameters to my SearchTxt function.

I create a function like:
   function TfrMain.SearchtTxt(const Path, Rec : String);

I declare that the parameters are strings.

I also declare it in the private section under Type as:
   function SearchtTxt(const Path, Rec : String);

However, when I compile, I get an error:
Syntax error, ":" expected but ";" found.

I think it's something to do with the way I write the function.

Any thoughts.

By the way, thanks for persevering - I'm learning heaps; but I realize that it is taking up your time.

Regards

[tjpren]

Hello,

i've completed my little project for for searching directories based on the extension type, and/or a search string.

Many thanks to those who helped.

I'd like to show my appreciation by posting my program  (as a zip file).

Is there any facility on the forum to post files - I tried before, but noticed there is a 250KB size limit.

Regards

[tjpren]

Sorry, Answered my own question - here's the code without the compiled executable.

It should run in the standard Lazarus IDE - there are no special packages.

Regards