Lazarus

Programming => General => Topic started by: TonnyB on September 11, 2017, 06:49:47 pm

Title: [TRegExpr] How to get all uses library's from pascal file ?
Post by: TonnyB on September 11, 2017, 06:49:47 pm
Hello there. I try to get all uses library from pascal file ?

My regular expression:

Code: Pascal  [Select][+][-]
  1. uses[\W+]{0,}([0-9A-Za-z_]{0,})[,\s]{0,};

h**ps://regex101.com/r/rfQtQg/1


But if there is several library then my regular expression not works.

h**ps://regex101.com/r/rfQtQg/4


Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Leledumbo on September 11, 2017, 09:22:17 pm
Use the syntax diagram (https://www.freepascal.org/docs-html/ref/refse105.html) as your base.
Your regex:
Code: [Select]
uses[\W+]{0,}([0-9A-Za-z_]{0,})[,\s]{0,};
has too many errors. Written in a more common notation (WSN / EBNF, implied whitespace), the grammar would be:
Code: [Select]
uses-clause = "uses" uses-ident { "," uses-ident } ";" .
uses-ident = identifier [ "in" pascal-string-literal ] .
You can simplify uses-ident to identifier if the former is too difficult to recognize and you're sure none will appear in the project you scan uses clause for.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: marcov on September 11, 2017, 09:34:17 pm
There might be preprocessor directives in there too (and there will be, in most professional codebases, if only to cope with FPC/Delphi versioning and OS dependence)

Regex are too weak to parse programming languages.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Thaddy on September 11, 2017, 09:57:58 pm
Indeed. Parsing used units (including those mentioned in includes) is like parsing a language.
Since regular expressions are not Turing complete this is not even possible.

There are tools, however, that will resolve each and every unit used, based on a Pascal parser.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: ASerge on September 11, 2017, 10:05:44 pm
Do not forget about //uses..., {uses...} and (*uses...*) and the possibility of placing on several lines with the comments between them.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Wosi on September 11, 2017, 10:16:45 pm
As others have mentioned before: Don't try to get any useful information from source code using regular expressions. You will faill.

DelphiAST  (https://github.com/RomanYankovsky/DelphiAST)is a really great parser as long as your files don't contain FPC related keywords. It will do a much better job than any regular expression.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: marcov on September 11, 2017, 10:24:05 pm
FPC comes with a similar parser, fcl-passrc. It has improved enormously in the past year.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Thaddy on September 11, 2017, 10:28:12 pm
Quote from: marcov on September 11, 2017, 10:24:05 pm
FPC comes with a similar parser, fcl-passrc. It has improved enormously in the past year.
Yes, but requires trunk? It will however parse everything and in all fpc modes. at least regarding the units. Something the other tools mentioned can't do at all.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: marcov on September 11, 2017, 10:30:26 pm
Quote from: Thaddy on September 11, 2017, 10:28:12 pm
Yes, but requires trunk?

Did you test fixes?
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Jurassic Pork on September 12, 2017, 12:54:21 am
hello,
Quote from: Thaddy on September 11, 2017, 10:28:12 pm
Yes, but requires trunk?
the package fcl-passrc is there in my Lazarus release 1.8RC3. To see all the units used in a file you can use the unit passsrcutil from this package.
Exemple to  read the units used in a file :
Code: Pascal  [Select][+][-]
  1. implementation
  2. uses passrcutil;
  3. {$R *.lfm}
  4. { TForm1 }
  5. procedure TForm1.Button1Click(Sender: TObject);
  6.  var filename: string;
  7.      PasAnalyser: TPasSrcAnalysis;
  8.      PasList : TStrings;
  9. begin
  10. if OpenDialog1.Execute then
  11.    begin
  12.    filename := OpenDialog1.Filename;
  13.    end
  14.    else exit;
  15.     PasAnalyser:=TPasSrcAnalysis.Create(Nil);
  16.     PasList:=TStringList.Create;
  17.     PasAnalyser.FileName:=filename;
  18.     Memo1.Clear;
  19.     Memo1.Append('File : ' + filename);
  20.     Memo1.Append('======  Interface Units =======');
  21.     PasAnalyser.GetInterfaceUnits(PasList);
  22.     Memo1.Append(PasList.Text);
  23.     Memo1.Append('======  Implementation Units =======');
  24.     PasList.Clear;
  25.     PasAnalyser.GetImplementationUnits(PasList);
  26.     Memo1.Append(PasList.Text);
  27.     FreeAndNil(PasAnalyser);
  28.     FreeAndNil(PasList);
  29. end;
  30. end.

Friendly, J.P
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: Thaddy on September 12, 2017, 09:17:09 am
Quote from: Jurassic Pork on September 12, 2017, 12:54:21 am
hello,
Quote from: Thaddy on September 11, 2017, 10:28:12 pm
Yes, but requires trunk?
the package fcl-passrc is there in my Lazarus release 1.8RC3. To see all the units used in a file you can use the unit passsrcutil from this package.

I was referring to the current implementation and improvements that Marco was referring to.
You are right that even the older implementations can parse the "used units" correctly, but the trunk implementation is now very close to what the compiler parses.
And that is only in trunk.
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: marcov on September 12, 2017, 01:30:48 pm
Quote from: Thaddy on September 12, 2017, 09:17:09 am
And that is only in trunk.

Everything up to a few weeks before the RC1 release was merged to trunk. IOW the bulk is available in fixes and 3.0.4rc1
Title: Re: [TRegExpr] How to get all uses library's from pascal file ?
Post by: soerensen3 on September 12, 2017, 03:20:29 pm
Another option would be to use CodeTools, which is also used by the CodeExplorer of the IDE.

Don't forget to add the CodeTools package to the project dependencies!
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   {$IFDEF UNIX}{$IFDEF UseCThreads}
  7.   cthreads,
  8.   {$ENDIF}{$ENDIF}
  9.   Classes,
  10.   sysutils,
  11.   CodeToolManager,
  12.   CodeTree, CodeCache
  13.   { you can add units after this };
  14.  
  15. var
  16.   Code: TCodeBuffer;
  17.   Tool: TCodeTool;
  18.   i: Integer;
  19.  
  20.   procedure ExtractUsesSectionFromFile( AFileName: String );
  21.   var
  22.     Main, Impl: TStrings;
  23.     j: Integer;
  24.   begin
  25.     Code:= CodeToolBoss.LoadFile( ExpandFileName( AFileName ), False, False );
  26.     if ( not Assigned( Code )) then
  27.       raise Exception.Create( 'Could not load the unit file: ' + AFileName );
  28.     Tool:= nil;
  29.     if ( not CodeToolBoss.Explore( Code, Tool, False, True )) then
  30.       raise Exception.Create( 'The code of the unit file could not be parsed as it contains errors: ' + AFileName );
  31.  
  32.     if ( not Assigned( Tool )) then
  33.       Tool:= TCodeTool( CodeToolBoss.GetCodeToolForSource( Code, False, False ));
  34.  
  35.     Tool.FindUsedUnitFiles( Main, Impl );
  36.     WriteLn( 'Used units of ', AFileName );
  37.     WriteLn( 'Used units in uses section:' );
  38.  
  39.     for j:= 0 to Main.Count - 1 do
  40.       WriteLn( Main[ j ]);
  41.     //WriteLn( Main.Text );
  42.     WriteLn();
  43.     WriteLn( 'Used units in implementation section:' );
  44.     WriteLn( Impl.Text );
  45.     Main.Free;
  46.     Impl.Free;
  47.   end;
  48.  
  49. begin
  50.   if ( ParamCount > 0 ) then
  51.     for i:= 1 to ParamCount do
  52.       ExtractUsesSectionFromFile( ParamStr( i ))
  53.   else
  54.     WriteLn( 'Error: no files passed as parameter' );
  55. end.
  56.  
It requires that there are no syntax errors in the file. However CodeTools is a little less strict than the compiler. Please note that you do not need to free instances of TCodeBuffer or TCodeTool as they are managed by CodeTools itself. If you use it with units that use includes not in the same directory, this method will fail. That is because the package or project file is not parsed by default.
You can add an include directory manually like this:

Code: Pascal  [Select][+][-]
  1. procedure AddIncludePath( Directory: String );
  2.   var
  3.     IncPathTemplate: TDefineTemplate;
  4.   begin
  5.     Directory:= ExpandFileName( Directory );
  6.  
  7.     // add a sub template to extend the include search path #IncPath.
  8.     IncPathTemplate:= TDefineTemplate.Create(
  9.       '', // optional: the name of the template, useful for finding it later
  10.       '', // optional: a description
  11.       IncludePathMacroName,
  12.       IncludePathMacro + ';' + Directory
  13.       ,da_DefineRecurse
  14.       );
  15.     // add the include path template to the tree
  16.     CodeToolBoss.DefineTree.Add( IncPathTemplate );
  17.   end;
TinyPortal © 2005-2018