Recent

Author Topic: Wiki stub: Make your own compiler, interpreter, parser, or expression analyzer  (Read 20217 times)

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
I wanted to make a starting point for anyone interested in doing it with FreePascal and Lazarus:

http://wiki.freepascal.org/Make_your_own_compiler,_interpreter,_parser,_or_expression_analyzer

Lex/Yacc and Gold are covered. Help is needed for AntLR, Coco-R or any other tool not on the list.

Feel free to update, expand and correct.

Have fun!    :D 8-) :D
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Edson

  • Hero Member
  • *****
  • Posts: 1296
Interesting. Maybe I can include information about SynFacilSyn wich is a highlighter and lexer, I have used in my compilers.
Lazarus 2.2.6 - FPC 3.2.2 - x86_64-win64 on Windows 10

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Interesting. Maybe I can include information about SynFacilSyn wich is a highlighter and lexer, I have used in my compilers.
Please do. I would appreciate if you follow the current layout: you add short info about SynFacilSyn on the main page, then if needed you add a separate page with all the SynFacilSyn details. Please let me know if you have a better proposal then this.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Thank you marcov for adding fcl-passrc, fcl-xml, symbolic, and TFPExpressionParser to the wiki stub. I believe that those are specialized, not general tools. For example first is specialized for pascal sources, and second for XML files. Therefore maybe we should make 2 categories: General Tools, and Specialized Tools. If anyone has a better idea, please say so.
« Last Edit: September 15, 2018, 05:25:52 pm by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Edson

  • Hero Member
  • *****
  • Posts: 1296
Updated with SynFacilSyn information.  :)
Lazarus 2.2.6 - FPC 3.2.2 - x86_64-win64 on Windows 10

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Updated with SynFacilSyn information.
Thank you!
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Just couldn't resist adding to wiki  :-[
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11353
  • FPC developer.
Thank you marcov for adding fcl-passrc, fcl-xml, symbolic, and TFPExpressionParser to the wiki stub. I believe that those are specialized, not general tools. For example first is specialized for pascal sources, and second for XML files. Therefore maybe we should make 2 categories: General Tools, and Specialized Tools. If anyone has a better idea, please say so.

Parsing is only part of making a compiler. And LALR parsers only a part of that. Most commercial compilers have recursive descent parsers afaik.
And as said fcl-passrc is being used for a pascal-to-js compiler.

Maybe the title is simply wrong for what you had in mind.  Make a topic "parser generators" or so, move the parser generators there, and then link that on the "compiler creation" page?
« Last Edit: September 19, 2018, 09:23:22 pm by marcov »

Leledumbo

  • Hero Member
  • *****
  • Posts: 8744
  • Programming + Glam Metal + Tae Kwon Do = Me
You make me wanna finish my handcoded compiler tutorial, as what's currently covered in the wiki are all generated by tools instead of coded from scratch by hand (except the fpc packages, of course, those are nice handwritten ones).

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Parsing is only part of making a compiler. And LALR parsers only a part of that.
I am aware of that, but such parsers can be used for much more then just compilers....

Most commercial compilers have recursive descent parsers afaik.
Speed is probably the most important reason. Maybe wiki should mention that somewhere...

Maybe the title is simply wrong for what you had in mind.  Make a topic "parser generators" or so, move the parser generators there, and then link that on the "compiler creation" page?
I felt like such segregation would make too much distraction for already loose coupled wiki topics. That's why I wanted to make a single starting place, and that's why I called it a wiki stub. The idea was that when in the future anyone wants to know more about anything from the title we point him to this wiki as a first step. Anyway, I don't mind anyone changing a title or making any deeper reorganization. All improvements are welcome. Please give it any shape you think is better then current one. I started the wiki in the first place because I wanted to preserve and share some of my experience with GOLD Parser Builder. The rest was simply an extension to that.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
You make me wanna finish my handcoded compiler tutorial
If you do then please add it to the wiki. I will be looking forward to take a look at it. I started with hand coding everything. Then lex/yacc. Then some GUI parser generators. I ended up with GOLD, which spoilt me with it's code and grammar separation, visual and step by step grammar testing and ambiquities finder capabilities, and running it's nice FreePascal engines with parsed tree output. It's not a runtime speed champ and has it's quirks, but it automates most of my work and for what I need is more then enough.
« Last Edit: September 20, 2018, 11:04:26 am by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11353
  • FPC developer.
Speed is probably the most important reason.

Speed, but also backwards compatibility requirements with older versions can often more easily be handled in code then adapting the grammar which often leads to multiplication of rules for the related part.

Also error generation is said to be finer and more to the point (less syntax error, more naming the exact issue).

It is not that commercial compilers don't use grammars and generators. Just the flagship ones often do not.

I know a guy that did/does compiler work for a living (mostly delivering C compilers for new DSP parts with some extensions for the C), and they worked very topdown and grammar based, and proof generation etc.

But the whole feature set was signed off on before final implementation started, and rarely iterated.
« Last Edit: September 20, 2018, 11:08:40 am by marcov »

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
backwards compatibility requirements with older versions can often more easily be handled in code then adapting the grammar which often leads to multiplication of rules for the related part.
The more such shortcuts exist, the harder it is to publish complete BNF/EBNF grammar at the end. Many times you do have to put some things in code, but unfortunatelly sometimes that code is hard to translate back to formal grammar.

Btw, I have just updated the wiki with 'Let's build a compiler' PDF from your site.  ;)
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11353
  • FPC developer.
Btw, I have just updated the wiki with 'Let's build a compiler' PDF from your site.  ;)

I didn't do it myself, because my site goes dark october 1st :-)

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Btw, I have just updated the wiki with 'Let's build a compiler' PDF from your site.  ;)

I didn't do it myself, because my site goes dark october 1st :-)
I have just tried to make PDF attachment to wiki, but it is not allowed. Do you plan to preserve it on github/bitbucket alike repo, so I can link it from the wiki? I remember there were other nice PDFs on your site. It would be a shame to loose them.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

 

TinyPortal © 2005-2018