Recent

Author Topic: "vintage" BASIC highlighter  (Read 6814 times)

Ladywasky

  • New Member
  • *
  • Posts: 20
"vintage" BASIC highlighter
« on: October 18, 2018, 04:56:55 pm »
It seems that was never done with SynEdit, I don't know why but I have my idea about it.

In short I am writing an code editor for Oric (ATMOS and Oric-1), and I really need an highliter for it's BASIC language.

Problem, I may be wrong but, since a BASIC program does not require whitespace, I am not sure that can be done with Synedit. For instance, this :
Code: [Select]
10 PRINT"HELLO":REM a comment
20 FOR I=1 TO 10:PRINT I:NEXT

can be written like this, without any whitespace :
Code: [Select]
10PRINT"HELLO":REM a comment
20 FORI=1TO10:PRINTI:NEXT

Do you think it's possible to highlight the code even if spaces are stripped out with synedit ?

Any help is much appreciated.

example of BASIC grammar (taken from Gold Builder) to have an idea of the challenge :
Code: [Select]
! -----------------------------------------------------------------------
! BASIC '64
!
! Beginner's All-purpose Symbolic Instruction Code
!
!    "It is practically impossible to teach good programming style to students
!     that have had prior exposure to BASIC; as potential programmers they are
!     mentally mutilated beyond hope of regeneration."
!
!     - Edsger W. Dijkstra
!
! BASIC is one of the oldest programming language and one of the most popular.
! It was developed in 1964 by John G. Kemeny and Thomas E. Kurtz to teach
! students the basics of programming concepts. At the advent of the microcomputer,
! BASIC was implemented on numerous platforms such as the Commodore, Atari,
! Apple II, Altair, IBM PC computers. Over time, BASIC evolved into GW-BASIC,
! QBasic, Visual Basic, and recently Visual Basic .NET.
!
! In practically all programming languages, the reserved word/symbol that denotes
! a comment is treated as a form of whitespace - having no effect in the manner in
! which the program runs. Once such type of comment is used to indicate the remainder
! of the line is to be ignored. These can be added to the end of any line without
! changing the meaning of the program. In C++, it is the '//' symbol;
! in BASIC '64 it is 'REM'.
!
! However, in the BASIC programming language, the line comment is treated like a
! statement. For instance, if 'REM' was a normal line comment:
!
!    10  PRINT "Hello World" REM Common first program
!
! would be a valid statement. However, in BASIC, this is illegal. In the example
! above, the comment must be added as an additional statement.
!
!    10  PRINT "Hello World" : REM Common first program
!
! As a result, the Line Comment terminal that is used in the GOLD Parser cannot be
! used here. In the grammar below, a 'Remark' terminal was created that accepts all
! series of printable characters starting with the characters "REM ". In some
! implementations of BASIC, any string starting with "REM" is a comment statement.
! Examples include "REMARK", "REMARKABLE" and "REMODEL". This grammar requires the space.
!
! This grammar does not include the editor statements such as NEW, SAVE, LOAD, etc...
!
! Note: This is an ad hoc version of the language. If there are any flaws, please
! e-mail GOLDParser@DevinCook.com and I will update the grammar. Most likely I have
! included features not available in BASIC '64.
! -----------------------------------------------------------------------


"Name"    = 'BASIC (Beginners All-purpose Symbolic Instruction Code)'
"Author"  = 'John G. Kemeny and Thomas E. Kurtz'
"Version" = '1964 - Original - before Microsoft enhanced the language for the IBM PC.'
"About"   = 'BASIC is one of most common and popular teaching languages ever created. '

"Case Sensitive" = False
"Start Symbol"   = <Lines>

{String Chars} = {Printable} - ["]
{WS}           = {Whitespace} - {CR} - {LF}

NewLine        = {CR}{LF}|{CR}
Whitespace     = {WS}+

Remark         = REM{Space}{Printable}*
ID             = {Letter}[$%]?
String         = '"'{String Chars}*'"'
Integer        = {digit}+
Real           = {digit}+.{digit}+

<Lines>       ::= Integer <Statements> NewLine <Lines>
                | Integer <Statements> NewLine

<Statements>  ::= <Statement> ':' <Statements>
                | <Statement>

<Statement>   ::= CLOSE '#' Integer
                | DATA <Constant List>
                | DIM ID '(' <Integer List> ')'
                | END         
                | FOR ID '=' <Expression> TO <Expression>     
                | FOR ID '=' <Expression> TO <Expression> STEP Integer     
                | GOTO <Expression>
                | GOSUB <Expression>
                | IF <Expression> THEN <Statement>         
                | INPUT <ID List>       
                | INPUT '#' Integer ',' <ID List>       
                | LET Id '=' <Expression>
                | NEXT <ID List>               
                | OPEN <Value> FOR <Access> AS '#' Integer
                | POKE <Value List>
                | PRINT <Print list>
                | PRINT '#' Integer ',' <Print List>
                | READ <ID List>           
                | RETURN
                | RESTORE
                | RUN
                | STOP
                | SYS <Value>
                | WAIT <Value List>
                | Remark

<Access>   ::= INPUT
             | OUPUT
                   
<ID List>  ::= ID ',' <ID List>
             | ID

<Value List>      ::= <Value> ',' <Value List>
                    | <Value>

<Constant List>   ::= <Constant> ',' <Constant List>
                    | <Constant>

<Integer List>    ::= Integer ',' <Integer List>
                    | Integer
                 
<Expression List> ::= <Expression> ',' <Expression List>
                    | <Expression>

<Print List>      ::= <Expression> ';' <Print List>
                    | <Expression>
                    | 

<Expression>  ::= <And Exp> OR <Expression>
                | <And Exp>

<And Exp>     ::= <Not Exp> AND <And Exp>
                | <Not Exp>
 
<Not Exp>     ::= NOT <Compare Exp>
                | <Compare Exp>

<Compare Exp> ::= <Add Exp> '='  <Compare Exp>
                | <Add Exp> '<>' <Compare Exp>
                | <Add Exp> '><' <Compare Exp>
                | <Add Exp> '>'  <Compare Exp>
                | <Add Exp> '>=' <Compare Exp>
                | <Add Exp> '<'  <Compare Exp>
                | <Add Exp> '<=' <Compare Exp>
                | <Add Exp>

<Add Exp>     ::= <Mult Exp> '+' <Add Exp>
                | <Mult Exp> '-' <Add Exp>
                | <Mult Exp>

<Mult Exp>    ::= <Negate Exp> '*' <Mult Exp>
                | <Negate Exp> '/' <Mult Exp>
                | <Negate Exp>

<Negate Exp>  ::= '-' <Power Exp>
                | <Power Exp>

<Power Exp>   ::= <Power Exp> '^' <Value>
                | <Value>

<Value>       ::= '(' <Expression> ')'
                | ID
                | ID '(' <Expression List> ')'
                | <Constant>

<Constant> ::= Integer
             | String
             | Real

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5573
    • wiki
Re: "vintage" BASIC highlighter
« Reply #1 on: October 18, 2018, 05:59:51 pm »
Sure, why not. You still can distinguish between a..z and others. Ever "FORI" if you detext the FOR.

There is a tutorial for writing your own HL: http://wiki.lazarus.freepascal.org/SynEdit_Highlighter
« Last Edit: October 18, 2018, 06:01:31 pm by Martin_fr »

Edson

  • Hero Member
  • *****
  • Posts: 1041
Re: "vintage" BASIC highlighter
« Reply #2 on: October 18, 2018, 07:26:09 pm »
You can use SynFacilSyn (https://github.com/t-edson/SynFacilSyn), with this syntax definition:

Code: XML  [Select]
  1. <?xml version="1.0"?>
  2. <Language >
  3.   <Identifiers CharsStart= "$" Content = "a..z"></Identifiers>
  4.   <Comment Start="REM" ></Comment>
  5.   <Symbols>
  6.     <Keyword> FOR PRINT REM TO NEXT </Keyword>
  7.   </Symbols>
  8. </Language>
  9.  

Maybe not perfect but do the work.
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Thaddy

  • Hero Member
  • *****
  • Posts: 8679
Re: "vintage" BASIC highlighter
« Reply #3 on: October 18, 2018, 07:34:56 pm »
Yes. But note, just like Basic or C, whitespace is not part of the language in most languages apart from e.g. Python.
So Synedit will work for BASIC I have a PowerBASIC(successor to TurboBASIC)  editor written using it, but I'll have to look it up in my archives. I rewrote it in PowerBasic..for fun.
(wasn't spoiled: I started with Fortran)
« Last Edit: October 18, 2018, 08:12:52 pm by Thaddy »
Most people that want to use threading should learn to patch their jeans first: use a needle.

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #4 on: October 18, 2018, 08:10:07 pm »
Thank you very much for your answers everybody,
At first, I think I will go for the SynFacil solution, then I will make my own highlighter is more is needed.

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #5 on: October 18, 2018, 09:45:33 pm »
Yes. But note, just like Basic or C, whitespace is not part of the language in most languages apart from e.g. Python.
So Synedit will work for BASIC I have a PowerBASIC(successor to TurboBASIC)  editor written using it, but I'll have to look it up in my archives. I rewrote it in PowerBasic..for fun.
(wasn't spoiled: I started with Fortran)
The example is bad, because PowerBASIC has nothing to do with old 8bit vintage BASIC :
Code: Pascal  [Select]
  1. PRINTAOR1
This, in powerBASIC, pascal or C is considered as a one variable.
In BASIC from 80's it is very different, it is a keyword, a variable, an operator and a number, it has to be treated like this :
Code: Pascal  [Select]
  1. PRINT A OR 1

More difficult to handle with syneditfacil :
A=1.3E4
B=2.75E-5


Edson

  • Hero Member
  • *****
  • Posts: 1041
Re: "vintage" BASIC highlighter
« Reply #6 on: October 18, 2018, 10:34:25 pm »
More difficult to handle with syneditfacil :
A=1.3E4
B=2.75E-5

In SynFacilSyn, to recognize numbers, you can add this definition to the XML:

Code: XML  [Select]
  1.  
  2. <Token regex = '[0-9][0-9]*[\.]?[0-9]*[eE][+-]?[0-9]+' Attribute='NUMBER'/>
  3.  

Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #7 on: October 18, 2018, 11:56:56 pm »
Edson, you are a genius !

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #8 on: October 19, 2018, 01:55:38 pm »
I did a huuuuuuuuuuuuuuuge progress but ! face up two problems SynFacil "environment" :
- completion do not work
- Line Numbers are treated like regular Numbers

Here id my BASIC.xml language definition :
Code: XML  [Select]
  1. <?xml version="1.0"?>
  2. <Language name="BASIC" ext="bas" CaseSensitive="TRUE">
  3.   <Completion>
  4.     <List>
  5.         "ABS|ABS(\_)"
  6.         AND
  7.         "ASC|ASC(\_)"
  8.         "ATN|ATN(\_)"
  9.         AUTO
  10.         CALL
  11.         "CHR$|CHR$(\_)"
  12.         CIRCLE
  13.         CLEAR
  14.         CLOAD
  15.         CLS
  16.         CONT
  17.         "COS|COS(\_)"
  18.         CSAVE
  19.         CURMOV
  20.         CURSET
  21.         DATA
  22.         "DEEK|DEEK(\_)"
  23.         DEF
  24.         DIM
  25.         DOKE
  26.         DRAW
  27.         EDIT
  28.         ELSE
  29.         END
  30.         "EXP|EXP(\_)"
  31.         EXPLODE
  32.         FALSE
  33.         FILL
  34.         FN
  35.         FOR
  36.         "FOR..TO|FOR \_=  TO "
  37.         "FOR..TO..STEP|FOR \_ =  TO  STEP "
  38.         FRE(0)|FRE(0)
  39.         "FRE&quot;&quot;|FRE&quot;&quot;"
  40.         GET
  41.         GOSUB
  42.         GOTO
  43.         GRAB|
  44.         HEX$|HEX$(\_)
  45.         HIMEM
  46.         HIRES
  47.         IF
  48.         INK
  49.         INPUT|
  50.         "INT|INT(\_)"
  51.         KEY$
  52.         LEFT$|LEFT$(\_,)
  53.         LEN|LEN(\_)
  54.         LET
  55.         "LET..=|LET \_="
  56.         LIST
  57.         LLIST
  58.         LN|LN(\_)
  59.         LOG|LOG(\_)
  60.         LORES
  61.         LPRINT|
  62.         "MID$|MID$(\_,,)"
  63.         MUSIC|
  64.         NEW
  65.         NEXT
  66.         NOT
  67.         ON
  68.         "ON..GOTO|ON \_ GOTO"
  69.         "ON..GOSUB|ON \_ GOSUB"
  70.         OR
  71.         PAPER
  72.         PATTERN
  73.         PEEK|PEEK(\_)
  74.         PI
  75.         PING
  76.         PLAY
  77.         PLOT
  78.         "POINT|POINT(\_,)"
  79.         POKE
  80.         POP
  81.         "POS|POS(\_)"
  82.         PRINT
  83.         PULL
  84.         READ
  85.         RECALL
  86.         RELEASE
  87.         REM
  88.         REPEAT
  89.         RESTORE
  90.         RETURN
  91.         "RIGHT$|RIGHT$(\_,)"
  92.         "RND|RND(\_)"
  93.         RUN
  94.         "SCRN|SCRN(\_,)"
  95.         "SGN|SGN(\_)"
  96.         SHOOT
  97.         SIN|SIN(\_)"
  98.         SOUND
  99.         "SPC|SPC(\_)"
  100.         "SQR|SQR(\_)"
  101.         STEP
  102.         STOP
  103.         STORE
  104.         "STR$|STR$(\_)"
  105.         "TAB|TAB(\_)"
  106.         "TAN|TAN(\_)"
  107.         TEXT
  108.         THEN
  109.         TO
  110.         TROFF
  111.         TRON
  112.         TRUE
  113.         UNTIL
  114.         USR
  115.         "USR()|USR(\_)"
  116.         "VAL|VAL(\_)"
  117.         WAIT
  118.         ZAP
  119.     </List>
  120.     <OpenOn AfterPattern='"DEF",space' FilterBy="None">
  121.         "FN|FN \_ ="
  122.         "USR|USR="
  123.     </OpenOn>
  124.   </Completion>
  125.   <Attribute Name="Keyword"   ForeCol="#00A4F2" Bold="True"  Italic="False" Underline="False"/>
  126.   <Attribute Name="Comment"   ForeCol="#8D8D8D" Bold="False" Italic="False" Underline="False"/>
  127.   <Attribute Name="Number"    ForeCol="#FF00FF" Bold="False" Italic="False" Underline="False"/>
  128.   <Attribute Name="LineNumber"    ForeCol="#000000" Bold="True" Italic="False" Underline="False"/>
  129.   <Attribute Name="Delimiter"    ForeCol="#000000" Bold="True" Italic="False" Underline="False"/>
  130.   <Attribute Name="Operator"    ForeCol="Blue" Bold="False" Italic="False" Underline="False"/>
  131.   <Attribute Name="Variable"    ForeCol="Teal" Bold="True" Italic="False" Underline="False"/>
  132.   <Identifiers CharsStart= "£" Content = "a..z"/>
  133.   <Token CharsStart="#" Content = '0..9ABCDEF' Attribute='Number'/>
  134.   <Token regex = '[0-9]+[\.]?[0-9]*[E][+-]?[0-9]+' Attribute='Number'/>
  135.   <String Start="&quot;" End="&quot;"> </String>
  136.   <Comment Start="REM" ></Comment>
  137.   <Comment Start="&apos;" ></Comment>
  138.   <Symbols>
  139.     <Keyword> END EDIT STORE RECALL TRON TROFF POP PLOT PULL LORES DOKE
  140.         REPEAT UNTIL FOR LLIST LPRINT NEXT DATA
  141.         INPUT DIM CLS READ LET GOTO RUN IF RESTORE GOSUB RETURN
  142.         REM HIMEM GRAB RELEASE TEXT HIRES SHOOT EXPLODE ZAP PING
  143.         SOUND MUSIC PLAY CURSET CURMOV DRAW CIRCLE PATTERN FILL
  144.         CHAR PAPER INK STOP ON WAIT CLOAD CSAVE DEF POKE PRINT
  145.         CONT LIST CLEAR GET CALL ! NEW TAB TO FN SPC @
  146.         AUTO ELSE THEN NOT STEP
  147.         SGN INT ABS USR FRE POS HEX$ &amp; SQR RND LN EXP COS
  148.         SIN TAN ATN PEEK DEEK LOG LEN STR$ VAL ASC CHR$ PI
  149.         TRUE FALSE KEY$ SCRN POINT LEFT$ RIGHT$ MID$</Keyword>
  150.     <Operator>
  151.       + - * / ^ AND OR &gt; = &lt; ( )
  152.     </Operator>
  153.     <Delimiter>
  154.       :
  155.     </Delimiter>
  156.     <LineNumber TokPos="1">
  157.        <Token regex='[0-9]+'/>
  158.     </LineNumber>
  159.         <Variable>
  160.                 <Token regex='[A-Z][A-Z0-9]*[$%]?'/>
  161.         </Variable>
  162.   </Symbols>
  163.  
  164. </Language>

Attached, program sample :

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #9 on: October 19, 2018, 05:40:51 pm »
Well, this is an updated xml for the BASIC grammar file of synfacil :
Code: XML  [Select]
  1. <?xml version="1.0"?>
  2. <Language name="BASIC" ext="bas" CaseSensitive="true">
  3.   <Completion CaseSensitive="false">
  4.        "ABS|ABS(\_)"
  5.         AND
  6.         "ASC|ASC(\_)"
  7.         "ATN|ATN(\_)"
  8.         AUTO
  9.         CALL
  10.         "CHR$|CHR$(\_)"
  11.         CIRCLE
  12.         CLEAR
  13.         CLOAD
  14.         CLS
  15.         CONT
  16.         "COS|COS(\_)"
  17.         CSAVE
  18.         CURMOV
  19.         CURSET
  20.         DATA
  21.         "DEEK|DEEK(\_)"
  22.         DEF
  23.         DIM
  24.         DOKE
  25.         DRAW
  26.         EDIT
  27.         ELSE
  28.         END
  29.         "EXP|EXP(\_)"
  30.         EXPLODE
  31.         FALSE
  32.         FILL
  33.         FN
  34.         FOR
  35.         "FOR..TO|FOR \_=  TO "
  36.         "FOR..TO..STEP|FOR \_ =  TO  STEP "
  37.         FRE(0)|FRE(0)
  38.         "FRE&quot;&quot;|FRE&quot;&quot;"
  39.         GET
  40.         GOSUB
  41.         GOTO
  42.         GRAB|
  43.         HEX$|HEX$(\_)
  44.         HIMEM
  45.         HIRES
  46.         IF
  47.         INK
  48.         INPUT|
  49.         "INT|INT(\_)"
  50.         KEY$
  51.         LEFT$|LEFT$(\_,)
  52.         LEN|LEN(\_)
  53.         LET
  54.         "LET..=|LET \_="
  55.         LIST
  56.         LLIST
  57.         LN|LN(\_)
  58.         LOG|LOG(\_)
  59.         LORES
  60.         LPRINT|
  61.         "MID$|MID$(\_,,)"
  62.         MUSIC|
  63.         NEW
  64.         NEXT
  65.         NOT
  66.         ON
  67.         "ON..GOTO|ON \_ GOTO"
  68.         "ON..GOSUB|ON \_ GOSUB"
  69.         OR
  70.         PAPER
  71.         PATTERN
  72.         PEEK|PEEK(\_)
  73.         PI
  74.         PING
  75.         PLAY
  76.         PLOT
  77.         "POINT|POINT(\_,)"
  78.         POKE
  79.         POP
  80.         "POS|POS(\_)"
  81.         PRINT
  82.         PULL
  83.         READ
  84.         RECALL
  85.         RELEASE
  86.         REM
  87.         REPEAT
  88.         RESTORE
  89.         RETURN
  90.         "RIGHT$|RIGHT$(\_,)"
  91.         "RND|RND(\_)"
  92.         RUN
  93.         "SCRN|SCRN(\_,)"
  94.         "SGN|SGN(\_)"
  95.         SHOOT
  96.         SIN|SIN(\_)"
  97.         SOUND
  98.         "SPC|SPC(\_)"
  99.         "SQR|SQR(\_)"
  100.         STEP
  101.         STOP
  102.         STORE
  103.         "STR$|STR$(\_)"
  104.         "TAB|TAB(\_)"
  105.         "TAN|TAN(\_)"
  106.         TEXT
  107.         THEN
  108.         TO
  109.         TROFF
  110.         TRON
  111.         TRUE
  112.         UNTIL
  113.         USR
  114.         "USR()|USR(\_)"
  115.         "VAL|VAL(\_)"
  116.         WAIT
  117.         ZAP
  118.   </Completion>
  119.   <Attribute Name="Keyword"   ForeCol="#00A4F2" Bold="True"  Italic="False" Underline="False"/>
  120.   <Attribute Name="Comment"   ForeCol="#8D8D8D" Bold="False" Italic="False" Underline="False"/>
  121.   <Attribute Name="Number"    ForeCol="#FF00FF" Bold="False" Italic="False" Underline="False"/>
  122.   <Attribute Name="LineNumber"    ForeCol="#000000" Bold="True" Italic="False" Underline="False"/>
  123.   <Attribute Name="Delimiter"    ForeCol="#000000" Bold="True" Italic="False" Underline="False"/>
  124.   <Attribute Name="Operator"    ForeCol="Blue" Bold="False" Italic="False" Underline="False"/>
  125.   <Attribute Name="Variable"    ForeCol="Teal" Bold="True" Italic="False" Underline="False"/>
  126.   <Identifiers CharsStart= "A..Z" Content = "A..Z1-9$%">
  127.      <Keyword> END EDIT STORE RECALL TRON TROFF POP PLOT PULL LORES DOKE
  128.         REPEAT UNTIL FOR LLIST LPRINT NEXT DATA
  129.         INPUT DIM CLS READ LET GOTO RUN IF RESTORE GOSUB RETURN
  130.         REM HIMEM GRAB RELEASE TEXT HIRES SHOOT EXPLODE ZAP PING
  131.         SOUND MUSIC PLAY CURSET CURMOV DRAW CIRCLE PATTERN FILL
  132.         CHAR PAPER INK STOP ON WAIT CLOAD CSAVE DEF POKE PRINT
  133.         CONT LIST CLEAR GET CALL NEW TAB TO FN SPC @
  134.         AUTO ELSE THEN NOT STEP
  135.         SGN INT ABS USR FRE POS HEX$ SQR RND LN EXP COS
  136.         SIN TAN ATN PEEK DEEK LOG LEN STR$ VAL ASC CHR$ PI
  137.         TRUE FALSE KEY$ SCRN POINT LEFT$ RIGHT$ MID$</Keyword>
  138.         <Variable>
  139.                 <Token regex='[A-Z][A-Z0-9]*[$%]?'/>
  140.         </Variable>
  141.   </Identifiers>
  142.   <Token CharsStart="#" Content = '0..9ABCDEF' Attribute='Number'/>
  143.   <Token regex = '[0-9]+[\.]?[0-9]*[E][+-]?[0-9]+' Attribute='Number'/>
  144.   <String Start="&quot;" End="&quot;"> </String>
  145.   <Comment Start="REM" ></Comment>
  146.   <Comment Start="&apos;" ></Comment>
  147.   <Symbols>
  148.    
  149.     <Operator>
  150.       + - * / ^ AND OR &gt; = &lt; ( )
  151.     </Operator>
  152.     <LineNumber TokPos="1">
  153.        <Token regex='[0-9]+'/>
  154.     </LineNumber>
  155.         <keyword>
  156.           ! &amp;
  157.         </keyword>
  158.         <Keyword> END EDIT STORE RECALL TRON TROFF POP PLOT PULL LORES DOKE
  159.         REPEAT UNTIL FOR LLIST LPRINT NEXT DATA
  160.         INPUT DIM CLS READ LET GOTO RUN IF RESTORE GOSUB RETURN
  161.         REM HIMEM GRAB RELEASE TEXT HIRES SHOOT EXPLODE ZAP PING
  162.         SOUND MUSIC PLAY CURSET CURMOV DRAW CIRCLE PATTERN FILL
  163.         CHAR PAPER INK STOP ON WAIT CLOAD CSAVE DEF POKE PRINT
  164.         CONT LIST CLEAR GET CALL NEW TAB TO FN SPC @
  165.         AUTO ELSE THEN NOT STEP
  166.         SGN INT ABS USR FRE POS HEX$ SQR RND LN EXP COS
  167.         SIN TAN ATN PEEK DEEK LOG LEN STR$ VAL ASC CHR$ PI
  168.         TRUE FALSE KEY$ SCRN POINT LEFT$ RIGHT$ MID$</Keyword>
  169.   </Symbols>  
  170. </Language>

If I remove the keywords from the symbols part, completion is working, but spaces are needed for highlighting completely a line, written without any spaces kills the highlight:
Code: [Select]
10IFA=3THENGOSUB100...otherelse, completion is not working.

I have tried to figure it out by creating my own Highlighter, deriving the TSynCustomHighlighter class :
Same limitations.

May be I am doing something wrong, may be Synedit or Synfacil are not the good candidates for Old school programming languages.


Pleaaaaaaaaaaase, help !
:/
« Last Edit: October 19, 2018, 05:45:21 pm by Ladywasky »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5573
    • wiki
Re: "vintage" BASIC highlighter
« Reply #10 on: October 19, 2018, 05:56:13 pm »
Completion in TSynEdit is independent of the Highlighter. So you have to write that yourself.

Especially if you need "10PRI" to complete to "10PRINT"

Subclassing the HL should work. Show what you got.

Basically you need to parse in the same way, as the basic interpreter does.
So in "procedure THighLighter.Next" you do not look for spaces.
- You start by flagging that you are at the start of the line.
- Skip spaces before the line number
- get any digits as line number
- now set the flag to "after line number", so further calls to next know that the line number is gone
- now go one char at a time. test for each position if
  - it is a keyword
  - a number
  - a symbol
  something else....

-------------------
Another approach (may be slower, uses more mem) is to scan the entire line at once when "next" is called the first time (or in setline).
You can then store the results in an array, that for each pos of the line stores the TokenKind and lenght of that token.
That way you can write one procedure that scans the entire line at once, which may be easier to write, as you do not need state-keeping for re-entrance of "Next"

Edson

  • Hero Member
  • *****
  • Posts: 1041
Re: "vintage" BASIC highlighter
« Reply #11 on: October 19, 2018, 06:55:46 pm »
- completion do not work
I see you're using SynFacilCompletion. Good. It's a SynFacilSyn with completion.
The syntax file is not common because of the particular token definitions. The keywords are not defined as identifiers but as symbols. If you want to implement a completion you could use the Openning Event:

Code: XML  [Select]
  1.   <OpenOn AfterPattern="Symbol">
  2. END EDIT STORE RECALL TRON TROFF POP PLOT PULL LORES DOKE
  3.   </OpenOn>
  4.  

Completion events can be controlled by code too. Then you can make a smarter completion.

- Line Numbers are treated like regular Numbers

Numbers are defined as tokens by content and in this kind of tokens, there is not a way to specify a position. There is parameter "TokPos" but is not accessible here.
One solution could be modify the highlighter if this is so crucial. The method is

Code: Pascal  [Select]
  1. procedure TSynFacilSyn.Next;
  2.   ...
  3.   if fRange = nil then begin
  4.       charIni:=fLine[posFin]; //guardar para tenerlo disponible en el método que se va a llamar.
  5.       fProcTable[charIni];    //Se ejecuta la función que corresponda.
  6.       if posTok = 1 then fTokenID := tnComment;   //<----- THIS MAKE ALL FIRST TOKEN ARE COMMENT
  7.   end else begin
  8.     if posFin = tamLin then begin  //para acelerar la exploración
  9.   ...
  10. end;
  11.  
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #12 on: October 19, 2018, 08:33:59 pm »
Thank you Martin, I better understand now how it is working.

Edson, definetly, you are really impressive.

About the usage of OpenOn, I did not think about the  AfterPattern="Symbol", in fact I had impletented it for someting else in my completion list, understood it, but I removed it for testing purpose. Just, I did not have the idea to use Symbol as a pattern.

Code: Pascal  [Select]
  1. <OpenOn AfterPattern='"DEF",space' FilterBy="None">
  2.         "FN|FN \_ ="
  3.         "USR|USR="
  4. </OpenOn>

For the Line numbers,
I even tried to define eachLine as a block with sections, blocks beginning with a number, sections ending with a column (":"), but I was oblied to finish my lines with a column to be coherent with the idea.
Definetly again, many thanks, I take this option.

Once finished I will post my xml here so that you could use it as an example for your SynFacil, it will be the first generic highlighter for fpc/Delphi for vintage BASIC : easy to adapt for many other micro-computers : ZX Spectrum, Amstrad CPC, Atari 2600, MSX, GWBasic, etc...

After that, it will be easier to do, there will be an xml for C and XA (6502 assembler).

To tell you, I develop for Oric 1/Atmos with a cross platform environment (OSDK : http://www.defence-force.org/ )

The OSDK is dos/windows set of tools using a lot bat files, and our IDE is Note-pad or NotePad++.
There is a necessity there to modernize all of this and make it cross-platform.
Coming from Delphi, lazarus was the good choice to make things working for Linux, MacOS, etc... too
The scripting/plugin part is important and lua is definetly my choice, with xml for configuration files.

In short, the compiler used is RCC16, an Oric targeted version of the LCC Compiler.
This compiler is decently ANSI compliant, and event support C++ comments as well as standard C comments. And the assembler is XA, it supports both the standard 6502 opcodes as well as the CMOS versions (Rockwell 65c02).

Here is my Github with a few of my projects if you wan't to have a look to what can be done with OSDK/Delphi/Lazarus for this Oric Atmos :
https://github.com/DJChloe?tab=repositories

I noticed that you are involved in 6502 programming too, more over I was surprised to discover that you develop a whole compiler for the Pascal language. How fast or compact produced by your pascal ?
Do you think it can be adapted to Oric's world ?

Thank you very much indeed to both of you. I learned a lot of things in a few days.

Edson

  • Hero Member
  • *****
  • Posts: 1041
Re: "vintage" BASIC highlighter
« Reply #13 on: October 19, 2018, 10:34:35 pm »
Hi. I'm glad to hear SynFacilCompletion is useful for you.

Here is my Github with a few of my projects if you wan't to have a look to what can be done with OSDK/Delphi/Lazarus for this Oric Atmos :
https://github.com/DJChloe?tab=repositories

:O I didn't know there is people still developing for the Oric-1

I noticed that you are involved in 6502 programming too, more over I was surprised to discover that you develop a whole compiler for the Pascal language.

Yes. I'm working in a 6502 compiler/IDE/debugger: https://github.com/t-edson/P65Pas
This a work in progress. Just some basic functionalities implemented. It's a lot of work and I must work in my free time. The assembler support is very good but the Pascal implementation is very poor by now.

How fast or compact produced by your pascal ?
Do you think it can be adapted to Oric's world ?
Well. I always include optimization to my compilers :) (check my PIC compiler https://github.com/t-edson/PicPas) but I haven't had time to work in it for the 6502.

My project is a 6502 compiler. Not specific for a particular system but I frequently use the Commodore 64 to some tests. The idea is to do the compiler the most general possible.
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Ladywasky

  • New Member
  • *
  • Posts: 20
Re: "vintage" BASIC highlighter
« Reply #14 on: October 20, 2018, 12:41:47 am »
OK ! I achieved what I needed !!!

I modified the Next method but differently, since it could break other languages I will use in my EDI.
I did something transparent that will activate itself just by creating a new attribute for the Language.
(FirstGen="true", wich is false by default).
This attribute means that we are dealing with a first generation interpreter dealing with Line numbers (Unstructured BASIC such like Dartmouth BASIC are 1st Generation BASIC : https://en.wikipedia.org/wiki/BASIC#Unstructured_BASIC )

I had to add a few code into SynFacilBasic.pas and SynFacilHighLighter.

First, this is the xml grammar (included in the zip file attached) :
Code: XML  [Select]
  1. <?xml version="1.0"?>
  2. <Language name="BASIC" ext="bas" FirstGen="true" CaseSensitive="true">
  3.   <Completion CaseSensitive="false" SelectOnEnter="true">
  4.         <OpenOn AfterPattern="Symbol">
  5.        "ABS|ABS(\_)"
  6.         AND
  7.         "ASC|ASC(\_)"
  8.         "ATN|ATN(\_)"
  9.         AUTO
  10.         CALL
  11.         "CHR$|CHR$(\_)"
  12.         CIRCLE
  13.         CLEAR
  14.         CLOAD
  15.         CLS
  16.         "COS|COS(\_)"
  17.         CSAVE
  18.         CURMOV
  19.         CURSET
  20.         DATA
  21.         "DEEK|DEEK(\_)"
  22.         DEF
  23.                 "DEF FN ... =|DEF FN \_ ="
  24.                 "DEF USR=|DEF USR="
  25.         DIM
  26.         DOKE
  27.         DRAW
  28.         EDIT
  29.         ELSE
  30.         END
  31.         "EXP|EXP(\_)"
  32.         EXPLODE
  33.         FALSE
  34.         FILL
  35.         FN
  36.         FOR
  37.         "FOR ... TO|FOR \_=  TO "
  38.         "FOR ... TO ... STEP|FOR \_ =  TO  STEP "
  39.         FRE(0)|FRE(0)
  40.         "FRE&quot;&quot;|FRE&quot;&quot;"
  41.         GET
  42.         GOSUB
  43.         GOTO
  44.         GRAB|
  45.         HEX$|HEX$(\_)
  46.         HIMEM
  47.         HIRES
  48.         IF
  49.         INK
  50.         INPUT|
  51.         "INT|INT(\_)"
  52.         KEY$
  53.         LEFT$|LEFT$(\_,)
  54.         LEN|LEN(\_)
  55.         LET
  56.         "LET ... =|LET \_="
  57.         LIST
  58.         LLIST
  59.         "LN|LN(\_)"
  60.         "LOG|LOG(\_)"
  61.         LORES
  62.         LPRINT|
  63.         "MID$|MID$(\_,,)"
  64.         MUSIC|
  65.         NEW
  66.         NEXT
  67.         NOT
  68.         ON
  69.         "ON ... GOTO|ON \_ GOTO"
  70.         "ON ... GOSUB|ON \_ GOSUB"
  71.         OR
  72.         PAPER
  73.         PATTERN
  74.         "PEEK|PEEK(\_)"
  75.         PI
  76.         PING
  77.         PLAY
  78.         PLOT
  79.         "POINT|POINT(\_,)"
  80.         POKE
  81.         POP
  82.         "POS|POS(\_)"
  83.         PRINT
  84.         PULL
  85.         READ
  86.         RECALL
  87.         RELEASE
  88.         REM
  89.         REPEAT
  90.         RESTORE
  91.         RETURN
  92.         "RIGHT$|RIGHT$(\_,)"
  93.         "RND|RND(\_)"
  94.         RUN
  95.         "SCRN|SCRN(\_,)"
  96.         "SGN|SGN(\_)"
  97.         SHOOT
  98.         SIN|SIN(\_)"
  99.         SOUND
  100.         "SPC|SPC(\_)"
  101.         "SQR|SQR(\_)"
  102.         STEP
  103.         STOP
  104.         STORE
  105.         "STR$|STR$(\_)"
  106.         "TAB|TAB(\_)"
  107.         "TAN|TAN(\_)"
  108.         TEXT
  109.         THEN
  110.         TO
  111.         TROFF
  112.         TRON
  113.         TRUE
  114.         UNTIL
  115.         USR
  116.         "USR()|USR(\_)"
  117.         "VAL|VAL(\_)"
  118.         WAIT
  119.         ZAP
  120.         </OpenOn>
  121.         <OpenOn AfterPattern='"DEF",space' FilterBy="None" Action="Insert">
  122.         "FN ... =|FN \_ ="
  123.         "USR=|USR="
  124.     </OpenOn>
  125.   </Completion>
  126.   <Attribute Name="Keyword1"   ForeCol="#00A4F2" Bold="false"  Italic="False" Underline="False"/>
  127.   <Attribute Name="Keyword2"   ForeCol="Purple" Bold="false"  Italic="False" Underline="False"/>
  128.   <Attribute Name="Comment"   ForeCol="#8D8D8D" Bold="False" Italic="False" Underline="False"/>
  129.   <Attribute Name="Number"    ForeCol="#FF00FF" Bold="False" Italic="False" Underline="False"/>
  130.   <Attribute Name="Delimiter"    ForeCol="#000000" Bold="false" Italic="False" Underline="False"/>
  131.   <Attribute Name="Operator"    ForeCol="Blue" Bold="False" Italic="False" Underline="False"/>
  132.   <Attribute Name="Variable"    ForeCol="Teal" Bold="false" Italic="False" Underline="False"/>
  133.   <Identifiers CharsStart= "£" Content = "a..z">
  134.   </Identifiers>
  135.  
  136.   <Token CharsStart="#" Content = '0..9ABCDEF' Attribute='Number'/>
  137.   <Token regex = '[0-9]+[\.]?[0-9]*[E][+-]?[0-9]+' Attribute='Number'/>
  138.   <Token regex='[A-Z][A-Z0-9]*[$%]?' Attribute='Variable'/>
  139.   <String Start="&quot;" End="&quot;"> </String>
  140.   <Comment Start="REM"></Comment>
  141.   <Comment Start="&apos;" ></Comment>
  142.   <Symbols>
  143.    
  144.     <Operator>
  145.       + - * / ^ AND OR &gt; = &lt; ( )
  146.     </Operator>
  147.         <keyword>
  148.           ! &amp; :
  149.         </keyword>
  150.         <Keyword1>EDIT STORE RECALL TRON TROFF PLOT LORES DOKE
  151.         LLIST LPRINT DATA
  152.         INPUT DIM CLS READ LET RUN RESTORE
  153.         REM HIMEM GRAB RELEASE TEXT HIRES SHOOT EXPLODE ZAP PING
  154.         SOUND MUSIC PLAY CURSET CURMOV DRAW CIRCLE PATTERN FILL
  155.         CHAR PAPER INK WAIT CLOAD CSAVE DEF POKE PRINT
  156.         LIST CLEAR GET CALL NEW TAB FN SPC @ AUTO NOT
  157.         SGN INT ABS USR FRE POS HEX$ SQR RND LN EXP COS
  158.         SIN TAN ATN PEEK DEEK LOG LEN STR$ VAL ASC CHR$ PI
  159.         TRUE FALSE KEY$ SCRN POINT LEFT$ RIGHT$ MID$</Keyword1>
  160.         <Keyword2> END POP PULL
  161.         REPEAT UNTIL FOR NEXT GOTO GOSUB RETURN
  162.         ON TO IF ELSE THEN STEP STOP
  163.         </Keyword2>
  164.        
  165.   </Symbols>
  166.  
  167. </Language>
  168.  

The core of the mod for TSynFacilSyn.Next :
Code: Pascal  [Select]
  1. procedure TSynFacilSyn.Next;
  2.   ...
  3.   if fRange = nil then begin
  4.       charIni:=fLine[posFin]; //guardar para tenerlo disponible en el método que se va a llamar.
  5.       fProcTable[charIni];    //Se ejecuta la función que corresponda.
  6.       //
  7.      if (FirstGen and isLineID) then
  8.      begin
  9.         if ((charIni<=' ') or (charIni in ['0'..'9']))
  10.         then fTokenID := tnComment
  11.         else isLineID:=false;
  12.      end;
  13.       //
  14.   end else begin
  15.     if posFin = tamLin then begin  //para acelerar la exploración
  16.   ...
  17. end;

That permits to comment out anything that is a number, a space, or a whitespace like typing in your program with real BASIC (I replace spaces by points to see it ) :
Code: [Select]
..10 PRINT
.3...56 FOR I=1TO5

The first line (line 10) was not commented because the first token is a space char with the suggestion you made in your post above.
The second line will be validated by the Microsoft BASIC of the Oric as line 356, as if it was accepted on the Oric command line.

OK, the colors are ugly, but we have for the first time something functionnal for vintage BASIC with the SynEdit environment. More important, it is very flexible and can be reused for other old school BASIC.

Try to find such an highlighter on the net : even NotePad++ cannot handle it !

Files and Screenshot in attachment (it is not the IDE I develop, it is the sample editor coming with SinFacilUtils) :
« Last Edit: October 20, 2018, 01:39:55 am by Ladywasky »