Recent

Author Topic: If you've ever wanted to be able to use multi-line strings in FPC...  (Read 7960 times)

BrunoK

  • Full Member
  • ***
  • Posts: 191
  • Retired programmer
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #60 on: July 13, 2019, 12:41:18 pm »
That works just as you've written it already, BTW:
Maybe  a {$MultiLineStringTrimLeft auto} would be convenient. It would decide, based on the position of the first backtick, what is the  count of character to consider in the {$MultiLineStringTrimLeft n_characters}. It could also check that there aren't any characters under/left  of the initial backtick.

Changing the paragraph horizontal (sorry could not find the word in english) 'retrait de paragraphe'  is pretty frequent during editing and having to correct the n_characters after any change would be a nuisance.

EDIT :

Changing the paragraph horizontal indent is pretty frequent during editing and having to correct the n_characters after any change would be a nuisance.
« Last Edit: July 13, 2019, 01:24:08 pm by BrunoK »
Lazarus trunk r. 62137/27.10.2019 (+/- patches regarding TScrollBar, IntitalSetupDialog, Options.Environment options, SearchResults).  Lazarus 3.0.6 raw from svn.
FPC 3.0.4 32 bits. (+heaptrc with leaked ClassName+Revisited TList) , Windows 10 Pro x64 (v. 1903 / 18362.418)

PascalDragon

  • Hero Member
  • *****
  • Posts: 712
  • Compiler Developer
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #61 on: July 13, 2019, 04:01:43 pm »
Do I understand correctly that this then wouldn't be trimmed at all, because the first line doesn't have any spaces?

You do not I'm afraid. It is trimmed like you'd expect (so, in that case, completely trimmed such that there is no leading space left anywhere.)

I.E. it would look like this if displayed with WriteLn:

Code: Pascal  [Select]
  1. A
  2. B
  3. C
  4. D
Ah, then it's not what I expected, but what I hoped for. ;)

That works just as you've written it already, BTW:

Code: Pascal  [Select]
  1. program SQL;
  2.  
  3. {$modeswitch MultiLineStrings}
  4.  
  5. {$MultiLineStringTrimLeft 10}
  6.  
  7. const
  8.   Str1 = `SELECT o.*, C.Company
  9.           from Orders O
  10.           join Customer C
  11.             on o.CustNo=C.ID
  12.           where
  13.             O.saledate=DATE '2001.03.20'`;
  14.  
  15. {$MultiLineStringTrimLeft 5}
  16.  
  17. const
  18.   Str2 =
  19.     `SELECT o.*, C.Company
  20.      from Orders O
  21.      join Customer C
  22.        on o.CustNo=C.ID
  23.      where
  24.        O.saledate=DATE '2001.03.20'`;
  25.  
  26. begin
  27.   WriteLn(Str1);
  28.   WriteLn(Str2);
  29. end.
Looking at this I wonder if some {$MultiLineStringTrimLeft Auto} would be useful, so that it would simply automatically trim both statements to look the same...

That said, the idea behind {$LINEBREAKS ON} is to direct the scanner to a new routine that handles multiline strings.  What Akira triggers with a ` (backtick), I would trigger with a directive.   Just as simple, just a different path to getting to the same place.

At least at first blush, there doesn't seem to be anything preventing such an implementation and it would not require adding new elements (such as the backtick) to the language.
No, it's definitely not that simple, because the + operator could be overloaded and as I said all the operator stuff is handled inside the parser not inside the scanner where strings are handled.
Look at this:
Code: Pascal  [Select]
  1. // assume an operator overload for String + Integer is in scope
  2. begin
  3.   somestr := 'str content' +
  4.                    someint;
  5. end.
Your code would break this if you'd do this inside the scanner.
And the parser sees the code as this:
Code: Pascal  [Select]
  1. begin somestr := 'str content' + someint; end.
So it can't (and should) not handle things like new lines.

I still don’t understand, why we just don’t introduce a new compiler switch, that’ll eliminate the
Code: Text  [Select]
  1. Fatal: String exceeds line
error. No need for new syntax and achieved our goal with minimal efforts.
Yeah, sure, a string line with a length of thousands of characters is soo much easier to read...  ::)

440bx

  • Hero Member
  • *****
  • Posts: 1267
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #62 on: July 13, 2019, 05:21:32 pm »
No, it's definitely not that simple, because the + operator could be overloaded and as I said all the operator stuff is handled inside the parser not inside the scanner where strings are handled.
You're right.  I didn't think about the fact that the "+" operator might be overloaded which forces the processing of the construct to be done by the parser.

Yes, Akira's implementation, without using "+" (or any other operator for that matter) makes implementation simpler and more localized.
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #63 on: July 13, 2019, 05:47:22 pm »
Looking at this I wonder if some {$MultiLineStringTrimLeft Auto} would be useful, so that it would simply automatically trim both statements to look the same...

A few people have indicated they'd like something like this. Would not be difficult to implement at all, as again the main logic for trimming is simply this, currently:

Code: Pascal  [Select]
  1. #32,#9,#11 :
  2.   if (had_newline or first_multiline) and (current_settings.whitespacetrimcount > 0) then
  3.     begin
  4.       trimcount:=current_settings.whitespacetrimcount;
  5.       while (c in [#32,#9,#11]) and (trimcount > 0) do
  6.         begin
  7.           readchar;
  8.           dec(trimcount);
  9.         end;
  10.       had_newline:=false;
  11.       first_multiline:=false;
  12.       goto quote_label;
  13.     end;

I'm unsure if there is precedent in the compiler for directives that take both numeric values and "string" values, such as would be the case here, however.

I know how to implement it properly, so there's no concern there, but I'm just not sure if it would be considered more "idiomatic FPC compiler code" to have a separate {$MultiLineStringTrimLeftAuto} directive that just took ON or OFF as input.

In the case of having the separate directive, the above code would become something roughly like this (untested for side effects!) block:

Code: Pascal  [Select]
  1. #32,#9,#11 :
  2.   if had_newline or first_multiline then
  3.     begin
  4.       if current_settings.whitespacetrimauto then
  5.         begin
  6.           while (c in [#32,#9,#11]) do
  7.             readchar;
  8.           had_newline:=false;
  9.           first_multiline:=false;
  10.           goto quote_label;
  11.         end
  12.       else if current_settings.whitespacetrimcount > 0 then
  13.         begin
  14.           trimcount:=current_settings.whitespacetrimcount;
  15.           while (c in [#32,#9,#11]) and (trimcount > 0) do
  16.             begin
  17.               readchar;
  18.               dec(trimcount);
  19.             end;
  20.           had_newline:=false;
  21.           first_multiline:=false;
  22.           goto quote_label;
  23.         end;
  24.     end;

In the case of "one directive that took either a number or AUTO", though, the code wouldn't change much at all as AUTO would likely just set current_settings.whitespacetrimcount to the maximum possible numeric value (currently 65535, as right now current_settings.whitespacetrimcount is a word field, chosen because all the four-byte max values seemed excessively huge to me while all the one-byte ones seemed a bit too small.)

This would work correctly and not slow anything down, because of course I wrote it so that the scanner always stops when there's no actual leading whitespace left anyways.

Option three would be "have the separate directive, but still just make it set the existing current_settings.whitespacetrimcount to the highest value possible."

All ways would work, and all are quite easy to pull off. I'd need to add a bit more logic to make it actually base itself on the source position of the first backtick, though. Which I think is possible, but perhaps slightly more "involved" to implement.

Apart from that, do you have any particular thoughts on my implementation as it exists so far? E.G, is there anything that stands out to you as obviously "wrong" / e.t.c? Very interested in any input at all I can get on this to make sure it's as useful as possible in the long run.
« Last Edit: July 14, 2019, 02:41:47 am by Akira1364 »

PascalDragon

  • Hero Member
  • *****
  • Posts: 712
  • Compiler Developer
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #64 on: July 13, 2019, 07:59:44 pm »
I'm unsure if there is precedent in the compiler for directives that take both numeric values and "string" values, such as would be the case here, however.
Take a look at for example dir_align.

Apart from that, do you have any particular thoughts on my implementation as it exists so far? E.G, is there anything that stands out to you as obviously "wrong" / e.t.c? Very interested in any input at all I can get on this to make sure it's as useful as possible in the long run.
Sorry, I've been too busy with the reintegration of attributes, so I haven't looked at your patch. Maybe I'll find the time tomorrow, but no promises!

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #65 on: July 13, 2019, 08:58:12 pm »
Take a look at for example dir_align.

Ah, that's right.

Sorry, I've been too busy with the reintegration of attributes, so I haven't looked at your patch. Maybe I'll find the time tomorrow, but no promises!

Oh yeah, no worries. I just wasn't sure if you had perhaps already looked at it to some extent or not.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #66 on: July 14, 2019, 02:07:40 am »
test the feature, and report any other bugs you might find! (Which of course is hopefully few, or even better none, haha.)
Finding a bug is much harder now, but I did manage to find one more, probably this is the last one.

When left-side trimming is enabled, and a string starts with anything but a backtick, it should not trim. Here are a few examples:
Code: Pascal  [Select]
  1. {$MultiLineStringTrimLeft 5}
  2.  
  3. middleSpaceBug: array[0..5] of string = (
  4. #$41` `#$42   //<--- this becomes #$41#$42 instead of #$41#$20#$42
  5. ,
  6. #$41` - `#$42 //<--- this becomes #$41#$2D#$20#$42 instead of #$41#$20#$2D#$20#$42
  7. ,
  8. #$41`      -      `#$42  //<--- one space left instead of six spaces
  9. ,
  10. #$41`      -      `#$42`  --  `  //<---- the same bug twice
  11. ,
  12. '  '#$41` `#$42   //<--- the space between backticks disappears: #$20#$20#$41#$42
  13. ,
  14. ^T' e s'` t`  //<--- last two: $73$74 instead of $73$20$74
  15. );

You can get the expected string by replacing backticks with apostrophe.

Well done, it did take me a while to find this one.

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #67 on: July 14, 2019, 02:22:00 am »
When left-side trimming is enabled, and a string starts with anything but a backtick, it should not trim. Here are a few examples:

Oh man, I use "literal-pound-sign-character-followed-immediately-by-quoted-string" so rarely that it absolutely did not occur to me to test that. I am glad it works normally with backticks, at least, apart from the trimming issue (although with the good design of FPC's scanner, there's no reason it wouldn't.)

Really good find, thanks so much. This is what open source is all about!

Once again, also, I definitely know how to fix that (now that I'm aware it's a problem), so it shouldn't take very long.
« Last Edit: July 14, 2019, 02:27:43 am by Akira1364 »

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #68 on: July 14, 2019, 02:31:03 am »
Without you keeping your Github fork branch updated it would be harder for me to keep testing. Thank you for working on what I believe is the last bug in this feature.

VTwin

  • Hero Member
  • *****
  • Posts: 793
  • Former Turbo Pascal 3 user
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #69 on: July 14, 2019, 03:45:11 am »

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?

Second. This syntax is simple and readable.
“Talk is cheap. Show me the code.” -Linus Torvalds

macOS 10.13.6: Lazarus 2.0.7 fixes svn 62300 (64 bit Cocoa)
Ubuntu 18.04.3: Lazarus 2.0.6 (64 bit on VBox)
Windows 7 Pro SP1: Lazarus 2.0.6 (64 bit on VBox)
fpc 3.0.4

jamie

  • Hero Member
  • *****
  • Posts: 2159
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #70 on: July 14, 2019, 04:02:16 am »
I third it …!

  I already have an editing tool that will build that for me, I just paste it in afterwards.

  The tool even reads it back for re-editing..
 
  The only thing I can see that maybe an advantage is for the compiler to carry to the next line looking
for the  closing " ' ".

   But I can see that getting complicated with the ability of inserting quotes etc..

  I can think of something else to be using this valuable time on for compiler enhancements.
Number 1 at blue screen app creations!

lucamar

  • Hero Member
  • *****
  • Posts: 2135
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #71 on: July 14, 2019, 04:33:06 am »
I already have an editing tool that will build that for me, I just paste it in afterwards.

Yeah, I have a mini-editor too that does that and other nifty things. Guess most people end up building one; it's a good (and productive!) exercise.

  I can think of something else to be using this valuable time on for compiler enhancements.

Let's be realistic: whatever someone does with his time is his own affair and if it ends up helping (part of) the community, it's time well spent.

I don't like much this feature but that is basically for personal reasons. I'm one of those persons who goes to add a resource string and forgets to close it or add the terminal ; , in my hurry to go back to the "real" code so I can well imagine myself writing a multiline string and a week later "closing" all those dangling strings without realizing it's a multiline one. or viceversa, trying to do a multiline with the normal '.

I love Pascal in part because it's very "unforgiving" and that forces you to be more careful (or compile three times in a row :)), so I dislike features that make it less so or force *you* to be inordinately attent to sintax.

I know I'm not explaining myself very well, so feel free to disregard all this as the ramblings of an (still not very) old coder. :D
« Last Edit: July 14, 2019, 04:38:28 am by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.4/2.0.6  - FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #72 on: July 14, 2019, 04:35:10 am »
I third it …!
I hear you.

  I already have an editing tool that will build that for me, I just paste it in afterwards.

  The tool even reads it back for re-editing..
Why not make it part of Lazarus?
How does it handle line endings on different systems?
What about cross compiling?
Can we see a sample? can you share its source code?
People want their text to stay formatted and beautiful. Can your tool fulfill all the requirements that Akira's multi-line strings are able to do today?

I am really eager to see the results. Can you at least post a screenshot?

That fact that you had to make a tool is by itself a strong indication of a missing feature. Moreover, Lazarus is NOT the only IDE people are using, so adding this function to Lazarus does not solve the problem for others.

 
  The only thing I can see that maybe an advantage is for the compiler to carry to the next line looking
for the  closing " ' ".
And this is a HUGE advantage if you write SQL/HTML/JS/LUA/..etc inside your code.
We are not forced to have long lines of strings, nor forced to add ' + LineEnding + so we can continue on to the next line.

The feature is not intrusive at all. It is off by default.

 
   But I can see that getting complicated with the ability of inserting quotes etc..
Not sure what you mean here. This is even simpler, as you do not need to escape apostrophe. Simple test:
Code: Text  [Select]
  1.   Test1 := `You can use apostrophe ', and double quote " without escaping`;
  2.   Test2 := `You only need to escape backticks, like this one ``, see?`;

 
  I can think of something else to be using this valuable time on for compiler enhancements.
You make it sound like everyone is working on this feature, this is not the case. Here is a list of new features:
https://wiki.freepascal.org/FPC_New_Features_Trunk

Akira spent more time arguing with people than coding. He coded it amazingly in a very short time, and bugs are fixed very quickly.
« Last Edit: July 14, 2019, 04:37:53 am by engkin »

PascalDragon

  • Hero Member
  • *****
  • Posts: 712
  • Compiler Developer
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #73 on: July 14, 2019, 11:33:01 am »

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?
Second. This syntax is simple and readable.
Yes, but the discussion to replace this syntax with backtick-strings only came later. The feature that is really talked about would replace this:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' + sLineBreak +
  2.            ' in the same line (provided your' + sLineBreak +
  3.            ' terminal is wide enaough)';

 
  I can think of something else to be using this valuable time on for compiler enhancements.
You make it sound like everyone is working on this feature, this is not the case. Here is a list of new features:
https://wiki.freepascal.org/FPC_New_Features_Trunk

Akira spent more time arguing with people than coding. He coded it amazingly in a very short time, and bugs are fixed very quickly.
Also keep in mind that Akira1364 is an independant developer who's not a core developer (I'm not trying to belittle his contribution, just to clear up his relationship with the core team to avoid any misunderstandings)

kupferstecher

  • Sr. Member
  • ****
  • Posts: 324
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #74 on: July 14, 2019, 01:42:27 pm »
Yes, but the discussion to replace this syntax with backtick-strings only came later. The feature that is really talked about would replace this:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' + sLineBreak +
  2.            ' in the same line (provided your' + sLineBreak +
  3.            ' terminal is wide enaough)';
A string operator could be introduced that combines the "+ sLineBreak +" for example using the "/":
Code: Pascal  [Select]
  1. MyString = 'This will not appear all' /
  2.            ' in the same line (provided you' /
  3.            ' use / instead of +)';

This could be useful even when Akira's approach gets implemented.

--
Several people have mentioned wanting a "no newlines at all" option for {$MultiLineStringLineEnding} (which would mean whatever is written would be interpreted as one long horizontal string), which would not be difficult to add, and so I may very well do so. The option would most likely be called NONE, I think.
In case of trimming a space would have to be inserted for each new line.

--
Quote from: Akira1364
{$MultiLineString[...]}
I would suppose not to use too many mode switches, i.e. not to provide too many options. Trimming on or off should be enough.
Either the format is not important like for the database accesses then a complete trim is suitable, or a intendation is needed for better human readability in the target application then a manual formatting with no trimming at all is imho better than some fancy options where you have to study several pages of documentation to understand whats happening. Keep things simple.