Recent

Author Topic: If you've ever wanted to be able to use multi-line strings in FPC...  (Read 8010 times)

Akira1364

  • Hero Member
  • *****
  • Posts: 539
You now can! After a long discussion on the FPC-devel mailing list, I spent some time over the last week implementing this functionality, and have now opened an issue on the bugtracker for it which is currently awaiting review:

https://bugs.freepascal.org/view.php?id=35827

If this is something that interests you, please feel free to test the patch (or directly test a build of FPC from my Github fork branch, which is linked on the tracker issue) as I'd love to get some additional real-world user feedback on this.

Edit:

(A basic explanation of the feature, that I also just posted on the tracker issue)

Here's a simple example adapted from one of my tests, that I think clearly explains how the trim directive works, that I'm posting here to make it more directly accessible:

Code: Pascal  [Select]
  1. program Example;
  2.  
  3. // You must set the below modeswitch to use the feature,
  4. // and will get an "illegal char" error at the opening
  5. // backtick of the first multi-line string encountered if you don't.
  6. // To be clear: multi-line strings are exclusively denoted with backticks,
  7. // not single quotes.
  8.  
  9. {$modeswitch MultiLineStrings}
  10.  
  11. {$MultiLineStringTrimLeft 1}
  12.  
  13. // There's two leading spaces on each line
  14. // of the multi-line string below. One will
  15. // be removed from each line, based on what we
  16. // just set for the trim directive.
  17.  
  18. const A = `
  19.   A
  20.   B
  21.   C
  22.   D
  23. `;
  24.  
  25. {$MultiLineStringTrimLeft 3}
  26.  
  27. // There's four leading spaces on each line
  28. // of the multi-line string below. Three will
  29. // be removed from each line, based on what we
  30. // just set for the trim directive.
  31.  
  32. const B = `
  33.     A
  34.     B
  35.     C
  36.     D
  37. `;
  38.  
  39. begin
  40.   Write(A);
  41.   Write(B);
  42. end.

The output of that is, thus:

 A
 B
 C
 D

 A
 B
 C
 D

since we first remove one space from where only two existed, and then remove three spaces from where four existed, leaving us with identical strings that have one leading space on each line.

Note that the default setting for MULTILINESTRINGTRIMLEFT is 0, meaning, by default no whitespace is removed.

If you set the directive to a higher number than the amount of leading whitespace present, the scanner will just stop skipping characters when it hits a non-whitespace one anyways, so you don't need to worry about mangling your strings or anything like that.

Additionally (as was an actual requirement specified for this feature by Michael van Canneyt) there exists a {$MULTILINESTRINGLINEENDING} directive, which takes one of CR, CRLF, LF, PLATFORM, or RAW as valid options. This specifies which particular line ending characters (or combination of characters in the case of CRLF) should be used at the end of each line in a multi-line string.

For anyone not aware:

CR = #13
CRLF = #13#10
LF = #10

PLATFORM means the compiler will use whichever line ending type is native to your operating system. RAW means the compiler will use the line endings exactly as present in the physical source file. RAW is the default.
« Last Edit: July 11, 2019, 09:39:03 pm by Akira1364 »

jshah

  • Full Member
  • ***
  • Posts: 200
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #1 on: July 11, 2019, 08:38:12 pm »
Good Addition

Is this change accepted by Core Team???

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #2 on: July 11, 2019, 09:10:21 pm »
Based on the long mailing list discussion, I'm under the impression that it is, essentially, yes. There were a few concerns expressed, and some things that were laid out as explicit requirements, but I've addressed all of them to the best of my ability.

Glad to hear you like it!
« Last Edit: July 11, 2019, 09:52:42 pm by Akira1364 »

avra

  • Hero Member
  • *****
  • Posts: 1736
    • Additional info
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #3 on: July 11, 2019, 10:17:37 pm »
Code: Pascal  [Select]
  1. {$MultiLineStringTrimLeft 40}
  2. {$MultiLineStringLineEnding PLATFORM}
  3.  
  4. const X = `
  5.     A
  6.     B
  7.     C
  8.     D
  9. `;
  10.  
  11. const Y = `A
  12.            B
  13.            C
  14.            D`;

1. Can we say that X and Y from above have identical content?
2. Can we say that if we compile on Windows then line endings would be CRLF and if we cross compile to Linux then it will be LF?
3. Could you explain {$MultiLineStringLineEnding RAW}?
4. What about {$MultiLineStringTrim 99} to trim 99 spaces from both left and right? And {$MultiLineStringTrimRight 99}?
« Last Edit: July 11, 2019, 10:34:40 pm by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #4 on: July 11, 2019, 10:56:50 pm »
1. Can we say that X and Y from above have identical content?

No. There is a small difference: the first one contains a blank line above the "A", and below the "D".

2. Can we say that if we compile on Windows then line endings would be CRLF and if we cross compile to Linux then it will be LF?

If using {$MultiLineStringEnding PLATFORM}, that would be the case, yes, as internally "target_info.newline" is what's checked to determine what to use.

3. Could you explain {$MultiLineStringLineEnding RAW}?

It just means the compiler does not alter anything with regards to newlines (i.e. the operating system is irrelevant, and what's actually in the file is all that matters.)

So for example, if you're on Windows, but you like to save your files with Unix-style LF newlines, the compiler will not implicitly change that, and the strings will still have LF newlines. Or vice versa, and so on.

This is the least intrusive approach, and also what people would usually expect and want I believe, and thus I made it the default setting.

4. What about {$MultiLineStringTrim 99} to trim 99 spaces from both left and right? And {$MultiLineStringTrimRight 99}?

Right-side trimming is certainly technically possible.

However, the implementation would be significantly more complicated, and I am not sure that it is nearly as useful as left-side trimming. (For example, no one's preferred code formatting style is thrown off by whitespace that may exist between the last visible character and the newline. The same cannot be said for whitespace that exists at the beginning of a line, before any visible characters.)
« Last Edit: July 11, 2019, 11:12:21 pm by Akira1364 »

jamie

  • Hero Member
  • *****
  • Posts: 2168
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #5 on: July 11, 2019, 11:02:53 pm »
Its ugly ..

What if I wanted to create a constant with multiple words in it and like columnize them?

Const = '
One [options ]
Two [options ]
three
Four
Ect'

For me it reads its clearer to read out..

Maybe you could suggest for the IDE/Editor code tools to have a popup option that would generate the v
visible line endings for you ?
'
One [ options ] '#13#10

etc...

 I would go for that but please , this isn't a single line language!

Number 1 at blue screen app creations!

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #6 on: July 11, 2019, 11:11:27 pm »
Its ugly ...

Looks pretty much just like a string does to me. Here's what it looks like in Lazarus, with a patched SynEdit (which I'll submit as a Lazarus patch separately at some point):

https://i.imgur.com/w1veLwg.png

What if I wanted to create a constant with multiple words in it and like columnize them?

Const = '
One [options ]
Two [options ]
three
Four
Ect'

I'm afraid I'm unsure exactly what your concern is here.

Maybe you could suggest for the IDE/Editor code tools to have a popup option that would generate the v
visible line endings for you ?

This is not a remotely equivalent solution, for a variety of reasons.

I would go for that but please , this isn't a single line language!

I don't know what you meant here.

jamie

  • Hero Member
  • *****
  • Posts: 2168
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #7 on: July 11, 2019, 11:19:37 pm »
You obviously don't see it.

Number 1 at blue screen app creations!

avra

  • Hero Member
  • *****
  • Posts: 1736
    • Additional info
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #8 on: July 11, 2019, 11:21:17 pm »
3. Could you explain {$MultiLineStringLineEnding RAW}?
...
This is the least intrusive approach, and also what people would usually expect and want I believe, and thus I made it the default setting.
I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing. I would rather have {$MultiLineStringLineEnding PLATFORM} as a default setting, and {$MultiLineStringLineEnding RAW} as optional. I also find word RAW not intuitive enough in this case. SOURCE sounds a little better to me.

I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

jamie

  • Hero Member
  • *****
  • Posts: 2168
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #9 on: July 11, 2019, 11:30:18 pm »
You do as you please,. I'll never ever ever enable that feature if that is, it ever gets considered...

If  such a feature is hard coded I'll leave fpc and spend the money for the latest Delphi, even though its
expensive. I need something that actually produces exactly what I tell it to in any manner I select of editing
style..

 Like I said, if you have that much ambition there are much better ideas to implement in the compiler...

 and one at the top of my list is Anonymous Records within Record defines.

 

 
Number 1 at blue screen app creations!

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #10 on: July 11, 2019, 11:33:08 pm »
You do as you please,. I'll never ever ever enable that feature if that is, it ever gets considered...

If  such a feature is hard coded I'll leave fpc and spend the money for the latest Delphi, even though its
expensive. I need something that actually produces exactly what I tell it to in any manner I select of editing
style..

 Like I said, if you have that much ambition there are much better ideas to implement in the compiler...

 and one at the top of my list is Anonymous Records within Record defines.

Whole lot of rather extravagant hyperbole going on here. Regardless, you are indeed free to not ever set

Code: Pascal  [Select]
  1. {$modeswitch MultiLineStrings}

which again, you have to in order to use the feature anywhere at all.

I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.

This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them. It's also how text and textfiles in general work, in all cases...

Having PLATFORM as the default would be much less expected IMO, and likely frustrating for anyone who did not want the compiler to impose a certain line ending that may have been not what was actually written.

The option name "RAW" is perhaps improvable, though. "SOURCE" may indeed be clearer.

I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.

What you seem to want here could be solved by having a directive or command-line option that worked globally instead of locally, I think, which is certainly possible.
« Last Edit: July 12, 2019, 12:14:05 am by Akira1364 »

lucamar

  • Hero Member
  • *****
  • Posts: 2145
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #11 on: July 12, 2019, 12:25:13 am »
I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.

What you seem to want here could be solved by having a directive or command-line option that worked globally instead of locally, I think, which is certainly possible.

Perhaps {$MultiLineStringTrim}, without parameter, should be made to mean "Trim All"?

I don't really like this feature (too many possible "unexpected" behaviours) but I see why someone may want to use it.

ETA: Regarding this:

I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.

This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them.

The only way now to add a line-ending to a constant string is either coding it as characters, that is: 'MyString'#10'My other string' or as 'MyString'+LineEnding+'MyOtherString'. No VCS will touch those as it may if they were real, embedded line-endings.

That's the crux: that the constants may change without you meaning it--although I'd say, in this case, it's more a question of VCS misconfiguration.
« Last Edit: July 12, 2019, 12:38:09 am by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.4/2.0.6  - FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

avra

  • Hero Member
  • *****
  • Posts: 1736
    • Additional info
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #12 on: July 12, 2019, 12:33:15 am »
I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.
This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them. It's also how text and textfiles in general work, in all cases...
Most editors show expected code layout no matter what line endings are in the code, and no matter what platform you're on. They handle it properly and do not force you to do a conversion. Therefore I do not mind if GIT/SVN/CVS change line endings in my sources since currently it can not affect output of my code. However with {$MultiLineStringEnding RAW} as default, it is easy to overlook that my code might have strings that have line endings not compatible with target platform and therefore produce unexpected and wrong result. That is the reason why I see {$MultiLineStringEnding PLATFORM} much more appropriate as default. And GIT/SVN/CVS are not the only ones who can change line endings in the source. Let's say I give source of some program to someone who opens just one file in an editor that tries to be smart and changes line endings on saving. After that user compiles my program but program does not work properly. Can you see how much of a problem that can be? Do I have to start making notes what are line endings in my sources? Do I have to put {$MultiLineStringEnding PLATFORM} in all files just because anyone can accidentally change line endings in my sources and by doing that make output of my program completely different?

Perhaps {$MultiLineStringTrim}, without parameter, should be made to mean "Trim All"?
{$MultiLineStringTrim All} sounds ok.
« Last Edit: July 12, 2019, 12:40:57 am by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Edson

  • Hero Member
  • *****
  • Posts: 1056
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #13 on: July 12, 2019, 12:47:39 am »
Good. Maybe not the best implementation, but it's a nice feature I always ask to FPC: https://forum.lazarus.freepascal.org/index.php/topic,22777.msg135091.html#msg135091
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Akira1364

  • Hero Member
  • *****
  • Posts: 539
Re: If you've ever wanted to be able to use multi-line strings in FPC...
« Reply #14 on: July 12, 2019, 01:12:51 am »
Maybe not the best implementation

I'm happy to hear any suggestions you might have!

Do note again though that there were specific "must-haves" and "must-not-dos" that I was required to consider when writing it in order for it to have any chance of being accepted as a patch.

Can you see how much of a problem that can be? Do I have to start making notes what are line endings in my sources? Do I have to put {$MultiLineStringEnding PLATFORM} in all files just because anyone can accidentally change line endings in my sources and by doing that make output of my program completely different?

I think you're vastly overstating the amount of scenarios where this would be a problem, and also expecting the compiler to essentially babysit your revision control configuration. The compiler directives are directives. Use them as you wish, like you would any existing directive.

Note once more, also: this feature is not hypothetical. It works, and you can test it right now. Please feel free to do so and give actual feedback based on actual use.

I don't really like this feature (too many possible "unexpected" behaviours) but I see why someone may want to use it.

What do you see as unexpected, exactly? You type in text. The compiler reads the text. The end. It's a very, very, simple feature.
« Last Edit: July 12, 2019, 01:27:51 am by Akira1364 »