Lazarus

Programming => General => Topic started by: Akira1364 on July 11, 2019, 03:36:34 pm

Title: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 11, 2019, 03:36:34 pm
You now can! After a long discussion on the FPC-devel mailing list, I spent some time over the last week implementing this functionality, and have now opened an issue on the bugtracker for it which is currently awaiting review:

https://bugs.freepascal.org/view.php?id=35827 (https://bugs.freepascal.org/view.php?id=35827)

If this is something that interests you, please feel free to test the patch (or directly test a build of FPC from my Github fork branch, which is linked on the tracker issue) as I'd love to get some additional real-world user feedback on this.

Edit:

(A basic explanation of the feature, that I also just posted on the tracker issue)

Here's a simple example adapted from one of my tests, that I think clearly explains how the trim directive works, that I'm posting here to make it more directly accessible:

Code: Pascal  [Select]
  1. program Example;
  2.  
  3. // You must set the below modeswitch to use the feature,
  4. // and will get an "illegal char" error at the opening
  5. // backtick of the first multi-line string encountered if you don't.
  6. // To be clear: multi-line strings are exclusively denoted with backticks,
  7. // not single quotes.
  8.  
  9. {$modeswitch MultiLineStrings}
  10.  
  11. {$MultiLineStringTrimLeft 1}
  12.  
  13. // There's two leading spaces on each line
  14. // of the multi-line string below. One will
  15. // be removed from each line, based on what we
  16. // just set for the trim directive.
  17.  
  18. const A = `
  19.   A
  20.   B
  21.   C
  22.   D
  23. `;
  24.  
  25. {$MultiLineStringTrimLeft 3}
  26.  
  27. // There's four leading spaces on each line
  28. // of the multi-line string below. Three will
  29. // be removed from each line, based on what we
  30. // just set for the trim directive.
  31.  
  32. const B = `
  33.     A
  34.     B
  35.     C
  36.     D
  37. `;
  38.  
  39. begin
  40.   Write(A);
  41.   Write(B);
  42. end.

The output of that is, thus:

 A
 B
 C
 D

 A
 B
 C
 D

since we first remove one space from where only two existed, and then remove three spaces from where four existed, leaving us with identical strings that have one leading space on each line.

Note that the default setting for MULTILINESTRINGTRIMLEFT is 0, meaning, by default no whitespace is removed.

If you set the directive to a higher number than the amount of leading whitespace present, the scanner will just stop skipping characters when it hits a non-whitespace one anyways, so you don't need to worry about mangling your strings or anything like that.

Additionally (as was an actual requirement specified for this feature by Michael van Canneyt) there exists a {$MULTILINESTRINGLINEENDING} directive, which takes one of CR, CRLF, LF, PLATFORM, or RAW as valid options. This specifies which particular line ending characters (or combination of characters in the case of CRLF) should be used at the end of each line in a multi-line string.

For anyone not aware:

CR = #13
CRLF = #13#10
LF = #10

PLATFORM means the compiler will use whichever line ending type is native to your operating system. RAW means the compiler will use the line endings exactly as present in the physical source file. RAW is the default.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jshah on July 11, 2019, 08:38:12 pm
Good Addition

Is this change accepted by Core Team???
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 11, 2019, 09:10:21 pm
Based on the long mailing list discussion, I'm under the impression that it is, essentially, yes. There were a few concerns expressed, and some things that were laid out as explicit requirements, but I've addressed all of them to the best of my ability.

Glad to hear you like it!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 11, 2019, 10:17:37 pm
Code: Pascal  [Select]
  1. {$MultiLineStringTrimLeft 40}
  2. {$MultiLineStringLineEnding PLATFORM}
  3.  
  4. const X = `
  5.     A
  6.     B
  7.     C
  8.     D
  9. `;
  10.  
  11. const Y = `A
  12.            B
  13.            C
  14.            D`;

1. Can we say that X and Y from above have identical content?
2. Can we say that if we compile on Windows then line endings would be CRLF and if we cross compile to Linux then it will be LF?
3. Could you explain {$MultiLineStringLineEnding RAW}?
4. What about {$MultiLineStringTrim 99} to trim 99 spaces from both left and right? And {$MultiLineStringTrimRight 99}?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 11, 2019, 10:56:50 pm
1. Can we say that X and Y from above have identical content?

No. There is a small difference: the first one contains a blank line above the "A", and below the "D".

2. Can we say that if we compile on Windows then line endings would be CRLF and if we cross compile to Linux then it will be LF?

If using {$MultiLineStringEnding PLATFORM}, that would be the case, yes, as internally "target_info.newline" is what's checked to determine what to use.

3. Could you explain {$MultiLineStringLineEnding RAW}?

It just means the compiler does not alter anything with regards to newlines (i.e. the operating system is irrelevant, and what's actually in the file is all that matters.)

So for example, if you're on Windows, but you like to save your files with Unix-style LF newlines, the compiler will not implicitly change that, and the strings will still have LF newlines. Or vice versa, and so on.

This is the least intrusive approach, and also what people would usually expect and want I believe, and thus I made it the default setting.

4. What about {$MultiLineStringTrim 99} to trim 99 spaces from both left and right? And {$MultiLineStringTrimRight 99}?

Right-side trimming is certainly technically possible.

However, the implementation would be significantly more complicated, and I am not sure that it is nearly as useful as left-side trimming. (For example, no one's preferred code formatting style is thrown off by whitespace that may exist between the last visible character and the newline. The same cannot be said for whitespace that exists at the beginning of a line, before any visible characters.)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 11, 2019, 11:02:53 pm
Its ugly ..

What if I wanted to create a constant with multiple words in it and like columnize them?

Const = '
One [options ]
Two [options ]
three
Four
Ect'

For me it reads its clearer to read out..

Maybe you could suggest for the IDE/Editor code tools to have a popup option that would generate the v
visible line endings for you ?
'
One [ options ] '#13#10

etc...

 I would go for that but please , this isn't a single line language!

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 11, 2019, 11:11:27 pm
Its ugly ...

Looks pretty much just like a string does to me. Here's what it looks like in Lazarus, with a patched SynEdit (which I'll submit as a Lazarus patch separately at some point):

https://i.imgur.com/w1veLwg.png (https://i.imgur.com/w1veLwg.png)

What if I wanted to create a constant with multiple words in it and like columnize them?

Const = '
One [options ]
Two [options ]
three
Four
Ect'

I'm afraid I'm unsure exactly what your concern is here.

Maybe you could suggest for the IDE/Editor code tools to have a popup option that would generate the v
visible line endings for you ?

This is not a remotely equivalent solution, for a variety of reasons.

I would go for that but please , this isn't a single line language!

I don't know what you meant here.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 11, 2019, 11:19:37 pm
You obviously don't see it.

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 11, 2019, 11:21:17 pm
3. Could you explain {$MultiLineStringLineEnding RAW}?
...
This is the least intrusive approach, and also what people would usually expect and want I believe, and thus I made it the default setting.
I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing. I would rather have {$MultiLineStringLineEnding PLATFORM} as a default setting, and {$MultiLineStringLineEnding RAW} as optional. I also find word RAW not intuitive enough in this case. SOURCE sounds a little better to me.

I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 11, 2019, 11:30:18 pm
You do as you please,. I'll never ever ever enable that feature if that is, it ever gets considered...

If  such a feature is hard coded I'll leave fpc and spend the money for the latest Delphi, even though its
expensive. I need something that actually produces exactly what I tell it to in any manner I select of editing
style..

 Like I said, if you have that much ambition there are much better ideas to implement in the compiler...

 and one at the top of my list is Anonymous Records within Record defines.

 

 
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 11, 2019, 11:33:08 pm
You do as you please,. I'll never ever ever enable that feature if that is, it ever gets considered...

If  such a feature is hard coded I'll leave fpc and spend the money for the latest Delphi, even though its
expensive. I need something that actually produces exactly what I tell it to in any manner I select of editing
style..

 Like I said, if you have that much ambition there are much better ideas to implement in the compiler...

 and one at the top of my list is Anonymous Records within Record defines.

Whole lot of rather extravagant hyperbole going on here. Regardless, you are indeed free to not ever set

Code: Pascal  [Select]
  1. {$modeswitch MultiLineStrings}

which again, you have to in order to use the feature anywhere at all.

I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.

This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them. It's also how text and textfiles in general work, in all cases...

Having PLATFORM as the default would be much less expected IMO, and likely frustrating for anyone who did not want the compiler to impose a certain line ending that may have been not what was actually written.

The option name "RAW" is perhaps improvable, though. "SOURCE" may indeed be clearer.

I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.

What you seem to want here could be solved by having a directive or command-line option that worked globally instead of locally, I think, which is certainly possible.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 12, 2019, 12:25:13 am
I also find awkward to have something like {$MultiLineStringTrim 99} in many of my sources, when I actually want something like {$MultiLineStringTrimAll} since {$MultiLineStringTrim} is already occupied for completely different use. I would then expect {$MultiLineStringTrimAll} to be default setting, which does not seam to be the case.

What you seem to want here could be solved by having a directive or command-line option that worked globally instead of locally, I think, which is certainly possible.

Perhaps {$MultiLineStringTrim}, without parameter, should be made to mean "Trim All"?

I don't really like this feature (too many possible "unexpected" behaviours) but I see why someone may want to use it.

ETA: Regarding this:

I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.

This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them.

The only way now to add a line-ending to a constant string is either coding it as characters, that is: 'MyString'#10'My other string' or as 'MyString'+LineEnding+'MyOtherString'. No VCS will touch those as it may if they were real, embedded line-endings.

That's the crux: that the constants may change without you meaning it--although I'd say, in this case, it's more a question of VCS misconfiguration.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 12, 2019, 12:33:15 am
I find it very strange and disturbing that a source file going through some version control system can change content of it's multi line strings based on line ending changes in it self. Pretty unexpected and confusing.
This is literally how single-line strings work currently, though. The compiler just uses whatever is actually present when reading them. It's also how text and textfiles in general work, in all cases...
Most editors show expected code layout no matter what line endings are in the code, and no matter what platform you're on. They handle it properly and do not force you to do a conversion. Therefore I do not mind if GIT/SVN/CVS change line endings in my sources since currently it can not affect output of my code. However with {$MultiLineStringEnding RAW} as default, it is easy to overlook that my code might have strings that have line endings not compatible with target platform and therefore produce unexpected and wrong result. That is the reason why I see {$MultiLineStringEnding PLATFORM} much more appropriate as default. And GIT/SVN/CVS are not the only ones who can change line endings in the source. Let's say I give source of some program to someone who opens just one file in an editor that tries to be smart and changes line endings on saving. After that user compiles my program but program does not work properly. Can you see how much of a problem that can be? Do I have to start making notes what are line endings in my sources? Do I have to put {$MultiLineStringEnding PLATFORM} in all files just because anyone can accidentally change line endings in my sources and by doing that make output of my program completely different?

Perhaps {$MultiLineStringTrim}, without parameter, should be made to mean "Trim All"?
{$MultiLineStringTrim All} sounds ok.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Edson on July 12, 2019, 12:47:39 am
Good. Maybe not the best implementation, but it's a nice feature I always ask to FPC: https://forum.lazarus.freepascal.org/index.php/topic,22777.msg135091.html#msg135091
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 01:12:51 am
Maybe not the best implementation

I'm happy to hear any suggestions you might have!

Do note again though that there were specific "must-haves" and "must-not-dos" that I was required to consider when writing it in order for it to have any chance of being accepted as a patch.

Can you see how much of a problem that can be? Do I have to start making notes what are line endings in my sources? Do I have to put {$MultiLineStringEnding PLATFORM} in all files just because anyone can accidentally change line endings in my sources and by doing that make output of my program completely different?

I think you're vastly overstating the amount of scenarios where this would be a problem, and also expecting the compiler to essentially babysit your revision control configuration. The compiler directives are directives. Use them as you wish, like you would any existing directive.

Note once more, also: this feature is not hypothetical. It works, and you can test it right now. Please feel free to do so and give actual feedback based on actual use.

I don't really like this feature (too many possible "unexpected" behaviours) but I see why someone may want to use it.

What do you see as unexpected, exactly? You type in text. The compiler reads the text. The end. It's a very, very, simple feature.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Edson on July 12, 2019, 01:37:07 am
Maybe not the best implementation

I'm happy to hear any suggestions you might have!

As I posted in the link, I think multilines string could be:

Code: Pascal  [Select]
  1. s='hello
  2. world'
  3.  

Leading spaces, and appearence could work of the editor/IDE.

Anyway, directives could help on setting spaces and line endings.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 01:50:04 am
As I posted in the link, I think multilines string could be:
Code: Pascal  [Select]
  1. s='hello
  2. world'
  3.  

It already is though? Just with backticks instead of single quotes.

Code: Pascal  [Select]
  1. const s = `hello
  2. world`;

I've attached an image (that I also posted earlier) of what it looks like with a patched version of SynEdit in Lazarus.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 12, 2019, 02:28:07 am
I've attached an image (that I also posted earlier) of what it looks like with a patched version of SynEdit in Lazarus.

Ah ... Off-topic but ... for future reference, would you mind cropping your images to the minimum needed for the purpose? In this case, maybe, screenshooting just the editor window?

Some of us still use (relatively) small screens, so the less scrolling the better :-[

ETA - Oh, didn't see this:
What do you see as unexpected, exactly? You type in text. The compiler reads the text. The end. It's a very, very, simple feature.

I didn't mean in the feature itself--although that too regarding the EOLs, trimming, etc.--but the possibilities of I myself (and people like me, few as there are) making lots of mistypings, and  the potentially multiple cycles of "compile-correct typo-compile- correct-typo-..." which tend to multiply inordinately the number of files in the "backup" folder. Note that in most non-US keyboards the "backtick" is a tilde key (i.e. a dead key), so it's easy to mistype it.

I kind of like the feature per se; only I would prefer some other method, like an operator or something like that. After all it's almost only a "want-to type-less" feature.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 12, 2019, 03:29:29 am
The problem here is that allowing multi lines for strings building is great, most editors allow it and most
compilers allow it but I don't like the idea of it changing the content of the string I specify..

 Changing the content means removing left spaces, inserting CR etc... that isn't ideal..

 I made a mini editor as a tool to create a string using multi-lines and then I have a button that translate
it to a C or Pascal string.., place it on the clipboard and drop it in the source editor.

 Its all human readable and the compiler accepts it. I keep my edited part of the text in the mini editor as a
data base for the current project...
 it works out well.
 
 I think Lazarus could benefit from having a code tool like that..
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 12, 2019, 03:29:56 am
After all it's almost only a "want-to type-less" feature.
Personally, I think it's not even "almost", that's exactly what it is.

I have mixed feelings about that feature.  There are things I like about it and things I don't like.

I like the clean, uncluttered look of the multiline string.

I am not fond of the need to have a new token in the language (presumably "(#") and, I like even less the need to have  directives to control the trimming of the strings.   Most, if not all the savings in typing, go out the window with those directives.

Here is a thought, it's a small variation on the concept Akira has presented, using current Pascal syntax:
Code: Pascal  [Select]
  1.  const
  2.    s = 'hello ' +
  3.        'world'  ;    // would print "hello world" (just as it does now.)
  4.  
  5.    {$LINEBREAKS ON}
  6.    b = 'hello'  +
  7.        'world'  ;
  8.  
  9.    // would print (imagine "// " is the beginning of the line)
  10.    // hello
  11.    // world
  12.  

{$LINEBREAKS would be active ONLY for 1 string, the very next one, to prevent having unintended line breaks in other strings.  I think this is a feature that is used rarely enough that having to type {$LINEBREAKS ON} for each one of them is within reason.

Basically, teach a new "trick" to the "+" pony.

That method would make "xxxTRIM" directives unnecessary since the programmer would still be responsible to put whatever number of spaces he/she desires within the quotes.  It also gives the programmer very precise control, which may be needed/wanted when the purpose is to have fixed-length strings on separate lines.

Those are my two (2) counterfeit cents on the subject.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 03:39:17 am
Akira, thank you for working on this feature.

I saw a problem. It could be the way I applied the patch, as I was eager to test it so I was not careful enough. So just in case, this code:
Code: Pascal  [Select]
  1. program example;
  2.  
  3. {$modeswitch MultiLineStrings}
  4. {$MultiLineStringTrimLeft 5}
  5.  
  6. const Test = `ThisSpace>>>>     <<<<diappeared`;
  7.  
  8. begin
  9.   Write(Test);
  10. end.

Removed the space in the middle and gave me:
Code: Text  [Select]
  1. ThisSpace>>>><<<<diappeared

Do you get the same result?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 12, 2019, 03:42:09 am
My little mini editor makes nice like formatted paragraphs  and fits it to  pascal editor/compiler format.

Its just a click and drop in the editor with all the needed + for multilines etc..
I suppose an open quote that carries to the next line would be nice but that is the extent of it.

 I think maybe this feature is better suited for an option in the IDE not the compiler.. a simple little memo type
editor to build your bulk lines in a design look.

 Its gets hard to read code at times with strings going out side the editor window on smaller screens.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 12, 2019, 03:45:32 am
Most likely the editor in the options list is setup to remove the trailing spaces, so basically it broke the
string up and there you...
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 03:55:08 am
Akira, thank you for working on this feature.

I saw a problem. It could be the way I applied the patch, as I was eager to test it so I was not careful enough. So just in case, this code:
Code: Pascal  [Select]
  1. program example;
  2.  
  3. {$modeswitch MultiLineStrings}
  4. {$MultiLineStringTrimLeft 5}
  5.  
  6. const Test = `ThisSpace>>>>     <<<<diappeared`;
  7.  
  8. begin
  9.   Write(Test);
  10. end.

Removed the space in the middle and gave me:
Code: Text  [Select]
  1. ThisSpace>>>><<<<diappeared

Do you get the same result?

Oh no! That is in fact a bug. Looking at my FPC-test-suite tests now, that particular manner of "single-line-like" use seems to be the one thing I overlooked, syntactically speaking. I will fix that literally right now. Thanks for the report, and for testing!

Most likely the editor in the options list is setup to remove the trailing spaces, so basically it broke the
string up and there you...

No. It's an oversight in my implementation of the feature, which is entirely a part of FPC's scanner logic. It has absolutely nothing to do with Lazarus, at all.

I am not fond of the need to have a new token in the language (presumably "(#") and, I like even less the need to have  directives to control the trimming of the strings. Most, if not all the savings in typing, go out the window with those directives.

Where are you getting the pound sign from? The syntax is exactly what's written in my example.

I like even less the need to have  directives to control the trimming of the strings

I personally don't even think there is a need so to speak. I added that because people seemed to really be concerned about maintaining "pretty" indentation while not having spaces actually appear in the string when displayed / written to file / e.t.c. at runtime.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 03:56:21 am
Most likely the editor in the options list is setup to remove the trailing spaces, so basically it broke the
string up and there you...

I just opened the file with a different editor, the space is there, or did I misunderstand you.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 12, 2019, 04:03:50 am
No you didn't miss understand me, I screwed that one up, I guess the editor in  your case makes
no difference but control characters and UTF8 need to be tested.

 For me all this is going to do is remove the use of the '+ at the end of the line.

 that is all I'll ever use it for.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 04:07:22 am
@jamie, do you use LineEnding anywhere in your code?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 12, 2019, 04:36:26 am
Where are you getting the pound sign from? The syntax is exactly what's written in my example.
I admit to having lost track of how you've implemented the feature.  At the time I wrote that, I was thinking about the discussion in FPC-devel which used "(##" (if I'm not mistaken again.)

I personally don't even think there is a need so to speak. I added that because people seemed to really be concerned about maintaining "pretty" indentation while not having spaces actually appear in the string when displayed / written to file / e.t.c. at runtime.
I understand.  The one thing that concerns me, if I've understood the description correctly, is that with that construct, there is no way to have fixed-length multi-line strings.    I think the ability of having fixed-length multi-line strings should not be lost (it can be done using current syntax but, obviously more typing and more clutter but, it's there, it's available if you need it.)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 04:42:50 am
Akira, thank you for working on this feature.

I saw a problem. It could be the way I applied the patch, as I was eager to test it so I was not careful enough. So just in case, this code:
Code: Pascal  [Select]
  1. program example;
  2.  
  3. {$modeswitch MultiLineStrings}
  4. {$MultiLineStringTrimLeft 5}
  5.  
  6. const Test = `ThisSpace>>>>     <<<<diappeared`;
  7.  
  8. begin
  9.   Write(Test);
  10. end.

Removed the space in the middle and gave me:
Code: Text  [Select]
  1. ThisSpace>>>><<<<diappeared

Do you get the same result?

Oh no! That is in fact a bug. Looking at my FPC-test-suite tests now, that particular manner of "single-line-like" use seems to be the one thing I overlooked, syntactically speaking. I will fix that literally right now. Thanks for the report, and for testing!

Ok, I've now fixed this, and added your example to my "FPC test suite folder" tests with the filename "tmultilinestring22.pp".

Literally a one-line fix, believe it or not! (A boolean value needed to be set to false in a specific place, but was not.)

I've pushed the changes to my github fork, and also uploaded a new pair of patch files on the bugtracker issue.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 04:51:17 am
That was quick, thank you!!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 04:56:47 am
I admit to having lost track of how you've implemented the feature.  At the time I wrote that, I was thinking about the discussion in FPC-devel which used "(##" (if I'm not mistaken again.)

Ah, that was my very initial suggestion, yes. Backtick was decided as being better quite quickly though. And it is, in hindsight, by quite a lot, especially from an implementation standpoint.

I understand.  The one thing that concerns me, if I've understood the description correctly, is that with that construct, there is no way to have fixed-length multi-line strings.    I think the ability of having fixed-length multi-line strings should not be lost (it can be done using current syntax but, obviously more typing and more clutter but, it's there, it's available if you need it.)

Multi-line strings are just strings in every way. So the following is valid code with this feature, that works like you'd expect:

Code: Pascal  [Select]
  1. type StringX = String[7];
  2.  
  3. const S: StringX = `
  4.  2
  5.  34
  6. `;

I even made sure in my tests that things like multi-line deprecation messages:

https://github.com/Akira13641/freepascal/blob/83def7b8ab3aabbe748bd127cf92ede7b5f1447c/tests/test/tmultilinestring18.pp (https://github.com/Akira13641/freepascal/blob/83def7b8ab3aabbe748bd127cf92ede7b5f1447c/tests/test/tmultilinestring18.pp)

and multi-line dispatch strings:

https://github.com/Akira13641/freepascal/blob/83def7b8ab3aabbe748bd127cf92ede7b5f1447c/tests/test/tmultilinestring17.pp (https://github.com/Akira13641/freepascal/blob/83def7b8ab3aabbe748bd127cf92ede7b5f1447c/tests/test/tmultilinestring17.pp)

work properly.

That was quick, thank you!!

No problem.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 06:18:46 am
I think I hit another bug. It doubles the line ending.

The following code:
Code: Pascal  [Select]
  1. program doubleLineEndingBug;
  2.  
  3. {$mode objfpc}
  4. {$modeswitch MultiLineStrings}
  5. {$MultiLineStringTrimLeft 15}
  6. {$MultiLineStringLineEnding Platform}
  7.  
  8. var
  9. {$MultiLineStringLineEnding CR}
  10.   a: array[0..3] of string = (
  11. ``
  12. ,
  13. `
  14. `
  15. ,
  16. `
  17.  
  18. `
  19. ,
  20. `
  21.  
  22.  
  23. `);
  24.  
  25.   {$MultiLineStringLineEnding CRLF}
  26. b: array[0..3] of string = (
  27. `1`
  28. ,
  29. `1
  30. 2`
  31. ,
  32. `1
  33. 2
  34. 3`
  35. ,
  36. `1
  37. 2
  38. 3
  39. 4`);
  40.  
  41. procedure Test(StrArray:array of string);
  42. var
  43.   s,sHex: string;
  44.   c: char;
  45. begin
  46.   for s in StrArray do
  47.   begin
  48.     WriteLn('Length: ',Length(s));
  49.     sHex := ``;
  50.     for c in s do
  51.       sHex := sHex+`$`+hexStr(ord(c),2)+` `;
  52.     WriteLn(sHex);
  53.   end;
  54.   WriteLn('---------------');
  55. end;
  56.  
  57. begin
  58.   Test(a);
  59.   Test(b);
  60. end.

Gives for array a:
Code: Text  [Select]
  1. Length: 0
  2.  
  3. Length: 2
  4. $0D$0D
  5. Length: 4
  6. $0D$0D$0D$0D
  7. Length: 6
  8. $0D$0D$0D$0D$0D$0D

I expected it to be:
Code: Text  [Select]
  1. Length: 0
  2.  
  3. Length: 1
  4. $0D
  5. Length: 2
  6. $0D$0D
  7. Length: 3
  8. $0D$0D$0D

and for the second array b it also doubles the line endings:
Code: Text  [Select]
  1. Length: 1
  2. $31
  3. Length: 6
  4. $31$0D$0A$0D$0A$32
  5. Length: 11
  6. $31$0D$0A$0D$0A$32$0D$0A$0D$0A$33
  7. Length: 16
  8. $31$0D$0A$0D$0A$32$0D$0A$0D$0A$33$0D$0A$0D$0A$34

See the second string in this array:
`1
2`

Turned to be:
$31$0D$0A$0D$0A$32

Assuming I have the trunk patched right, of course.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 06:59:46 am
I think I hit another bug. It doubles the line ending.

I get your expected results with an LF encoded file, but not with a CRLF one. Let me take a closer look at what's going on there. Thanks for really digging in to the testing!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 12, 2019, 08:23:40 am
Can you see how much of a problem that can be? Do I have to start making notes what are line endings in my sources? Do I have to put {$MultiLineStringEnding PLATFORM} in all files just because anyone can accidentally change line endings in my sources and by doing that make output of my program completely different?
I think you're vastly overstating the amount of scenarios where this would be a problem, and also expecting the compiler to essentially babysit your revision control configuration. The compiler directives are directives. Use them as you wish, like you would any existing directive.
Are you saying that people using GIT or SVN is an overstated scenario? I do not expect compiler to babysit my strings, I expect it to be in line with current behavior. Current behavior is that if you have String1 in SomeStringList and add String2 to SomeStringList, then SomeStringList will have different line endings in Windows and Linux, and SomeStringList will be printable on both without any changes. I expect the same behavior form this new feature, and anything else is in collision with what are we used to. Otherwise we will have a code that might produce unusable strings on some platforms unless we tell it explicitly to the compiler that we don't want that. And it doesn't matter if we have to put a project wide switch or unit wide switch. Both are bad.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 12, 2019, 09:45:31 am
Here is a thought, it's a small variation on the concept Akira has presented, using current Pascal syntax:
Code: Pascal  [Select]
  1.  const
  2.    s = 'hello ' +
  3.        'world'  ;    // would print "hello world" (just as it does now.)
  4.  
  5.    {$LINEBREAKS ON}
  6.    b = 'hello'  +
  7.        'world'  ;
  8.  
  9.    // would print (imagine "// " is the beginning of the line)
  10.    // hello
  11.    // world
  12.  

{$LINEBREAKS would be active ONLY for 1 string, the very next one, to prevent having unintended line breaks in other strings.  I think this is a feature that is used rarely enough that having to type {$LINEBREAKS ON} for each one of them is within reason.

Basically, teach a new "trick" to the "+" pony.
This would mean that the parser would need to handle new lines which it currently is totally ignorant of. The multiline string implemented  by Akira1364 is solely implemented inside the scanner which is already dealing with lines and such. Thus your idea is much more intrusive for the compiler and thus harder to maintain in the long run and thus much less likely to get accepted than Akira1364's idea.

No you didn't miss understand me, I screwed that one up, I guess the editor in  your case makes
no difference but control characters and UTF8 need to be tested.

 For me all this is going to do is remove the use of the '+ at the end of the line.

 that is all I'll ever use it for.
You are aware that the feature replaces more than just the +?
Code: Pascal  [Select]
  1. const
  2.   Str1 = 'Currently a String ' + sLineBreak +
  3.          'with a linebreak';
  4.  
  5.   Str2 = `A new multi
  6.           line string`;
(yes, I'm aware that there'd be spaces between the start and line if the trim option is not used)

The problem here is that allowing multi lines for strings building is great, most editors allow it and most
compilers allow it but I don't like the idea of it changing the content of the string I specify..

 Changing the content means removing left spaces, inserting CR etc... that isn't ideal..

Look at this code:
Code: Pascal  [Select]
  1. procedure Something;
  2. begin
  3.   if whatever then
  4.     if somethingelse then
  5.       somestring := `Some multiline
  6.                      string`;
  7. end;
Without the ability to trim spaces on the left I'd either have a bunch of spaces before string or I'd need to write it like this:
Code: Pascal  [Select]
  1. procedure Something;
  2. begin
  3.   if whatever then
  4.     if somethingelse then
  5.       somestring := `Some multiline
  6. string`;
  7. end;
Which is ugly as hell.

And the ability to control the new line characters is important as well cause in normal strings you can control it by inserting the corresponding characters manually (see the sLineBreak above)

Having PLATFORM as the default would be much less expected IMO, and likely frustrating for anyone who did not want the compiler to impose a certain line ending that may have been not what was actually written.
Considering that the idea is to replace StrConst = 'Foo' + sLineBreak + 'Bar' making PLATFORM the default might indeed be the most sensible approach.

Another thing regarding the left trimming: I think you wrote that the trim option will trim spaces of the first line, but in the example mentioned by avra there was this:
Code: Pascal  [Select]
  1. {$MultiLineStringTrimLeft 40}
  2.  
  3. const Y = `A
  4.            B
  5.            C
  6.            D`;
Do I understand correctly that this then wouldn't be trimmed at all, because the first line doesn't have any spaces?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: BrunoK on July 12, 2019, 10:19:17 am
I would gladly use something like :
Code: Pascal  [Select]
  1.     const
  2.       Str2 = `SELECT o.*, C.Company
  3.               from Orders O
  4.               join Customer C
  5.                 on o.CustNo=C.ID
  6.               where
  7.                 O.saledate=DATE '2001.03.20'`;
  8. or
  9.  
  10.     const
  11.       Str2 =
  12.         `SELECT o.*, C.Company
  13.          from Orders O
  14.          join Customer C
  15.            on o.CustNo=C.ID
  16.          where
  17.            O.saledate=DATE '2001.03.20'`;
  18.  
would be interpreted by the scanner as  :
Code: Pascal  [Select]
  1.       Str2 = 'SELECT o.*, C.Company' + LineEnding +
  2.              'from Orders O' + LineEnding +
  3.              'join Customer C' + LineEnding +
  4.              '  on o.CustNo=C.ID' + LineEnding +
  5.              'where' + LineEnding +
  6.              '  O.saledate=DATE ''2001.03.20''';
  7.  
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: JernejL on July 12, 2019, 10:24:29 am
An unpopular suggestion: What about simply introducing an alternate syntax system for strings?
 
php has single quoted and double quoted strings:
https://www.php.net/manual/en/language.types.string.php
 
The difference is, that single quote strings are as-is (kinda like pascal strings) while double-quote strings support escape sequences and other magical things including multiline behavior.

 
If we could simply introduce secondary string syntax, it could conform and immitate this - already seen behavior, we could then also add things like escape sequences to whole thing - something present in practically every single other language out there.
 
This would also make double quoted strings that would be practically identical in how they'd behave in C-languages, making c code portability even easier.
 
And the best part is, this can simply not break any other thing in pascal syntax, double quotes are simply not used for anything currently.
 
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 02:50:48 pm
Considering that the idea is to replace StrConst = 'Foo' + sLineBreak + 'Bar' making PLATFORM the default might indeed be the most sensible approach.

If that's what people really want, I'm not too picky either way.

Do I understand correctly that this then wouldn't be trimmed at all, because the first line doesn't have any spaces?

You do not I'm afraid. It is trimmed like you'd expect (so, in that case, completely trimmed such that there is no leading space left anywhere.)

I.E. it would look like this if displayed with WriteLn:

Code: Pascal  [Select]
  1. A
  2. B
  3. C
  4. D

Upon encountering characters #32, #9, or #11, my code in the scanner checks two boolean values to determine whether or not to trim, which it toggles between true and false where appropriate: "first_multiline" (meaning we've just seen an opening backtick) and also "had_newline" (meaning we've just seen a newline character and are at the very beginning of the next line.)

The main block of code relevant to that is simply this:

Code: Pascal  [Select]
  1. #32,#9,#11 :
  2.   if (had_newline or first_multiline) and (current_settings.whitespacetrimcount > 0) then
  3.     begin
  4.       trimcount:=current_settings.whitespacetrimcount;
  5.       while (c in [#32,#9,#11]) and (trimcount > 0) do
  6.         begin
  7.           readchar;
  8.           dec(trimcount);
  9.         end;
  10.       had_newline:=false;
  11.       first_multiline:=false;
  12.       goto quote_label;
  13.     end;

"quote_label" there being exactly the top of the case statement containing that branch (which must be re-entered after trimming in order to properly avoid EOF.)

I think I hit another bug. It doubles the line ending.

I get your expected results with an LF encoded file, but not with a CRLF one. Let me take a closer look at what's going on there. Thanks for really digging in to the testing!

Ok, I've identified the issue with your second bug-find here. The problem is the specific combination of "MultiLineStringLineEnding CRLF" with a source file that is itself already CRLF (or "MultiLineStringEnding PLATFORM" when "target_info.newline" is specifically #13#10, with a file that is already itself CRLF).

After reviewing it, I realized that my code does not currently quite properly account for a contiguous sequence of #13 and #10, one after another, in those cases, and so both the #13 and #10 are replaced with a full #13#10.

("MultiLineStringLineEnding RAW" is not affected by this regardless of the operating system or the nature of the file, however, because it does not ever intentionally write more than one character at a time.)

I'm fairly confident that I know what the best way to remedy this is, but I might not have a chance to do so until later today (am at work right now, haha.) As soon as I can though, I'll fix it, push the changes to github, and upload another new pair of patches.

Thanks again for really trying to find the edge cases! It's precisely the kind of thing I was hoping for when I asked for testers. Hopefully that's the last big-ish bug, but of course let me know if you find anything else.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 03:14:00 pm
I'm fairly confident that I know what the best way to remedy this is, but I might not have a chance to do so until later today (am at work right now, haha.) As soon as I can though, I'll fix it, push the changes to github, and upload another new pair of patches.
Take your time. I am sure you can find the best way to correct it. Thank you for putting all that effort into this nice feature.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 12, 2019, 06:10:06 pm
This would mean that the parser would need to handle new lines which it currently is totally ignorant of. The multiline string implemented  by Akira1364 is solely implemented inside the scanner which is already dealing with lines and such. Thus your idea is much more intrusive for the compiler and thus harder to maintain in the long run and thus much less likely to get accepted than Akira1364's idea.
First, most important, you may very well be right since I have not spent any real amount of time analyzing the ways the feature can be implemented.

That said, the idea behind {$LINEBREAKS ON} is to direct the scanner to a new routine that handles multiline strings.  What Akira triggers with a ` (backtick), I would trigger with a directive.   Just as simple, just a different path to getting to the same place.

At least at first blush, there doesn't seem to be anything preventing such an implementation and it would not require adding new elements (such as the backtick) to the language.

I threw that out there, not even as a suggestion but, as an idea in case keeping syntax just the way it currently is would be appealing to those who would really like to see the feature implemented.

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lainz on July 12, 2019, 09:11:08 pm
I like the idea, didn't read all the opinions btw.

Also I like the backtick usage, is like is done in JavaScript for example.

Keep going and get it bug free in time.  :)

in javascript
https://flaviocopes.com/how-to-create-multiline-string-javascript/

and template literals can be good to have too
https://flaviocopes.com/javascript-template-literals/
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Thaddy on July 12, 2019, 09:14:10 pm
I am never going to use it because I really do not see the point. (Even after reading all discussions on the mailing list)
You can't solve bad programming with a compiler switch.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 12, 2019, 09:26:47 pm
You can't solve bad programming with a compiler switch.
That's true but, this feature isn't about solving a "bad programming" problem.

@BrunoK above gave a very nice example of how multiline strings make a SQL statement much cleaner and easier to read.

Depending on context, it could be a feature used very little or a whole lot (for instance when interacting with a SQL database.)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 12, 2019, 10:44:59 pm
I am beginning to think they want Fpc to be a database compiler.., this aint Cobol.


For once Thaddy actually said something that I can agree with.. I don't want it modifying my
strings..

 Multi lines is a great idea for the compiler to read but I want it to assemble exactly what it see's …

 If formatted text within the source code is a desired then maybe a utility tool to paste the results
in after the fact.

I was just looking at the prices for the latest Embarcadero Delphi products (professional). I guess I
could swing it but I find looking for forums like we have with Lazarus is not showing much fruit..

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 12, 2019, 10:49:11 pm
For once Thaddy actually said something that I can agree with..
When that happens, the probability of being wrong is infinitesimally close to being 1.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 12, 2019, 11:48:17 pm
I would gladly use something like :
Code: Pascal  [Select]
  1.     const
  2.       Str2 = `SELECT o.*, C.Company
  3.               from Orders O
  4.               join Customer C
  5.                 on o.CustNo=C.ID
  6.               where
  7.                 O.saledate=DATE '2001.03.20'`;
  8. or
  9.  
  10.     const
  11.       Str2 =
  12.         `SELECT o.*, C.Company
  13.          from Orders O
  14.          join Customer C
  15.            on o.CustNo=C.ID
  16.          where
  17.            O.saledate=DATE '2001.03.20'`;
  18.  
would be interpreted by the scanner as  :
Code: Pascal  [Select]
  1.       Str2 = 'SELECT o.*, C.Company' + LineEnding +
  2.              'from Orders O' + LineEnding +
  3.              'join Customer C' + LineEnding +
  4.              '  on o.CustNo=C.ID' + LineEnding +
  5.              'where' + LineEnding +
  6.              '  O.saledate=DATE ''2001.03.20''';
  7.  

That works just as you've written it already, BTW:

Code: Pascal  [Select]
  1. program SQL;
  2.  
  3. {$modeswitch MultiLineStrings}
  4.  
  5. {$MultiLineStringTrimLeft 10}
  6.  
  7. const
  8.   Str1 = `SELECT o.*, C.Company
  9.           from Orders O
  10.           join Customer C
  11.             on o.CustNo=C.ID
  12.           where
  13.             O.saledate=DATE '2001.03.20'`;
  14.  
  15. {$MultiLineStringTrimLeft 5}
  16.  
  17. const
  18.   Str2 =
  19.     `SELECT o.*, C.Company
  20.      from Orders O
  21.      join Customer C
  22.        on o.CustNo=C.ID
  23.      where
  24.        O.saledate=DATE '2001.03.20'`;
  25.  
  26. begin
  27.   WriteLn(Str1);
  28.   WriteLn(Str2);
  29. end.

I like the idea, didn't read all the opinions btw.

Also I like the backtick usage, is like is done in JavaScript for example.

Keep going and get it bug free in time.  :)

Thanks! I intend to. Don't think we're too far out. I should have the second bug engkin reported fixed later tonight.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 11:49:46 pm
Multi lines is a great idea for the compiler to read but I want it to assemble exactly what it see's …
Where did you get this idea from? This feature does *not* affect any strings in your old source code, and will not change strings in your new source code. Moreover, it is turned off by default. To turn it on, you need to use {$modeswitch MultiLineStrings}. Even then, it will *not* change the strings that you already have.

Your misunderstanding probably comes from MultiLineStringLineEnding. As is today, it leaves the line endings the way they are. It is default value is {$MultiLineStringLineEnding RAW} which does not make any changes to line endings. The other possibility of your misunderstanding comes from another switch MultiLineStringTrimLeft, which, again, by default it does nothing. You need to use either switch to affect multi-line strings, and please notice they only affect strings enclosed in the newly introduced character *backtick* ` usually found on the same key with the tilde mark ~.

As you can see, it will not affect the usual Pascal strings enclosed in between two apostrophe characters:

'**This will not get affected**'  <--- your usual string

It only affects the new string enclosed in between two backticks:

`This might get affected`  <-- the new string


Let me zoom in --> Apostrophe '  Backtick `
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: kupferstecher on July 12, 2019, 11:50:57 pm
Typing 's and +es each line is really no fun. And it's not only about typing, the (necessary) formatting disturbes my thinking.
BUT, in the cases a singleline is not enough for me, I often don't want a line break to be inserted.
So this new syntax wouldn't cover my cases.

Existing:
Code: Pascal  [Select]
  1. const
  2.   HelpText = 'This is some example text like it may look like in a help text. There is no'
  3.            + ' content here, you don''t need to go on reading. Just a string that doesn''t'
  4.            + ' fit into one line.'+#13+#10
  5.            + 'Now this is a new line.';

Akira's syntax
Code: Pascal  [Select]
  1. const
  2.   HelpText = `I''m not sure if it is even possible to have such a long line. This is some example text like it may look like in a help text. There is no content here, you don''t need to go on reading. Just a string that doesn''t fit into one line.
  3.               Now this is a new line.`;

Something I'd prefer (## for new line)
Code: Pascal  [Select]
  1. const
  2.   HelpText = 'About the trims I''m not sure. I could live with it to bring it to the front
  3.              of the line, as such strings are normally as constants in the interface section
  4.              and not distributed throughout the code.##
  5.              Now this is a new line';
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 12, 2019, 11:59:02 pm
@kupferstecher,

Akira's syntax
Code: Pascal  [Select]
  1. const
  2.   HelpText = `I''m not sure if it is even possible to have such a long line. This is some example text like it may look like in a help text. There is no content here, you don''t need to go on reading. Just a string that doesn''t fit into one line.
  3.               Now this is a new line.`;
You don't need to escape the apostrophe if used in between two backticks. But you do need to escape the backtick if you need it in this text:
Code: Pascal  [Select]
  1.   HelpText = `I'm not escaping this apostrophe ' but I have to escape this back tick```
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 13, 2019, 12:01:56 am
BUT, in the cases a singleline is not enough for me, I often don't want a line break to be inserted.
So this new syntax wouldn't cover my cases.

Several people have mentioned wanting a "no newlines at all" option for {$MultiLineStringLineEnding} (which would mean whatever is written would be interpreted as one long horizontal string), which would not be difficult to add, and so I may very well do so. The option would most likely be called NONE, I think.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 13, 2019, 12:36:35 am
Several people have mentioned wanting a "no newlines at all" option for {$MultiLineStringLineEnding} (which would mean whatever is written would be interpreted as one long horizontal string), which would not be difficult to add, and so I may very well do so. The option would most likely be called NONE, I think.

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 13, 2019, 01:06:58 am
This is pure curiosity...

why the backtick (which is slightly awkward to reach on the keyboard) and not simply a double quote ?  (which is really easy to reach)

The only reason I can see for not choosing the double quote is that it could potentially be a source of confusion for programmers migrating from another language.



Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: GAN on July 13, 2019, 01:07:23 am

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?

No.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lainz on July 13, 2019, 01:13:16 am
This is pure curiosity...

why the backtick (which is slightly awkward to reach on the keyboard) and not simply a double quote ?  (which is really easy to reach)

The only reason I can see for not choosing the double quote is that it could potentially be a source of confusion for programmers migrating from another language.

````````````

I just press a single key both on windows and mac, english and spanish keyboard as well.

but to double quote i need to use shift """"""""""""""""  :D
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 13, 2019, 01:21:11 am
This is pure curiosity...

why the backtick (which is slightly awkward to reach on the keyboard) and not simply a double quote ?  (which is really easy to reach)

Michael proposed it (https://www.mail-archive.com/fpc-devel@lists.freepascal.org/msg38347.html).

Probably escaping double quote would be more frequent if it were to be used instead of backtick:
Code: Pascal  [Select]
  1.   Sample = """It is unfair"" he said.";
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 13, 2019, 01:22:43 am
I just press a single key both on windows and mac, english and spanish keyboard as well.

Huh?  Which "Spanish" keyboards? On all of mine throughout the ages the "backtick" (acute? tilde) has always been a dead-key so it has to be pressed and then you have to press the space-bar (system-dependent, this) to obtain the tilde by itself.

Presumably so we poor Spaniards can write French with some easy, I guess :D
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lainz on July 13, 2019, 01:40:24 am
Chinese assembled Spanish keyboard  :)

Really I press the backtick always I want to add a tilde, I'm used to American Spanish keyboard not international Spanish keyboard (I currently have this one).
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 13, 2019, 01:43:57 am
Michael proposed it (https://www.mail-archive.com/fpc-devel@lists.freepascal.org/msg38347.html).
<snip>
Probably escaping double quote would be more frequent if it were to be used instead of backtick:
Code: Pascal  [Select]
  1.   Sample = """It is unfair"" he said.";
Thanks.  The rational for it, know that I know it, seems quite reasonable.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 13, 2019, 04:34:37 am
Ok! I've now fixed the second bug engkin reported. The changes have been pushed to my github fork-branch of the compiler, and once again I've uploaded a new pair of patch files on my bugtracker issue for the feature (including engkin's example of the second bug, as a test file to ensure the problem stays fixed, with the name "tmultilinestring24.pp".)

Again, if you're someone who doesn't mind building FPC from source, please feel to free to apply the patch, test the feature, and report any other bugs you might find! (Which of course is hopefully few, or even better none, haha.)

Extra bonus: attributes were merged into trunk FPC today. Of course, once I noticed this, I right away wanted to see how they work with multi-line strings. Turns out: just fine! And they look right at home together in (my patched) Lazarus, if you ask me (see the attached image.)

Doesn't that quite defeates the whole purpose of this feature?

Not necessarily. I could see certain things being much easier to format without being separated with single-quotes and plus signs, while not specifically requiring newlines in the output.

Also, again, it's trivial enough to add that I see no reason not to.
Title: Re: If you’ve ever wanted to be able to use multi-line strings in FPC...
Post by: Kays on July 13, 2019, 12:24:25 pm
Michael proposed it (https://www.mail-archive.com/fpc-devel@lists.freepascal.org/msg38347.html).
<snip>
Probably escaping double quote would be more frequent if it were to be used instead of backtick:
Code: Pascal  [Select]
  1.   Sample = """It is unfair"" he said.";
Thanks.  The rational for it, know that I know it, seems quite reasonable.
Also, " is already known in the (Pascal) programming universe as a string delimiter of strings containing escaped characters. (E. g. the GNU Pascal compiler allowed that.)

I still don’t understand, why we just don’t introduce a new compiler switch, that’ll eliminate the
Code: Text  [Select]
  1. Fatal: String exceeds line
error. No need for new syntax and achieved our goal with minimal efforts.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: BrunoK on July 13, 2019, 12:41:18 pm
That works just as you've written it already, BTW:
Maybe  a {$MultiLineStringTrimLeft auto} would be convenient. It would decide, based on the position of the first backtick, what is the  count of character to consider in the {$MultiLineStringTrimLeft n_characters}. It could also check that there aren't any characters under/left  of the initial backtick.

Changing the paragraph horizontal (sorry could not find the word in english) 'retrait de paragraphe'  is pretty frequent during editing and having to correct the n_characters after any change would be a nuisance.

EDIT :

Changing the paragraph horizontal indent is pretty frequent during editing and having to correct the n_characters after any change would be a nuisance.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 13, 2019, 04:01:43 pm
Do I understand correctly that this then wouldn't be trimmed at all, because the first line doesn't have any spaces?

You do not I'm afraid. It is trimmed like you'd expect (so, in that case, completely trimmed such that there is no leading space left anywhere.)

I.E. it would look like this if displayed with WriteLn:

Code: Pascal  [Select]
  1. A
  2. B
  3. C
  4. D
Ah, then it's not what I expected, but what I hoped for. ;)

That works just as you've written it already, BTW:

Code: Pascal  [Select]
  1. program SQL;
  2.  
  3. {$modeswitch MultiLineStrings}
  4.  
  5. {$MultiLineStringTrimLeft 10}
  6.  
  7. const
  8.   Str1 = `SELECT o.*, C.Company
  9.           from Orders O
  10.           join Customer C
  11.             on o.CustNo=C.ID
  12.           where
  13.             O.saledate=DATE '2001.03.20'`;
  14.  
  15. {$MultiLineStringTrimLeft 5}
  16.  
  17. const
  18.   Str2 =
  19.     `SELECT o.*, C.Company
  20.      from Orders O
  21.      join Customer C
  22.        on o.CustNo=C.ID
  23.      where
  24.        O.saledate=DATE '2001.03.20'`;
  25.  
  26. begin
  27.   WriteLn(Str1);
  28.   WriteLn(Str2);
  29. end.
Looking at this I wonder if some {$MultiLineStringTrimLeft Auto} would be useful, so that it would simply automatically trim both statements to look the same...

That said, the idea behind {$LINEBREAKS ON} is to direct the scanner to a new routine that handles multiline strings.  What Akira triggers with a ` (backtick), I would trigger with a directive.   Just as simple, just a different path to getting to the same place.

At least at first blush, there doesn't seem to be anything preventing such an implementation and it would not require adding new elements (such as the backtick) to the language.
No, it's definitely not that simple, because the + operator could be overloaded and as I said all the operator stuff is handled inside the parser not inside the scanner where strings are handled.
Look at this:
Code: Pascal  [Select]
  1. // assume an operator overload for String + Integer is in scope
  2. begin
  3.   somestr := 'str content' +
  4.                    someint;
  5. end.
Your code would break this if you'd do this inside the scanner.
And the parser sees the code as this:
Code: Pascal  [Select]
  1. begin somestr := 'str content' + someint; end.
So it can't (and should) not handle things like new lines.

I still don’t understand, why we just don’t introduce a new compiler switch, that’ll eliminate the
Code: Text  [Select]
  1. Fatal: String exceeds line
error. No need for new syntax and achieved our goal with minimal efforts.
Yeah, sure, a string line with a length of thousands of characters is soo much easier to read...  ::)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 13, 2019, 05:21:32 pm
No, it's definitely not that simple, because the + operator could be overloaded and as I said all the operator stuff is handled inside the parser not inside the scanner where strings are handled.
You're right.  I didn't think about the fact that the "+" operator might be overloaded which forces the processing of the construct to be done by the parser.

Yes, Akira's implementation, without using "+" (or any other operator for that matter) makes implementation simpler and more localized.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 13, 2019, 05:47:22 pm
Looking at this I wonder if some {$MultiLineStringTrimLeft Auto} would be useful, so that it would simply automatically trim both statements to look the same...

A few people have indicated they'd like something like this. Would not be difficult to implement at all, as again the main logic for trimming is simply this, currently:

Code: Pascal  [Select]
  1. #32,#9,#11 :
  2.   if (had_newline or first_multiline) and (current_settings.whitespacetrimcount > 0) then
  3.     begin
  4.       trimcount:=current_settings.whitespacetrimcount;
  5.       while (c in [#32,#9,#11]) and (trimcount > 0) do
  6.         begin
  7.           readchar;
  8.           dec(trimcount);
  9.         end;
  10.       had_newline:=false;
  11.       first_multiline:=false;
  12.       goto quote_label;
  13.     end;

I'm unsure if there is precedent in the compiler for directives that take both numeric values and "string" values, such as would be the case here, however.

I know how to implement it properly, so there's no concern there, but I'm just not sure if it would be considered more "idiomatic FPC compiler code" to have a separate {$MultiLineStringTrimLeftAuto} directive that just took ON or OFF as input.

In the case of having the separate directive, the above code would become something roughly like this (untested for side effects!) block:

Code: Pascal  [Select]
  1. #32,#9,#11 :
  2.   if had_newline or first_multiline then
  3.     begin
  4.       if current_settings.whitespacetrimauto then
  5.         begin
  6.           while (c in [#32,#9,#11]) do
  7.             readchar;
  8.           had_newline:=false;
  9.           first_multiline:=false;
  10.           goto quote_label;
  11.         end
  12.       else if current_settings.whitespacetrimcount > 0 then
  13.         begin
  14.           trimcount:=current_settings.whitespacetrimcount;
  15.           while (c in [#32,#9,#11]) and (trimcount > 0) do
  16.             begin
  17.               readchar;
  18.               dec(trimcount);
  19.             end;
  20.           had_newline:=false;
  21.           first_multiline:=false;
  22.           goto quote_label;
  23.         end;
  24.     end;

In the case of "one directive that took either a number or AUTO", though, the code wouldn't change much at all as AUTO would likely just set current_settings.whitespacetrimcount to the maximum possible numeric value (currently 65535, as right now current_settings.whitespacetrimcount is a word field, chosen because all the four-byte max values seemed excessively huge to me while all the one-byte ones seemed a bit too small.)

This would work correctly and not slow anything down, because of course I wrote it so that the scanner always stops when there's no actual leading whitespace left anyways.

Option three would be "have the separate directive, but still just make it set the existing current_settings.whitespacetrimcount to the highest value possible."

All ways would work, and all are quite easy to pull off. I'd need to add a bit more logic to make it actually base itself on the source position of the first backtick, though. Which I think is possible, but perhaps slightly more "involved" to implement.

Apart from that, do you have any particular thoughts on my implementation as it exists so far? E.G, is there anything that stands out to you as obviously "wrong" / e.t.c? Very interested in any input at all I can get on this to make sure it's as useful as possible in the long run.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 13, 2019, 07:59:44 pm
I'm unsure if there is precedent in the compiler for directives that take both numeric values and "string" values, such as would be the case here, however.
Take a look at for example dir_align.

Apart from that, do you have any particular thoughts on my implementation as it exists so far? E.G, is there anything that stands out to you as obviously "wrong" / e.t.c? Very interested in any input at all I can get on this to make sure it's as useful as possible in the long run.
Sorry, I've been too busy with the reintegration of attributes, so I haven't looked at your patch. Maybe I'll find the time tomorrow, but no promises!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 13, 2019, 08:58:12 pm
Take a look at for example dir_align.

Ah, that's right.

Sorry, I've been too busy with the reintegration of attributes, so I haven't looked at your patch. Maybe I'll find the time tomorrow, but no promises!

Oh yeah, no worries. I just wasn't sure if you had perhaps already looked at it to some extent or not.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 14, 2019, 02:07:40 am
test the feature, and report any other bugs you might find! (Which of course is hopefully few, or even better none, haha.)
Finding a bug is much harder now, but I did manage to find one more, probably this is the last one.

When left-side trimming is enabled, and a string starts with anything but a backtick, it should not trim. Here are a few examples:
Code: Pascal  [Select]
  1. {$MultiLineStringTrimLeft 5}
  2.  
  3. middleSpaceBug: array[0..5] of string = (
  4. #$41` `#$42   //<--- this becomes #$41#$42 instead of #$41#$20#$42
  5. ,
  6. #$41` - `#$42 //<--- this becomes #$41#$2D#$20#$42 instead of #$41#$20#$2D#$20#$42
  7. ,
  8. #$41`      -      `#$42  //<--- one space left instead of six spaces
  9. ,
  10. #$41`      -      `#$42`  --  `  //<---- the same bug twice
  11. ,
  12. '  '#$41` `#$42   //<--- the space between backticks disappears: #$20#$20#$41#$42
  13. ,
  14. ^T' e s'` t`  //<--- last two: $73$74 instead of $73$20$74
  15. );

You can get the expected string by replacing backticks with apostrophe.

Well done, it did take me a while to find this one.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 14, 2019, 02:22:00 am
When left-side trimming is enabled, and a string starts with anything but a backtick, it should not trim. Here are a few examples:

Oh man, I use "literal-pound-sign-character-followed-immediately-by-quoted-string" so rarely that it absolutely did not occur to me to test that. I am glad it works normally with backticks, at least, apart from the trimming issue (although with the good design of FPC's scanner, there's no reason it wouldn't.)

Really good find, thanks so much. This is what open source is all about!

Once again, also, I definitely know how to fix that (now that I'm aware it's a problem), so it shouldn't take very long.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 14, 2019, 02:31:03 am
Without you keeping your Github fork branch updated it would be harder for me to keep testing. Thank you for working on what I believe is the last bug in this feature.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 14, 2019, 03:45:11 am

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?

Second. This syntax is simple and readable.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 14, 2019, 04:02:16 am
I third it …!

  I already have an editing tool that will build that for me, I just paste it in afterwards.

  The tool even reads it back for re-editing..
 
  The only thing I can see that maybe an advantage is for the compiler to carry to the next line looking
for the  closing " ' ".

   But I can see that getting complicated with the ability of inserting quotes etc..

  I can think of something else to be using this valuable time on for compiler enhancements.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 14, 2019, 04:33:06 am
I already have an editing tool that will build that for me, I just paste it in afterwards.

Yeah, I have a mini-editor too that does that and other nifty things. Guess most people end up building one; it's a good (and productive!) exercise.

  I can think of something else to be using this valuable time on for compiler enhancements.

Let's be realistic: whatever someone does with his time is his own affair and if it ends up helping (part of) the community, it's time well spent.

I don't like much this feature but that is basically for personal reasons. I'm one of those persons who goes to add a resource string and forgets to close it or add the terminal ; , in my hurry to go back to the "real" code so I can well imagine myself writing a multiline string and a week later "closing" all those dangling strings without realizing it's a multiline one. or viceversa, trying to do a multiline with the normal '.

I love Pascal in part because it's very "unforgiving" and that forces you to be more careful (or compile three times in a row :)), so I dislike features that make it less so or force *you* to be inordinately attent to sintax.

I know I'm not explaining myself very well, so feel free to disregard all this as the ramblings of an (still not very) old coder. :D
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 14, 2019, 04:35:10 am
I third it …!
I hear you.

  I already have an editing tool that will build that for me, I just paste it in afterwards.

  The tool even reads it back for re-editing..
Why not make it part of Lazarus?
How does it handle line endings on different systems?
What about cross compiling?
Can we see a sample? can you share its source code?
People want their text to stay formatted and beautiful. Can your tool fulfill all the requirements that Akira's multi-line strings are able to do today?

I am really eager to see the results. Can you at least post a screenshot?

That fact that you had to make a tool is by itself a strong indication of a missing feature. Moreover, Lazarus is NOT the only IDE people are using, so adding this function to Lazarus does not solve the problem for others.

 
  The only thing I can see that maybe an advantage is for the compiler to carry to the next line looking
for the  closing " ' ".
And this is a HUGE advantage if you write SQL/HTML/JS/LUA/..etc inside your code.
We are not forced to have long lines of strings, nor forced to add ' + LineEnding + so we can continue on to the next line.

The feature is not intrusive at all. It is off by default.

 
   But I can see that getting complicated with the ability of inserting quotes etc..
Not sure what you mean here. This is even simpler, as you do not need to escape apostrophe. Simple test:
Code: Text  [Select]
  1.   Test1 := `You can use apostrophe ', and double quote " without escaping`;
  2.   Test2 := `You only need to escape backticks, like this one ``, see?`;

 
  I can think of something else to be using this valuable time on for compiler enhancements.
You make it sound like everyone is working on this feature, this is not the case. Here is a list of new features:
https://wiki.freepascal.org/FPC_New_Features_Trunk

Akira spent more time arguing with people than coding. He coded it amazingly in a very short time, and bugs are fixed very quickly.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 14, 2019, 11:33:01 am

Doesn't that quite defeates the whole purpose of this feature? An it can already be done very easily:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' +
  2.            ' in the same line (provided your' +
  3.            ' terminal is wide enaough)';

Do people really find that sintax so unuseable that they need another?
Second. This syntax is simple and readable.
Yes, but the discussion to replace this syntax with backtick-strings only came later. The feature that is really talked about would replace this:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' + sLineBreak +
  2.            ' in the same line (provided your' + sLineBreak +
  3.            ' terminal is wide enaough)';

 
  I can think of something else to be using this valuable time on for compiler enhancements.
You make it sound like everyone is working on this feature, this is not the case. Here is a list of new features:
https://wiki.freepascal.org/FPC_New_Features_Trunk

Akira spent more time arguing with people than coding. He coded it amazingly in a very short time, and bugs are fixed very quickly.
Also keep in mind that Akira1364 is an independant developer who's not a core developer (I'm not trying to belittle his contribution, just to clear up his relationship with the core team to avoid any misunderstandings)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: kupferstecher on July 14, 2019, 01:42:27 pm
Yes, but the discussion to replace this syntax with backtick-strings only came later. The feature that is really talked about would replace this:
Code: Pascal  [Select]
  1. MyString = 'This, of course, will appear all' + sLineBreak +
  2.            ' in the same line (provided your' + sLineBreak +
  3.            ' terminal is wide enaough)';
A string operator could be introduced that combines the "+ sLineBreak +" for example using the "/":
Code: Pascal  [Select]
  1. MyString = 'This will not appear all' /
  2.            ' in the same line (provided you' /
  3.            ' use / instead of +)';

This could be useful even when Akira's approach gets implemented.

--
Several people have mentioned wanting a "no newlines at all" option for {$MultiLineStringLineEnding} (which would mean whatever is written would be interpreted as one long horizontal string), which would not be difficult to add, and so I may very well do so. The option would most likely be called NONE, I think.
In case of trimming a space would have to be inserted for each new line.

--
Quote from: Akira1364
{$MultiLineString[...]}
I would suppose not to use too many mode switches, i.e. not to provide too many options. Trimming on or off should be enough.
Either the format is not important like for the database accesses then a complete trim is suitable, or a intendation is needed for better human readability in the target application then a manual formatting with no trimming at all is imho better than some fancy options where you have to study several pages of documentation to understand whats happening. Keep things simple.



Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 14, 2019, 04:44:04 pm
Why can't we have alternate C escape strings on the fly and simply allow the compiler to assemble the
string line by line until it encounters the closing Quote in use ?

 There is many locations in code where this would benefit everyone..

 I've seen versions of Pascal compilers, not FPC that can do this. it makes it clean looking to insert
line feeds  and also easy to port code from other languages.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 14, 2019, 10:53:24 pm
Ok, engkin's third bug-find is fixed! Same as before: changes have been pushed to my github fork-branch, and there's also a new pair of patches on the bugtracker issue.

Additionally, I've now implemented both "ALL" and "AUTO" for MultiLineStringTrimLeft. To quote myself from a comment I just left on the tracker issue:

"ALL is basically just a more user-friendly way of passing the highest possible value, meaning it just guarantees that literally all leading whitespace is trimmed from each line (essentially by ensuring that the scanner runs out of whitespace to actually trim long before it decrements the temporary local trim count to zero, at which point it of course always stops immediately anyways).

AUTO, however, is based on the column position of the opening backtick of the current multi-line string in the source file, as some people had suggested would be what they'd want."

So, for example, BrunoK's SQL example can now be written like this:

Code: Pascal  [Select]
  1. program tmultilinestring25;
  2.  
  3. {$modeswitch MultiLineStrings}
  4. {$MultiLineStringTrimLeft Auto}
  5.  
  6. // ↓↓↓ interpreted the same as if {$MultiLineStringTrimLeft 10} was set
  7.  
  8. const
  9.   Str1 = `SELECT o.*, C.Company
  10.           from Orders O
  11.           join Customer C
  12.             on o.CustNo=C.ID
  13.           where
  14.             O.saledate=DATE '2001.03.20'`;
  15.  
  16. // ↓↓↓ interpreted the same as if {$MultiLineStringTrimLeft 5} was set
  17.  
  18. const
  19.   Str2 =
  20.     `SELECT o.*, C.Company
  21.      from Orders O
  22.      join Customer C
  23.        on o.CustNo=C.ID
  24.      where
  25.        O.saledate=DATE '2001.03.20'`;
  26.  
  27. begin
  28.   WriteLn(Str1);
  29.   WriteLn(Str2);
  30. end.
  31.  

Which will give identical output for both strings:

Code: Text  [Select]
  1. SELECT o.*, C.Company
  2. from Orders O
  3. join Customer C
  4.   on o.CustNo=C.ID
  5. where
  6.   O.saledate=DATE '2001.03.20'
  7. SELECT o.*, C.Company
  8. from Orders O
  9. join Customer C
  10.   on o.CustNo=C.ID
  11. where
  12.   O.saledate=DATE '2001.03.20'

Hopefully this will make the feature a bit more convenient to use overall.

Lastly, again, if you're someone who is willing / able to do so, please feel free to test the feature and report any bugs you might find!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 15, 2019, 08:58:33 pm
AUTO, however, is based on the column position of the opening backtick of the current multi-line string in the source file, as some people had suggested would be what they'd want."

The AUTO feature does make it more intuitive, and probably less susceptible to typing errors.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 15, 2019, 09:56:29 pm
Having PLATFORM as the default would be much less expected IMO, and likely frustrating for anyone who did not want the compiler to impose a certain line ending that may have been not what was actually written.
Considering that the idea is to replace StrConst = 'Foo' + sLineBreak + 'Bar' making PLATFORM the default might indeed be the most sensible approach.

Using PLATFORM as default seems logical to me.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 15, 2019, 10:45:26 pm
Not a replacement for Akira's very nice work, but perhaps an option for some:

Code: Pascal  [Select]
  1. operator/ (const a, b: string): string;
  2. begin
  3.   result := a + LineEnding + b;
  4. end;
  5.  
  6. procedure TForm1.Button1Click(Sender: TObject);
  7. var
  8.   s: string;
  9. begin
  10.   s :=
  11.     'Hello' /
  12.     'World';
  13.   Memo1.Append(s);
  14. end;  
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 15, 2019, 11:42:56 pm
Using PLATFORM as default seems logical to me.

This seems to be the consensus, or something close to one at least. Will probably change it over to PLATFORM sometime tonight.

I also plan to re-use the variable that tracks the column position of the opening backticks for AUTO to add a multi-line-string specific error, shown in the event that one is "unbalanced" or "unterminated", that tells you where exactly it started (as opposed to the more general syntax error you'd get in that case currently.) Probably the last real addition to the feature I'll do for the time being.

Not a replacement for Akira's very nice work, but perhaps an option for some:

Code: Pascal  [Select]
  1. operator/ (const a, b: string): string;
  2. begin
  3.   result := a + LineEnding + b;
  4. end;
  5.  
  6. procedure TForm1.Button1Click(Sender: TObject);
  7. var
  8.   s: string;
  9. begin
  10.   s :=
  11.     'Hello' /
  12.     'World';
  13.   Memo1.Append(s);
  14. end;  

That works, but I would just... not recommend that you do it, ever. It's adding a non-trivial amount of runtime overhead to your application for no reason (as opposed to single-line or multi-line string concatenation with either the '+' operator or the actual Concat intrinsic function, which is always resolved entirely at compile time and results in a single constant string.)

Take a look at this Compiler Explorer link to get an idea of what I mean. (https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(j:1,lang:pascal,source:'unit+output%3B%0A%0A%7B$mode+ObjFPC%7D%7B$H%2B%7D%0A%0A//+%E2%86%93%E2%86%93%E2%86%93%E2%86%93+it!'s+even+worse+without+this%0A%7B$ImplicitExceptions+Off%7D%0A%0Ainterface%0A%0Aoperator/+(const+a,+b:+string):+string%3B+inline%3B%0A%0Aimplementation%0A%0Aoperator/+(const+a,+b:+string):+string%3B%0Abegin%0A++result+:%3D+a+%2B+LineEnding+%2B+b%3B%0Aend%3B%0A%0Aprocedure+TestOverload%3B%0Avar+s:+string%3B%0Abegin%0A++s+:%3D%0A++++!'Hello!'+/%0A++++!'world!'+/%0A++++!'goodbye!'+/%0A++++!'world!'+/%0A++++!'Hello!'+/%0A++++!'world!'+/%0A++++!'goodbye!'+/%0A++++!'world!'%3B%0A++writeln(s)%3B%0Aend%3B%0A%0Aend.%0A'),l:'5',n:'0',o:'Pascal+source+%231',t:'0')),k:39.52941176470588,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:fpc304,filters:(b:'0',binary:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'1',intel:'0',libraryCode:'1',trim:'0'),lang:pascal,libs:!(),options:'-Ci-+-Cr-+-g-+-CX+-XXs+-O4+-CfAVX2+-CpCOREAVX2+-OpCOREAVX2',source:1),l:'5',n:'0',o:'x86-64+fpc+3.0.4+(Editor+%231,+Compiler+%231)+Pascal',t:'0')),k:60.47058823529412,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 15, 2019, 11:58:37 pm
That works, but I would just... not recommend that you do it, ever. It's adding a non-trivial amount of runtime overhead to your application for no reason (as opposed to single-line or multi-line string concatenation with either the '+' operator or the actual Concat intrinsic function, which is always resolved entirely at compile time and results in a single constant string.)

 "ever" :D Good point, and fair enough.
Title: Re: If you’ve ever wanted to be able to use multi-line strings in FPC...
Post by: JernejL on July 16, 2019, 08:01:37 am
Also, " is already known in the (Pascal) programming universe as a string delimiter of strings containing escaped characters. (E. g. the GNU Pascal compiler allowed that.)

Interesting, i cannot find any mention of that, i assume that FPC doesn't support that at all?
 
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: BrunoK on July 16, 2019, 09:26:00 am
Using PLATFORM as default seems logical to me.

This seems to be the consensus, or something close to one at least. Will probably change it over to PLATFORM sometime tonight.

I also plan to re-use the variable that tracks the column position of the opening backticks for AUTO to add a multi-line-string specific error, shown in the event that one is "unbalanced" or "unterminated", that tells you where exactly it started (as opposed to the more general syntax error you'd get in that case currently.) Probably the last real addition to the feature I'll do for the time being.

Not a replacement for Akira's very nice work, but perhaps an option for some:

Code: Pascal  [Select]
  1. operator/ (const a, b: string): string;
  2. begin
  3.   result := a + LineEnding + b;
  4. end;
  5.  
  6. procedure TForm1.Button1Click(Sender: TObject);
  7. var
  8.   s: string;
  9. begin
  10.   s :=
  11.     'Hello' /
  12.     'World';
  13.   Memo1.Append(s);
  14. end;  

Akira, as I wrote somewhere above, the idea of having an easier looking way of expressing long string literals is very engaging.

But : we have only JEDI code formatter to salvage strangely indented code and this I need more than this new feature.

So you could slightly reconsider the syntax using kupferstecher / VTwin formulation.
It, of course, is not as easy to use as just having a backtick at the beginning and end, but pascal in general does not use position in the source code layout and a formulation that would remain 'conventional' seems to be a better idea.
The inconvenience is that it still would require quoting strings (and '' single quotes); in this regard this is not as good as the back tick.

Since you are in it, you could parse the literal applying this rule :
Any non terminated (terminator being ;) literal string that meets operator / outside a the single quoted literal is interpreted by the scanner as LineEnding +.
This would have the additional quality of letting // and {} comments appear in the source stream. This should also be simpler for adapting the JEDI code formatter taking in account the new syntax.

Now just a slightly more complicated SQL script straight out of a text file with the back tick idea, would it pass ?
Code: Pascal  [Select]
  1.   DBIScript =
  2.    `/* ImportEcr2012.sql : Reprendre toutes écritures 2012,
  3.                            injecter dans écritures
  4.                            mettre à jour références comptes DB/CR
  5.                            Mettre à jour code catégorie selon Categorie de compte */
  6.     /* SQL-92 Table Creation Script with DBISAM Extensions */
  7.     DROP TABLE IF EXISTS "\Memory\EcrituresNew";
  8.     CREATE TABLE IF NOT EXISTS "\Memory\EcrituresNew"
  9.     (
  10.        "UId" AUTOINC,
  11.        "DateEcr" DATE,
  12.        "DB" VARCHAR(10),
  13.        "CR" VARCHAR(10),
  14.        "Ref" VARCHAR(10),
  15.        "Libelle" VARCHAR(100),
  16.        "MntEcr" MONEY,
  17.     PRIMARY KEY ("RecordID") COMPRESS NONE
  18.     TEXT INDEX ("Libelle")
  19.     STOP WORDS ':TEXT_INDEXPARAMS=_DBI_TIPS_FR :TEXT_INDEXRULES=CH_FR'
  20.     SPACE CHARS #1+#2+#3+#4+#5+#6+#7+#8+#9+#10+#11+#12+#13+
  21.     #14+#15+#16+#17+#18+#19+#20+#21+#22+#23+
  22.     #24+#25+#26+#27+#28+#29+#30+#31+#32+'*+'+
  23.     ',-./:;<=>\`'
  24.     INCLUDE CHARS '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_ab' +
  25.     'cdefghijklmnopqrstuvwxyz€‚ƒ„…†‡ˆ‰Š‹ŒŽ'+
  26.     '‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·'+
  27.     '¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß'+
  28.     'àáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþ'
  29.     LOCALE CODE 4108
  30.     USER MAJOR VERSION 1
  31.     );
  32.  
  33.     import table "\Memory\EcrituresNew"
  34.       from
  35.         "C:\DBI_DB\Cresus08\Ecritures12.txt"
  36.       DELIMITER #9
  37.       Columns (DateEcr,DB,CR,Ref,Libelle,MntEcr)
  38.       DATE 'dd.mm.yy'
  39.       DECIMAL '.';
  40.  
  41.     DROP INDEX IF EXISTS Ecritures.ixDB;
  42.     DROP INDEX IF EXISTS Ecritures.ixCR;
  43.     DROP INDEX IF EXISTS Ecritures.ixRef;
  44.  
  45.     delete  FROM Ecritures
  46.       where DateEcr>='2012-01-01';
  47.  
  48.     INSERT INTO
  49.       "Ecritures" (DateEcr,DB,CR,Ref,Libelle,MntEcr)
  50.     SELECT DateEcr,DB,CR,Ref,Libelle,MntEcr
  51.     FROM "\Memory\EcrituresNew";
  52.  
  53.     CREATE NOCASE INDEX IF NOT EXISTS "ixDB" ON "Ecritures" ("DB");
  54.     CREATE NOCASE INDEX IF NOT EXISTS "ixCR" ON "Ecritures" ("CR");
  55.  
  56.     update Ecritures E
  57.       set E.DBUId=P1.UId,
  58.           E.CRUId=P2.UId
  59.     from
  60.       Ecritures
  61.     left join Plan P1 on (E.DB=P1.CompteUnique)
  62.     left join Plan P2 on (E.CR=P2.CompteUnique);
  63.  
  64.     update Ecritures E
  65.        set CatCodes=P1.CatCode+P2.CatCode
  66.     from
  67.       Ecritures
  68.     left join Plan P1 on (E.DBUId=P1.UId)
  69.     left join Plan P2 on (E.CRUId=P2.UId);
  70.  
  71.     update Ecritures E
  72.        set CatCodes=P1.CatCode+P2.CatCode
  73.     from
  74.       Ecritures
  75.     left join Plan P1 on (E.DBUId=P1.UId)
  76.     left join Plan P2 on (E.CRUId=P2.UId);
  77.  
  78.     update Ecritures
  79.        set Ref='0'+Ref
  80.     where length(Ref)<6 and SUBSTRING(Ref,1,1) between '1' and '9';
  81.  
  82.     CREATE INDEX IF NOT EXISTS "ixRef" ON "Ecritures" ("Ref");
  83.     /* Clean up */
  84.     DROP TABLE IF EXISTS "\Memory\EcrituresNew";`;  
  85.  
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 16, 2019, 09:41:05 am
Code: Pascal  [Select]
  1. operator/ (const a, b: string): string;
  2. begin
  3.   result := a + LineEnding + b;
  4. end;
  5.  
  6. procedure TForm1.Button1Click(Sender: TObject);
  7. var
  8.   s: string;
  9. begin
  10.   s :=
  11.     'Hello' /
  12.     'World';
  13.   Memo1.Append(s);
  14. end;  

<snipped>

So you could slightly reconsider the syntax using kupferstecher / VTwin formulation.
If I understood what you're proposing correctly, that is, use "/" (without quotes of course) to indicate a line continuation, that seems to have the same problem as using "+" for the same purpose.  The possibility of overloading the operator would force the entire construction to be parsed instead of just scanned.

Refer to post #34 by PascalDragon, in this thread, for a more complete explanation of the problems it would create.

if a line-break continuation character is to be used, it cannot be a character that is already in use as an operator (or one which may be used in the future as an operator.)


Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: kupferstecher on July 16, 2019, 11:05:08 am
If I understood what you're proposing correctly, that is, use "/" (without quotes of course) to indicate a line continuation, that seems to have the same problem as using "+" for the same purpose.  The possibility of overloading the operator would force the entire construction to be parsed instead of just scanned.

Refer to post #34 by PascalDragon, in this thread, for a more complete explanation of the problems it would create.
This would mean that the parser would need to handle new lines which it currently is totally ignorant of.
I understand that sentence that the problem is the Parser can't distinguish, if the tokenized "+" is at the end of a line which would result in a inserted linebreak or it is between two strings in the same line, which wouldn't result in a inserted linebreak. Making fpc able to distinguish both cases of course would be intrusive.
But with a new operator "/" this is not necessary. The token "/" is just passed to the parser, the lexer perhaps may be even untouched (But I don't know). It's actually desirable that the parser deals with it, because then it also can treat string variables, not only source strings.

The result is a clean syntax without mode switches, different types of strings, trimming mechanisms etc.
Code: Pascal  [Select]
  1. var
  2.   s,s2: string;
  3. begin
  4.   s2:= 'sweet'
  5.  
  6.   s := 'Hello,' / s2 + 'world'
  7.   Memo1.Append(s);
  8. end;

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 16, 2019, 11:59:08 am
But with a new operator "/" this is not necessary.
The problem is "/" is not a new operator, it is currently used to indicate division and it is legal, though it would look a bit strange, to define division between two strings. There is nothing stopping/preventing a programmer from doing that which would cause an ambiguity, i.e, is it a division or a line continuation ?.


Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 16, 2019, 05:16:27 pm
But : we have only JEDI code formatter to salvage strangely indented code and this I need more than this new feature.

I'm afraid I'm not quite sure what you're getting at with this.

Now just a slightly more complicated SQL script straight out of a text file with the back tick idea, would it pass ?

You'd need to escape the backtick literal (by adding a second one) on the line that looks like this, if you wanted to have it in the output:

Code: Text  [Select]
  1. ',-./:;<=>\`'

But apart from that, yes.

I understand that sentence that the problem is the Parser can't distinguish, if the tokenized "+" is at the end of a line which would result in a inserted linebreak or it is between two strings in the same line, which wouldn't result in a inserted linebreak. Making fpc able to distinguish both cases of course would be intrusive.
But with a new operator "/" this is not necessary. The token "/" is just passed to the parser, the lexer perhaps may be even untouched (But I don't know). It's actually desirable that the parser deals with it, because then it also can treat string variables, not only source strings.

It doesn't really work like you think it does.

FPC uses exactly one completely context-free procedure (tscannerfile.readtoken) to do basically all reading of strings (whether they be initialized variables, or "true constants", or resourcestrings, or a standalone literal in a function call, and so on.)

This is why multiline strings "just work" everywhere single-line strings do: it boils down to doing exactly the same thing, except between two backticks instead of between two single-quotes, and without intentionally stopping when it hits a newline character.

The result is a clean syntax without mode switches, different types of strings, trimming mechanisms etc.

A) The mode switch was laid out as an explicit requirement for this feature to ever be considered at all, by the core team.

B) This feature doesn't introduce a new type of string. It just enables the compiler to read strings the way it always has, but across multiple lines.

C) IMO the "/" operator thing is just not really any better than the "+" we've always had, and provides none of the advantages that real unseparated multi-line string constants do.

D) At this point I simply have no intention of changing the core way the feature works, regardless.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: kupferstecher on July 16, 2019, 05:54:29 pm
The problem is "/" is not a new operator, it is currently used to indicate division and it is legal, though it would look a bit strange, to define division between two strings. There is nothing stopping/preventing a programmer from doing that which would cause an ambiguity, i.e, is it a division or a line continuation ?.
The same holds for "+".
Because the "+" operator is already used, thus it's not far-fetched to use "/" as well.

What has "+" to do with strings? Well, it "adds" them together.
What has "/" to do with strings? It "devides" them to two lines.
Quite intuitive, I'd say.

Anyways, that was only a suggestion.

It doesn't really work like you think it does.

FPC uses exactly one completely context-free procedure (tscannerfile.readtoken) to do basically all reading of strings (whether they be initialized variables, or "true constants", or resourcestrings, or a standalone literal in a function call, and so on.)

This is why multiline strings "just work" everywhere single-line strings do: it boils down to doing exactly the same thing, except between two backticks instead of between two single-quotes, and without intentionally stopping when it hits a newline character.
I didn't assume any different. My comment was about the claim, that a "/" would be intrusive.
I can see the charm of your solution that everything is covered in the Lexer, but it also means there is still no better way to insert linebreaks concatenated with variable strings.


Quote
A) The mode switch was laid out as an explicit requirement for this feature to ever be considered at all, by the core team.
Well a mode switch for enabling a feature is one thing. I was talking about configuration mode switches: trim 5, trim auto...

Quote
B) This feature doesn't introduce a new type of string. It just enables the compiler to read strings the way it always has, but across multiple lines.
New delimiters, new syntax. The syntax issue I guess is what BrunoK meant with "JEDI code formatter". Tools and syntax highlighters may stumble over the new syntax. (But thats evolution...)
A freepascal programmer then has to learn both the standard strings and multiline strings.

Quote
C) IMO the "/" operator thing is just not really any better than the "+" we've always had, and provides none of the advantages that real unseparated multi-line string constants do.
As said before, this doesn't contradict to your proposal, they could coexist.

Quote
D) At this point I simply have no intention of changing the core way the feature works, regardless.
Thats clear. And I can understand that its not very motivating to get contra. But such implementations have an effect on all freepascal users, so it should be discussed.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 16, 2019, 06:14:57 pm
Well a mode switch for enabling a feature is one thing. I was talking about configuration mode switches: trim 5, trim auto...

I mean, it's just one directive. IMO the "Auto" option I added should pretty much cover the common use case for anyone who doesn't want to worry about indentation at all, also.

New delimiters, new syntax. The syntax issue I guess is what BrunoK meant with "JEDI code formatter". Tools and syntax highlighters may stumble over the new syntax. (But thats evolution...)
A freepascal programmer then has to learn both the standard strings and multiline strings.

Adapting formatters and syntax highlighters to the backticks is certainly far more straightforward than it would be with "/". In the case of a formatter, all it has to do is treat the column position of the opening backtick as the "left side" (much like {$MultiLineStringLeft Auto} does) and reposition the entire string block based on that (while just not altering anything that comes between the two backticks at all.)

I also don't think there's anything to "learn", really. Anywhere you can use a single-line string denoted with single quotes, you can use a multi-line string denoted with backticks. They are exactly the same thing apart from the fact that one of them can span multiple lines while the other cannot.

As said before, this doesn't contradict to your proposal, they could coexist.

Well, of course. I guess my real point is that I just don't really understand why or where the "/" thing would ever be preferable.

Thats clear. And I can understand that its not very motivating to get contra. But such implementations have an effect on all freepascal users, so it should be discussed.

I don't mind the "contra" at all, really. We did discuss it at length on the mailing list though, and I specifically made this thread afterwards to give the whole thing additional visibility.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 17, 2019, 03:58:36 am
Just finished up a round of refactoring on this:

- The RAW option for MultiLineStringLineEnding is now called SOURCE.
- PLATFORM is now the default MultiLineStringLineEnding option.
- I've implemented the multi-line-string-specific error message that I mentioned in an earlier comment.

Here's an example that shows how the error works / is displayed:

Code: Pascal  [Select]
  1. { %FAIL }
  2.  
  3. { Will show:
  4.   tmultilinestring28.pp(20,1) Fatal: Unterminated multi-line string beginning at line 11, column 7. }
  5.  
  6. program tmultilinestring28;
  7.  
  8. {$modeswitch MultiLineStrings}
  9.  
  10. const
  11.   a = `this will be unterminated
  12. with some
  13. lines in it.
  14.  
  15. var
  16.   B : String;
  17.  
  18. begin
  19.   B:=`
  20. again
  21. something
  22. end backticked`;
  23.  
  24. end.
  25.  

Again, I've pushed the changes to my github branch of the compiler, and there's new patches on the bugtracker issue.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: engkin on July 17, 2019, 08:09:48 am
Akira, just want to let you know. Not hearing from me, means I did not find another bug.  :)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 17, 2019, 08:51:07 am
- The RAW option for MultiLineStringLineEnding is now called SOURCE.
- PLATFORM is now the default MultiLineStringLineEnding option.
- I've implemented the multi-line-string-specific error message that I mentioned in an earlier comment.
Thank you.

I was wondering if we really have to stick with the backtick, or we can keep the "old school" single quote instead?
Question stands first from implementation point of view, and then from "Is that really a more desirable syntax for multi line strings?" point of view.
I mean, everything is already there, and I might even prefer keeping single quotes if possible, while keeping all code switches as they are.

This renders to
Code: Pascal  [Select]
  1. const
  2.   Str1 = `this is
  3.           a multi line
  4.           string.`
versus
Code: Pascal  [Select]
  1. const
  2.   Str1 = 'this is
  3.           a multi line
  4.           string.'

And I like second example a little more.

And second example already renders nicely in the forum  ;)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 17, 2019, 09:12:29 am
I was wondering if we really have to stick with the backtick, or we can keep the "old school" single quote instead?
This was a requirement from the core team to even remotely consider this for inclusion. The normal strings stay untouched. Period.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 17, 2019, 03:09:28 pm
Akira, just want to let you know. Not hearing from me, means I did not find another bug.  :)

Good to hear!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Imants on July 18, 2019, 08:28:12 am
Is your patch working on FPC 3.2.0 or only on trunk?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 18, 2019, 09:12:35 am
Considering that nothing significant changed in the scanner since then I think that it should work. Just give it a try. *shrugs*
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Imants on July 18, 2019, 09:28:34 am
I gave it a try and it did not compile in 3.2.0 and it compiles in trunk. I used fpcupdeluxe

Console output:
Code: Text  [Select]
  1. echo '42444' > revision.inc
  2. C:/fpcupdeluxe/fpcbootstrap/ppcx64.exe -Ur -Xs -O2 -n -Fux86_64 -Fusystems -FuC:/fpcupdeluxe/fpcsrc/rtl/units/x86_64-win64 -Fix86_64 -FE. -FUx86_64/units/x86_64-win64 -dRELEASE -vw-n-h-l-d-u-t-p-c- -dFPC_SOFT_FPUX80   -dREVINC -dx86_64 -dGDB -dBROWSERLOG -Fux86 -Sew version.pas
  3. C:/fpcupdeluxe/fpcbootstrap/ppcx64.exe -Ur -Xs -O2 -n -Fux86_64 -Fusystems -FuC:/fpcupdeluxe/fpcsrc/rtl/units/x86_64-win64 -Fix86_64 -FE. -FUx86_64/units/x86_64-win64 -dRELEASE -vw-n-h-l-d-u-t-p-c- -dFPC_SOFT_FPUX80   -dREVINC -dx86_64 -dGDB -dBROWSERLOG -Fux86 -Sew pp.pas
  4. scanner.pas(4252,46) Error: Identifier not found "m_multiline_strings"
  5. scanner.pas(4476,53) Error: Identifier not found "m_multiline_strings"
  6. scanner.pas(5150,30) Error: Identifier not found "m_multiline_strings"
  7. scanner.pas(5684,40) Error: Identifier not found "m_multiline_strings"
  8. scanner.pas(5949) Fatal: There were 4 errors compiling module, stopping
  9. Fatal: Compilation aborted
  10.  

I was using multi_line_strings_main_rev5.patch
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 18, 2019, 02:54:32 pm
I gave it a try and it did not compile in 3.2.0

Seems like the changes to "globtype.pas", specifically, were not applied successfully against 3.2.0.

This makes sense actually, as I've been developing directly against the latest trunk, which contains several other new modeswitches (that I do not believe would exist in the 3.2.0 sources) in the list before m_multiline_strings:

m_multi_helpers, m_array2dynarray, and m_prefixed_attributes.

If you really wanted to use it with 3.2.0 though, I think it would be pretty easy to just do the changes to "globtype.pas" manually (without the other modeswitches), as for that particular file it is not very much code being added.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 19, 2019, 09:28:57 am
I gave it a try and it did not compile in 3.2.0 and it compiles in trunk. I used fpcupdeluxe
The first step after applying a patch is to check whether all changes applied cleanly. In this case you would have noticed that adding m_multiline_strings to the modeswitch enum failed. For a solution see Akira1364's remark about adding it manually.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Imants on July 19, 2019, 12:18:45 pm
I gave it a try and it did not compile in 3.2.0 and it compiles in trunk. I used fpcupdeluxe
The first step after applying a patch is to check whether all changes applied cleanly. In this case you would have noticed that adding m_multiline_strings to the modeswitch enum failed. For a solution see Akira1364's remark about adding it manually.
I used fpcupdelux to build compiler I never build compiler myself. And patching happens there automatically.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 20, 2019, 10:55:35 am
I gave it a try and it did not compile in 3.2.0 and it compiles in trunk. I used fpcupdeluxe
The first step after applying a patch is to check whether all changes applied cleanly. In this case you would have noticed that adding m_multiline_strings to the modeswitch enum failed. For a solution see Akira1364's remark about adding it manually.
I used fpcupdelux to build compiler I never build compiler myself. And patching happens there automatically.
Then this should be considered a bug in fpcupdeluxe if it starts building after applying the patch failed.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 20, 2019, 04:02:57 pm

I am surprised that, no one noticed that there is an old Pascal feature which gets broken with introducing these multiline strings -- until now, in Pascal sources, all trailing spaces at the end of each line have always been insignificant.
Introducing these multiline strings breaks this rule.

See:
Code: Pascal  [Select]
  1. var
  2.   S1, S2: String;
  3. begin
  4.   S1 := 'this is a two-line string which has   ' // you can see the spaces before the quote.
  5.      + LineEnding + 'three ending spaces in first line.';
  6.  
  7.   // in the next line there are three trailing spaces; you can not see them:
  8.   S2 := `another string which has  
  9. three ending spaces in first line, but we don't know it just by looking at the code`;
  10.  
  11.  //
  12. end;
  13.  

Although having multiline strings can be useful, I really don't like breaking this old Pascal feature -- trailing spaces on a line should always be insignificant!
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 20, 2019, 04:31:26 pm
Another thing which bothers me is the choice of "backtick" character.
It is too similar to quote character, not very careful reader will not notice that it is not a single quote.

Better choice would be some character which does not resemble the single quote, which is used for "normal" strings.

I wonder if all latin keyboards even have this character.
I found that I actually have two very similar characters on my keyboard -- "`" (right-alt+7) and "´" (right-alt+9), which makes further confusion.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 20, 2019, 10:25:51 pm
I wonder if all latin keyboards even have this character.
I found that I actually have two very similar characters on my keyboard -- "`" (right-alt+7) and "´" (right-alt+9), which makes further confusion.

I don't know if all, but most* should have them (both the "grave" ^ and "acute" ' tildes, different from the apostrophe: '). They are both used as tildes rather extensively throughout most of Europe, West, Central and East, along with ^, " and ~.


* Actually looked (a quick look) in an old IBM guide of international keyboard layouts. It's a bit old and doesn't have all, but enough of them to get the idea.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 20, 2019, 10:42:49 pm
I wonder if all latin keyboards even have this character.
It's an ASCII character (https://en.wikipedia.org/wiki/ASCII#Printable_characters) and thus it's very likely that all keyboards have it available somewhere.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: winni on July 20, 2019, 10:48:15 pm
Beside the different internationale keyboard layouts:

The backtick is eye powder! Not everyone has good eyes. Or a monster screen!

Winni
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: VTwin on July 20, 2019, 11:04:11 pm
The problem is "/" is not a new operator, it is currently used to indicate division and it is legal, though it would look a bit strange, to define division between two strings. There is nothing stopping/preventing a programmer from doing that which would cause an ambiguity, i.e, is it a division or a line continuation ?.
The same holds for "+".
Because the "+" operator is already used, thus it's not far-fetched to use "/" as well.

What has "+" to do with strings? Well, it "adds" them together.
What has "/" to do with strings? It "devides" them to two lines.
Quite intuitive, I'd say.

Anyways, that was only a suggestion.

It makes sense to me. "/" is not defined as a string operator now, and, if desired, could be redefined. Anyway, I was just pointing out a simple option.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 12:16:16 am

I am surprised that, no one noticed that there is an old Pascal feature which gets broken with introducing these multiline strings -- until now, in Pascal sources, all trailing spaces at the end of each line have always been insignificant.
Introducing these multiline strings breaks this rule.


I'm sorry, but this makes no sense whatsoever. That is absolutely not an "old Pascal feature" or "rule" by any sense of the term (more like a random, insignificant detail that you're pointing out), nor is introducing multiline strings going to "break" anything.

The nature of trailing whitespace is just something that has literally been completely irrelevant in the past due to there being no common scenario where anyone would have any reason to think about it. Which is not going to change. It still doesn't matter at all outside the specific context of a multiline string.

Beside the different internationale keyboard layouts:

The backtick is eye powder! Not everyone has good eyes. Or a monster screen!

Winni

It's like, exactly the same size as the single-quote we've all been happily using for years (and will happily continue to use, as once again, it's not going anywhere.)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: winni on July 21, 2019, 12:30:18 am
Quote
It's like, exactly the same size as the single-quote we've all been happily using for years (and will happily continue to use.)

Yes, but then we got two different types of fruit flies.

Winni
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 12:35:37 am
Yes, but then we got two different types of fruit flies.

This means absolutely nothing and proves no point to speak of, so... ok?

Sorry for getting a bit frustrated, but I quite frankly find the level of continued opposition to the entire concept of multi-line strings, despite the fact that I've gone way out of my way to address all of the reasonable concerns with them that I could, to be utterly mind-boggling.

It is an incredibly simplistic piece of functionality to add to the language (or more specifically, to add to FPC's implementation of the language) that is not going to have any negative impact whatsoever on anyone's existing workflow.

Some of the comments in this thread are far too uncannily similar to the ridiculous non-arguments I often hear from backwards-thinking Golang users about how generics are somehow guaranteed to "ruin Go" if ever added to it, for my tastes. (Ironically Go does however have multiline strings, BTW!)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: winni on July 21, 2019, 01:29:38 am
@Akira1364

Ok, I got your point of view. Accepted.

It's just about the eye powder.

I've got bad eyes and glasses. I know the problem to distinguish the back tick and the apostroph from a lot of working with the bash. If it's your own stuff then you can avoid the problem, but if it's system stuff or from other person, I really have always to look twice what kind of nothing that is.

So - I just wanted to avoid that problem. Nothing more.

Winni
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 01:37:32 am
@Akira1364

Ok, I got your point of view. Accepted.

It's just about the eye powder.

I've got bad eyes and glasses. I know the problem to distinguish the back tick and the apostroph from a lot of working with the bash. If it's your own stuff then you can avoid the problem, but if it's system stuff or from other person, I really have always to look twice what kind of nothing that is.

So - I just wanted to avoid that problem. Nothing more.

Winni

Don't get me wrong: I totally understand the eyesight thing, if that's the case. I just do not believe that it is, by itself, a significant enough reason to be "universally against" the feature to the extent that it prevents everyone else on Earth from using it.

Do note also:

Once this functionality gets merged to FPC, I then intend to submit a patch I have for Lazarus SynEdit which makes it highlight multi-line strings much the way it does multi-line comments right now.
(Of course, I know not everyone uses Lazarus, but it's something.)

The overall point is though that the ideal "end-game" tooling-and-editor-wise means that it should be very obvious in all (or at least the strong majority of) cases what is and is not a multi-line string, when reading source files.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 21, 2019, 02:35:30 am
It's an ASCII character (https://en.wikipedia.org/wiki/ASCII#Printable_characters) and thus it's very likely that all keyboards have it available somewhere.

Hah, really! I wouldn't expect this to be ascii.

Okay, being ascii is really a strong point for it
 However, it is very similar to single quote, which is used for "old style" string literals, so can't we use some other character instead? Perhaps tilde ("~" -- also ASCII)?


I am surprised that, no one noticed that there is an old Pascal feature which gets broken with introducing these multiline strings -- until now, in Pascal sources, all trailing spaces at the end of each line have always been insignificant.
Introducing these multiline strings breaks this rule.


I'm sorry, but this makes no sense whatsoever. That is absolutely not an "old Pascal feature" or "rule" by any sense of the term (more like a random, insignificant detail that you're pointing out), nor is introducing multiline strings going to "break" anything.


I am very surprised that you don't see that syntax which allows trailing spaces affects readability of code. These spaces are literally invisible, so how readable that can be?

Readability is an important language feature.

Some different syntax which would require some (visible) character to mark the end-of-line and continuation on the next line would be much better way to implement this.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 02:43:20 am
Hah, really! I wouldn't expect this to be ascii.

...What? Of course it is. Character number 96.

Perhaps tilde ("~" -- also ASCII)?

Oh, so I can waste even more time arguing with all of the Nitpick McGees out there who would definitely be very vocal with regards to complaining about how "weird" it looks to them or whatever? Yeah, no thanks.

Again, this is a moot point, now, anyways: I have literally zero intention of making that kind of fundamental change at this stage. It's not gonna happen. I've based my implementation around what was discussed (at length!) on the mailing list, and that's how at the very least the core of it is going to stay.

I am very surprised that you don't see that syntax which allows trailing spaces affects readability of code. These spaces are literally invisible, so how readable that can be?

Readability is an important language feature.

Some different syntax which would require some (visible) character to mark the end-of-line and continuation on the next line would be much better way to implement this.

There is literally no connection between multi-line strings and any hypothetical reason you would have cared in the slightest about "invisible spaces" in the first place (which is, none I can think of.) I don't understand why you think this is significant at all, let alone something that profoundly "affects readability".
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: SlightlyOutOfPhase on July 21, 2019, 05:17:38 am
There's just an unfortunately high amount of really, really stupid people who frequent these forums and fill them up with illiterate bullshit. Don't let them get you down Akira.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 21, 2019, 10:11:43 am

I am surprised that, no one noticed that there is an old Pascal feature which gets broken with introducing these multiline strings -- until now, in Pascal sources, all trailing spaces at the end of each line have always been insignificant.
Introducing these multiline strings breaks this rule.


I'm sorry, but this makes no sense whatsoever. That is absolutely not an "old Pascal feature" or "rule" by any sense of the term (more like a random, insignificant detail that you're pointing out), nor is introducing multiline strings going to "break" anything.


I am very surprised that you don't see that syntax which allows trailing spaces affects readability of code. These spaces are literally invisible, so how readable that can be?

Readability is an important language feature.
In Lazarus you can enable whitespace to be shown as most editors allow. It's also very useful to find out if a tab was smuggled in somewhere...

Some different syntax which would require some (visible) character to mark the end-of-line and continuation on the next line would be much better way to implement this.
That would prohibit another purpose of this feature: to be able to copy some piece of code (e.g. some SQL script) and insert it without having to worry about adjusting it (aside from maybe escaping backticks, but that would be necessary with apostrophe as well).
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: lucamar on July 21, 2019, 10:24:22 am
There is literally no connection between multi-line strings and any hypothetical reason you would have cared in the slightest about "invisible spaces" in the first place (which is, none I can think of.) I don't understand why you think this is significant at all, let alone something that profoundly "affects readability".

Imagine this situation:
Code: Pascal  [Select]
  1. MyString :=
  2.   'Some string that, for whatever reason.' + LF
  3.   '      needs                           ' + LF
  4.   ' to have all these spaces.            ';

As a multiline:
Code: Pascal  [Select]
  1. OtherString :=
  2.   `Some string that, for whatever reason.
  3.          needs                          
  4.     to have all these spaces.            `;

Will both strings be the same? And you have to recognize that the multiline string is much less readable (how many spaces are there?) than the standard one and runs the risk of some editor "pruning" the final spaces in the second and third lines.

So yeah, it's probably nitpicking and a not very common situation, but it exists and some people may think it's important, whatever you or I or anyone else thinks, so it's not unreasonable.

Of course, one could always answer;: "In those situations, use standard strings" :)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jshah on July 21, 2019, 11:04:40 am
I think these feature is very nice.

if any one has issue they should avoid using it.

But we should encourage people who contributing to FPC development.

More contribution to FPC means much much better PROGRAMMING LANGUAGE/PLATFORM
it benefit to EVERYONE.


Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 21, 2019, 01:29:28 pm
if any one has issue they should avoid using it.

Reading and understanding code (not only the code you had written yourself) is important.

Then, please understand that programming is not only writing new code, but also maintaining code.

So, please avoid arguments like "If you don't like it, don't use it".

Of course, one could always answer;: "In those situations, use standard strings" :)

Yes, I knew this "argument" was just about to appear. ;)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 21, 2019, 01:38:49 pm

Hah, really! I wouldn't expect this to be ascii.

...What? Of course it is. Character number 96.

I am sorry I hadn't checked that in the first place. As I already admitted, the fact that it is ASCII is a strong point for it.

Anyway, I'm still saying that it is too similar to single quote which is used for "old style" strings, but I also have to admit we don't have much ASCII chars that are not already part of Pascal syntax left.


Perhaps tilde ("~" -- also ASCII)?

Oh, so I can waste even more time arguing with all of the Nitpick McGees out there who would definitely be very vocal with regards to complaining about how "weird" it looks to them or whatever? Yeah, no thanks.

Again, this is a moot point, now, anyways: I have literally zero intention of making that kind of fundamental change at this stage. It's not gonna happen. I've based my implementation around what was discussed (at length!) on the mailing list, and that's how at the very least the core of it is going to stay.

Akira, there is no need for this tone. I want to say what I think, that is all.

You are the one who took the effort to implement this, so from the moment I entered this topic, I have understood quite well that it is you and the core team who will decide what syntax will enter the language, it won't be me.

And, I would expect that now is still not too late for some changes, but of course only if you and the core fpc team agree with my points; otherwise you just won't accept them.

Nevertheless, I think that this topic is the right place to say what I think about it.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 21, 2019, 04:04:24 pm
  Personally I think this whole concept and what is being done is just a push to make fpc a database  scripting compiler which in my opinion takes it away from what it should be..

 I views as with many others here have no weight in the decision making so i'll sit and watch the many kacks
being done to turn it into a scripting compiler.

  I can think of much better things to be doing with the compiler than making it a script kiddy editor..

 I just pray there will always be switches to turn it off and keep it off..
 
 My self, I've always had a tool around that created a pascal line by line block of text like that I could edit
off the side and then simply paste it into the editor.. It works for Delphi and it works for Lazarus thus keeping
it the pascal style and also, it will back paste into the side editor for rework...

 I really think this should be done in Lazarus as a tool option and not in the compiler but who am I right? Just another out there like many here. := Noise;



Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 21, 2019, 05:50:40 pm
Do note also:

Once this functionality gets merged to FPC, I then intend to submit a patch I have for Lazarus SynEdit which makes it highlight multi-line strings much the way it does multi-line comments right now.
(Of course, I know not everyone uses Lazarus, but it's something.)

I hope you will also take care of "Trim trailing spaces" IDE option, which has always been safe to use with Pascal source, but now it will have to take care not to brake new style string literals.

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 06:10:48 pm
Akira, there is no need for this tone. I want to say what I think, that is all.

You are the one who took the effort to implement this, so from the moment I entered this topic, I have understood quite well that it is you and the core team who will decide what syntax will enter the language, it won't be me.

Sorry if that came off overly harsh. It's just that you need to keep in mind the "big picture":

For example, I am 100% sure that if, for some reason, we were actually planning on using the tilde character to denote multiline strings, the amount of controversy around the whole thing would be an order of magnitude larger than it is now.

I hope you will also take care of "Trim trailing spaces" IDE option, which has always been safe to use with Pascal source, but now it will have to take care not to brake new style string literals.

That IDE option does not need to change in any way at all. I don't understand why you think it would "break" multi-line strings. I've specifically tested it, and it does in fact trim the trailing space if turned on (as it should) but this does not alter the way a multiline string is ultimately formatted / appears when displayed with WriteLn, e.t.c.

If you were someone who actually literally wanted real trailing whitespace (despite the fact that, again, the string will look exactly the same whether it is there or not), the answer is obviously "turn that IDE option off when necessary."

Note: it is probably feasible to tweak that feature a bit so that it does indeed actually "ignore" multi-line strings. This might be a useful thing to do, at some point. I'm sure I could probably do it myself, and may well do so at some point down the line. One thing at a time however.

This general idea that I've implemented the multi-line strings functionality in some fragile way full of "gotchas" that's ready to fall apart at any moment is rather annoying, though.

Will both strings be the same?

For the last time, if using {$MultiLineStringTrimLeft Auto}, yes, the output would be completely identical in that case.

And you have to recognize that the multiline string is much less readable

I really do not.

how many spaces are there?

This is not some mysterious edge case. I have explained exactly how it works over, and over, and over, and over again. There's exactly as many spaces as what's actually present, unless you're trimming the leading ones in a way of your choosing with the MultiLineStringTrimLeft directive.

Also, @Jamie:

Sorry, but I'm not going to waste any more of my time responding at length to your childish hyperbole.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on July 21, 2019, 06:45:51 pm
Personally I wish you would stop making the compiler what it isn't and move that work over where it should
be and that is the IDE.

  if you feel I am being childish about this, I think you're in for a rude awaking. I am only posting what many others are thinking. I wasn't brought up to be politically correct just to appease those that would go against
the many to please a few.

  I fully agree with many here that have had negative views about what is going to take place.. And I see the code tools along with some components used in Lazarus taking a major overhaul just to make this work.

  You may find it hard to believe, but there are actually very knowable users posting here that uses the tools and can point out many issues that may or will take place , especially in the IDE code tools. But you claim you have it all under control as you indicated in  your replies.

 I can see I'll be sticking with the 2.0.2 and 3.0.2 for some time now. I can't say what others will be doing but for me "Since I've written compilers and IDE's " this isn't happening to soon.

 And btw, I use the SynEdit extensively in some critical editing tools where line formatting and function of the component as it is now is very important.

 Have a good day, and thank you for letting me express myself, as worthless as it is..

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 21, 2019, 07:15:18 pm
- snip -

You're just oozing FUD here. Nobody who actually understood how I'd implemented the multi-line strings functionality (something they can of course see for themselves, as the source is very available!) would think any tool would require anything resembling a "major overhaul" in order to support it. It took me literally less than 10 minutes to make SynEdit highlight them properly in my local checkout of Lazarus, for example.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 21, 2019, 07:48:29 pm

That IDE option does not need to change in any way at all. I don't understand why you think it would "break" multi-line strings.

I would expect it to trim trailing spaces from the first line of two-line string literal.
You are sure it will not? Why not? We haven't had significant trailing spaces in Pascal source until now, so I would not expect this utility to have a clever way to decide which trailing spaces should be kept in source.

This general idea that I've implemented the multi-line strings functionality in some fragile way full of "gotchas" that's ready to fall apart at any moment is rather annoying.

I am sorry to annoy you. I just would really expect this IDE option to trim all trailing spaces from the source, even when they finish the line inside multiline string.
I would never say a small "gotcha" like this could lead the whole new feature to fall apart. ;)

For example, I am 100% sure that if, for some reason, we were actually planning on using the tilde character to denote multiline strings, the amount of controversy around the whole thing would be an order of magnitude larger than it is now.


Yes, I believe it would.
I admit that the only thing I really don't like is the possibility to have significant spaces on the end of the line, for the first time in Pascal history. I find it much more annoying than the choice of "backtick" character, which I criticized for resembling single quote too much, but I can accept it, actually it is eventually quite fine with me.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 22, 2019, 12:16:09 am
I would expect it to trim trailing spaces from the first line of two-line string literal.

It does. Why wouldn't it?

I am sorry to annoy you. I just would really expect this IDE option to trim all trailing spaces from the source, even when they finish the line inside multiline string.
I would never say a small "gotcha" like this could lead the whole new feature to fall apart. ;)

Again, that option does still do that in that scenario, currently.

My point was that this does not "break" the appearance of the multi-line string when displayed at runtime and is overall essentially irrelevant, because you can't actually visually tell if there are or are not trailing spaces in it regardless.

If you actually wanted trailing spaces, for some reason, as it stands now you would just turn that IDE option off, as has always been the case.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on July 22, 2019, 01:40:11 am
I would expect it to trim trailing spaces from the first line of two-line string literal.

It does. Why wouldn't it?

Why?
Because that changes the string. The trailing spaces are the part of the string, they get lost...
Are you serious? :o

My point was that this does not "break" the appearance of the multi-line string when displayed at runtime and is overall essentially irrelevant, because you can't actually visually tell if there are or are not trailing spaces in it regardless.
You say when you display string on the screen, it does not matter if trailing spaces are cut off, it appears the same.
You surely understand that strings can be used for purposes not so trivial as to be just displayed on the screen.

It changes the string literal, and the fact that you can't actually visually tell if there are or not trailing spaces is the essence of the problem - you will not notice the change.

If you actually wanted trailing spaces, for some reason, as it stands now you would just turn that IDE option off, as has always been the case.
If for some reason I actually want trailing spaces in the multiline string literal, I will surely not use the new style syntax. But someone will.

And some day I will open some unit from some library written by someone -- a unit which has a new syntax string literal with trailing spaces, and the IDE will just change this string. I will open that unit to see some other code, some function far from that string, I will not be aware of the string existence.
And of course that could seriously affect the programme behaviour.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 22, 2019, 02:13:15 am
If for some reason I actually want trailing spaces in the multiline string literal, I will surely not use the new style syntax. But someone will.
And some day I will open some unit from some library written by someone -- a unit which has a new syntax string literal with trailing spaces, and the IDE will just change this string. I will open that unit to see some other code, some function far from that string, I will not be aware of the string existence.
And of course that could seriously affect the programme behaviour.

Ah, the old "contrived problem that almost certainly will never, ever actually occur as described in real life, and is highly unlikely to be a significant issue if it did due to what the vast majority of likely use cases for the feature are" tailor-made scare-mongering. Yawn.

The bottom line is:

I have literally said I could probably modify that feature of Lazarus to ignore multi-line strings without too much of an issue, and that I would be willing to do so (despite the fact I generally disagree that FPC needs to specifically cater to arbitrary Lazarus IDE features. They are two separate things.)

I do not however believe that 99% of the people who will ever use multi-line strings will do so in a way where the existence or non-existence of trailing whitespace between the last visible character and the newline character will actually matter to them in any way whatsoever, due to the fundamental nature of what strings are used for in the vast majority of cases (i.e. displaying information!)

This is of course very different from leading whitespace, which does indeed affect the formatting (hence why I introduced the MultiLineStringTrimLeft directive.)

We are talking about a programming language where anybody can, for example, just casually drop into inline assembly in the middle of their Button.OnClick() regardless of whether or not they have any idea what they're doing, and cannot reasonably expect the compiler or Lazarus to "save them from themselves" in the event they screw something up. And this is fine! It is what separates Pascal from nominally "safer" but objectively less powerful languages such as Java, e.t.c.

This is mind, do you not see how ridiculous it is to then apply this massive double standard that treats strings as way more critical than they ever commonly are in reality, and goes "it is unacceptable if every editor that anyone might ever code Pascal in does not fully and entirely account for this new language feature IMMEDIATELY!"

Am I supposed to go and start writing Vim / Emacs / Sublime Text / Visual Studio Code / e.t.c. plugins just to really make sure no poor, innocent programmer blows their leg off while using these dangerous, complicated multi-line strings (because it's not like their editor configuration is their own responsibility! That's crazy!) then? Where does it end?

Even if no editor ever directly supported multi-line strings in any capacity, the chances of their use in a codebase causing you any kind of serious problem is remarkably low, and it is disingenuous to claim otherwise.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: SlightlyOutOfPhase on July 22, 2019, 03:33:06 am
I think these feature is very nice.

if any one has issue they should avoid using it.

But we should encourage people who contributing to FPC development.

More contribution to FPC means much much better PROGRAMMING LANGUAGE/PLATFORM
it benefit to EVERYONE.

+1.

if you feel I am being childish about this, I think you're in for a rude awaking. I am only posting what many others are thinking. I wasn't brought up to be politically correct just to appease those that would go against
the many to please a few.

If we are all not "being politically correct to please people" here, then I'm just going to straight up say that EVERYTHING you have said in this thread makes you sound like a fucking idiot. All of your comments are embarrassing, and anyone who agrees with you is equally stupid. You are NOT "the many".

People like you are DIRECTLY what causes FPC's slow, inevitable decline in popularity. It's the same fucking thing every fucking time anybody wants to introduce anything new: a bunch of illiterate morons spouting off conspiracy-theory nonsense about how the sky is falling, who are ALWAYS, ALWAYS, ALWAYS clearly wrong in hindsight. You can go wayyyyy back on the developer mailing list for FPC and find people who unironically argued against the introduction of for-in loops the same fucking way you're arguing against this strings feature, back in like 2006. I'm not kidding. For-in loops! How fucking insane does that sound now that they've been a normal part of the language for years? Very!

So yeah, for the love of god stop fucking driving away smart people who both want to improve FPC and can also actually do the work themselves, or sometime real soon there won't be anyone left working on it at all.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Imants on July 22, 2019, 07:48:43 am
I think these feature is very nice.

if any one has issue they should avoid using it.

But we should encourage people who contributing to FPC development.

More contribution to FPC means much much better PROGRAMMING LANGUAGE/PLATFORM
it benefit to EVERYONE.

+1.

if you feel I am being childish about this, I think you're in for a rude awaking. I am only posting what many others are thinking. I wasn't brought up to be politically correct just to appease those that would go against
the many to please a few.

If we are all not "being politically correct to please people" here, then I'm just going to straight up say that EVERYTHING you have said in this thread makes you sound like a fucking idiot. All of your comments are embarrassing, and anyone who agrees with you is equally stupid. You are NOT "the many".

People like you are DIRECTLY what causes FPC's slow, inevitable decline in popularity. It's the same fucking thing every fucking time anybody wants to introduce anything new: a bunch of illiterate morons spouting off conspiracy-theory nonsense about how the sky is falling, who are ALWAYS, ALWAYS, ALWAYS clearly wrong in hindsight. You can go wayyyyy back on the developer mailing list for FPC and find people who unironically argued against the introduction of for-in loops the same fucking way you're arguing against this strings feature, back in like 2006. I'm not kidding. For-in loops! How fucking insane does that sound now that they've been a normal part of the language for years? Very!

So yeah, for the love of god stop fucking driving away smart people who both want to improve FPC and can also actually do the work themselves, or sometime real soon there won't be anyone left working on it at all.

Totally agree with your statement.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 24, 2019, 04:14:04 am
Came across another area today where I think multi-line strings would be quite helpful to a lot of people: writing tests!

For example, the FCL-Passrc package that comes with FPC has a rather large test suite with hundreds of procedures that take source code in string form as input. Currently, almost all of these procedures have to do this via an overload of the test input method that takes an open array of many single-line strings. You can see what this looks like in the first image I've attached to this comment "Current.png".

However, with multi-line strings, you can simply pass a lone constant to the overload of the test input method that actually takes a string directly, which is easier to write, and also means less runtime overhead for the test as well as a smaller executable for the test. You can see what that looks like (in my patched Lazarus) in the second image I've attached, "WithMultiLine.png".

There's an additional benefit also: since multi-line strings can be folded in Lazarus, this allows you to reduce a lot of screen clutter when / where you want to. You can see what that looks like in the third image I've attached, "Folded.png."
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Awkward on July 24, 2019, 05:55:33 am
Than farther, then more looks like you want to realize just your wishes but not worrying about other people and language. Please, stop to lobby your wishes! Keep pascal as pascal.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 24, 2019, 06:33:46 am
@Akira

Came across another area today where I think multi-line strings would be quite helpful to a lot of people: writing tests!
There is no doubt that the feature you're adding makes multiline strings significantly more readable and easier to insert in code.


@Awkward

Keep pascal as pascal.
could you explain how this feature leads to not "keep pascal as pascal" ?
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Awkward on July 24, 2019, 06:56:12 am
Pascal as itself don't have this multiline feature. But have mechanic to use multiline strings by standard way.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on July 24, 2019, 07:40:19 am
Pascal as itself don't have this multiline feature. But have mechanic to use multiline strings by standard way.
Yes, multiline strings can be done in Pascal without the feature Akira is adding but, that doesn't mean that adding a mechanism that makes multiline strings easier to read and code is detrimental to the language, on the contrary.

Using your logic, the case statement is "not Pascal" because any case statement can be built with standard Pascal's "if/then/else" construct and, not only that, the "if/then/else" construct allows testing variables against variables which cannot be done with a standard case statement.

Be happy that a feature that allows a cleaner construction when needed is being added to the language.  It doesn't make "pascal not pascal", it makes Pascal better.



Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Awkward on July 24, 2019, 08:13:29 am
i not sure what this feature making pascal better and easier. at least coz many people uses editors which cut trailing spaces which can be part of these multiline strings. and it makes harder to recognize "occasional strings" when you just wrote wrong (quote) char. 100% we will got some new problems with existing editing frameworks if this thing will be implemented. I'm not against language changes but if it really helpfull, necessary and don't makes new problems. but not this case (anyway, it just my personal opinion)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Imants on July 24, 2019, 08:47:30 am
i not sure what this feature making pascal better and easier. at least coz many people uses editors which cut trailing spaces which can be part of these multiline strings. and it makes harder to recognize "occasional strings" when you just wrote wrong (quote) char. 100% we will got some new problems with existing editing frameworks if this thing will be implemented. I'm not against language changes but if it really helpfull, necessary and don't makes new problems. but not this case (anyway, it just my personal opinion)

Everything can make new problems but that is not reason to not go forward and evolve. Problems needs to be solved not run away from them!

This feature is like ages in c++ and python (c++ user R("text") and python uses """text""") and I do not see anybody bashing it. Even more this feature is widely used to write complex SQL-s in our code so that you can stop using annoying "'text' +" construction witch makes code harder to read and maintain. And am already testing testing this feature using Lazarus and it works quite nice. And yes there are some hiccups like if I write word "type" inside new construction Lazarus will protest that type can't be there. But still it is not reason to not implement this feature.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Dio Affriza on July 24, 2019, 08:52:46 am
People like you are DIRECTLY what causes FPC's slow, inevitable decline in popularity. It's the same fucking thing every fucking time anybody wants to introduce anything new: a bunch of illiterate morons spouting off conspiracy-theory nonsense about how the sky is falling, who are ALWAYS, ALWAYS, ALWAYS clearly wrong in hindsight. You can go wayyyyy back on the developer mailing list for FPC and find people who unironically argued against the introduction of for-in loops the same fucking way you're arguing against this strings feature, back in like 2006. I'm not kidding. For-in loops! How fucking insane does that sound now that they've been a normal part of the language for years? Very!

So yeah, for the love of god stop fucking driving away smart people who both want to improve FPC and can also actually do the work themselves, or sometime real soon there won't be anyone left working on it at all.
Absolutely agree, absolutely.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 24, 2019, 09:15:17 am
i not sure what this feature making pascal better and easier. at least coz many people uses editors which cut trailing spaces which can be part of these multiline strings. and it makes harder to recognize "occasional strings" when you just wrote wrong (quote) char. 100% we will got some new problems with existing editing frameworks if this thing will be implemented. I'm not against language changes but if it really helpfull, necessary and don't makes new problems. but not this case (anyway, it just my personal opinion)
As Akira1364 wrote, there are already enough more critical features in Object Pascal that allows a user to shoot himself in the foot if they're not careful (and more spectacular as well). There are always ways to abuse or misuse a feature, so to simply use that to outweigh the potential uses of this feature feels like the easy way out.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 24, 2019, 09:31:49 am
Came across another area today where I think multi-line strings would be quite helpful
Speaking of ideas, would something like this be possible eventually:
Code: Pascal  [Select]
  1. const
  2.    MultiLineTemplateStr = {$i '/codegen/template1.pas'};
where template1.pas would be regular pascal file without opening and closing multilinestring character at the beginning and end of file?

Of course, probably with something new replacing $i from given example.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on July 24, 2019, 09:39:25 am
Came across another area today where I think multi-line strings would be quite helpful
Speaking of ideas, would something like this be possible eventually:
Code: Pascal  [Select]
  1. const
  2.    MultiLineTemplateStr = {$i '/codegen/template1.pas'};
where template1.pas would be regular pascal file without opening and closing multilinestring character at the beginning and end of file?

Of course, probably with something new replacing $i from given example.
Look here (https://bugs.freepascal.org/view.php?id=25536).
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: avra on July 24, 2019, 10:01:53 am
Look here (https://bugs.freepascal.org/view.php?id=25536).
Nice! I missed it somehow...  :-X
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: kupferstecher on July 24, 2019, 10:42:00 am
One thing (I didn't test, perhaps it is already designed like that).

In my opinion the following two notations should result in the same string, i.e. in the same number of line breaks.

Code: Pascal  [Select]
  1. SomeString:=
  2. `First Line
  3. Second Line`;

Code: Pascal  [Select]
  1. SomeString2:=
  2. `
  3. First Line
  4. Second Line`;

This on the other hand should insert an additional line break:
Code: Pascal  [Select]
  1. SomeString3:=
  2. `
  3.  
  4. First Line
  5. Second Line`;

This means the first line break is ommitted in case the line containing the backtick is empty (no character following). Perhaps the same could be done with the closing backtick, but I'm not sure how useful this woud be. Pasting between "untouched" backticks would be possible, but it would be less intuitive to get a line break removed.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on July 24, 2019, 06:30:00 pm
i not sure what this feature making pascal better and easier. at least coz many people uses editors which cut trailing spaces which can be part of these multiline strings.

Think about it this way: the reason that so many editors remove trailing whitespace automatically is because pretty much nobody ever specifically wants trailing whitespace. It's something that most of the time just winds up as a part of the line accidentally, and that serves no purpose I can think of in the vast majority of cases other than to unnecessarily increase the file size of the document in question.

Also, to be clear, as I've said in earlier comments, as far as my implementation of multi-line strings for FPC goes the presence or non-presence of trailing whitespace does not affect them in any way you could physically notice unless you were actively checking to see if it was there.

And am already testing testing this feature using Lazarus and it works quite nice. And yes there are some hiccups like if I write word "type" inside new construction Lazarus will protest that type can't be there. But still it is not reason to not implement this feature.

Glad you like it! As far as the "type" thing: yes, that would be due to Lazarus "CodeTools" (which is separate from SynEdit, to be clear) not currently being aware of multi-line strings. I'm quite willing to (and probably will) work up a patch for it myself, but I don't see much point in spending a lot of time on it until the actual language feature is merged to FPC trunk.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on August 04, 2019, 03:31:39 pm
I don't see much point in spending a lot of time on it until the actual language feature is merged to FPC trunk.
Just curious... how is the "future" of that feature looking like ?... has a decision been made on whether or not it will eventually be added ?

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Thaddy on August 04, 2019, 06:22:54 pm
My hunch is - and I would support that - a big no.
I really don't see the point. It is already supported as many people explained.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on August 04, 2019, 06:29:29 pm
I really don't see the point.
That is no surprise.

It is already supported as many people explained.
You must be very happy driving your Model T, it supports everything a car is supposed to do. 
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on August 09, 2019, 03:00:08 pm
My hunch is - and I would support that - a big no.

If that was the case I think it would have been made much more explicit early on (rather than laying out requirements for it to be considered and even giving words of encouragement due to how quickly I implemented them, which is how things actually went.)

Either way I intend to keep the patch up to date, and maintain the open issue on the bugtracker for as long as it takes. I'm not going to "get bored" of waiting and forget about it after a while or anything like that.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jamie on August 09, 2019, 04:13:33 pm
as I stated before, I have a side tool I use since the days of Delphi and it also works in Lazarus that allows me to edit, save, put to clipboard, back from clipboard blocked text with all the standard string '+ etc...

 This tool also uses (optional) C escape text so that I can quickly build a pascal string with all the pascal methods of inserting special characters. It has two views the raw initial view and the final pascal view..

  I really think a tool like this should be in the LCL tools instead of the compiler, but who am I ? Just some code junky born and raised in the foot hills of Maine, USA where everyone is related! ;)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: SlightlyOutOfPhase on August 10, 2019, 01:30:47 am
I really don't see the point. It is already supported as many people explained.

No it's not. Not even close. The fact that there's anyone at all who would be "against" something like this is literally insane. It makes no sense.

Look at the stuff Akira does in his tests and tell me that's possible right now. (https://github.com/graemeg/freepascal/compare/master...Akira13641:master?diff=split#files_bucket)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Thaddy on August 10, 2019, 07:39:20 am
For the record: I am not against it, but I really do not see the point.

I only take exception to code like this:
Code: Pascal  [Select]
  1. if (c in ['1'..'9']) then // <------ sigh
  2.           begin
  3.             count:=current_scanner.readval;
  4.             if (count<0) or (count>65535) then // <------- sigh some more
Both the range and the limit. See if you can spot what I mean  :o

In my view the functionality is already covered by the string helper in sysutils and without an artificial limit. Let alone the meaning of "word"..
In my subsequent view this should not be a compiler feature but a high level library feature. In that sense I consider this a design flaw, and I am not the only one.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on August 10, 2019, 01:16:14 pm
I only take exception to code like this:
Code: Pascal  [Select]
  1. if (c in ['1'..'9']) then // <------ sigh
  2.           begin
  3.             count:=current_scanner.readval;
  4.             if (count<0) or (count>65535) then // <------- sigh some more
Both the range and the limit. See if you can spot what I mean  :o

In my view the functionality is already covered by the string helper in sysutils and without an artificial limit. Let alone the meaning of "word"..
Okay, the range for c should start from '0' as other directives allow leading zeros as well, but that's all we're talking about here: detecting leading zeros. The complete number is read by readval.
And using High(Word) instead of the literal 65535 would indeed be nice as well.

That said there has been no decision yet whether we want to allow the use of helpers (or generics for that matter) inside the compiler. Also the string helper would be useless as a) the compiler uses mainly ShortString and b) there is no Char helper.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Thaddy on August 10, 2019, 01:57:31 pm
@Sven
You were not supposed to bite...
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on August 10, 2019, 05:45:41 pm
I only take exception to code like this:

"c" is one character, Thaddy. (As in, the first character of what may or may not be a valid number.) "readval" then attempts to read an actual complete number into the "count" variable.

As far as this being provided by some kind of library, that would be strictly worse, as it would be slower and could not have the capabilities it does now. As far as TStringHelper being related to this kind of thing at all, I don't get what you mean.

Okay, the range for c should start from '0' as other directives allow leading zeros as well, but that's all we're talking about here: detecting leading zeros. The complete number is read by readval.
And using High(Word) instead of the literal 65535 would indeed be nice as well.

I'll make those changes.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on August 10, 2019, 05:55:58 pm
This is a dirty feature.
Don't you understand -- in Pascal, trailing spaces at the end of a line have always been insignificant.

This is important -- trailing spaces are not visible.
You cannot see them! A programming language in which spaces in the end of line can be significant is not clean, as Pascal should be.

It is very bad.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on August 10, 2019, 06:03:13 pm
- snip -

Read my previous comment as to why I fundamentally disagree with you.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Thaddy on August 10, 2019, 06:06:21 pm
"c" is one character, Thaddy. (As in, the first character of what may or may not be a valid number.) "readval" then attempts to read an actual complete number into the "count" variable.
Well, Sven knew what I meant... :D
And as far as the helper goes, look at the interface. You can achieve the same  with few lines of code. (syshelph.inc)
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Zoran on August 10, 2019, 06:10:05 pm
- snip -

Read my previous comment as to why I fundamentally disagree with you.

I read them, of course... Nevertheless, I still hope for a little common sense...
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: 440bx on August 10, 2019, 07:14:19 pm
Nevertheless, I still hope for a little common sense...
What I'm going to state isn't associated with the feature being discussed here, just a truism which is, common sense is the least common of all senses.  (as clearly evidenced in a number of threads completely unrelated to this new feature.)


Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: Akira1364 on August 10, 2019, 07:50:59 pm
Nevertheless, I still hope for a little common sense...

Right, as do I. There's mountains of evidence in various places (and various languages) as to how this kind of feature does not cause anything resembling kind of issues you're speculating about.

Also, as far as I know, my implementation of this actually goes farther than any other language's implementation does with regards to end-user customization and error handling in the case of "unterminated" multi-line strings.
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: ASBzone on August 11, 2019, 06:25:41 am
This is a dirty feature.
Don't you understand -- in Pascal, trailing spaces at the end of a line have always been insignificant.
...


Perhaps I am missing something, but this feature doesn't change that at all.

For those who do not implement it, there is zero change to trailing space significance.


To those who implement the feature, there is zero change to trailing space significance.


If someone needs trailing spaces for a regular or multi-line string, then they have to pay attention to it for the strings in question, but it will not automatically affect every string.

Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: PascalDragon on August 11, 2019, 02:00:38 pm
@Sven
You were not supposed to bite...
It's about compiler related code, so of course I'll bite. And Akira1364 said that he'd make some adjustments...
And as far as the helper goes, look at the interface. You can achieve the same  with few lines of code. (syshelph.inc)
I still don't get though what you're trying to get at with the helpers...
Title: Re: If you've ever wanted to be able to use multi-line strings in FPC...
Post by: jshah on September 14, 2019, 01:22:32 pm
Is this FEATURE ACCEPTED BY CORE TEAM???