Character Spacing in SynEdit

[Astral]

I'm wondering how SynEdit is able to edit my source code and display nicely space characters.

When I display natural language text using Arial Unicode MS font, it seems to display the characters with a fixed spacing that is wider than the one used for editing source code in the Lazarus IDE.

Does it use a worst case spacing for the entire font, taking into account the widest possible character for the given font?

Is there any way to control the width of the characters and the intercharacter spacing?

Even English text in this font seems to be oddly spaced.

For instance the M is jammed up next to the I, as if the characters all start on the left side of an imaginary character grid.  M happens to be one of the widest characters in English.  There are some Cyrillic characters which are also very wide.

I tried to install TRichMemo as an alternative, but it did not compile or install correctly.  I can't remember what it said, but it didn't work.  I will try it again and write down what it says.

I particularly need the capabilities of SynEdit for highllighting my output text in different colors, for different purposes, flagging unknown words, part of speech display in different colors, etc.

BTW, I have been working exclusively under Lazarus for the past week or more, without going back to Delphi.  That says a lot of good things about Lazarus/IDE.  I am trying hard to take advantage of the free compiler and IDE to make my life simpler and cheaper.  Delphi should have a free version of Delphi 2009.  I know they have had free versions in the past.  I just wish they would continue the Borland tradition.

I tried several other systems, including Eclipse and NetBeans, and Microsoft Visual C++ Express Edition (had a lot of good experience out of that).

I prefer Pascal, but I can get by in C++ if push comes to shove.

 

[Martin_fr]

Hi,

Yes, SynEdit takes in account the widest possible char (out of a few tested). In your case this is most likely the "@", wich in many proportional fonts is very wide.

SynEdit is a code editor, as such it is meant to work with fixed-width fonts and not with proportional fonts.
It enforces that by enforcing the cell grid.

And yes all chars are displayed on the left of there grid cell. Displaying monospaced text, under normal circumstance means that you only need to know, on char width (And not one per char) So SynEdit does not know how much smaller an "I" is than an "M". And without that knowledge it can not center it.

If you whish to use a non proportional font you can use:
  "Environment Menu" => Options => Editor => Display:
  There is afield "Extra Char Spacing". 
You can put in any value you want, even negative values. You probably need -2 or -3.

This will help you with the over-spacing (except "@" symbols may get overlapped or cut off)

If you want to spent time on doing real proportional, I can give you pointers where to start, but it will be quite some work.

[Martin_fr]

just to add

When I display natural language text using Arial Unicode MS font, it seems to display the characters with a fixed spacing that is wider than the one used for editing source code in the Lazarus IDE.

The IDE uses Courier new. (on windows)

Edit: fixed quoting tags.

[Martin_fr]

and also:
If you want to put in some work => SynEdit already supports 2 char widths (for eastern double width chars) (look for DoubleWidth View in the SynEdit dir).

You can base your own code on this making it a proportional output.

[Astral]

gmta     (great minds think alike)

I also thought of putting in a negative "extra spacing" but hadn't gotten around to trying it yet, awaiting your response.

I appreciate your response very much.  I guess you did realize I was trying to use Synedit to display foreign language text.

I guess it would be ideal to use proportional spacing, and I'm
willing to try anything, but I also like to take shortcuts and take advantage of what's already been done, where appropriate.

For a particular natural language I have strings I have built for all the lowercase characters and all the uppercase characters.  These strings are used for sorting.  Note that a few characters still
come out as ?, not sure exactly which ones.  I allow a lot of
characters in English to accomodate foreign words and to use the standard sort orders (collation sequences) defined on the site:


http://developer.mimer.com/charts/index.tml


Notice the big variation in widths!

Thanks again for your help.  If I use SynEdit, I will definitely need
to figure out the char widths and do proportional spacing.


English:

aàæbcçdeéèêëfghiïjklmnñoôö?pqrstuvwxyz

AÀÆBCÇDEÉÈÊËFGHIÏJKLMNÑOÔÖ?PQRSTUVWXYZ

Esperanto:

abcĉdefgĝhĥijĵklmnoprsŝtuŭvz

ABCĈDEFGĜHĤIJĴKLMNOPRSŜTUŬVZ

German:

aäbcdefghijklmnoöpqrs?tuüvwxyz

AÄBCDEFGHIJKLMNOÖPQRS?TUÜVWXYZ

Polish:

aąbcćdeęfghijklłmnńoópqrsśtuwxyzzż

AĄBCĆDEĘFGHIJKLŁMNŃOÓPQRSŚTUWXYZZŻ

Croatian:

abcčćddžđefghijklljmnnjoprsštuvzž

ABCČĆDDŽĐEFGHIJKLLJMNNJOPRSŠTUVZŽ

Russian:

абвгдежзийклмнопрстуфхцчшщъыьэюя

АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ

Finnish:

abcdeéêfghijklmnoóòôpqrsštuvwxyzžåäö

ABCDEÉÊFGHIJKLMNOÓÒÔPQRSŠTUVWXYZŽÅÄÖ

Estonian:

abcdefghijklmnopqrsšzžtuvõäöüxy

ABCDEFGHIJKLMNOPQRSŠZŽTUVÕÄÖÜXY

Latvian:

aābcčdeēfgģhiījkķlļmnņoprsštuūvzž

AĀBCČDEĒFGĢHIĪJKĶLĻMNŅOPRSŠTUŪVZŽ

Swedish:

aàbcdeéfghijklmnopqrstuvwxyzåäö

AÀBCDEÉFGHIJKLMNOPQRSTUVWXYZÅÄÖ

Danish:

abcdeéfghijklmnopqrstuvwxyzæøåaa

ABCDEÉFGHIJKLMNOPQRSTUVWXYZÆØÅAA

Norwegian:

aàbcdeéêfghijklmnoóòôpqrstuvwxyzæøå

AÀBCDEÉÊFGHIJKLMNOÓÒÔPQRSTUVWXYZÆØÅ

[Astral]

Actually, the spacing is a nuisance, but perhaps the use of a negative value for "extra spacing" will help enough that I don't need the
proportional spacing.

I do appreciate the suggestions very much.

I like to take the path of least resistance.  But I'm not lazy.

I just call it efficient, not doing what doesn't need to be done yet.

 

[Astral]

I'm still struggling trying to locate "DoubleWidth View" in the
SynEdit directory.

I'm searching in  C:\Lazarus and subs for it.

No sign of it yet.

I have a few questions about the way SynEdit works.

I want to be able to highlight a specific word in a line in a
different text color, to flag it as an unknown word or as
some particular category, like part-of-speech, noun, verb,
etc.

How do I change the text color for only part of a line?

Can I set the color and other attributes for each individual
character?  If so is it reasonably efficient to do so?

The reason I ask is that I have struggled for the last few years with various versions of TRichEdit and it allows me to change the text color in mid line, but it's a royal pain, and my highlighting code gets slower and slower and slower as it gets further down into the file being displayed.  That has to do with a linear search through the entire text from the top, counting line terminators.  The TNTware version has that behavior and I think the Delphi version also has that problem, but won't swear to it.

Setting the Font.Color seems to affect all lines within a given instance of SynEdit.

Is there some kind of finer control than setting the Font.Color, Font.Size, Font.Name, etc, and then called Lines.Add?

I need to be able to change color as many times in one
line as are required to highlight the text, based on user
options, including turning on/off highlighting altogether.

I hope my question makes sense?

I do want to pursue the proportional font idea, if you can point me to the right subdirectory under c:\lazarus or c:\fpc !  Perhaps I am searching in the wrong place.  The feature you mentioned ought to be in the latest version for Windows32, right?

For the set of languages I am using, I can probably figure out the
widths of the individual characters.

BTW, the Extra Char Spacing works, but I can't control it on a line by line basis, for instance when I am showing Interlinear translations in multiple languages.

Perhaps what I need is not a code editor, but a word processor.

TRichEdit provides all the capabilities that my program needs.

I will keep trying to use TSynEdit for the moment, but I am a little perturbed I could not get TRichMemo to install.  Is there a trick to it?

Thanks for your help!  I'm sorry if I seem a little dense.  I could probably have written my own editor/word processor in the time I've spent struggling with TRichEdit on Delphi.

The original version of my program was a command line program in Turbo Pascal, and in some ways I preferred those days.  I remember reading in files I created under DOS with KEDIT and having to convert the DOS character set to Windows version of ANSI.  For Cyrillic I used transliteration.  I've been working on the program and the ideas behind it since 1980, if that gives you any idea how much time I have put into it.  It's evolved from Atari OSS Personal Pascal to IBM Pascal to Turbo Pascal to TPW, BPW, Delphi 1, 2, 3, 7, 2007, and 2009.  It's about time it should be running on Lazarus.

I'm halfway tempted to try my hand at converting the last free TNTware version of TRichEdit to Lazarus.   Is there any fundamental showstopper that would prevent that?

Any suggestions for alternative approaches would be appreciated.

I'd like to stick to Pascal, since I've been using it so many years, but I could be persuaded to use C++.

I've never built any custom components under Delphi, and the whole business of installing packages and components is confusing to me.  Sometimes I can understand how things work if I just find the right person to give me a quick overview and point me to where to go look for detailed help.

I have tried to avoid using 3rd-party software in my program, but some things are really useful and some wheels don't need reinventing.  If TNTware worked under Lazarus, to provide WideChar/Unicode versions of components, I'd be happy to use it.  I don't want to have to worry about licensing issues and redistribution fees, if I can avoid it.

I can write my own editor if I must, but SynEdit seemed like a no-brainer at the time.  I first started out trying to port my code to Lazarus about two years ago, and had some luck then, but ended up abandoning the effort.  I'm now revisiting the idea and seeing how far I can get.

So far I've been working for more than a week with only Lazarus/FPC, without going back to Delphi, so that's a good sign.

My translator program is running pretty well, but the wide gaps between some characters makes the input and output hard to read.

Some words contain endings like -ij.   They get widely spaced and they look like separate words almost, which makes reading some languages difficult for human beings, who prefer proportional spacing in the general case.

I would very much like to have never heard of a codepage, because in my application I have to deal with mixed languages which would require different codepages, but Unicode/UTF-8 makes everything work.  I prefer the WideString approach, but a lot of people prefer UTF-8.  I suppose it's a concern about wasting a lot of bytes, filling them up with zeroes that have no real significance, other than to take up space.

I do handle conversions for a few languages, some transliterations, and codepage to UTF-8 transformations, but only because some languages are not always written in Unicode on websites I run across.

[Martin_fr]

ok, a lot of text to reply to

First things first. We never spoke about which version of SynEdit you use. SynEdit has undergone major changes since Lazarus 0.9.26; even since 0.9.26.2.

I suggest if you want to work on it to use the latest version from SVN. Which is what I have.
Downloading a nightly snapshot should do. You can have more than one lazarus installation, but must use --primary-config-path="..." to ensure they use different config dirs.

1)
> "Note that a few characters still come out as ?, not sure exactly which ones."

Do you mean SynEdit does not display them correct? I tried the German sharp s, which displays fine in my SynEdit. But it may have had issues in older revisions)

2)
> "I'm still struggling trying to locate "DoubleWidth View" in the SynEdit directory."

<lazdir> \components\synedit\synedittextdoublewidthchars.pas

There are several of those View classes, doing different work preparing the about displaying of the text. The are in something like a linked-list.

You would have to replace this one, by your one.

the function returns an array of how many display cells each char needs for display (some Chinese chars need 2 cells). In this case it does not depend on the font used. so it can be told by simply looking at the unicode.

Each unicode-byte is mapped into this len-string. The display width is always on the 1st byte of a char. Continues bytes of multibyte chars are filled with 0.

eg ä is one char, but 2 bytes => would result in 1,0 in the len-string ( 1 cell, continued byte)


Now if you force lazarus to assume that each cell is one pixel width (by overriding the font detection, well then you can return what you need here.
(Make sure you know what font overhang is, and do not include it)

You can call TextWidth or similar function each time, but it takes a lot of time. Best would be to find a smart way to store char widths.
If you do find a smart way (let say store pages of 128 or 256 chars, in a tree) then you can submit the result, and I look at integrating it into SynEdit for everyone, if you want to contribute)


3)
There is an example called SynPositionHighlighter. I thing this may help.

Otherwhis you can look at the different Markup-Modules, and write your own code as Markup module:
 syneditmarkupselection.pp for marking position based
 syneditmarkuphighall.pp for all instances of a word.

Note some times coordinates, are screen coords, sometimes bytes in the line.

I don't know all the conditions you need to test for your highligting. So I do not know if this will be out of the box, or how quick it will work... or if SynEdit is the best choice.

***
There is a RichEdit for Lazarus somewhere. You may want to give that a try. it may be easier
***

Good Luck

[Astral]

I have 0.9.27 Beta working now.  I am getting ready to try the
TRichMemo to see if it can do all the things I need it to do.

Thanks for your help.  I presume TRichMemo will do most of what I need, if not all, but it will require rewriting a bunch of code, and then I will not necessarily be able to use the same source on Delphi.  Source code management is a nightmare at times, on multiple machines, different operating system versions, etc.

That warning message on the Lazarus snapshots page is kind of
scary.  I made a backup of all my sources on a flash drive, just in case.  lol

It's convenient to have weekly or even daily snapshots, but scary if everything is suddenly broken and you have to spend hours putting it back together again.

I guess I am whining a bit, but for good reasons. 

(I have a long list of nits to pick later, especially about how the gdb debugger works, but I don't want to address that here and now)

I do appreciate the help.

Somehow I can't imagine having to recompile the Delphi environment, so having to recompile Lazarus is a bit intimidating.

But I remember working with the Berkeley Pascal compiler on an early version of Berkeley Unix on a VAX 11/780, and I had to make a source code change and do a make on the compiler.  That was scary.  That was in 1982, I think.

Plus ça change, plus c'est la même chose.

Was mich nicht umbringt, macht mich stärker.