Code editor — the ability to embed images

[Martin_fr]

You could add support for something like {%link "file://foo#bar"}, where a plugin could show the contents. (in case of a multisource file, show "bar")

I like it. It is similar to the already mentioned active comments idea, and avoids external file for storing additional info.

I hoped the "file:" hinted to use an external file. I'm in nowhere pro embedding image sources into code.

I don't think anyone wants to embed the actual binary data of the image.

The question is rather,
- are there special comments embedded that say "{%display-after-this-line img xyz.png}"?
- or is there an external map file "unit1.imgmap" that contains: "line=211;img=xyz.png"?

The latter meaning, that anyone who doesn't want the images, will not even be bothered by some cryptic comment.


Now, I have myself used the argument: It will affect me, because I may have to read the code.
Usually when it came to suggestions of replacing pascal grammar with c-like grammar.

So, if anyone is concerned about having to read those comments that are of on use to him.... I do understand.

I just also believe, that when it comes to comments, there is no control over that.
Those who would use image, would then (in absence of a plug-in) add a comment
   // See doc.pdf page 13 img 122
Then to those who don't use images that is the same as {%img xyz.png}

Same if it is some ascii chart instead, or ....

Therefore IMHO allowing a plug-in, that uses such special comments, is not going to worsen the situation for those who don't use it.
Potentially the opposite. => SynEdit could be (and with afaik rather little effort) be told to collapse comments (that match a pattern, like "%img"). And it could do that within a line. So even if the line is not folded away, it would just remove the text from the display of the line (or replace it with just a {} in a special color.)
Such a filter however works best if people use a fixed format for this sort of comment, and with a plugin they do.

[furious programming]

The question is rather,
- are there special comments embedded that say "{%display-after-this-line img xyz.png}"?
- or is there an external map file "unit1.imgmap" that contains: "line=211;img=xyz.png"?

The original idea, what I suggested myself, was not to touch the contents of the unit/file. The source code file was not supposed to contain any information about embedded images (no comments or special directives).

Information about the images would be in an external file (in serialized form, such as .lfm or .lrs), this file would be loaded by the IDE, the images would be deserialized and passed (as objects) to SynEdit along with information about where the image should be displayed (and other parameters).

Also the original idea was that the image should be visible all the time, like in PDF or Word documents. The image should be easily moveable (using the mouse + some keys) and also be rendered behind the text (in code editing mode, which is basic mode) or in front of the text (in image editing mode).

The latter meaning, that anyone who doesn't want the images, will not even be bothered by some cryptic comment.

Exactly. If handled images as I suggested, then someone who doesn't want to see images in code would never see them and wouldn't even know that images can be embedded. Also, would never deal with such a special image resource files, because if someone will not embed any image, then the IDE will not create an image resource file.


A few days ago I was looking at the SynEdit source code and I was able to get the component to render the image pretty quickly. Quick test, ugly hack (see hinghlighted line) in the TCustomSynEdit.Paint method:

Code: Pascal  [Select][+][-]

1.   Include(fStateFlags,sfPainting);
2.   Exclude(fStateFlags, sfHasScrolled);
3.   TSynPaintEventHandlerList(FPaintEventHandlerList).CallPaintEventHandlers(Self, peBeforePaint, rcClip);
4.   FScreenCaret.BeginPaint(rcClip);
5.   // Now paint everything while the caret is hidden.
6.   try
7.     FPaintArea.Paint(Canvas, rcClip);
8.     DoOnPaint;
9.     // paint images here
10.   finally
11.     UpdateCaret; // Todo: this is to call only ShowCaret() / do not create caret here / Issue 0021924
12.     FScreenCaret.FinishPaint(rcClip); // after update caret
13.     TSynPaintEventHandlerList(FPaintEventHandlerList).CallPaintEventHandlers(Self, peAfterPaint, rcClip);
14.     {$IFDEF EnableDoubleBuf}
15.     EndPaintBuffer(rcClip);
16.     {$ENDIF}
17.     Exclude(fStateFlags,sfPainting);
18.   Include(fStateFlags, sfHasPainted);
19.   end;
20. end;

The image was displayed nicely in the editor window in the right place and, interestingly, reacted to scrolling. However, the image was always rendered over the line content (over text, line selection, breakpoints, etc.) and the caret always over the image, which looked pretty funny (but at least the caret was always visible).

From my research, it turns out that my proposal can be implemented and act correctly, but the component code would have to be modified so that rendering the content of the line interacts with images. All the problems you raised, @Martin_fr, have a solution, so all you need to do is simply modify the code of the component — either permanently change its code, or rework it in such a way that you can overwrite some things (methods) from the plugin level and thus have a complete control over content rendering and content modification handling (the plugin would need to know when lines are added and removed). You'd also need to override mouse and keyboard handling to allow for two SynEdit modes — code editing and image editing.

The only problem that cannot be solved (taking into account my proposal) is the support for embedding images in conjunction with enabled word wrap. Therefore, to implement my proposal, word wrap would have to be disabled and locked in this state (which would be the only sensible solution). Anyway, I'm not going to use word wrap anyway, because I don't see any sense in wrapping source code lines, so disabling this function wouldn't bother me at all.

[Martin_fr]

Yes, all of it can be solved, of course. After all, we got the source code.

Off course code needs to be done as plug-in. But some hooks may be possible to be added.

I am still not clear on how you are solving line-height?
I assume (some/many) images are higher than a single text line?
Do you want to display them as background for the lines below? But what if those have also images?

SynEdit is painting line by line. And that is unlikely to change. So if you want any solution that uses space belonging to lines below, you likely need to slice the image and paint it slice by slice.
(Well at least as far as memory serves me here ....)

Code: Pascal  [Select][+][-]

1. unit LazSynTextArea;
2. procedure TLazSynTextArea.PaintTextLines
3.   procedure PaintLines;
4. ...
5.       // Delete the whole Line
6.       fTextDrawer.BackColor := colEditorBG;
7.       SetBkColor(dc, ColorToRGB(colEditorBG));
8.       rcLine.Left := EraseLeft;
9.       InternalFillRect(dc, rcLine);
10. ...
11.       while FTokenBreaker.GetNextHighlighterTokenEx(TokenInfoEx) do begin
12.         DrawHiLightMarkupToken(TokenInfoEx);
13.       end;
14.  

Despite the "delete the whole line", the background is painted again with each token in:
(Well maybe there are 2 or 3 pixels at the side that are distance to the gutter, and not painted by tokens)

Code: Pascal  [Select][+][-]

1.   procedure DrawHiLightMarkupToken(ATokenInfo: TLazSynDisplayTokenInfoEx);
2. ...
3.     (* rcToken.Bottom may be less that crLine.Bottom. If a Divider was drawn, then RcToken will not contain it *)
4.     TxtFlags := ETO_OPAQUE;
5.  

So basically there does not exist a place where you could hook between the background and the foreground being painted.
Never mind, finding out if background is a markup color, because the DrawHiLightMarkupToken does just know the background color for that token, it does not know if that is the editor background or a highlight.

Adding a hook in the middle of DrawHiLightMarkupToken => No. Because this code will eventually be rewritten (there is at least one existing bug, that requires this). And I have no plan to make that any harder than it needs to be.

Without having spent to much thought....

If a plug-in using TLazSynDisplayView modifies the token background color, and sets it to transparent, the the code in DrawHiLightMarkupToken could be adjusted to honour that.

And in that case, it may be possible to have a hook in PaintLines: OnBeforePaintTokens
And there you could paint the correct slice of the image.

Personal note... I am ok with having an image as background to the line...

(In case it applies: ) I don't like that image can overlap into lines below. But what happens in your plug-in, is your business...

[furious programming]

Off course code needs to be done as plug-in. But some hooks may be possible to be added.

Or as a fork of the current component, with the rendering engine reworked to handle images.

I am still not clear on how you are solving line-height?

The only sensible solution is to remember the scale of the image (at the moment of adding the image to the code and its modification) in relation to the height of the line. Thanks to it, it will be possible to calculate the target size (size on the screen), and thus the number of lines in which the image is to be rendered. If the image was moved by the user or resized/scaled, the component would update the calculations related to the number of lines it occupies.

Given that SynEdit is rendered line by line (like ListBox and so on), image rendering must be done in chunks. When rendering a given line, the component would calculate the part of the image visible in it and render it, respectively, behind or on-top of the text (depending on which mode is active). If the user would change the position or size of the image, the component would first recalculate its area (as above), and then repaint all lines where a fragment of the image should be visible. However, when writing code, the component repaints only edited lines, so it would repaint the part of the image visible in the edited lines.

A long time ago I was playing around with the ListBox component where I did something like this — the component had a graphical background and items were rendered over it. The difference, however, was that the background was rendered (in chunks) underneath the item text, and also that they were plain text with a transparent background, which was easy to render.

I assume (some/many) images are higher than a single text line?

The vast majority of images would be higher than a single text line.

Do you want to display them as background for the lines below?

Yes, but that's what the single line rendering code would do, so that each line can only paint the relevant portion of the image. If you had a situation where many lines had to be repainted, you would repaint those lines by calling the appropriate method for each such line — that's not a problem.

But what if those have also images?

This is no problem either. Since the component would know how many images there are in the document, where they are (coordinates of the upper left corner, expressed in line/character number) and how many lines to occupy, the method that paints a single line would simply check how many images should be visible on a line and would paint a fragment of each, taking into account their internal Z-order (e.g. the order of objects in the full list of images of a given document).

It is possible, but to make it possible, the entire rendering process of the component must be programmed in such a way that each painting activity can be separated:

• line background — solid color rectangle,
• image chunk — if it's code edit mode, the image must be painted as first, below the entire content of the line,
• line contents — everything that is currently rendered in a line, i.e. all tokens with syntax highlighting, selections, highlights, frames, breakpoints, etc.,
• image chunk — if it's image edit mode (manipulating its position, size and settings), it must be painted last, above the entire content of the line.

To make this possible, tokens and rectangular selections must be painted in a way that uses transparency. Otherwise, the text in front of the image will be rendered with a rectangular, single-color background, which will look ugly.

[avra]

Now, I have myself used the argument: It will affect me, because I may have to read the code.

It will not affect you if it is implemented using just "http:" and "file:" in the comments. Without IDE plugin everyone would read it as a comment, and will be able to locate links manually if needed. IDE plugin could just provide CTRL+CLIK event to open link via external application (URL, PDF, JPG, MD...), and optionally display mouse over visual hint for supported image formats. All source files stay as they are, no additional external files, and no need to add any new data into project files. I can not imagine it being less intrusive then that.

I would not like it if it plugin could show images to look embedded in the source, no matter how it is implemented, so in such case I would also like code to stay readable without installed plugin.

[Martin_fr]

Or as a fork of the current component, with the rendering engine reworked to handle images.

Well that have to be determinated when you have more details what you want to change, and where you need access.
I pointed you in the general direction.

If you have questions on how specific stuff currently works... Or if a hook (or virtual method)  can be introduced...
Please ask.

[furious programming]

Ok, thanks for the replies and for the offered help. Nevertheless, since it turns out that almost no one is interested in the functionality of embedding images in the source code, there is rather no point in digging into this topic and trying to do anything serious. Maybe someday I'll come back to this topic.