Code editor — the ability to embed images

[furious programming]

Treat the following proposal rather as my thoughts on the code editor functionality I desire, because I doubt that it will ever be actually implemented. I've been thinking about it a lot, which I think is a pretty cool idea, so I decided to share it with you.


I'm currently working on video game code that does more complicated things and calculations in many places. I create the code in such a way that I add comments to it describing more or less what a given fragment does. Unfortunately, in many cases it is difficult to explain in words what a given code does, which forces long descriptions (and which ultimately do not allow to clearly explain what is going on).

Such long descriptions packed with complicated nomenclature in many places would not be necessary if they could be replaced with a picture — a graphical diagram illustrating what it is all about. It would be useful not only to create graphical descriptions of how code fragments work, but also to avoid the need to create e.g. tables in ASCII art and other similar things.


How would it look and work?

SynEdit would have to implement the ability to embed images that would be rendered under the text (source code) as a background for the code. The z-order of such an image (in the rendering pipeline) would be the lowest, so that currently supported content will always be rendered over such an image (text, line highlight, text etc. etc.) — the area of the control would first be filled with a solid color, then the images would be rendered, and finally the source code. Popular image file formats with raster graphics (.png, .bmp, .jpg) should be supported, but it would also be nice if there was support for vector graphics, at least .svg (they could be scaled nicely when zooming in the editor code).

Handling embedded images would require two modes — background mode and edit mode.

The first mode would mean that the image would just be rendered as the editor background, below the text (source code). You should be able to write code over that image, and clicking on that image wouldn't cause any interaction with it — you'd just place the caret there, and you'd be able to select the text over the image, so edit the code as you normally would.

The second mode would allow you to modify the embedded image. Activating the image editing mode would mean holding down the appropriate key combination and clicking the appropriate mouse button on the image. When clicked in this way, the image would be rendered in the foreground (above the editor content) and would have visual frame and anchors to stretch it (just like in the forms designer). Left-clicking on the image would allow you to move the image around in the editor. Right-clicking on the edited image would open a context menu with useful options — copy, cut, delete the image; z-order settings relative to other embedded images; priority settings (rendering under or over code), scale settings etc. etc. The visual representation of the edited image and the content of the context menu would be more or less in line with the functionality of the current form designer, so it will be simple to use and intuitive.

To maintain backward compatibility and compatibility with other IDEs and text editors, embedded images would not be a physical part of the source code file. The source files would still be plain text files, but the embedded images would be stored in separate files, one for each module (if a module doesn't have a single image embedded, its resource file shouldn't exist either). Assuming we have a unit named MyUnit.pp, after embedding the first image in it, a file named MyUnit.pp.res should be created next to it (this is just an example) and the image should be saved in it, in an encoded form (just like in the .lfm files).

Implementing such functionality should not be difficult. Rendering an image on a SynEdit canvas is just adding another step between rendering a solid background and highlighting and line content (text). The image panning and stretching mode can be implemented so that the image will be converted to a visual component (windowed so that the LCL is able to display it in the foreground), embedded in SynEdit. Then it will be easy to implement mouse support and context menu. The rest is the implementation of handling files with resources, on the side of the IDE code.


Overall, it would be nice to have this functionality. I would certainly use it myself, and whoever did not need to use it would not have to and their source files would have the same form as they are now (backwards compatibility would be preserved). I am curious about your opinions on this topic. Would you like to be able to embed images? Would you use it? Or maybe not only images, but also other types of resources?

[KodeZwerg]

Would you like to be able to embed images?

No.

Would you use it?

Hell no, but that is just my personal opinion.

Or maybe not only images, but also other types of resources?

Not a resource but a coloring for methods, example method A is yellow backgrounded, method B is blue and so on to have an easy overview where I am.

//edit
With background color I really do mean a backgroundcolor, like basecolor of IDEs background with a slight transparent overdraw of my wished color, to keep contrast in a readable good way

[440bx]

I am curious about your opinions on this topic. Would you like to be able to embed images? Would you use it? Or maybe not only images, but also other types of resources?

Being able to use images in the source to document its operation would definitely be a very nice plus (honestly, I've thought about it many times in the past.)

I haven't given the idea a whole lot of thought but, something tells me it wouldn't be easy to implement smoothly and in a platform independent way.  For instance, even if the source file had images, I would still want to be able to edit it using any regular text editor, which means the images cannot get in the way of the "vanilla" editors out there, not to mention that the image data cannot get in the compiler's way either.

[furious programming]

For instance, even if the source file had images, I would still want to be able to edit it using any regular text editor […]

That's why I suggested that images embedded in the code editor should be saved to a text file, e.g. in the same format as images and other non-text resources used in visual components and stored in .lfm files. The API for parsing such files already exists, so it can be used right away.

[…] which means the images cannot get in the way of the "vanilla" editors out there, not to mention that the image data cannot get in the compiler's way either.

And it would be if only these images were stored in separate files. You would still be able to edit the source code file in any test editor as no metadata about the embedded images would be stored in the code file. The compiler, on the other hand, would not take such files into account at all, it would have nothing to do with them, so you could easily compile the code as it is today (using FPC or any other compiler).

In summary, image embedding support would only apply to SynEdit and Lazarus, the compiler would not use these additional files at all.

[progmokus]

Maybe the link embedded in the comment that Synedit can handle? That way it doesn't bother the translator. That way you can add anything.

for example:
   // [read.txt] [image.jpg]
ctrl+mouse click to activate...

[VisualLab]

Maybe the link embedded in the comment that Synedit can handle? That way it doesn't bother the translator. That way you can add anything.

for example:
   // [read.txt] [image.jpg]
ctrl+mouse click to activate...

This is already some concrete (but not construction concrete Maybe some IDE directive that attaches an image. Another solution is (loose example, nothing definitive):

• a directive is added in the code, let's say: {$#IMG12},
• in the XML file associated with the project (program, library, package) there are details related to the directives from the source files.

But really, I agree with CodeZwerg

As for diagrams (CASE). Newer versions of Delphi (I think from 2005) had a nested UML editor (but that didn't work in the source code editor). It only supported class and module diagram. I almost never used it (only a few times). I preferred separate editors. But creating such diagrams is useful when designing complex software. And it makes sense when it allows you to work both ways:

• from diagrams -> source code,
• from source code -> diagram.

Most of these programs were written in Java. And they were terribly cumbersome to use (this is my opinion, maybe others were satisfied). The more decent ones are Enterprise Architect (Sparx Systems) or Model Maker (supported Delphi). The free ones are Umbrello for Linux.

However, an external diagram editor may suffice. Otherwise, Lazarus will become a mare similar to Visual Studio

[furious programming]

@VisualLab: I have a feeling you didn't understand what's going on.

The point is that the images should be displayed along with the code, that is, be embedded in SynEdit and rendered as SynEdit content is rendered. Secondly, image handling should not modify the content of the source code file, nor should it force modification of the compiler code, therefore, the implementation of image embedding cannot be done in the form of new directives. Thirdly, I am interested in embedding images of any content (freehand drawings, diagrams, tables, screenshot clippings, etc., etc.), not just diagrams.

I would also like to point out that it would be important that after the implementation of such functionality, it does not interfere with the work of those who do not need it. Lazarus has a million different functions, but if someone doesn't need one, they turn it off (or don't activate it) and have nothing to do with it. In this case, it should be the same — if someone doesn't want to support images in SynEdit, they don't activate this functionality, the images will not be displayed in the code, and the IDE will ignore these additional files (if the project already contains them).

[KodeZwerg]

@VisualLab: I have a feeling you didn't understand what's going on.

The point is that the images should be displayed along with the code , that is, be embedded in SynEdit and rendered as SynEdit content is rendered. Secondly, image handling should not modify the content of the source code file, nor should it force modification of the compiler code, therefore, the implementation of image embedding cannot be done in the form of new directives. Thirdly, I am interested in embedding images of any content (freehand drawings, diagrams, tables, screenshot clippings, etc., etc.), not just diagrams.

I would also like to point out that it would be important that after the implementation of such functionality, it does not interfere with the work of those who do not need it. Lazarus has a million different functions, but if someone doesn't need one, they turn it off (or don't activate it) and have nothing to do with it. In this case, it should be the same — if someone doesn't want to support images in SynEdit, they don't activate this functionality, the images will not be displayed in the code, and the IDE will ignore these additional files (if the project already contains them).

Just another 2ct of me, I understand your wish to have a fancy image behind parts of code but in my thinking:
 - give your source to somebody (with images) = not everbody uses a dark or bright IDE color, but image is either dark or bright colored = readable content will become unreadable (contrast)
 - wasting time to make your image looking good on your machine does not mean that it look everywhere the same, ie: width and height proportion may look very strange
 - programmers should program and not design pictures for their methods 

Anyway, the wish for individualism I support 

[Fred vS]

You may insert ASCII art, as comment, into your source:

Code: Pascal  [Select][+][-]

1. If blablabla > blablabli then drawthis
2.  
3. {
4. @@@@@@@@@@@@@@@@@@@@@@@@&%%%%%%%%%%&@@@@@@@@@@@@@@@@@@@@@@@@
5. @@@@@@@@@@@@@@@@@@@@@@&%#*.      .*#%&@@@@@@@@@@@@@@@@@@@@@@
6. @@@@@@@@@@@@@@@@@@@@@&%#*          ,#%&@@@@@@@@@@@@@@@@@@@@@
7. @@@@@@@@@@@@@@@@@@@@@%%/.          .*%%&@@@@@@@@@@@@@@@@@@@@
8. @@@@@@@&&&&&&@@@&&%%#*. ........ ... .*(%%%&@@@&&&&&&@@@@@@@
9. @@@@&%#(,.,*/((((*...........,,...........*((((/*,.,(#%&@@@@
10. @@@%%(.         .............//.............         .(%%@@@
11. @@%#/     ,/(#%%//(&%#/......//......*###&(/%%#(/,     /#%@@
12. @@%#*       ..((. **.*#*.....,,.....*#*.,*..((..       *#%@@
13. @@@%%#*    ......,,,,/%/............*%(*,,,..........*#%%@@@
14. @@@@@&%#* ...........,(/...........,/#*,******,,,,.*#%&@@@@@
15. @@@@@@&%(............*(*..,,,,*/////(#(*******,,,,,(%&@@@@@@
16. @@@@@@&%(,..........*#/...,*/////////(%(******,,,,,(%&@@@@@@
17. @@@@@@%%/.........,##,...,*//////////*/#%/****,,,,,/%%@@@@@@
18. @@@@&%#/. .......(%*...,#&&&&&&&&&&%/***/%#**,,,,,.,/#%&@@@@
19. @@%%#*     .....,%*....*#%&&&&&&&&%#*****/%/*,,,,.....*#%%@@
20. @@%#,        ....%/.,,*****/%&&%/********(%*,,,,.....  ,#%@@
21. @@%%#,       ..,,(%/*********(#*********/%(,,,.....   .(%%@@
22. @@@@%#/    .....,,,,,,,,*(%%####%%(*,,,,,,,,......   *#%@@@@
23. @@@@@&%#(((#%###%#(*,,,,,,,,,,,,,,,,,,,,*(#%%%%%#(((#%&@@@@@
24. @@@@@@@@@@@@@@@@@@%%%#/,...,,,,,....,/#%%%@@@@@@@@@@@@@@@@@@
25. @@@@@@@@@@@@@@@@@@@@@&%(,..........,(%&@@@@@@@@@@@@@@@@@@@@@
26. @@@@@@@@@@@@@@@@@@@@@@&%/.        .*%%@@@@@@@@@@@@@@@@@@@@@@
27. @@@@@@@@@@@@@@@@@@@@@@@%%#/*,,,,,/#%%&@@@@@@@@@@@@@@@@@@@@@@
28. }
29.  
30. else drawthat;
31.  
32. {
33. MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
34. MMMMMMMMMMMMMMMMMWWWWWMMMMMMMMMMMMMMMMWWWWMMMMMMMMMMMMMMMMMM
35. MMMMMMMMMMMMMMWNKOkxxkO0XXNWWWWWWNXK0OkxxkOKNWMMMMMMMMMMMMMM
36. MMMMMMMMMMMMMN0dl:,,,,;clooddddddolc:;,,,;:lkKWMMMMMMMMMMMMM
37. MMMMMMMMMMMWN0o;,,,,,,,,,,,,,,,,,,,,,,,,,,,,:dKNMMMMMMMMMMMM
38. MMMMMMMMMMMWKdc,',,,,,,,,,,,,,,,,,,,,,,,,,,',lkXWMMMMMMMMMMM
39. MMMMMMMMMMMW0d:,,,,,,,,,,,,,,,,,,,,,,,,,,,,',cxKWMMMMMMMMMMM
40. MMMMMMMMMMMWKxc,',,,,,,,,,,,,,,,,,,,,,,,',,',lOXWMMMMMMMMMMM
41. MMMMMMMMMMMMN0d;,,,',,,,,,,,,,,,,,,,,,,,,,,,:xKWMMMMMMMMMMMM
42. MMMMMMMMMMMMMN0d:,,,,,,,,,,,,,,,,,,,,,,,,,;cxKWMMMMMMMMMMMMM
43. MMMMMMMMMMMMMMWKkl;,,,,,,,,,,,,,,,,,,,,,;:oOXWMMMMMMMMMMMMMM
44. MMMMMMMMWNNNNWWWNKxl;,,,,,,,,,,,,,,',,,:okKWWWWNNNWWMMMMMMMM
45. MMMMMMNXOxoodkKNWWNOo;,,,,,,,,,,',,,,,:d0NWWNKkdddk0NWMMMMMM
46. MMMMWXOo:;;;;:d0NWMWKxc,,,,,,,,,,,,,;ckKWMWNOo:;;;:lx0NWMMMM
47. MMMWXkl;,,,,,;lkXWMMWKxc;,,,,,',,,,;lkXWMMWKxc;,,,,,:d0NMMMM
48. MMMN0l;,,,,,,;lxXWMMMWXOl;,',,,,,,:o0NWMMMWKdc;,,,,,;:dKWMMM
49. MMWNOl,,,,,,,;oONWMMMMWNKkoc::::cokKWMMMMMWXkl;,,,,,,,l0WMMM
50. MMWNOl;,,,,,;ckXWMMMMWWWWNXK0OO0KXNWWNWWMMMWKxc;,,,,,;o0WMMM
51. MMMWKxc;,,;cdOXWMMWNXK0KXNWMMMMMWN0xoldkKWMMWXOoc;,,:lkXWMMM
52. MMMMWX0xddk0XWMMWNKOkkkkO0XWMMMWXkc....'ckXWMWNXOkxdkKNWMMMM
53. MMMMMMWWNNWWMMMWNKOkkkkkkO0XWMMNk:..   ..:kXWMMMWWWWWWMMMMMM
54. MMMMMMMMMMMMMMMWKOkkkkkkkkOXWMMXo..     ..cONMMMMMMMMMMMMMMM
55. MMMMMMMMMMMMMMWN0kkkkkkkkkOXWMMXo..      .'dXMMMMMMMMMMMMMMM
56. MMMMMMMMMMMMMMWN0kkkkkkkkO0XWMMNk:..     ..oXMMMMMMMMMMMMMMM
57. MMMMMMMMMMMMMMMNKOkkkkkkk0XWMMMWKx;..   ..;xNMMMMMMMMMMMMMMM
58. MMMMMMMMMMMMMMMWNKOkkkkO0XWMMMMMWXx:'....;xKWMMMMMMMMMMMMMMM
59. MMMMMMMMMMMMMMMMWNX0000XNWMMMMMMMWN0xl::oOXWMMMMMMMMMMMMMMMM
60. MMMMMMMMMMMMMMMMMMWWWWWWMMMMMMMMMMMWWNXXNWMMMMMMMMMMMMMMMMMM
61. MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
62. MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
63.  
64. }
65.  
66.  

[furious programming]

- give your source to somebody (with images) = not everbody uses a dark or bright IDE color, but image is either dark or bright colored = readable content will become unreadable (contrast)

There are many problems with the implementation of image embedding, but this one — the background color of the editor under the embedded image — I have no idea how to solve yet. The syntax colors wouldn't matter too much, because the idea is to have the images next to the code (like comments), not below it.

- wasting time to make your image looking good on your machine does not mean that it look everywhere the same, ie: width and height proportion may look very strange

Not if information about the scale of the image was stored in the code resource file. This way, regardless of system DPI, code editor font and zoom, etc., the image could be displayed at the same size relative to the code. It could be done, but it is needed to take into account any code editor settings that could affect the position of embedded images.

[KodeZwerg]

because the idea is to have the images next to the code (like comments), not below it.

AHHHH  That information I somehow missed, I really thought you mean behind code you want do display an image, stretched from begin to end statements over full screen size.
With that new information... maybe it be better to display a small "box" when editor found [imageXYZ.png] for current codeblock, and when you hover over box some info displayed [readme.txt] and your image (to keep loading and scrolling smooth)

[furious programming]

[…] maybe it be better to display a small "box" when editor found [imageXYZ.png] for current codeblock, and when you hover over box some info displayed [readme.txt] and your image […]

An intermediate solution would be an icon on the gutter, and after hovering over it, it would display a window with an image. But that wouldn't be as convenient to use as the image displayed between the lines of code, visible all the time, without any user's actions.

Something similar is implemented by the Image Preview plugin for Visual Studio Code (see attachment, it has support for .svg graphics). The icons are displayed as-is, regardless of the color of the gutter, but nevertheless this plugin has been downloaded well over a million times.

(to keep loading and scrolling smooth)

Displaying a component's background image does not require complex calculations and lots of CPU time, so it won't burden the IDE significantly. Well, unless it is implemented in an absolutely suboptimal way. I suggest displaying relatively small images, but there are plugins for VSC that allow to set the wallpaper of the code editor, which forces IDE to constantly repaint the entire component, and yet people use it anyway. So don't worry about performance.

[VisualLab]

@VisualLab: I have a feeling you didn't understand what's going on.

The point is that the images should be displayed along with the code, that is, be embedded in SynEdit and rendered as SynEdit content is rendered. Secondly, image handling should not modify the content of the source code file, nor should it force modification of the compiler code, therefore, the implementation of image embedding cannot be done in the form of new directives. Thirdly, I am interested in embedding images of any content (freehand drawings, diagrams, tables, screenshot clippings, etc., etc.), not just diagrams.

But I understood you very well what you want to achieve. Only my suggestion was a bit off the mark. Because linking image files to places in the source code so that the code doesn't have any links to those images will be complicated. Because what would be saved in the proposed resource file? Line numbers in the source code next to which the image should be displayed? Probably not, because if someone changes the code in the source file (adds or removes some fragment), these previous line numbers will be out of date. You can try to associate it with some identifier (constant, variable, type, subroutine, etc.). But it will fail when someone makes changes to the code. I'm just analyzing this case. There may be a trigger point, but I don't see it yet.

[VisualLab]

Displaying a component's background image does not require complex calculations and lots of CPU time, so it won't burden the IDE significantly. Well, unless it is implemented in an absolutely suboptimal way. I suggest displaying relatively small images, but there are plugins for VSC that allow to set the wallpaper of the code editor, which forces IDE to constantly repaint the entire component, and yet people use it anyway. So don't worry about performance.

But these are just small icons (glyphs). I don't think you meant the graphics.

Visual Studio Code actually has no problem with displaying graphics, because it is a modified web browser with lots of HTML, CSS and JS files (especially the last ones). It is known that WebKit has long supported raster graphics (JPG, PNG, GIF) and one vector graphics (SVG). God forbid that Lazarus should be such a "Frankenstein" in IT (I mean the WebKit embedded inside it with adjacencies).

However, if the solution you propose would be an extension (plug-in), then I don't see a problem.

[avra]

Visit this link: https://www.mikroe.com/mikropascal-arm

Then go to "IDE" tab, and then finally go to "Active Comments" section:

Active Comments

We have developed Active Comments - a unique feature of mikroPascal PRO for ARM®. Any comment can become your multimedia event hook.

Add images, files and URLs to any active comment and assign events as you like.

Right Mouse Click can open browser with URL, Mouse Over can display the image, and Double Click can open a specified file.

You can add:

Images
Files
URLs

This is the best implementation I have seen of an idea somewhat similar to what you want.

https://download.mikroe.com/documents/compilers/mikropascal/pic/help/active_comments.htm