Code editor — the ability to embed images

[flowCRANE]

Because linking image files to places in the source code so that the code doesn't have any links to those images will be complicated.

It wouldn't, because the resources would be managed by the IDE, while SynEdit would be used to display them (as it is currently used to display text). This means that SynEdit would have a list of images (just like e.g. TImageList has image list data) and would know how and where to display them, and the IDE would handle the files, load into memory images and passed them to SynEdit along with data on where they should be displayed.

Because what would be saved in the proposed resource file? Line numbers in the source code next to which the image should be displayed?

Exactly. Just like in the .lfm file, there is information about what components the form should contain and where they should be located.

Probably not, because if someone changes the code in the source file (adds or removes some fragment), these previous line numbers will be out of date.

It's not a problem, because it is solvable. If you remove the line above the image (above its upper left corner position), the image should be moved up the size of one line. So exactly the same as in the case of breakpoints — if you remove the code above the breakpoint, it will move up, if you add few lines, the breakpoint will move down.

The position of the image in the code should be defined by coordinates in the form of two values - the line number and the character number (upper left corner of the image). Additional information should also be stored if pixel-precise positioning of the image is to be possible. This can be done by specifying the position of the image with four numbers, two of which would define the coordinates in lines/characters, and the next two numbers would define the normalized position of the upper left corner in the area of one character (two floats). In this way, not only would it be possible to set the image in a pixel-by-pixel position, but also the position would be relative, i.e. independent of the set font, editor zoom, DPI, etc.


@avra: yeah, nice feature! But still, in order to see the image, some action needs to be done (create a comment with a link and hover over it to see the image). What I want is for the images to be visible all the time, without any user's action and without modifying the content of the project's source files.

[440bx]

What I want is for the images to be visible all the time, without any user's action and without modifying the content of the project's source files.

I'm not sure having the image visible all the time is actually desirable.  For instance, a few lines of code that implement a somewhat sophisticated concept may require an image that takes more space than what a few lines of code take, thereby requiring blank lines to account for the additional vertical size of the image.

As I mentioned earlier, I have not given the idea much thought but, I would be inclined to just have a line "indicator" that an image is present and have that "indicator" turn into the real/full image when the caret is on that line. 

Also, I'm not sure that having a separate file for the images is a desirable solution because it opens the possibility of having source code reference non existent images (particularly if the image file is inadvertently deleted or corrupted).  Instead, I would consider using an alternate data stream in the same (source) file to store the images.

[Thaddy]

Somehow I missed this thread, but it is not such a bad idea. Back in 1998, while working for a company called PerfectView (a database engine and design environment, very popular in the Netherlands ) /Cyco software, later acquired by accruent, we wrote an editor based on a Windows TRichEdit (rtf format) derived editor that could contain images based on RTF for the purpose of visualizing database diagrams. It worked, but never made it into production. What we did instead was displaying the diagrams alongside the source code linked to line numbers. As far as I recall, our customers who saw a alpha/beta did not like it too much, so the feature was dropped. Probably due to us (me, mea culpa) not thinking the design through good enough .
The second approach would be a viable option alongside synedit. The first approach was dropped because of parsing issues. (There were no TOM interfaces at the time, at least not documented). I still must have some sourcecode based on this that can embed images in the sourcecode based on rtf and a separate parser to create the plain text. Not the production code, but my proof of concept.  Synedit can't do that. Although possible to embed, I think the second approach is better design and much easier to implement.

At the time we had access to WitLab, a behavioural science lab associated with the University of Delft, and concluded that embedding was confusing. I have no real opinion about the current possibilities, although I still think that the second option would be better.
This was one of my Hey cool! experiences (everybody at the company said so)  and then walking away from it like a dog with his tail between the legs. I could not pull it off and used up a huge amount of time and effort chasing the wrong solution. At least the company allowed me to do that and never punished me for it. Proper scientists, that lot! In the kindest, most sincere sense!

So maybe I have an opinion after all: use no embedding, but the logic from collapsible regions + line numbers to display images based on context (line numbers?) and store in and display from a separate file.

(MS TOM interfaces are available from RICHEDIT2 and later MSFT: https://learn.microsoft.com/en-us/windows/win32/controls/about-text-object-model  )

My code was a very old ancestor to the attached image, and yes, that is eventually my grand-grand-grand child - now web based, but the idea lives on! The functional specs from 1998 live on, Meridian is a direct derivative from PerfectView-:

[Marc]

The IDE already supports inline hints starting with % in a comment. The most used one is {%MainUnit  to point the IDE to the unit where from an include file is used.

You could add support for something like {%link "file://foo#bar"}, where a plugin could show the contents. (in case of a multisource file, show "bar")

Marc

[flowCRANE]

I'm not sure having the image visible all the time is actually desirable.

For me is. And this is the main reason why I started this topic and shared my thoughts, since the ”necessity is the mother of invention”.

For instance, a few lines of code that implement a somewhat sophisticated concept may require an image that takes more space than what a few lines of code take, thereby requiring blank lines to account for the additional vertical size of the image.

You don't need to make a special place for the image, because typical source code has a lot of empty space that can be used. My main point is that if I have some code that does something that is not obvious and that is difficult to describe in words, there is always empty space around it, where you could insert a small picture illustrating what is going on. It was these empty spaces that gave me the idea of embedding images.

Also, I'm not sure that having a separate file for the images is a desirable solution because it opens the possibility of having source code reference non existent images (particularly if the image file is inadvertently deleted or corrupted).

That is not an argument. After all, we already have exactly the same situation, for example, in the case of forms — the .lfm file with the form content and the .pp file with its source code. There is still a fear that the data in both files will not match, that e.g. in the .lfm file there will be an event assigned whose body does not exist in the .pp file, but the IDE can detect and handle this situation.

In the case of embedding images, there would be no such problems, because the .pp file (with the source code) would not contain any data about where to embed the images — the IDE would handle the resource files and pass the images to SynEdit. There is no resource file, so there is nothing to pass to the component. The resource file is corrupted, the IDE displays an error when loading its content with the buttons to abort/ignore loading it, just as it displays an error when loading a corrupted .lfm file.


SynEdit can handle and render anything we want, because its content is fully rendered by this component's code — every fragment (gutter, background, every line of text and every token of it). The problem is not whether it is possible to embed images and render them above/under text (because it can be done), but how to do it so that it is useful and does not hinder those who do not want to use image embedding. Well, I have already described how to do it so that these assumptions are met.

[avra]

You could add support for something like {%link "file://foo#bar"}, where a plugin could show the contents. (in case of a multisource file, show "bar")

I like it. It is similar to the already mentioned active comments idea, and avoids external file for storing additional info.

[Thaddy]

I like it. It is similar to the already mentioned active comments idea, and avoids external file for storing additional info.

And introduce sourcecode bloat and incompatibily with sane editors, like Geany? or notepad or nano 
Anyway, this will run into the same issues we identified back in 1998.

Note that it CAN be done, but it is not as easy as you think it is, not even after 25 years.

[Edson]

I was really thinking on something similar to this for a long time. Although my goal was some different. I face this problem several time when documenting software and having to review the code many times.

So I conclude a source code should be (and can be) more than simple program instructions. In fact it's. Source code includes comments and they are not program instructions.

My idea was to make a source code includes some kind of script language, that can be included in comments (could be in the  inline hints %%), without interfering with Pascal (or other programming language) program. Something like Javadoc but much more advanced. Something more like a Latex syntax that will be able to include:
- Text
- Formulas
- Diagrams (flow, UML, network, syntax, )
- Image (embedded or linked)

This "like LaTeX" metalanguage (I can't think of a better name) should be plain text and should be part of the source code without interfering. Something similar to this could be the Markdown language.

Having this information on a file, it should later parsed or rendered to obtain a document view (could be an HTML output) with text, image or diagrams.

About modifying SynEdit I think it's not recommended. SynEdit is a text editor and showing images (or something non-text) is not part of its objective. For image a simple link with a "on the fly preview" should work.

For generating a document from this "metalanguage" included in comments it should be better to create a new View in Lazarus. Maybe a parsing the metalanguage to HTML and then show it on a HTML viewer. So Lazarus just need to include a new document view and don't touch SynEdit.

[avra]

I like it. It is similar to the already mentioned active comments idea, and avoids external file for storing additional info.

And introduce sourcecode bloat and incompatibily with sane editors, like Geany? or notepad or nano 

I am sorry but I do not see how would this lead to incompatibilities. Source files remain editor compatible, even with Notepad, and since part of the comments - pascal parsing would still work in any editor.

Let's imagine some form of active comments in source files which I would like to see in IDE:

Code: Pascal  [Select][+][-]

1. // BLAHBLAH PROJECT V2.0
2. // http://www.blahblah.net
3. //
4. // Documentation:
5. // file://../doc/schematic.pdf
6. // file://../doc/board.jpg
7. // file://../doc/readme.md
8.  

In it's simplest form I can imagine IDE parsing finds comments, then additionally finds files and URLs inside of comments, and let's me CTRL+CLICK on them for opening files locally (converting relative to absolute paths first if needed) or opening URLs via default browser. There can even be an option to show floating image in mouse over event over BMP, JPG, PNG, SVG and similar supported image formats.

It doesn't matter if we use "ac:" as Mikroelektronika did, or "http:" and "file:" which I would prefer - there is nothing new and alien in shown pascal comments that external editors would have trouble with when parsing such pascal source files, so compatibility would not be threatened at all.

[440bx]

Just thought I'd mention this...

I've been trying, unsuccessfully unfortunately, to remember the name of a programming language I came across a few years ago that uses html file as its source code. 

The actual source code is simply text in a section (a specifically named div if I remember correctly.)  You use any tool that allows you to create html pages/documents (notepad can be used for that) to create the source code which means anything that can be in a web page can be source code.

[Zoran]

What would the image inside the source do? What would you use it for?
Perhaps I have no imagination, but I have no idea what would be the purpose of these images.

[440bx]

What would the image inside the source do? What would you use it for?
Perhaps I have no imagination, but I have no idea what would be the purpose of these images.

Sometimes a diagram showing the relationships between data elements can shed light on the why and how a task is carried out.

Imagine a complex "select" statement to get data from a database.  A diagram could show the tables/sets accessed as well as the keys and foreign keys being used.  A single picture/diagram would show _why_ the select statement is constructed as it is.

Also, any complex relationships between data structures created in memory can greatly benefit from a diagram that shows their inter-relationships.

[flowCRANE]

What would the image inside the source do?

They would be... displayed and act like a sticky notes.

What would you use it for?

They can contain anything, and I would use them to illustrate what the code does, if a verbal explanation of how it works in a comment would not be enough. Small drawings (something like cliparts) placed in some places, next to the code, the operation of which is difficult to explain, but very easy to visualize.

[Martin_fr]

Ok having read through (or partly just looked over) the discussion...

The beginning feels a little bit like: Lets write an add-on to "whatever software", let's start with where to but the enable-button. Important, but the last step you usually do.

- How does it get the image (ex- and in-cluding IDE integration)
- Who does it affect

=> Synedit is pretty modular. All that is the work of a plug-in (and in the very end one or two opts in the IDE)

1) Anyone using SynEdit will never know until they use the plug-in.
2) Anyone in the IDE won't be bothered, unless they activate the option.
  Well, they will, if they are forced to read sources with special comments, then those comments will be there. (But so will be other silly comments, that may be put in instead)



Ok, so let's see where to start

Pre-claimer:
1) I do not have any time of my own to do anything on this.
2) References to "word wrap". This exists in very alpha. It's code will still change a lot. So even small changes like "make this protected and/or virtual" may not be acceptable. if there is a faint possibility that "this" might not exist past the alpha.


Painting:
Hooking the paint handler (somewhere in TLazSynSurface / SyneditTextArea should be no issue. A plug-in can then use those hooks, and ok.

It's still good to learn about SynEdit's token based panting (including that future optimization may include collectively pre-painting background for several tokens).
At the same time, familiarize yourself with "text-views" / and "TLazSynDisplayView". May net be needed, but could be helpful.

Image location (to the right, below, background, even text-flow)
Almost entirely a decision of the plug-in.

I am not sure, but it might be that "background / behind the text" may be the hardest.
Because IIRC(!) to the painter the background of un-marked text (i.e. the editor background color, usually white) in handled exactly like the background of any markup.
And if that "normal white" is not distinguishable from the dark-blue of a selection, then it won't be easy to put something in between.

Image height - allocating space/lines
This is the hard one. Albeit maybe/probably already solved-ish.

An image does not fit into a single textline (in most cases). I don't know if the original idea is to paint it behind the text of lines below, but if that is the idea, then sorry but it is crab: There may be the next image already.

Here is where the beta of the word wrap comes in. Add word wrap. hook the calculation of the word count. increase it to get the blank lines to fit the image.

The hook can decide => if the image isn't shown, then don't add the height. (and don't paint the image).
Folding wont offer help here. Even if wrapped folding only sees ONE line, and ONE line can not be folded.

   Alternative image at end of line: Paint image only for the active line (avoid conflict with next line), and show then on the right, after EOL of the longest visible line.
       However to get horiz-scrolling to work, you need to hook the calculation of the actual longest line. So there is enough space on the scrollbar.

The rest
When you got painting nailed, you can add finding/loading/retrieving images.

And you can add user interaction. Any SynEdit plug-in can add keycombos and mouse actions.

[Marc]

You could add support for something like {%link "file://foo#bar"}, where a plugin could show the contents. (in case of a multisource file, show "bar")

I like it. It is similar to the already mentioned active comments idea, and avoids external file for storing additional info.

I hoped the "file:" hinted to use an external file. I'm in nowhere pro embedding image sources into code.