*.lpi XML-Structure - why not better way?

[DimProfi]

Well the question is:
Do you want to look at it, and intend to write a patch?

Exactly 

I will get feedback from the other developers, to see what can be accepted (e.g. how additions to the xml parser should look like).

If you plan to do something, I will get  back on those details. (And maybe if you are on the mail list, discussion can be moved there)

That's was what I need it!
To get a feedback from the main developers, who really decide, what is allowed to come in to the code. I want to write patch only if it will be accepted. It's not a question of code here. It's a question of XML-Structure and compatibility handling (Save As / Export / default format). This should be a mutual descision.

[taazz]

Just don't, the number of options is getting bigger by the minute there is no need to add options for file formats as well.

May it should be the fundemental requirement in XML-Structures in Lazarus to avoid this kind of naming Item1, Item2, Item3, ... ? 

Personally I do not care one way or an other, it makes no difference if the item tag is ITEM or ItemN although the ItemN is closer to the INI way of thinking and it will make it easier to revert back to an INI type of file if needed. If I had to choose then binary files all the way for me. less disk space wasted, more compact files, less disk activity on read/write, more complex to support (big/small endian etc).

[Martin_fr]

That's was what I need it!
To get a feedback from the main developers, who really decide, what is allowed to come in to the code. I want to write patch only if it will be accepted. It's not a question of code here. It's a question of XML-Structure and compatibility handling (Save As / Export / default format). This should be a mutual descision.

Well then again, you reach more of the developers on the mail list.
Anyway, I posted the Idea to the others in the team. Give it a couple of days for feedback.

Sofar, one reply, general idea OK. Depends on exact implementation.

First steps will be what to add to the xml units, so the data in the new structure can be accessed:
- conveniently
- fast

More on that later.

[Martin_fr]

Ok, sorry for the wait.... Seems everyone is ok, or does not mind. So go ahead.

Feel free to discuss this idea. IT is mean to give you a broad idea what would be a possible way of implementing this, and what requirements should be met.


First an idea, on how it could look for the IDE (though this is the last bit to be implemented / see below)

Currently there is code like
   FooSubConfObject.LoadFormXMLConf(XMLConfig, Path + 'Foo' + IntToStr(i) + '/');
   
Code like this may be nested, So Path may already Contain Unit1 or Unit2

Now if we change Unit1, Unit2, ... to Unit, Unit, Unit we need to be able to pass the selected node to those procedures.
I would suggest to keep doing that as part of the path
  Config/Path/Unit[1]/Foo[2]
 
The [] are not allowed in xml names, so we can use them. This will also make it very easy to update the IDE code.


IMPORTANT:
When reading on please keep in mind that the following must also be supported (even if not used yet, but may be in future)
  Config/Path/Unit[1]/
  Config/Path/Unit[2]/
  Config/Path/AllUnitInfo
A node like "Config/Path/" may contain children of different names, of which one or more children-names may repeat and be accessible via index
In fact any node, even if not repeated may be accessed with "[0]" in the end.

When searching through "Config/Path/" for "unit[174]" then this must be done in an efficient way. That is if "unit[173]" was already accessed (and no nodes deleted/inserted or moved then accessing "unit[174]" MUST NOT search through all "unit[n]" from index 0 again. It must use the knowledge from "unit[173]" (or have a filtered list of all "unit[n]" without other siblings, see using a hash for cache below).
If nodes where deleted/inserted, then if possible the cache should be kept/adjusted. BUt in some cases may be lost.
 
 
Where to start:
TXMLConfig uses Laz2_DOM, the latter should already support the required structure.

In TXMLConfig there is InternalFindNode  which is probably where most of the changes will be needed.
In any case before doing the IDE, TXMLConfig must support that.
And it will probably need some testcases (unittests). It has a cache, and it must be ensured this does not go wrong. Especially if in future the node name (of the node in Laz2_DOM can no longer be used as check, if the cache is valid.
In fact a testcase for proper working of the cache may be a requirement for acceptance.

Currently InternalFindNode caches the last accessed node. So if the next node is related (as a common prefix in the path), it can be accessed faster.

But if you cache a node like "Config/Path/Unit[3]"  and nodes gets inserted/deleted before "Unit[3]" then the cache must be invalidated (or set to the parent).
This is very important, otherwise user config will be badly damaged. And this is why a testcase is needed. (look up where and how the IDE code inserts/ deletes nodes, replicate them in the test. And/Or add scenarios that are likely to occur at some point in the future.

An Alternative to just caching the last entry, is to keep all known paths in a hash (and for "Config/Path/Unit[x]" the has node is "Config/Path/Unit" and contains a list of nodes.
That would be best, as it allows fast iteration in any direction, and random access of indexed nodes.
But doing the hash, is not mandatory for acceptance.

What is mandatory is, that in no occasion in which the IDE does repeated access of "unit[x]" (which x either increasing, or decreasing) there is a search for "unit[1]" and goes over all units. (see "important" above)

[Martin_fr]

Given the amount of information, I do feel this will be hard to discuss on the forum, where the display of large text  is liwited by the surrounding webpage.

Also writing replies with proper quotes is substatially harder than in a good email client.

For that reason, if you whish to continue this, it should be discussed mail based.

This can be on the mail list. But if you do not whish to use the mail list, feel free to mail me directly.  To do so correct the spelling in the first word. Lazatuss@mfriebe.de

[hinst]

How is the discussion going?

One of the reasons why units should not be stored like
Unit1, Unit2, Unit3
is that: comparing files in version control system does not work like it should.
Consider we had units:

Code: [Select]

Unit1: A
Unit2: B
Unit3: C
Unit4: D
Unit5: E
Then we remove unit B, so now we have:

Code: [Select]

Unit1: A
Unit2: C
Unit3: D
Unit4: E
Diff tool thinks than: that many lines changed, like

Code: [Select]

-Unit2: B
-Unit3: C
-Unit4: D
-Unit5: E
+Unit2: C
+Unit3: D
+Unit4: E
It's like 7 lines, and when project contains many units, there are lots of "changed" lines, and unit declarations in XML file actually take 5 lines each.
This happens because 'Unit3: C' "magically" becomes 'Unit2: C': the number changes
Actually only one line changed:

Code: [Select]

-Unit2: B......... So numbers are "spoiling the view".
If we don't have numbers, then modification will be detected correctly:

Code: [Select]

Unit: A
-Unit: B
Unit: C
Unit: D
Unit: E
So when developer looks at commit info, he can immediately tell what changed: a unit reference was removed from project
Same goes for other indexed XML elements in project file
---
Addition: indexed XML elements make impossible manual editing project files in text editor
For example, in QT project file one declares files like this:

Code: [Select]

HEADERS += menu.h
HEADERS += window.h
In case of Lazarus if I want to add a unit to project file manually, I have to change number of each unit below, and also do not forget to change the "count" value