Recent

Author Topic: Exploring FPC/Lazarus units and packages  (Read 9491 times)

TRon

  • Hero Member
  • *****
  • Posts: 3770
Re: Exploring FPC/Lazarus units and packages
« Reply #45 on: July 18, 2024, 07:47:44 am »
For the record should anyone stumble across this:  include also

https://gitlab.com/freepascal.org/fpc/documentation.git

As a source of useful documentation (thanks to dsiders for the link).
I would rephrase that to: that is the source (code) for all existing official documentation that is listed here.

The existing documentation (PDF, HTML, CHM helpfiles) gets build from that. Though keep in mind it is a repository so more up to date than (existing) generated documentation.
« Last Edit: July 18, 2024, 09:43:29 am by TRon »
I do not have to remember anything anymore thanks to total-recall.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5802
  • Compiler Developer
Re: Exploring FPC/Lazarus units and packages
« Reply #46 on: July 18, 2024, 09:25:55 pm »
The official location of the FPC documentation is here.

You haven't read the thread, have you?

That exhibits /precisely/ the crap indexing problems that I'm highlighting.

I have read the thread, but my reply isn't about any indexing problem, but that Curt Carpenter mentioned something as “first step” that shouldn't be the “first step”.

Curt Carpenter

  • Hero Member
  • *****
  • Posts: 574
Re: Exploring FPC/Lazarus units and packages
« Reply #47 on: July 19, 2024, 12:03:43 am »
With respect and without seeking an argument, I stand by the link I provided

( https://wiki.freepascal.orgLazarus_Documentation#Free_Pascal_Compiler_Documentation )

as a "first step" to newcomers exploring the range of documentation for FPC/Lazarus.  Its contents begins with, for example, some links to tutorials for both pascal and lazarus, gives a very broad (and motivational I think) view of the rich variety of other sources of information available to someone new, and does in fact have links to the official reference documents (even though those links are sadly broken (a not uncommon thing in open source information, but just a part of the way it is).

The excellent and comprehensive formal documentation files are correctly identified as references -- and for reasons others have discussed, they are hard to use unless you have a good idea of what specific question or problem you are trying to address.   Even then, they can be problematic -- also for reasons discussed elsewhere here and in the forum generally. 

People with deep expertise I think -- and it's no criticism, just human nature -- tend to forget what it was to be a newcomer just starting up a learning curve (or maybe half way up but still trying to learn). 

Fair enough though:  the question is certainly open to a variety of opinions and perspectives.  Maybe the best place for a newcomer to start is right here in the forum with all its questions, answers and problem encounters.

 


TRon

  • Hero Member
  • *****
  • Posts: 3770
Re: Exploring FPC/Lazarus units and packages
« Reply #48 on: July 19, 2024, 03:07:12 pm »
People with deep expertise I think -- and it's no criticism, just human nature -- tend to forget what it was to be a newcomer just starting up a learning curve (or maybe half way up but still trying to learn). 
Although I do not consider myself being someone with deep expertise I am able to get by pretty well. At least I do not have to ask many questions anymore as I am usually able to find the answer myself (I already mentioned in one of my previous posts the paths I take in order to get there) .

But for sure I remember the time that nothing seem to work as advertised if there was any documentation to begin with. Very frustrating.

Most of the knowledge I gained wrt pascal programming comes form TP, BP and Delphi and the numerous books that where published back in the day. I still own books with titles as "the tome of Delphi" and "Delphi unleashed" amongst others. The Delphi accompanied Helpfiles literally thought me how to write relational databases with Delphi though I already had experience with RLDB's using DBase.

I can most certainly sympathize with newcomers because a lot has changed (added) over the years and it is a lot to digest. You simply can't learn all that in a couple of days (or even months) and it requires quite a lot of experimenting especially if you do not have something like a guide that is able to help you through that process. But even then, I you honestly think there is people out there that know every existing component from hearth then you got another thing coming. It is something I am still learning about to this day and it is something that will never end.

I more or less disagree with just simply pushing your question to the forums in the hope that will be able to teach you something because it really doesn't. Not unless you put in the time and effort to understand what solution was provided and do your own research based on that. In practice most people tend to just copy-paste an answer with no clue whatsoever with no intention to find that clue either (at least that is my experience though there are some that you can see that really do something with a provided answer and for whom you are able to see and watch them grow over time).

It is far better to try first and when stuck (for example when you read the reference of a component that is conflicting for you and you don't get things to work) ask specific questions about a topic. A simple "how do i use component x" is usually frowned upon especially since when you got the name for the component you can search for it. 9 out f 10 times there exist a wiki article or there is an example present that show how to use the component.

The only way that is ever going to improve is if someone writes a detailed documentation for each and every existing component showing all kind of use cases in which the component can be used. tedious, tiresome and boring work that nobody really wants to waste time on/with. Most wiki articles only mention some specific details about a component and which is mostly based on questions asked in the forums (or otherwise). Ofc there are exceptions such as for a simple example the rgba components but even there it is only the tip of the iceberg of what those components/classes can actually do.

2 cents.

I do not have to remember anything anymore thanks to total-recall.

Curt Carpenter

  • Hero Member
  • *****
  • Posts: 574
Re: Exploring FPC/Lazarus units and packages
« Reply #49 on: July 19, 2024, 08:03:04 pm »
Thanks for sharing your thoughts.

MarkMLl

  • Hero Member
  • *****
  • Posts: 8090
Re: Exploring FPC/Lazarus units and packages
« Reply #50 on: December 25, 2024, 02:04:41 pm »
Download links for all formats and versions:

https://sourceforge.net/projects/freepascal/files/Documentation/3.2.2/
https://sourceforge.net/projects/freepascal/files/Documentation/

Thanks, I'll investigate.

I don't promise I'll be able to do much even as a POC: from my POV the window of opportunity closed years ago. Also (noting Marco's stricture that I use the CHMs) a lot depends on wjat utilities etc. I can find to process the various formats.

FPC's chmls appears to violate the "Software Tools" philosophy by not having a facility to output to stdout so that it can be part of a pipeline.

(Debian's) version of archmage looks as though it's not been tested for a few years.

So I think I'm in the territory of needing to write a wrapper for chmls so that I can (pretend to) pipe its output through lynx or something else which will give me plaintext, so I can extract the synopsis (or en equivalent paragraph) and start processing it.

Which I think highlights why the last time round I started with the PDF files: much of the merging has already been done (apparently without much information loss), and extracting text is relatively simple.

After which I think I'll also look at older versions, so that I know in advance whether anything gross has changed in the format.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dsiders

  • Hero Member
  • *****
  • Posts: 1324
Re: Exploring FPC/Lazarus units and packages
« Reply #51 on: December 25, 2024, 04:11:21 pm »
I went on from there trying to generate a "this was added in version..." document, but that might have been when I got bogged down in file formats and the requirement of matching files and utilities... again, this was some years ago and I don't remember.

(it does assume everything was documented in the version it first appeared).

Quote
Specific points: A large number of fields, constants etc. are highly relevant but not indexed.

One of the problems is that the target that the document is generated (and the source is parsed with) influences the docs. So no careful annotation "this is only for windows" or so. If it is not visible on Linux, then it is not there. Period.

Quote
This also applies in spades to helpers: a few days ago somebody was insisting that /of/ /course/ you'd use the string .Contains helper for something BUT unless you already know that TStringHelper is in SysUtils it's quite simply not indexed.

But is that reference guide indexing, or lack of introductionary text? I think the latter.

Indexing primarily.

TStringHelper appears in the Index - so I'm not sure what the complaint is for that particular one.

But Contains() does not. Same for any method or property in a class, object, or record. They're not  in the the index... because FPDoc doesn't generate index entries for them.

Contains() method
  in FPCBitSet
  in TPropInfoList
  in TList<T>
  in TRect
  in TSortedSet<T>
  in TSortedHashSet<T>
  in TStringHelper
  in TValueAnsiStringHelper
  ...


I stand corrected. They are present in the CHM - just not in the HTML. My bad. I'll shut up now.
« Last Edit: December 25, 2024, 04:51:13 pm by dsiders »
Preview the next Lazarus documentation release at: https://dsiders.gitlab.io/lazdocsnext

MarkMLl

  • Hero Member
  • *****
  • Posts: 8090
Re: Exploring FPC/Lazarus units and packages
« Reply #52 on: December 25, 2024, 07:04:03 pm »
I stand corrected. They are present in the CHM - just not in the HTML. My bad. I'll shut up now.

I think they're in the HTML, but there's no mechanism for helpers to be crosslinked to the class to which they're applied. Now I /know/ that there's issues here: the optional nature of even the sysutils unit, and the (initially at least) linear structure of helpers, but I'm sure there's ways around it: if necessary artificial injection of the mere potential existence of a helper into the indexing of a class etc.

Is the consensus that the CHM files are the best place to start, short of going back and parsing the original source units?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dsiders

  • Hero Member
  • *****
  • Posts: 1324
Re: Exploring FPC/Lazarus units and packages
« Reply #53 on: December 26, 2024, 03:32:33 am »
I stand corrected. They are present in the CHM - just not in the HTML. My bad. I'll shut up now.

I think they're in the HTML, but there's no mechanism for helpers to be crosslinked to the class to which they're applied. Now I /know/ that there's issues here: the optional nature of even the sysutils unit, and the (initially at least) linear structure of helpers, but I'm sure there's ways around it: if necessary artificial injection of the mere potential existence of a helper into the indexing of a class etc.

Is the consensus that the CHM files are the best place to start, short of going back and parsing the original source units?

MarkMLl

All I know is that the HTML index does not have an entry for TStringHelper.Contains() and the CHM index does.  HTML does not include methods, properties, events, or members in the index. CHM does. I also know that the HTML docs do not have a search page that provides the searchable permuted index for every word in a help topic, but the CHM docs does. So draw your own conclusion about which would be best to work with.

The lack of linking for types like String, Integer, et. al. is problematic. But equally troublesome is the fact that source declarations in the Help output do not match the actual source code. TStringHelper for instance:

In generated Docs (both CHM and HTML):
Code: Pascal  [Select][+][-]
  1. type TStringHelper = type helper

The actual source declaration is:
Code: Pascal  [Select][+][-]
  1. type TStringHelper = type helper for String

There is your linkage (clickable or not). And that's an FPDoc issue that needs to be addressed.

The other stuff about version history and guided task completion is not an index issue. They require tagging metadata that just isn't present in the source or the help topics. Not that they aren't good ideas or useful... there's no way to automatically generate it.
Preview the next Lazarus documentation release at: https://dsiders.gitlab.io/lazdocsnext

MarkMLl

  • Hero Member
  • *****
  • Posts: 8090
Re: Exploring FPC/Lazarus units and packages
« Reply #54 on: December 26, 2024, 09:42:44 am »
The other stuff about version history and guided task completion is not an index issue. They require tagging metadata that just isn't present in the source or the help topics. Not that they aren't good ideas or useful... there's no way to automatically generate it.

In short this

Quote
Unlike other documentation tools, the documentation can be in a separate file (in XML format), so the sources aren’t cluttered with documentation.

from https://www.freepascal.org/docs-html/current/user/userse39.html#x138-1450008.3 is a very poor design philosophy. I agree that having a large amount of raw documentation embedded in function bodies etc. can be undesirable, but it should at the very least be possible to embed documentation /hints/: "this helper/constant/etc. must be xlinked to this class/function/etc.".

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

MarkMLl

  • Hero Member
  • *****
  • Posts: 8090
Re: Exploring FPC/Lazarus units and packages
« Reply #55 on: January 11, 2025, 10:15:11 pm »
I hope to have something "interesting" to contribute over the next couple of days: unpolished, but perhaps a good starting point. However I have some questions:

* Can anybody suggest a resource (possibly "AI") to recognise terms of art such as "end of file" which should be treated as a single token?

* Can anybody suggest a resource (possibly "AI") which advises on fillers such as "the", "a" etc. which can be safely discarded?

* Can anybody suggest a resource (possibly "AI") which advises on fillers such as "the result", "a parameter" etc. which should be preserved but not indexed?

* The example I gave earlier

Code: [Select]
Eof:                                 | Check for end of file
Eof:                       Check for | end of file
SeekEof:        Set file position to | end of file
Eof:                Check for end of | file
FilePos:                 Position in | file
Seek:                            Set | file position
SeekEof:                         Set | file position to end of file
SeekEof: Set file position to end of | file
Seek:                       Set file | position
FilePos:                             | Position in file
SeekEof:                    Set file | position to end of file
Seek:                                | Set file position
SeekEof:                             | Set file position to end of file

suggests that some of the "left hand" parts are verb-like and should be elided, for example "Set file position to" and "Set file". Can anybody provide any rules for this?

* Finally, Where could I usefully post my working scripts (probably unix shell plus Perl) so that others can contribute?

I'd rather that this remained a fairly un-forked effort, and feel that dumping it straight onto e.g. Github wouldn't look good: we should never have got(ten) ourselves into the position where such a retroactive hack is necessary.

MarkMLl
« Last Edit: January 11, 2025, 10:25:21 pm by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

MarkMLl

  • Hero Member
  • *****
  • Posts: 8090
Re: Exploring FPC/Lazarus units and packages
« Reply #56 on: January 13, 2025, 05:48:05 pm »
Attached is a script-generated permuted index for (only) the v3.2.2 RTL.

It's by no means complete: my code to collect significant text from .chm files is very crude, and it doesn't allow for inconsistencies in the way that functions etc. are described.

It also doesn't attempt to save the (package and) unit name as part of the function: these would obviously be useful /but/ output lines are already getting longer than I'm comfortable with and I think it's reasonable to assume that a user shouldn't need to be bludgeoned too thoroughly before he's prepared to use the existing documentation to find the untruncated description of a function: once he knows it exists and its True Name.

The number of input functions was around 6.5k, with fairly extensive use of the stopword list I've kept the index down to around 22k lines.

Updated: script files etc. uploaded to https://github.com/MarkMLl/fpc-ptx/tree/main

MarkMLl
« Last Edit: January 15, 2025, 12:07:17 pm by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018