Recent

Author Topic: Old chestnut: including a big block of raw text into a program  (Read 12626 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 3078
Re: Old chestnut: including a big block of raw text into a program
« Reply #15 on: July 31, 2021, 06:48:50 pm »
Thanks for that. So it's doable, but hardly painless.

That is a very opinionated and biassed statement. There are multiple sides to this, and if you have e.g. external editors or viewers for such included files, then suddenly the in-source view is extremely cumbersome and a poor man's solution.

But many languages actively limit themselves to the in source solution,, because they can't standardize tooling outside of the core language due to oppressive requirements for formal language definitions. Then doing it in source becomes the "my only tool is a hammer" kind of solution.

FPC and Delphi don't have that problem.

Well, maybe I'm an opinionated and biased person: after all, I use Pascal and have consistently argued its strengths compared with lesser languages for the last 40 years.

But I note that C++11 introduced inline text storage, which had never been implemented in older revisions or in C itself: whatever we think of the language there's smart people involved, and I'm sure that if they could see a better way they'd have used it.

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 9539
  • FPC developer.
Re: Old chestnut: including a big block of raw text into a program
« Reply #16 on: July 31, 2021, 06:53:23 pm »
But I note that C++11 introduced inline text storage, which had never been implemented in older revisions or in C itself: whatever we think of the language there's smart people involved, and I'm sure that if they could see a better way they'd have used it.

C and C++ are the archetypical development system of the "my only tool is a hammer" type, since they only standardize compiler and preprocessor, and it is enormously complicated to add another tool and file type to the standard. Going with the in source option is the direction of least resistance, and they probably hope they put the matter to rest for a while

Like us, when we implemented the resource solution.

MarkMLl

  • Hero Member
  • *****
  • Posts: 3078
Re: Old chestnut: including a big block of raw text into a program
« Reply #17 on: July 31, 2021, 06:53:47 pm »
There is a patch for FPC to add multi-line strings. Discussed here.

I don't know why I have to sign in to gitlab to view the bug report %)

I'd forgotten that debate, and viewed purely as a multiline string issue I think I wasn't enthusiastic... but I can see the overlap now that you've mentioned it.

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

avra

  • Hero Member
  • *****
  • Posts: 2230
    • Additional info
Re: Old chestnut: including a big block of raw text into a program
« Reply #18 on: July 31, 2021, 08:23:02 pm »
It is interesting to know that one embedded pascal I use allows this:

Code: Pascal  [Select][+][-]
  1. const
  2.   // 100 point.x point.y of integer, 3 bytes info
  3.   MuLookUp: array[1..(100 * 4) + 3] of byte = 'AVR Interpol.crvg';
  4.  
  5.   // 32*32 pixels = 128bytes + 2 bytes size info
  6.   MyBitMap: array[1..(32*32 div 8) + 2] of byte = 'logo.pbmp';
  7.  
  8.   // 128 chars = 128 * 7 bytes + 2 bytes size info
  9.   MyCharSet: array[1..(128 * 7) + 2] of byte = 'myCharSet.pchr';

I could not find info that strings could load content from file as arrays above, but anyway...
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

MarkMLl

  • Hero Member
  • *****
  • Posts: 3078
Re: Old chestnut: including a big block of raw text into a program
« Reply #19 on: July 31, 2021, 10:03:04 pm »
@avra: I see what you're getting at there, and loading pregenerated lookup tables etc. is obviously a fairly common requirement for embedded stuff... not to mention founts for LCD displays. I suppose that we need a preprocessor of some sort, which would mean provision for invoking one as a standard part of the compiler run since makefiles aren't used for most projects.

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 727
  • Professional amateur ;-P
Re: Old chestnut: including a big block of raw text into a program
« Reply #20 on: July 31, 2021, 11:12:08 pm »
Hey Mark,

I dunno if you still want a bit of code for this particular example.

If not, just ignore my post and All's good.

If you still want it, I've cobbled up a pure RTL version of the resource loading of a very simple shell script into a string and then shoving it into a TMemo, but you'll notice that I have it on a string, so the memo is just for visual aid.

I include the shell script and if you press CTRL+SHIT+F11 and click on the Resources item you'll find where I'm adding the resource and where I define it's name and type.

Hope this helps in any form or shape.

Cheers,
Gus

Edit:
Completely forgot to mention that this is from the stable version of Lazarus/FPC which is 2.0.12/3.2.2 on an Ubuntu 21.04.
My Install is via fpcupdeluxe and I've not added any of the resource linkers mentioned.
But I do have to admit that I don't know what resource builder it uses and if it was installed by fpcupdeluxe or it comes by default with Ubuntu.
I, personally, did nothing to install a resource builder.
« Last Edit: July 31, 2021, 11:25:42 pm by Gustavo 'Gus' Carreno »
Lazarus 2.3.0(trunk) FPC 3.3.1(trunk) Ubuntu 21.04 64b Dark Theme
Lazarus 2.0.12(stable) FPC 3.2.2(stable) Ubuntu 21.04 64b Dark Theme
http://github.com/gcarreno

Kays

  • Sr. Member
  • ****
  • Posts: 357
  • Whasup!?
    • KaiBurghardt.de
Re: Old chestnut: including a big block of raw text into a program
« Reply #21 on: July 31, 2021, 11:24:27 pm »
[…] LCD displays. […]
OMG. Did you just write “liquid crystal display displays”? :o

Is there a way that works with as-installed FPC on Linux of preloading a big block of raw text into a string? […]
No, but since fp-compiler depends on binutils, that means there is an assembler as(1), you could also (ab)use objects files and the {$link} directive. First create a wrapper file:
Code: ASM  [Select][+][-]
  1. global words
  2. global words.length
  3.  
  4. section .data
  5.  
  6. words:
  7. incbin '/usr/share/dict/words'
  8.  
  9. words.length:
  10.         dq $ - words
“Assemble” it (sorry, I only know nasm(1), but there definitely is some GNU Assembler equivalent):
Code: Bash  [Select][+][-]
  1. nasm -f elf64 words.asm
And then use it:
Code: Pascal  [Select][+][-]
  1. program foo(input, output, stdErr);
  2. {$link words}
  3. {$modeSwitch unicodeStrings-}{ensure single-Byte `char`}
  4. var
  5.         words: array[1..high(ptrInt)] of char; external name 'words';
  6.         words_length: qWord; external name 'words.length';
  7. begin
  8.         writeLn(words_length, ' Bytes');
  9.         write(words[1..80]);
  10. end.
Evidently that’s not a string constant, yet it’s technically less complicated than using resources, but it’s also just a really dumb character array. Not worth mentioning, using Assembly can be considered nasty.
Yours Sincerely
Kai Burghardt

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 9539
  • FPC developer.
Re: Old chestnut: including a big block of raw text into a program
« Reply #22 on: July 31, 2021, 11:35:22 pm »
Kays:  bin2obj and data2inc are two pre resource ways for including data.

Kays

  • Sr. Member
  • ****
  • Posts: 357
  • Whasup!?
    • KaiBurghardt.de
Re: Old chestnut: including a big block of raw text into a program
« Reply #23 on: August 01, 2021, 12:30:04 am »
Kays:  bin2obj and data2inc are two pre resource ways for including data.
Ah, of course! Why didn’t you mention it earlier? Although I hate typed pseudo-“constants”, data2inc(1) is the way to go.

Evidently that’s not a string constant, […]
To make it effectively a constant, put everything in
Code: ASM  [Select][+][-]
  1. section .rodata
At least the ELF supports non-executable read-only sections, yet any assignments to this “constant” are only detected at run-time. A typed pseudo-constant will still be mutable.
« Last Edit: August 01, 2021, 12:47:20 am by Kays »
Yours Sincerely
Kai Burghardt

avra

  • Hero Member
  • *****
  • Posts: 2230
    • Additional info
Re: Old chestnut: including a big block of raw text into a program
« Reply #24 on: August 01, 2021, 12:41:48 am »
loading pregenerated lookup tables etc. is obviously a fairly common requirement for embedded stuff
Well, I have only shown arrays which are common with standard pascal. That embedded implementation also has tables, which are a specialized 1-dimensional version of arrays containing look-up tables. The table length is limited on power of 2, to allow a very, very fast access:
Code: Pascal  [Select][+][-]
  1. structconst {constant in Rom, at startup copied into Ram}
  2.    Table1: Table[0..3] = (0, $45, $A5, $FF);
  3. var
  4.    tb1: Table[0..15] of char;
  5.    tb2: Table[0..127] of word;
Sometimes tricks like this mean a lot in embedded world...  8)
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

PascalDragon

  • Hero Member
  • *****
  • Posts: 3335
  • Compiler Developer
Re: Old chestnut: including a big block of raw text into a program
« Reply #25 on: August 01, 2021, 12:43:24 pm »
Simply include the textfile as a resource, and load it from the resource? Afaik resources work with Linux too.

https://wiki.freepascal.org/Lazarus_Resources says that doing so needs windres or GoRC, neither of which comes as standard. The current manual concurs, and the examples still have to be quoted.

windres is contained with the Windows distribution of FPC. On other platforms you need to make sure that your buildchain is set up correctly (just like you need to make sure that as and ld are available).

Though for manual resource compilation you can use fpcres from 3.2.2 and in 3.3.1 you can pass -FF to tell the compiler to use fpcres instead of windres/gorc (though that switch will soon be dropped soon as it's behavior will become the default).

I however load the data over windows unit functions, but it should be possible portably too (Lazarus uses it afaik)

Very easy:

Code: Pascal  [Select][+][-]
  1. program tres;
  2.  
  3. {$mode objfpc}
  4.  
  5. uses
  6.   Classes, resource;
  7.  
  8. {$R test.res}
  9.  
  10. var
  11.   rs: TResourceStream;
  12.   sl: TStringList;
  13. begin
  14.   rs := TResourceStream.Create(HINSTANCE, 'Default', MAKEINTRESOURCE(RT_RCDATA));
  15.   try
  16.     sl := TStringList.Create;
  17.     try
  18.       sl.LoadFromStream(rs);
  19.  
  20.       Writeln(sl.Text);
  21.     finally
  22.       sl.Free;
  23.     end;
  24.   finally
  25.     rs.Free;
  26.   end;
  27. end.

test.rc:

Code: [Select]
Default RCDATA Defaults.ini
Defaults.ini:

Code: [Select]
[test]
a=1
b=2

[foo]
c=3

Works on both Windows and Linux (and also other platforms). In trunk you can use -FF to force the use of fpcres.

If you don't have windres in FPC 3.2.2 or older you need to compile the resource manually using a FPC 3.2.2 fpcres.

Alextp

  • Hero Member
  • *****
  • Posts: 1383
    • UVviewsoft
Re: Old chestnut: including a big block of raw text into a program
« Reply #26 on: August 01, 2021, 12:47:35 pm »
@PascalDragon,
that was useful post, so I ask: is it (that info how to use TResourceStream with some text) in the Wiki?

PascalDragon

  • Hero Member
  • *****
  • Posts: 3335
  • Compiler Developer
Re: Old chestnut: including a big block of raw text into a program
« Reply #27 on: August 01, 2021, 12:53:16 pm »
@PascalDragon,
that was useful post, so I ask: is it (that info how to use TResourceStream with some text) in the Wiki?

The already provided link to the Wiki contains that information (though it could be improved a bit).

MarkMLl

  • Hero Member
  • *****
  • Posts: 3078
Re: Old chestnut: including a big block of raw text into a program
« Reply #28 on: August 01, 2021, 03:06:38 pm »
windres is contained with the Windows distribution of FPC. On other platforms you need to make sure that your buildchain is set up correctly (just like you need to make sure that as and ld are available).

I habitually use Debian (mostly AMD64) and build both FPC and Lazarus from scratch. There is no windres on these systems, and Debian has no package of (some variant of) that name in their repositories.

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 9539
  • FPC developer.
Re: Old chestnut: including a big block of raw text into a program
« Reply #29 on: August 01, 2021, 03:12:27 pm »
I habitually use Debian (mostly AMD64) and build both FPC and Lazarus from scratch. There is no windres on these systems, and Debian has no package of (some variant of) that name in their repositories.

IIRC it is part of cross win32 binutils. But as said better use trunk or 3.2.2 fpcres.

 

TinyPortal © 2005-2018