Lazarus

Programming => Operating Systems => Other => Topic started by: chobani on November 25, 2022, 02:08:20 pm

Title: [SOLVED] StringUtils in MSDOS
Post by: chobani on November 25, 2022, 02:08:20 pm
Is there a way to include stringutils when targeting MSDOS? I get cannot find BASE64, but if I change compile target to Win32/64 it compiles fine. I'm hoping I'm just doing something wrong, or I need to compile an older version of stringutils or something. I did some searches and didn't find anything.  Thank you!
Title: Re: StringUtils in MSDOS
Post by: KodeZwerg on November 25, 2022, 02:14:47 pm
I do not have MS-DOS to test, please try:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. uses
  4.   Base64;
  5.  
  6. var
  7.   s: string;
  8. begin
  9.   s := 'Test if it works';
  10.   WriteLn(s); // original
  11.   s := EncodeStringBase64(s);
  12.   WriteLn(s); // encoded
  13.   s := DecodeStringBase64(s);
  14.   WriteLn(s); // decoded
  15. end.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 25, 2022, 03:17:48 pm
Fatal: Cannot find Base64 used by Project1 of the Project Inspector.

I'm not sure why it's saying cannot find, but thank you that's a step in the right direction.  If I change the target to Win32 this sample program compiles fine. Any idea why just this wouldn't be found and only just for MSDOS target?  I can include sysutils for instance and it will compile to MSDOS.
Title: Re: StringUtils in MSDOS
Post by: KodeZwerg on November 25, 2022, 04:11:44 pm
Fatal: Cannot find Base64 used by Project1 of the Project Inspector.
I do not know whats wrong at your installation / usage, so here is a solution that need nothing.
Code: Pascal  [Select][+][-]
  1. program base64test;
  2.  
  3. var
  4.   EncodeTable: array[0..63] of Char =
  5.     'ABCDEFGHIJKLMNOPQRSTUVWXYZ' +
  6.     'abcdefghijklmnopqrstuvwxyz' +
  7.     '0123456789+/';
  8.  
  9.   DecodeTable: array[#0..#127] of Integer = (
  10.     Byte('='), 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64,
  11.     64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64,
  12.     64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 62, 64, 64, 64, 63,
  13.     52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 64, 64, 64, 64, 64, 64,
  14.     64,  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14,
  15.     15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 64, 64, 64, 64, 64,
  16.     64, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
  17.     41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 64, 64, 64, 64, 64);
  18.  
  19. type
  20.   PPacket = ^TPacket;
  21.   TPacket = packed record
  22.     case Integer of
  23.       0: (b0, b1, b2, b3: Byte);
  24.       1: (i: Integer);
  25.       2: (a: array[0..3] of Byte);
  26.       3: (c: array[0..3] of Char);
  27.   end;
  28.  
  29. procedure EncodePacket(const Packet: TPacket; NumChars: Integer; OutBuf: PChar);
  30. begin
  31.   OutBuf[0] := EnCodeTable[Packet.a[0] shr 2];
  32.   OutBuf[1] := EnCodeTable[((Packet.a[0] shl 4) or (Packet.a[1] shr 4)) and $0000003f];
  33.  
  34.   if NumChars < 2 then
  35.     OutBuf[2] := '='
  36.   else
  37.     OutBuf[2] := EnCodeTable[((Packet.a[1] shl 2) or (Packet.a[2] shr 6)) and $0000003f];
  38.  
  39.   if NumChars < 3 then
  40.     OutBuf[3] := '='
  41.   else
  42.     OutBuf[3] := EnCodeTable[Packet.a[2] and $0000003f];
  43. end;
  44.  
  45. function Base64Encode(const Input: string): string;
  46. var
  47.   I, K, J: Integer;
  48.   Packet: TPacket;
  49. begin
  50.   Result := '';
  51.   I := (Length(Input) div 3) * 4;
  52.   if Length(Input) mod 3 > 0 then Inc(I, 4);
  53.   SetLength(Result, I);
  54.   J := 1;
  55.   for I := 1 to Length(Input) div 3 do
  56.   begin
  57.     Packet.i := 0;
  58.     Packet.a[0] := Byte(Input[(I - 1) * 3 + 1]);
  59.     Packet.a[1] := Byte(Input[(I - 1) * 3 + 2]);
  60.     Packet.a[2] := Byte(Input[(I - 1) * 3 + 3]);
  61.     EncodePacket(Packet, 3, PChar(@Result[J]));
  62.     Inc(J, 4);
  63.   end;
  64.   K := 0;
  65.   Packet.i := 0;
  66.   for I := Length(Input) - (Length(Input) mod 3) + 1 to Length(Input) do
  67.   begin
  68.     Packet.a[K] := Byte(Input[I]);
  69.     Inc(K);
  70.     if I = Length(Input) then
  71.       EncodePacket(Packet, Length(Input) mod 3, PChar(@Result[J]));
  72.   end;
  73. end;
  74.  
  75. function DecodePacket(InBuf: PChar; var nChars: Integer): TPacket;
  76. begin
  77.   Result.a[0] := (DecodeTable[InBuf[0]] shl 2) or
  78.     (DecodeTable[InBuf[1]] shr 4);
  79.   NChars := 1;
  80.   if InBuf[2] <> '=' then
  81.   begin
  82.     Inc(NChars);
  83.     Result.a[1] := (DecodeTable[InBuf[1]] shl 4) or (DecodeTable[InBuf[2]] shr 2);
  84.   end;
  85.   if InBuf[3] <> '=' then
  86.   begin
  87.     Inc(NChars);
  88.     Result.a[2] := (DecodeTable[InBuf[2]] shl 6) or DecodeTable[InBuf[3]];
  89.   end;
  90. end;
  91.  
  92. function Base64Decode(const Input: string): string;
  93. var
  94.   I, J, K: Integer;
  95.   Packet: TPacket;
  96. begin
  97.   Result := '';
  98.   for I := 1 to Length(Input) div 4 do
  99.   begin
  100.     Packet := DecodePacket(PChar(@Input[(I - 1) * 4 + 1]), J);
  101.     K := 0;
  102.     while J > 0 do
  103.     begin
  104.       Result := Result + Packet.c[K];
  105.       Inc(K);
  106.       Dec(J);
  107.     end;
  108.   end;
  109. end;
  110.  
  111. var
  112.   s: string;
  113. begin
  114.   s := 'Test if it works';
  115.   WriteLn(s); // original
  116.   s := Base64Encode(s);
  117.   WriteLn(s); // encoded
  118.   s := Base64Decode(s);
  119.   WriteLn(s); // decoded
  120. end.
Title: Re: StringUtils in MSDOS
Post by: MarkMLl on November 25, 2022, 05:54:23 pm
OP hasn't told us what version of compiler he's running. There's a reasonable chance that for some reason that file hasn't been built as part of the RTL: I can see it mentioned in fpcsrc/packages/fcl-base/fpmake.pp for 3.2.2 but haven't checked other versions.

MarkMLl
Title: Re: StringUtils in MSDOS
Post by: chobani on November 25, 2022, 06:06:24 pm
Hi, and thanks. I am running 3.2.2, but maybe I'll try and just do a fresh install and report back -

Free Pascal Compiler version 3.2.2 [2021/05/19] for i8086
Title: Re: StringUtils in MSDOS
Post by: MarkMLl on November 25, 2022, 06:24:06 pm
I've had to look for missing packages etc. in the past when testing rarely-used combinations... I couldn't see anything obvious that would exclude it, but I'm the first to admit that I find the FPC build stuff pretty obscure.

MarkMLl
Title: Re: StringUtils in MSDOS
Post by: chobani on November 25, 2022, 07:47:16 pm
I removed all and reinstalled everything, made sure I only had 3.2.2 on my system and the cross compiler installed, and I still have the same exact issue. I am going to try and make my own base64 include file as suggested by KodeZwerg next and see how that goes.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 25, 2022, 08:08:31 pm
@KodeZwerg I think it's a little more complicated than that. Early in the base package of stringutils there's this for example: Decoder: TBase64DecodingStream;

Is it possible no one ever needed stringutils for an MSDOS compile yet? I find that really unlikely so I think it's something with my settings. But like I said, I can compile to MSDOS target no issue when stringutils is not included (or compile fine to other platforms when it is).

 I really appreciate the assitance!
Title: Re: StringUtils in MSDOS
Post by: KodeZwerg on November 25, 2022, 08:33:21 pm
Maybe its a 16bit problem? I just can guess... sorry.
Did my second code work for you?
Title: Re: StringUtils in MSDOS
Post by: chobani on November 26, 2022, 03:22:07 pm
Ah yeah, sorry, it did compile to MSDOS target just fine and I can run it in a 16bit environment -

https://imgur.com/a/wGNgNj6 (https://imgur.com/a/wGNgNj6)

Title: Re: StringUtils in MSDOS
Post by: marcov on November 26, 2022, 03:28:51 pm
Base64 is in fcl-base, which is one of the packages lowest in the packages/ history.

IOW if somehow the RTL is configured, but not packages/, you get such an error.

First step would be to see if you actually have  a fcl-base/ dir with base64.o and base64.ppu in it for msdos in the memory model that you use.
Title: Re: StringUtils in MSDOS
Post by: MarkMLl on November 26, 2022, 03:44:18 pm
Good point. So IOW: a good compiler capable of targeting MS-DOS, an RTL tailored for MS-DOS, but no packages built.

But since StrUtils is in the RTL, shouldn't any reliance on packages be optional?

MarkMLl
Title: Re: StringUtils in MSDOS
Post by: marcov on November 26, 2022, 03:55:28 pm
But since StrUtils is in the RTL, shouldn't any reliance on packages be optional?

Since 3.0 or so, Strutils is in rtl-objpas, which is a packages/

What is in the core RTL/ is roughly decided by if the compiler needs it, moving units out minimizes bootstrapping requirements  and speeds it up.
Title: Re: StringUtils in MSDOS
Post by: Thaddy on November 26, 2022, 04:17:36 pm
Good point. So IOW: a good compiler capable of targeting MS-DOS, an RTL tailored for MS-DOS, but no packages built.
That makes me cry, not even smurk.
Nothing to do with the compiler, but with libraries.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 26, 2022, 04:58:00 pm
Okay, I think I'm following everyone as best as I can. In my fcl-base directory I do see both base64.o and base64.ppu in my units directory for i386-win32. I do not have that in MSDOS unit directly and this is a fresh install where I haven't made any modifications other than to install the latest version of FPC along with the MSDOS cross compiler.

I see sysutils and strutils both in my MSDOS directory for all memory sizes. There is no FCL-BASE in any of the MSDOS unit directories though. Does this mean no one uses strutils out of the box for MSDOS target? That seems like it might be the case because it starts with the BASE64 include, but I don't know why that wouldn't be one of the next things you would want to have.

So I think you all have shown me the exact issue. I have no idea what to do about that though. I'm guessing I have to do some sort of compile that might be beyond me, hopefully not. Or maybe it's possible someone has already done this? I can't tell you how much I appreciate everyone helping, I'm just trying to help someone else who wants a 16 bit application version of something I wrote for a hobby. I really thought I might be able to just target MSDOS and call it a day hah, so thanks!

Title: Re: StringUtils in MSDOS
Post by: MarkMLl on November 26, 2022, 05:16:43 pm
This is basically an installation or installation instructions issue. Let's start off by trying to work out what you've got.

On my system I can see- for example-

/usr/local/lib/fpc/3.2.2/units/i386-linux/fcl-base/base64.ppu

If I step back a couple of directory levels to /usr/local/lib/fpc/3.2.2/units/i386-linux I can see


...
fcl-base
...
rtl
rtl-console
rtl-extra
rtl-generics
rtl-objpas
rtl-unicode
...


What do you have in the corresponding place in your MSDOS tree? I think it would be worthwhile C&Ping the entire list so that we can see the scale of the damage.

In any event, it might be that one of the maintainers can point out more detailed instructions that you should have been using (in which case why haven't the out of date ones been removed?) but at present I don't think it's any mistake on your part.

MarkMLl





Title: Re: StringUtils in MSDOS
Post by: Thaddy on November 26, 2022, 06:09:18 pm
maybe you are on the wrong foot about fcl-base and base64. fcl base does or should not contain a base64 at all.
Read fcl-base as fcl-basics (minimal required). Base 64 is an encoding to save - which  it does not do very good - space and work around non-printable characters. They are not related.
See https://en.wikipedia.org/wiki/Base64

If you need base64, You might want to shoot a bear - guns not allowed, just spear or bow and arrow - and cut down a tree for warmth.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 26, 2022, 06:12:49 pm
I do have a Linux VM I'll kick up and can test that install to see if it works. This is what the MSDOS directory looks like as you can see no fcl-base and this is true for all the memory module directories -

Directory of C:\lazarus\fpc\3.2.2\units\msdos\80286-tiny

11/25/2022  12:14 PM    <DIR>          .
11/25/2022  12:14 PM    <DIR>          ..
11/25/2022  12:14 PM    <DIR>          fcl-stl
11/25/2022  12:14 PM    <DIR>          fv
11/25/2022  12:14 PM    <DIR>          graph
11/25/2022  12:14 PM    <DIR>          hash
11/25/2022  12:14 PM    <DIR>          libtar
11/25/2022  12:14 PM    <DIR>          regexpr
11/25/2022  12:14 PM    <DIR>          rtl
11/25/2022  12:14 PM    <DIR>          rtl-console
11/25/2022  12:14 PM    <DIR>          rtl-extra
11/25/2022  12:14 PM    <DIR>          rtl-generics
11/25/2022  12:14 PM    <DIR>          rtl-objpas
Title: Re: StringUtils in MSDOS
Post by: Thaddy on November 26, 2022, 06:15:30 pm
read my reply above, posts crossed.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 26, 2022, 07:25:25 pm
read my reply above, posts crossed.

Ah yes! Even closer now.  Appreciate it, I see that I've added some confusion by mistake here. It's actually StringUtils a sub library that's getting imported, not strUtils which I've been using interchangeably and are obviously not the same thing (sincere apologies, was just kind of staring at it). The code for StringUtils (my version) starts with uses BASE64 and that's where the hang up is. I went through everything and it doesn't seem to have any other issues other than not being able to find it in the MSDOS install. I don't think I can work around not importing BASE64 here without significant pain though...  is this a feasible thing?

 
Title: Re: StringUtils in MSDOS
Post by: PascalDragon on November 27, 2022, 10:02:20 pm
maybe you are on the wrong foot about fcl-base and base64. fcl base does or should not contain a base64 at all.
Read fcl-base as fcl-basics (minimal required). Base 64 is an encoding to save - which  it does not do very good - space and work around non-printable characters. They are not related.

The Base64 unit is part of the FCL-Base package. The FCL-Base package is however not enabled for the MSDOS target (and some other more restricted targets).

@chobani: I have not tested it with target i8086-msdos, but you should probably be able to simply take the source file of the Base64 unit and copy it to your project.
Title: Re: StringUtils in MSDOS
Post by: chobani on November 28, 2022, 03:56:07 pm
Just simply saving as BASE642 and renaming the Unit BASE642 worked. I should have tried that. Thank you!
Title: Re: StringUtils in MSDOS
Post by: MarkMLl on November 28, 2022, 05:46:09 pm
The Base64 unit is part of the FCL-Base package. The FCL-Base package is however not enabled for the MSDOS target (and some other more restricted targets).

So where is that controlled: in fcl-base/fpmake.pp where it says

Code: Pascal  [Select][+][-]
  1.     P.SourcePath.Add('src/unix',AllUnixOSes);
  2.     P.SourcePath.Add('src/win',AllWindowsOSes);
  3.  

and so on? I admit to having trouble understanding how things hang together these days: the manual talks about compiling the compiler etc. manually and talks about either doing it manually or using a Makefile, and it also documents fpcmake... but doesn't say whether this is the ultimate starting point.

MarkMLl
Title: Re: StringUtils in MSDOS
Post by: PascalDragon on November 30, 2022, 10:43:33 pm
So where is that controlled: in fcl-base/fpmake.pp where it says

Code: Pascal  [Select][+][-]
  1.     P.SourcePath.Add('src/unix',AllUnixOSes);
  2.     P.SourcePath.Add('src/win',AllWindowsOSes);
  3.  

It's controller a few lines further up:

Code: Pascal  [Select][+][-]
  1.     P.OSes:=AllOSes-[embedded,msdos,win16,macosclassic,palmos,zxspectrum,msxdos,amstradcpc,sinclairql];
  2.     if Defaults.CPU=jvm then
  3.       P.OSes := P.OSes - [java,android];
  4.  

I admit to having trouble understanding how things hang together these days: the manual talks about compiling the compiler etc. manually and talks about either doing it manually or using a Makefile, and it also documents fpcmake... but doesn't say whether this is the ultimate starting point.

The overall building is controlled by the Makefiles which is also used for the compiler and the core RTL. The building of the packages themselves (and the utils) is controlled in the corresponding fpmake.pp files which is essentially a Pascal code building system. fpcmake (in contrast to fpmake) is used to convert Makefile templates (Makefile.fpc) to Makefiles.
Title: Re: [SOLVED] StringUtils in MSDOS
Post by: MarkMLl on December 01, 2022, 09:47:46 am
Thanks for that, so I was on the right track.

I've obviously used the makefiles, and continue to use them in other contexts. Apart from anything else I think it's good practice to have a working makefile for any published project... at least for stuff which is console-only.

I was vaguely aware of the fpmake/fpcmake "metalayer" and have tinkered with it minimally in the past- I think that was while fpmake was dominant. A summary like the one you've just given could usefully appear in the docs at some point.

MarkMLl
Title: Re: [SOLVED] StringUtils in MSDOS
Post by: PascalDragon on December 01, 2022, 10:01:08 pm
I was vaguely aware of the fpmake/fpcmake "metalayer" and have tinkered with it minimally in the past- I think that was while fpmake was dominant. A summary like the one you've just given could usefully appear in the docs at some point.

fpmake is younger than fpcmake and is also what powers fppkg.
Title: Re: [SOLVED] StringUtils in MSDOS
Post by: MarkMLl on December 01, 2022, 10:18:02 pm
fpmake is younger than fpcmake and is also what powers fppkg.

Ah. I clearly need to plough through whatever documentation is available... fppkg I'm chiefly aware of because of the insoluble question when trying to start a new instance of Lazarus.

MarkMLl
TinyPortal © 2005-2018