Recent

Author Topic: Can FreePascal be used to create an internal or embedded DSL?  (Read 8779 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #30 on: August 14, 2022, 10:26:35 pm »
This is likely to be moved to "Other" by a moderator.

Sorry if I was a bit brutal earlier, I was intentionally goading you.

Quoting selectively:

My end goal is to create a system that runs on modern hardware that can produce text-based programs that run on modern and retro hardware. ...

The main target audience for this system would be those who own retro 8-bit home computers or use emulators for them (possibly because they no longer own such machines).

Interpreters were common for these devices and compilers were much less so. Including one with the front end of the system seems natural, especially if the process for translating and compiling the code has several stages.

BASIC would have been the most commonly used language on these retro machines, as it was inluded with most of them. Pascal and C compilers were available, but much less widely used.

HiSoft sold versions of BASIC, Pascal and C, for example.

If you owned a Z80 machine with a disk drive (rare in the UK), you could use more languages via CP/M. I'm reasonably sure that this was how Turbo Pascal was made available for 8-bit micros.

I think that a sufficient proportion of retro users cut their teeth on BASIC machines such as the Spectrum and BBC Micro that selling any other language would be quite a job /unless/ it was as well integrated as the original Turbo Pascal. And for similar reasons that makes using a PC-hosted cross-compiler unrealistic, /unless/ you produced an entire product range of physical and emulated micros with some sort of "tube" arrangement to the host for program download (precedent: https://hackaday.io/Just4Fun for a recently-designed range of boards from the same stable, plus the Arduino IDE (which I'm sure you're aware supports multiple targets) for a way of developing using a slightly-cut-down C++ and downloading over a USB serial link).

I was, for a long time, an outspoken critic of BASIC and C. But the truth is that both language families have inherited a lot of features from Pascal and its successor Modula-2, notably strong type checking and the extension of that across compilation units. And even if BASIC is now, like just about everything else, compiled, the retro community still yearns for something with the immediate response of the classical implementations... I won't go on, you probably know that better than I do.

The only other things that give you that kind of immediacy are environments such as Lisp, Forth or Smalltalk... and quite frankly I think they're non-starters for what you're trying to do. There was at one time an incrementally-compiled Pascal called Mystic Pascal, I think I saw one copy sold and that it ended up on magazine-cover collections.

These days, there are far more people who know how to write a half-decent compiler than there were in the '80s. Unfortunately, they are intent on doing it "properly" by the standard of their computer science courses, and... well, even if they insist that they're using all the right techniques don't expect the result to be compact.

I'm pretty sure that you will not get FPC running on an 8-bit system. I've been trying during the day to remember whether Turbo Pascal on a CP/M-80 system actually switched between editor and compiler using an overlay... without success although I'd throw in that the much later VB for DOS did.

My suspicion is that it's easier to write a Pascal compiler than a BASIC one: recursive descent is adequate, and generally speaking it's possible to avoid having to consider the sort of hoops discussed in the well-regarded https://moam.info/writing-interactive-compilers-and-interpreters-pj-wiley-online-library_59b149bf1723ddd7c686d281.html

However, you need to consider two things: (a) do you need the level of immediacy that only BASIC really gives you and (b) if you decide to use Pascal are you prepared to stick to the language handled by the early versions (short strings, no try-finally construct, no exceptions, no objects and so on).

Oh, and I knew Hisoft and Dave Nutkins fairly well, as well as some of the people who wrote stuff for them.

Quote
The upshot of this is, that more users would know BASIC than any other language, so the system should use something similar. Pascal would be close enough for those used to structured BASIC.

But Structured BASIC is much younger than the 8-bit micros you're talking about: it's a 16-bit thing (QB4?).

Quote
Languages with a C-style syntax would be harder to adjust to. I know it caused me issues when I tried learning it.

There is very little difference between Pascal and C: both are direct descendants of ALGOL-60. I'd suggest that the problem you had with C was the number of idioms you were forced to learn- iterating a string using a pointer etc.- that were quite simply hidden in Pascal.

I'd suggest focussing not on what language you want, but on the degree of immediacy: what sort of IDE, and what sort of debugger. Compared with those, writing a compiler will be trivial.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Wysardry

  • Jr. Member
  • **
  • Posts: 69
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #31 on: August 15, 2022, 12:07:13 am »
And for similar reasons that makes using a PC-hosted cross-compiler unrealistic, /unless/ you produced an entire product range of physical and emulated micros with some sort of "tube" arrangement to the host for program download
I was hoping that anyone with the desire to target retro machines would either own one or find suitable emulators for themselves.

Including links to several of these wouldn't be an issue, but creating new ones or attempting to package some with the tools and keeping on top of updates would be too time consuming.

Quote
My suspicion is that it's easier to write a Pascal compiler than a BASIC one
I actually bought two books on writing an interpreter in Object Pascal and am waiting for the third to be published before diving in. The created language is a variation of BASIC.

Also, Decimal BASIC is written in FreePascal and has the source code available, so I would have some sort of guidance if I went that route.

Quote
However, you need to consider two things: (a) do you need the level of immediacy that only BASIC really gives you and (b) if you decide to use Pascal are you prepared to stick to the language handled by the early versions (short strings, no try-finally construct, no exceptions, no objects and so on).
I think the desire for immediacy would be sated by any interpreter. Python is a popular mainstream example.

There are going to be some unavoidable restrictions in the language given the number of platforms I would like to support. I'm hoping most users would accept that.

Quote
Oh, and I knew Hisoft and Dave Nutkins fairly well, as well as some of the people who wrote stuff for them.
Are you still in contact with them? I've used the contact form on their site a couple of times asking about legally obtaining HiSoft Pascal for 8-bit machines and haven't had a response.

Quote
But Structured BASIC is much younger than the 8-bit micros you're talking about: it's a 16-bit thing (QB4?).
By "structured BASIC" I meant ANSI Full BASIC or BBC BASIC that allowed for a more modular construction via procedures and provided more looping options. They were available somewhere around the late 80s. IS-BASIC on the Z80-based Enterprise 64 (released in 1985) was a variation of ANSI Full BASIC.

Quote
There is very little difference between Pascal and C: both are direct descendants of ALGOL-60. I'd suggest that the problem you had with C was the number of idioms you were forced to learn- iterating a string using a pointer etc.- that were quite simply hidden in Pascal.
Having to use braces was off-putting, but dealing more directly with memory allocation was also a factor, yes.

Quote
I'd suggest focussing not on what language you want, but on the degree of immediacy: what sort of IDE, and what sort of debugger. Compared with those, writing a compiler will be trivial.
Having a quick way to test code would be a high priority. Maybe that could be made available via a compile and run option on the host machine as the programs shouldn't become too large.

Many errors wouldn't be caught until compile time though, unless each line of code could be assessed by the system in the same way that most retro BASIC interpreters did.

I haven't thought about the IDE too deeply, as I was hoping to allow almost any existing tool to be used. I shall have to reconsider that if I want an integrated interpreter rather than a command line one.

The debugger I haven't considered at all, mostly because of not knowing the direction I will be taking. If I modify or reuse an existing system, it will likely already have one available.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #32 on: August 15, 2022, 09:14:02 am »
@Wysardry:

When you talk about "a compiler", you're actually talking about three separate things:

1. The actual compiler, that translates your program into machine code.
2. The support libraries, that handle the I/O.
3. The user-interface.

If you have a compiler that compiles to your target CPU architecture, you're done with 1. FPC scores high here. And if you are fine with a character-based UI, as you said, 3 is handled automatically when you have a (G)UI and 2.

What you are talking about is 2. And especially: video, keyboard (, mouse) and storage. Mostly, this is a list of pointers and constants, with some very basic functions. Like, where does the screenbuffer start, how many rows and columns, how do I print a char at that location. Like, an old BIOS. Compiled, it will fit a few KB of code.




MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #33 on: August 15, 2022, 09:20:37 am »
Quote
Oh, and I knew Hisoft and Dave Nutkins fairly well, as well as some of the people who wrote stuff for them.
Are you still in contact with them? I've used the contact form on their site a couple of times asking about legally obtaining HiSoft Pascal for 8-bit machines and haven't had a response.

Like other companies of the era, I suspect that the original entity no longer exists (i.e. there might be a successor, but they have no interest in the original business and no longer represent the same software authors etc.).

Quote
I haven't thought about the IDE too deeply, as I was hoping to allow almost any existing tool to be used. I shall have to reconsider that if I want an integrated interpreter rather than a command line one.

The debugger I haven't considered at all, mostly because of not knowing the direction I will be taking. If I modify or reuse an existing system, it will likely already have one available.

That's where you need to start. Writing a compiler is by now well-understood, but a half-decent IDE and debugger is still an art.

Frankly, I think that in effect you're out to reinvent Turbo Pascal or something of that era, and that you'll find it extremely difficult to fit it onto an 8-bit system without writing large parts in carefully-tuned assembler.

With respect to other members of this community, but high-level languages, particularly when you start using their standard libraries etc., have their limitations.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #34 on: August 15, 2022, 10:04:05 am »
What you are talking about is 2. And especially: video, keyboard (, mouse) and storage. Mostly, this is a list of pointers and constants, with some very basic functions. Like, where does the screenbuffer start, how many rows and columns, how do I print a char at that location. Like, an old BIOS. Compiled, it will fit a few KB of code.

/If/ the target has a screenbuffer. That was the difference between the MS-DOS and PC-DOS variants of early TP: the PC-DOS one had a memory mapped display (and was what matured into later versions) while the CP/M and MS-DOS ones assumed ASCII+escapes.

Then there was a small number of code generators and RTLs, and a small number of OSes.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #35 on: August 15, 2022, 11:00:42 am »
What you are talking about is 2. And especially: video, keyboard (, mouse) and storage. Mostly, this is a list of pointers and constants, with some very basic functions. Like, where does the screenbuffer start, how many rows and columns, how do I print a char at that location. Like, an old BIOS. Compiled, it will fit a few KB of code.

/If/ the target has a screenbuffer. That was the difference between the MS-DOS and PC-DOS variants of early TP: the PC-DOS one had a memory mapped display (and was what matured into later versions) while the CP/M and MS-DOS ones assumed ASCII+escapes.

Then there was a small number of code generators and RTLs, and a small number of OSes.

MarkMLl

Certainly. But that doesn't matter. You need to know how many rows and columns it can hold, and you need a function to write a char at that location. Optionally, a function to read the char at a location, but you can emulate that with an array. For a character-based display, everything else is optional, but functions for scrolling and such are nice to have, as those can be hardware-accelerated.

So, you end up with a screenbuffer and the same functions in either case. And that buffer probably holds two locations (bytes) for each char: one for the ASCII code and another one for the attributes (color, blink, underline, etc). Or two buffers, if you want to make it easier to support different displays. If you want it to be generic, you define those.

Or, for FPC, a crt unit.

avra

  • Hero Member
  • *****
  • Posts: 2514
    • Additional info
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #36 on: August 15, 2022, 03:39:34 pm »
If that was my project, I would go a slightly different path. Write a p-code compiler and interpreter.

The main change is that instead of creating different binaries (each time you change sources) for every target with cross compiler, you cross compile once to generate interpreter and compiler for each target (ok, compiler can be just on host if that suits your goal better), and then after each source change you generate single p-code file that will be the same for each target.

Depending on your goals, p-code idea may be better or worse approach to the problem.

Here are some useful links:
https://wiki.freepascal.org/Make_your_own_compiler,_interpreter,_parser,_or_expression_analyzer
https://en.wikipedia.org/wiki/P-code_machine
http://standardpascaline.org/p5.html

Going back to your original idea of just renaming WriteLn() to MyWriteLn(), I think that compared to any other idea it would be more realistic and feasible to just create a single INC file with all your macro renaming aliases that you would just have to include into all your units, and simply continue to use FreePascal with it's full power without limits that a one man solution would bring. You could start immediately writing apps for already supported FPC targets, and later implement missing targets for FPC. That seams to me far less work compared to other alternatives.
« Last Edit: August 15, 2022, 03:49:58 pm by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Wysardry

  • Jr. Member
  • **
  • Posts: 69
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #37 on: August 15, 2022, 05:38:17 pm »
@Wysardry:

When you talk about "a compiler", you're actually talking about three separate things:
Ideally, I'd like to reuse whatever is already available, to reduce the amount of time and effort it would take, plus a certain number of people would already know the original tools/language/system.

Like other companies of the era, I suspect that the original entity no longer exists (i.e. there might be a successor, but they have no interest in the original business and no longer represent the same software authors etc.).
I rechecked the HiSoft About Us page and it seems David Link still works there, but Dave Nutkins is barely mentioned.

The company now seems to provide services related to web design.

Quote
That's where you need to start. Writing a compiler is by now well-understood, but a half-decent IDE and debugger is still an art.
I think I'll start testing similar software to see what works and what doesn't.

Quote
Frankly, I think that in effect you're out to reinvent Turbo Pascal or something of that era, and that you'll find it extremely difficult to fit it onto an 8-bit system without writing large parts in carefully-tuned assembler.
I'm hoping to avoid creating tools that run on the 8-bit targets. I would much rather reuse what already works, by producing source code to feed into existing 8-bit compilers.

If that was my project, I would go a slightly different path. Write a p-code compiler and interpreter.
I had considered this idea, but discounted it due to the lack of memory on the 8-bit systems.

Interpreters and virtual machines often waste a lot of memory, as they have to contain code for every supported function even if the program it is currently running does not use them all.

Quote
Going back to your original idea of just renaming WriteLn() to MyWriteLn(), I think that compared to any other idea it would be more realistic and feasible to just create a single INC file with all your macro renaming aliases that you would just have to include into all your units...
Yes, storing macros in include files is similar to what I had in mind. I also envisioned creating a library of different files containing functions that could optionally be added if needed.

If this method would interfere with Pascal to other language translators though, that would be a problem. Although many of the 8-bit home computers had a Z80 processor, not all of them did, so FreePascal alone (with its current targets) would not be enough.

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #38 on: August 15, 2022, 06:40:24 pm »

If that was my project, I would go a slightly different path. Write a p-code compiler and interpreter.
I had considered this idea, but discounted it due to the lack of memory on the 8-bit systems.

Interpreters and virtual machines often waste a lot of memory, as they have to contain code for every supported function even if the program it is currently running does not use them all.

Hi!

Just remember:

The whole UCSD operating system including p-machine, editor, compiler, filer did fit into 64 kB of RAM on an Apple ][ with a 6502.  Or cheaper clones.

It was also called Apple Pascal. The Apple version of UCSD was also able to do graphics.

So this interpreter for a p-machine did not waste too much memory ....

Winni


winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #39 on: August 15, 2022, 06:53:21 pm »
@avra

Maybe interesting for you:

Derek Jones from Knowledge Software wrote the system.interpr(eter) for the UCSD p-system in the 80s in C. Until then it was written in Assembler and had to be adapted on every new processor.

Knowledge Software:  http://knosof.co.uk/

Winni




Wysardry

  • Jr. Member
  • **
  • Posts: 69
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #40 on: August 15, 2022, 07:23:44 pm »
I have come up with a set of existing open source tools that might work together to form the system that I'm aiming for:-
  • Decimal BASIC as the front end and BASIC interpreter
  • Decimal BASIC to FreePascal translator (two options available from the Decimal BASIC site)
  • Lazarus/FreePascal as Z80 compiler
  • Lazarus/FreePascal to WebAssembly for web pages
  • Pascal to Nim translator to use Nim output as C for 8-bit C compilers and as JavaScript for web pages

If I was able to get all these to play together nicely, I could then gradually create the following using FreePascal:-
  • A Decimal BASIC to ZX BASIC translator to use that as Z80 compiler (more targets than FreePascal)
  • A Decimal BASIC (or FreePascal) to Turbo Rascal translator to use it as 8-bit multi-system compiler
  • A Decimal BASIC to XC=BASIC translator to use XC=BASIC as Commodore 64 + 6502 compiler

The last would probably not be needed if the Turbo Rascal translator was done.

Handoko

  • Hero Member
  • *****
  • Posts: 5122
  • My goal: build my own game engine using Lazarus
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #41 on: August 15, 2022, 07:51:14 pm »
That seems great. But in case you haven't know, not all open source licenses can be combined with others. For example Decimal BASIC's GPLv2 is incompatible with zxbasic's GPLv3. I'm not a lawyer, but these are what I found:

https://opensource.stackexchange.com/questions/1777/why-is-gplv2-incompatible-with-gplv3

Quote
... the GPLv2 license by itself is not compatible with GPLv3 ...
Source: https://en.wikipedia.org/wiki/License_compatibility#Copyleft_licenses_and_GPL


Wysardry

  • Jr. Member
  • **
  • Posts: 69
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #42 on: August 15, 2022, 08:18:18 pm »
This shouldn't be an issue if each tool was kept separate and retained its own license though, right?

I don't have any intention to merge code.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #43 on: August 16, 2022, 09:44:18 am »
Writing a crt unit for each system is a lot less work then trying to make all those work together. I wouldn't do such a project, it is almost guaranteed to fail.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #44 on: August 16, 2022, 10:10:11 am »
I don't have any intention to merge code.

In that case what's the point? You seem to be aiming at a cross-compiled solution with no common download mechanism. You appear to be aiming at similar "look and feel" on different target platforms, but using completely different libraries to achieve that... do you have any idea at all what a maintenance nightmare that's likely to be?

You started the thread asking about a DSL, but what you're now talking about isn't. You're now talking about multiple layers of code translation, but don't appreciate that with every hop you will introduce inefficiency and miss optimisation possibilities.

If you really are prepared to accept a cross-compiler and non-native IDE etc., then start off with FPC or e.g. Turbo-Rascal (don't expect to be able to use both) and after identifying what desirable targets they don't support learn enough to help the project team add the desired functionality.

In any event, you've got a big job on your hands and the first thing you have to do is have a robust description of what /exactly/ you're trying to do: for your own benefit as much as anybody else.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018