Recent

Author Topic: Can FreePascal be used to create an internal or embedded DSL?  (Read 5386 times)

avra

  • Hero Member
  • *****
  • Posts: 2385
    • Additional info
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #45 on: August 16, 2022, 10:19:05 am »
If that was my project, I would go a slightly different path. Write a p-code compiler and interpreter.
I had considered this idea, but discounted it due to the lack of memory on the 8-bit systems.

Interpreters and virtual machines often waste a lot of memory, as they have to contain code for every supported function even if the program it is currently running does not use them all.
Are you aware that almost every mentioned 8-bit system has built in BASIC interpreter as a proof that your claims are wrong? Only essentials are embedded, the rest is provided through external units/ libraries.

Although many of the 8-bit home computers had a Z80 processor, not all of them did, so FreePascal alone (with its current targets) would not be enough.
I've already said what would be enough:
Quote
You could start immediately writing apps for already supported FPC targets, and later implement missing targets for FPC.
If that step is too much for you, then you should not waste time on this direction...
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Wysardry

  • Jr. Member
  • **
  • Posts: 53
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #46 on: August 16, 2022, 06:05:16 pm »
In that case what's the point? You seem to be aiming at a cross-compiled solution with no common download mechanism.
The point would be to provide multiple translation and cross compilation options, depending on what target platforms the end user is targetting. There is no existing direct route for all platforms.

It's a little early to be thinking about how everything would be made available for download.

Quote
You started the thread asking about a DSL, but what you're now talking about isn't. You're now talking about multiple layers of code translation, but don't appreciate that with every hop you will introduce inefficiency and miss optimisation possibilities.
The tools I suggested were for a proof of concept, so that I could learn what would work and what wouldn't.

Decimal BASIC could become the DSL, with the appropriate adjustments to its FreePascal code. The existing BASIC to Pascal translators might also need adjusting, depending on whether I added code to create the DSL or just removed some.

The main reasons that is the start of the testing chain is it has BASIC syntax, an existing interpreter and is written in FreePascal.

Quote
start off with FPC or e.g. Turbo-Rascal (don't expect to be able to use both) and after identifying what desirable targets they don't support learn enough to help the project team add the desired functionality.
It's entirely possible that I might do that in the future, especially if my tests go badly.

Quote
In any event, you've got a big job on your hands and the first thing you have to do is have a robust description of what /exactly/ you're trying to do: for your own benefit as much as anybody else.
What I'm trying to do is create (or reuse) tools to allow text-based programs to be written in a domain specific language and/or library on modern platforms and then compile or cross compile them for as many platforms as practical. Testing should be possible on the modern platforms before cross compilation is attempted to simplify the workflow.

Even if I was creating everything myself from scratch, each of the tools would be independent programs and not every project would use them all.

Are you aware that almost every mentioned 8-bit system has built in BASIC interpreter as a proof that your claims are wrong? Only essentials are embedded, the rest is provided through external units/ libraries.
Are you aware that many of the 64Kb machine had less than 40Kb available to write programs in? Much of the unavailable space was used by the BASIC interpreter (although some was taken by the BIOS of the day).

Quote
I've already said what would be enough:
Quote
You could start immediately writing apps for already supported FPC targets, and later implement missing targets for FPC.
If that step is too much for you, then you should not waste time on this direction...
I've already said that I'd be willing to learn enough to add new Z80 targets as I've been told it's relatively simple due to the work already done by others.

The problem is that creating new targets for unsupported CPUs would be too much for me, so that route would lead to a dead end.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #47 on: August 16, 2022, 07:11:13 pm »
It's a little early to be thinking about how everything would be made available for download.

I meant download from the development host to the target. You need to be considering that sort of thing early- very early- since accessibility of debugging symbols etc. is an important issue.

Quote
Are you aware that many of the 64Kb machine had less than 40Kb available to write programs in? Much of the unavailable space was used by the BASIC interpreter (although some was taken by the BIOS of the day).

40K? Luxury!

Note that the term "BIOS" came in with CP/M, in which context it was a part of the OS which was loaded into memory. Most systems either had a full BASIC (with MS copyright notices) or a boot loader which might or might not have remained accessible after the OS had been started... the latter was again more of a CP/M thing but there were still non-CP/M systems which in principle were RAM-only such as the Sharp MZ-80.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Wysardry

  • Jr. Member
  • **
  • Posts: 53
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #48 on: August 16, 2022, 07:54:53 pm »
In most cases, the last tool in the chain would decide the final export format. There are also various existing tools to convert from one format to another.

I'm reasonably sure the "BIOS of the day" varied enough that there wasn't one applicable term, or one that I recall anyway. I do remember "CMOS" being used a lot, but as I understand it that is just a type of chip that can have multiple uses.

I used to own a Sharp MZ-700 and that had a mostly empty memory when first switched on. BASIC was provided on tape or disk and had to be loaded before you could use it.

I think other programming languages could be purchased and loaded instead, but I didn't own any.

I also owned an Enterprise 128, which had BASIC on a cartridge. Again, other languages could be purchased, but I only ever bought a BASIC compiler (on tape).

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #49 on: August 16, 2022, 08:15:42 pm »
BIOS was, pre-PC, specifically a CP/M term. At least on CP/M-80, DR shipped a standard BDOS and the machine-specific BIOS was quite simply concatenated to it by the manufacturer... CP/M-86 might have had, and CCP/M-86 definitely did have, a slightly more complex arrangement.

CMOS describes a particular type of electronics rather than to a particular device. Refer to Wikipedia.

And the Sharps were unusual in loading almost everything from tape, I think the ROM was left in the memory map but didn't have much on it. I used one for my MPhil.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

y.ivanov

  • Hero Member
  • *****
  • Posts: 560
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #50 on: August 17, 2022, 02:36:41 am »
It's a little early to be thinking about how everything would be made available for download.

I meant download from the development host to the target. You need to be considering that sort of thing early- very early- since accessibility of debugging symbols etc. is an important issue.

*snip*

IMHO is the trickiest part of the whole venture. All of the vintage home computers I've had (Apple ][, Oric Atmos, SinclairQL) used some sort of tape cartridges/cassettes or (luxury) floppies with a specific format. Not even a serial port (except for QL).

@Wysardry
If I'm getting you right, the idea is to use BASIC as a primary language, then transpile it to Pascal, use FPC as a cross-compiler to the specific targets (mostly 8-bit vintage CPUs), eventually debug the product in some emulators, and then somehow "magically" send them to the actual items for execution. Correct me if I'm wrong on this.

And in case I'm not wrong, there are few things to consider:

The main BASIC virtue isn't the simplicity of the language, but convenience of use - its interactivity and simultaneous execution, see DTSS. Your workflow is about to nullify those advantages.   

Most if not all of those micros have BASIC included into ROM. Despite it is mostly a toy implementation, it gives the ability to write short programs without the need to load additional executables from the external storage. You don't waste precious RAM, time or effort. In your workflow, again, it requires some sort of downloading and the binary will reside into RAM together with the RTL.

8-bit micros where made with Z80 and 6502 CPUs. While the former is somewhat capable, the latter is a pathetic compiler-target for any HLL - 8-bit registers, just one accumulator. One way it can be used is with some sort of bytecode interpreter written in assembly. Not sure it is a disadvantage, though, since the bytecode can be carved to be more compact than the native one from a compiler. But someone has to write the interpreter in assembly.

Each micro has its own ways to work with the display/keyboard, character codes are weird, sometimes with inverse/blinking bits in them. Any idea how to match, i.e. how to display the same letters for all micros? Graphics mode?

Have you considered/investigated the FORTH system? It exploits things like RPN, threaded code and it is traditionally targeted on restricted environments. IMHO it can be used as in intermediate language or/and interpreter.



"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

Wysardry

  • Jr. Member
  • **
  • Posts: 53
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #51 on: August 17, 2022, 06:26:37 am »
IMHO is the trickiest part of the whole venture. All of the vintage home computers I've had (Apple ][, Oric Atmos, SinclairQL) used some sort of tape cartridges/cassettes or (luxury) floppies with a specific format. Not even a serial port (except for QL).

@Wysardry
If I'm getting you right, the idea is to use BASIC as a primary language, then transpile it to Pascal, use FPC as a cross-compiler to the specific targets (mostly 8-bit vintage CPUs), eventually debug the product in some emulators, and then somehow "magically" send them to the actual items for execution. Correct me if I'm wrong on this.
That is one of the possible workflows, yes. Apart from the magical aspect that is.

There are existing tools that allow suitable files to be converted into an audio format. Some retro compilers include the option as standard.

For example, the ZX Basic Compiler can create files in .bin and .tzx format for use with emulators and in .mp3 and .wav format to use with digital recorders or record to tape.

Some emulators also include an audio export option, and there are third party tools which do the same.

I doubt these options exist for every retro platform, but they do for most of the popular ones.

Quote
The main BASIC virtue isn't the simplicity of the language, but convenience of use - its interactivity and simultaneous execution, see DTSS. Your workflow is about to nullify those advantages.
On a modern machine you do lose the immediate feedback you would get from a retro machine when entering code a line at a time, but you gain a better editing environment.

Including an interpreter should allow you to test programs each time you complete a section of code.

BASIC is still used on modern platforms, possibly by more people than Pascal (if you include all dialects of both languages).

Quote
Most if not all of those micros have BASIC included into ROM. Despite it is mostly a toy implementation, it gives the ability to write short programs without the need to load additional executables from the external storage. You don't waste precious RAM, time or effort. In your workflow, again, it requires some sort of downloading and the binary will reside into RAM together with the RTL.
The BASIC interpreter would only be used on modern platforms with memory and storage space to spare.

At the other end of the toolchain a machine code file would be transferred to the retro machine, which would most likely overwrite any BASIC interpreter in RAM. Most commercial software was sold in machine code form rather than in BASIC (although there were some exceptions).

Quote
Each micro has its own ways to work with the display/keyboard, character codes are weird, sometimes with inverse/blinking bits in them. Any idea how to match, i.e. how to display the same letters for all micros? Graphics mode?
I would hope the differences would be handled by the final compiler, if it includes configuration settings for multiple platforms. Graphics would not be an issue as the system is only intended to be used to create text-based programs.

Quote
Have you considered/investigated the FORTH system?
I have absolutely no experience with FORTH and the only thing I know about it is it was included with the Jupiter Ace, which I have never seen a physical example of.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #52 on: August 17, 2022, 09:28:09 am »
There are existing tools that allow suitable files to be converted into an audio format. Some retro compilers include the option as standard.

For example, the ZX Basic Compiler can create files in .bin and .tzx format for use with emulators and in .mp3 and .wav format to use with digital recorders or record to tape.

Some emulators also include an audio export option, and there are third party tools which do the same.

I doubt these options exist for every retro platform, but they do for most of the popular ones.

Most of these things used audio, i.e. CUTS or something roughly equivalent, so in principle a bit of signal processing and you're there. There might have been one or two which were actually digital (i.e. PE or whatever), but in any case you should find designing something that looks like a cassette pretty easy... I presume you've got a 'scope etc. and know how to use it.

Quote
I have absolutely no experience with FORTH and the only thing I know about it is it was included with the Jupiter Ace, which I have never seen a physical example of.

In that case get off your backside and at least read about how it worked. That sort of ignorance is out of place for somebody trying to do what you are.

But I'm troubled that the use of a multi-layered cross-development platform with diminished immediacy on the target makes this a non-starter: there just isn't enough there to make it attractive to the average retro user who only cares about the computer and software he remembers from his teens. Are you /really/ serious about dedicating a big chunk of your life to doing this?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

avra

  • Hero Member
  • *****
  • Posts: 2385
    • Additional info
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #53 on: August 17, 2022, 10:57:06 am »
Are you aware that almost every mentioned 8-bit system has built in BASIC interpreter as a proof that your claims are wrong? Only essentials are embedded, the rest is provided through external units/ libraries.
Are you aware that many of the 64Kb machine had less than 40Kb available to write programs in? Much of the unavailable space was used by the BASIC interpreter (although some was taken by the BIOS of the day).
In the eighties ZX81 had 8Kb ROM and 1Kb RAM, and it had built in BASIC editor (which takes most space and not needed by you) and interpreter. Screen handling, cassete and cartridge handling and lots of other things also took ROM space but let's be gentle and not count them. ZX Spectrum had 16Kb ROM and 16Kb RAM. I remember that in my early ATMEL AVR days I have even seen a magazine publishing full sources of pascal interpreter for a microcontroller with 1Kb RAM and 8Kb flash. That is way below your available resources so implementation possibility is not in question. Did you look at p-machine implementations at all? P6 is fresh, but first few were built in the seventies when resources were even more limited, so I do not see a problem comming from that side.
« Last Edit: August 17, 2022, 11:18:36 am by avra »
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #54 on: August 17, 2022, 11:24:04 am »
Did you look at p-machine implementations at all? P6 is fresh, but first few were built in the seventies when resources were even more limited, so I do not see a problem comming from that side.

Noting that there are two different usages of "P" here: Wirth's original system described at http://pascal.hansotten.com/px-descendants/ and the UCSD P-system.

I think it's worth noting that chips which are currently considered to be "absolutely miniscule" like the ESP8266 hence barely supportable by Pascal are, in fact, substantially larger than most 1980s home computers.

TBH, I think that OP would do far better throwing his enthusiasm behind either FPC or Turbo Rascal, or identifying some comparable project with a common frontend language and multiple backends.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 4759
  • Compiler Developer
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #55 on: August 17, 2022, 01:34:26 pm »
I think it's worth noting that chips which are currently considered to be "absolutely miniscule" like the ESP8266 hence barely supportable by Pascal are, in fact, substantially larger than most 1980s home computers.

I wouldn't consider the ESP8266 even close as “miniscule”. It's simply an embedded platform just like the Pi Pico which Pascal can run very well on. More miniscule are the AVRs, especially the TinyAVRs and even there you can use Pascal ;)

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #56 on: August 17, 2022, 02:20:08 pm »
I wouldn't consider the ESP8266 even close as “miniscule”. It's simply an embedded platform just like the Pi Pico which Pascal can run very well on. More miniscule are the AVRs, especially the TinyAVRs and even there you can use Pascal ;)

I'd hardly call the Pico tiny, with 100s of Kb RAM and provision for several Mb Flash. And whenever one maxes-out an AVR one can simply move to the next one along the product range.

However I think it's worth emphasising for OP's benefit that FPC/Lazarus has decent debugging facilities for targets such as the Pico/RP2040, and that it's taken the collective development tool communities hundreds of man-years to get there.

If I had £1 for every suggestion I've seen over the years that a single under-resourced developer of limited experience could rewrite everything from scratch in a finite time, or that "it must be easy" to integrate a few dozen tools from competing suppliers to produce a result that would have well-heeled customers beating a path to his door, I'd be much richer than I am now.

And long-term members of this community will know that I've done some of this stuff myself, with some degree of success. But I have enormous respect for the FPC team- Florian, Jonas, yourself (PascalDragon) and the rest, who by and large manage to keep up with the minutiae and ramifications of an increasingly-large language and ecosystem.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 10399
  • FPC developer.
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #57 on: August 17, 2022, 02:48:00 pm »
Did you look at p-machine implementations at all? P6 is fresh, but first few were built in the seventies when resources were even more limited, so I do not see a problem comming from that side.

Which P6 ?

https://github.com/samiam95124/Pascal-P6     Last commit 4-5 years ago....

http://wirth-dijkstra-langs.org/  Hasn't processed FPC compatibility contributions sent 3 (4?) years ago.

Both seem to be quite stale.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5572
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #58 on: August 17, 2022, 03:46:27 pm »
Did you look at p-machine implementations at all? P6 is fresh, but first few were built in the seventies when resources were even more limited, so I do not see a problem comming from that side.

Which P6 ?

https://github.com/samiam95124/Pascal-P6     Last commit 4-5 years ago....

http://wirth-dijkstra-langs.org/  Hasn't processed FPC compatibility contributions sent 3 (4?) years ago.

Both seem to be quite stale.

https://sourceforge.net/projects/pascalp5/ appears to have seen action over the last year or so.

Why should they have any interest in FPC compatibility? My recollection is that their priority is to work on (emulations of) larger machines including IBM mainframes, so they might simply have ignored them

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Wysardry

  • Jr. Member
  • **
  • Posts: 53
Re: Can FreePascal be used to create an internal or embedded DSL?
« Reply #59 on: August 17, 2022, 07:32:12 pm »
In that case get off your backside and at least read about how it worked. That sort of ignorance is out of place for somebody trying to do what you are.
There are plenty of other rabbit holes for me to dive down that would be a more direct route to where I want to end up.

Most likely, Forth is one of the many languages I have discounted because they either don't run on the machines I would be targetting or the syntax looks too different to BASIC or Pascal.

Quote
But I'm troubled that the use of a multi-layered cross-development platform with diminished immediacy on the target makes this a non-starter: there just isn't enough there to make it attractive to the average retro user who only cares about the computer and software he remembers from his teens. Are you /really/ serious about dedicating a big chunk of your life to doing this?
This project wouldn't really be aimed at someone who wants to use a particular computer from their youth, as they can just use BASIC on a physical machine or in an emulator.

This would be for someone who likes or uses more than one machine and wants to produce software for all of them plus modern machines, using one set of code.

The choice of BASIC isn't really for the immediacy aspect, it's to make the system more familiar and to potentially ease translation to other BASIC dialects.

I don't want to dedicate a big chunk of my life to it, no. That's why I'm trying to reuse as many existing tools as possible.

Did you look at p-machine implementations at all? P6 is fresh, but first few were built in the seventies when resources were even more limited, so I do not see a problem comming from that side.
I actually bought a used book on creating text adventures in UCSD Pascal a couple of years back.

I haven't seen any implementations for 8-bit home computers though, so I haven't explored it too thoroughly.

TBH, I think that OP would do far better throwing his enthusiasm behind either FPC or Turbo Rascal, or identifying some comparable project with a common frontend language and multiple backends.
I will definitely be considering adding extra Z80 targets to FPC if it becomes part of the toolchain.

Turbo Rascal would be somewhat of a non-starter because AFAIK it has a small team who likely have their own workflow and wouldn't want a beginner who hates C messing with their existing code.

I could potentially fork it on Github, but my progress would be slow and I'd hate every minute of it due to the amount of C and C++ code I'd have to contend with.

If I actually liked C and knew more about it, I would have more options available to me.

If I had £1 for every suggestion I've seen over the years that a single under-resourced developer of limited experience could rewrite everything from scratch in a finite time, or that "it must be easy" to integrate a few dozen tools from competing suppliers to produce a result that would have well-heeled customers beating a path to his door, I'd be much richer than I am now.
I do know that it will be far from easy. I've been searching for potential solutions on and off for about two years now, trying to find compatible pieces of the puzzle.

The shortest existing toolchain I have found so far is Nim (via its C output) to SDCC or z88dk, a combination which apparently has been successfully used with micro boards with 32Kb ROM and 512 bytes of RAM. There is no existing option to use BASIC with that, but it might be possible to use macros to allow the use of some BASIC and/or domain specific keywords.

Although Nim was originally written in Pascal and there is a Pascal to Nim translator, Nim is now written in Nim and there is no Nim to Pascal translator available that I know of.

I'm not trying to do this because I expect it to sell like hot cakes (it would be free anyway), but rather because it would be a tool that I would find useful and I hope that at least some others would find useful too.

 

TinyPortal © 2005-2018