"Mario & Luigi" (1994-2001, DOS) - building and porting

[_skrzyp_]

Hello!

Little bit of introduction...

At first I should admit, that I'm not a Pascal expert of any kind. I fiddled around with TP7 when I was 8-10, only had access to very "academic" books about it and - at that age - considered it boring and moved forward.

Now, two decades later, I'm rediscovering FreePascal and I feel it's underrated and overlooked a little too much, considering it's clarity and features compared to C or C++, on the same level of native code generation.

However, there's someone who succeeded in Pascal a little more, and this is the major reason of this post.

Mike Wiering made a homebrew game called Mario & Luigi in 1994 and published it officially in 2001 with complete source code.

UPDATE: I made a GitHub repo with the unmodified source code unpacked for easy access. Additionally, there's a very simple build script there and files got put into separate subdirs for sources and sprite data.

Many of you (especially in Europe) probably played it, even before 2001. There was a leaked beta version which was spreading around on a floppies, but the game wasn't complete and you couldn't get past on some level. I also played that version back in the game, and the "official" 2001 release too.

The game runs in 16-bit real-mode under MS-DOS, the binary is ~57KB UPX'd and ~170KB uncompressed. It also ran properly on 9x/Me and NTVDM under 2000/XP. It stopped working for me on machines past Pentium 4 era even under the same NTVDM, still unsure if it was about the CPU or later GPUs not handling direct VGA access properly.

...but straight to the point...

After my latest encounter of FreePascal (I was looking for non-C environments which were able to generate native code on m68k/Amiga) the game got back on my mind and I recalled the website has original source code archives.

So I've got an idea: Could I try building the game with latest FPC version, then port it on other platforms?

And then, the whole journey started... It's not as flashy as it could be, and most likely I won't be able to progress forward without your help. But I'd like to post my present experiences.

What I did and where I am?

Okay, so I grabbed the "stable" native FPC 3.2 and built the i8086/msdos target from source. It was easier than expected, the internal build system isn't that well documented and a little messy, but still works better than bootstrapping cross GCC, for example.

Initially, I thought that binary isn't packed in any way so assumed I should target -WmTiny memory model, and straight crashed to the issues with "NearPointer" vs "FarPointer". Took me a bit to realize that TP7 defaulted to what FPC calls "-WmLarge" memory model.

However, the FPC 3.2 failed to build it even then.

One of the initial set of issues resulted from sources having ASCII >127 characters present as options in case statements, as well as other expressions. It's related to level data stored in raw format and using these characters for blocks, enemies, etc. Setting codepage directives to CP437 did not change anything, sadly.

I made a simple Python script to replace all occurrences of extra ASCII chars with direct #nnn statemens. It made the related errors go away, but then the final boss arrived:

Code: [Select]

ENEMIES.PAS(354,3) Fatal: Internal error 200309041
Fatal: Compilation aborted

The procedure StopEnemies looked quite normal and didn't show any suspicious behavior. I tried commenting it out and other lines calling it, but the problem persisted.

Someone told me to use latest 3.3.1-trunk compiler from snapshots instead. It changed mostly everything and made the application compile without any changes in source code.

So I started pulling these from snapshots (cross-ompilers for x86_64/linux, there are also win32 ones): https://downloads.freepascal.org/fpc/snapshot/trunk/i8086-msdos/

I am using the following command to build it:

Code: [Select]

$FPC_LARGE/bin/x86_64-linux/ppcross8086 \
  -XX \
  -Tmsdos \
  -Mtp \
  -WmLarge \
  -Fu'$FPC_LARGE/units/i8086-msdos/*' \
  -Fu'$FPC_LARGE/units/msdos/*' \
MARIO.PAS

It compiles nicely to ~190KB executable and ~70KB UPX'd. Great success.
However, it doesn't run.

Trying in DOSBox-X, it just shows blinking cursor, and the DOSBox throws this on standard output in a loop:

Code: [Select]

ERROR CPU:Illegal Unhandled Interrupt Called 6

Sometimes, when I flip the various build flags (like -Xs, -O-, -Cp8086, etc.) or just move the order of these flags around it also throws the Invalid opcode XXXX in a loop (interleaved with the previous error). Changing randomly every time I rebuild the code with different params.

Same happens on QEMU with FreeDOS 1.4 (boot floppy). But this time the FreeDOS just stops the application and prints "Invalid Opcode NNNN NNNN NNNN NNNN..." every time.

NOTE: The original binary from official website runs just fine without any issues on both of these platforms.

The README.TXT suggests that the application should work on at least 80486, so I tried -Cp80486 with no effect. What was I thinking, since TP7 doesn't perform such optimizations, I don't know

The game also uses FPU due to some divisions and Round()] calls here and there. Currently, i8086 target does not allow for soft-floats so I made sure that both DOSBox and QEMU had FPUs enabled and tested them, still no changes.

Someone even suggested LOADFIX -a MARIO.EXE which should fix memory access for 16-bit realmode code, even in DOSBox. Guess what, nothing changed

I found that the game does InitKeyboard on very beginning which is registering a custom keyboard interrupt handles. Makes sense, but the GetKey procedure is hardcoded assembly. Actually, the whole game has lots of hardcoded x86 asm in random places. Initially I thought there's some bug in how FPC handles such procedures or references in these, so commented it out. At least the game should run without keyboard, right? Not really, nothing changed.

The DOSBox-X has internal debugger, grandfathered from upstream DOSBox. However, it's atrocious and lacking many features. For example, you can't step through the code with commands, you need to use F<x> keys only (and these conflict with my terminal emulator and system hotkeys). You can't also load symbols or show backtrace stack respectively, like GDB.

Speaking of GDB, I tried to plug i386-elf-gdb to QEMU which supports it. But GDB generally doesn't work really well with real mode and segmented memory, mostly showing trash or unreliable data.

Sadly, I'm kinda out of options in regards to how to debug this properly. I see there's a big issue with debugging real-mode DOS stuff in general. The most respectable 86box/PCem emulators don't seem to provide any means of debugging, too.

I also tried various ways of commenting or flipping around some code blocks to make it at least go "somewhere", but unfortunately I can't really find any way to progress forward, as I'm definitely not an expert in low-level PC/DOS architecture. Mostly an Amiga guy in general, which is definitely simpler in basic stuff like this, plus has linear memory addressing without segmentation or XMS/EMS/UMB/whatever

The Plan

The ultimate objective is to make this game running on modern (and maybe some legacy) platforms without changing the gameplay and make it a faithful port. Similar to how Apotris has been done (but it's in C++).

But the current plan is:

• Make the current dump of original TP7.x sources compile under FPC-trunk (i8086/msdos) with only necessary changes to execute the game properly: We are here.
• Refactor the code a little and abstract out hardware access to separate unit, still making it work on DOS all the time forward.
• Drop -Mtp and fix issues resulting from that.
• Try adding at least one simple target which allows to open a framebuffer in indexed color mode (Lazarus canvas? BGRABitmap? SDL2? Any suggestions?)
• Write down the current experiences, refine the project and decide if we should go forward with FPC or move to any other environment
• If decided with Pascal, move to Modern Pascal / ObjFPC completely
• Add proper desktop (Windows, MacOS, Linux) support
• If the code is reliable and abstracted enough, add all platforms supported by FPC which are capable of running this game in separate platform units (Amiga, GBA, NDS, PSX, WebAssembly, etc...)
• Finally start adding improvements over the original feature set: save/restore (currently it just stores the last level and restarts from there), custom levels (maybe with editor), speedrun timer, and so on
• Refine and optimize with gained experience from previous steps

My questions

• How to progress forward with making current source code build properly? I am aware that FPC might not be able to replicate all of the TP7 quirks and some changes should be made, but I can't really find anything that should be patched here for now.
• Do you know any reliable ways to debug real-mode code under DOS, which are not a Turbo Debugger?

Sorry to be a little too broad and lacking specifics in some places, but as I said at the beginning, I am not a Pascal expert and probably might be overlooking something very obvious. Additionally, if I succeed with the initial issues I'd plan to keep this thread updated in the future with more information as I go forward.

Thank you in advance for any bits of information and suggestions.

[Gustavo 'Gus' Carreno]

Hey _skrzyp_,

First of all, I want to congratulate you on just the initial initiative!!! That does demonstrate BALLS!!
I also want to send emotional support for all the walls your head has banged into.

Allow me to give you some suggestions that may, or may not , help you along the way:

• Don't use the command line to compile:
  • In Lazarus, create a project for your main file.
  • This allows the means to have multiple build modes.
  • And per each, you can define different flags for the compiler.
• If you're into it, please use fpcupdeluxe:
  • This will allow you to install multiple cross-compilers with ease.
  • Pair that with the build modes from Lazarus and it's a drop-down away to compile with different cross-compilers, options and compiler flags.

After opting for creating a Lazarus Project file(.lpi), and you still want some command line freedom, you can use lazbuild. I abuse it quite frequently on my GitHub actions, paired with the setup-lazarus action.
I would also have a look at this post regarding lazbuild: https://forum.lazarus.freepascal.org/index.php/topic,71345.0.html

Now, this advice probably wont solve your current issues. But it will give you an amazing Ease of Life boost!! And we gamers are all for Ease of Life hacks, amirite?!

Hope that this advice eases you on your journey!!

If you opt for using fpcupdeluxe and run into trouble, please ask for assistance here on the forums or on the Discord Server!!

Cheers,
Gus

[_skrzyp_]

Updated with link to the Git repository in original post.

As for Lazarus, at this point I don't plan going that way. But I am pretty sure I'll be testing the waters when the time for modern port comes. I might, however, try to use fpcmake for CI/CD builds in the future.

Right now I'm mostly interested in fixing the original sources.

[paule32]

not to come to complexy ...
but take a look to Visual Code, too (as you said you are not a Pascal'er - maybe an other DSL can help you more ...)

as you wrote, you would support the Amiga 500 Line, you could take a look to the VC PlugIn from Bebbo - it is available under github, too...

I would not publish the Links because there is a Pascal Forum (and Bebbo Plugin is hold in C/C++)
But you can could try to make a Amiga Library that imports Symbols from a existing Code Base into your new FPC Application's optimized for Commodre Amiga Computers (since it is support for Amiga Images by FPC)

It give Emulator's for the Amiga, so you can compile and test your Image under Windows / Linux with the VICE (C-64) or others.

If you want to hold the Pascal Line, then take a Look to the RASCAL Compiler and IDE (which can produce many Console Application's with Pascal Code)

So, now, you have some Views or Options where you can go ...

Hope this helps to make a Decision for your future Development's

All mentioned Tools and minds are Hints and not for make Decisions for anybody - It is a kind of my advices For Looking For Solution before the journey begin ... WITH NO GURANTEES OR CLAIMS !!!

[_skrzyp_]

As you can see in the screenshot, we have a brand new game, IOMAR & GILUI!

It might hopefully make Nintendo less suspicious, if anything!
But, honestly, I see some issues with data alignment and overall instability.

Actually, what I did here is to comment out ReadConfig() procedure and now the game runs. But as you can see, the title and border tiles are garbled, game sprites/blocks are fine.

Not only that, the gameplay starts, but it's not quite there.
Pressing RIGHT makes Mario go LEFT(?)
Pressing LEFT... RESETS the level (fade out -> fade in, so it's triggering some procedure).
Jumping does not work.

If I un-define MENU, it starts without menu system and gameplay works in the same weird way.

If I un-define MENU and uncomment ReadConfig() back, the game goes into proper VGA mode but stays on black screen after that.

If I define MENU again, comment out ReadConfig() and add -CX and -Xs to build system, we're back to square one with same issue (cursor blinking, INT 6h fired in a loop).

Started thinking about that ReadConfig() procedure and its DOS calls for file access. I removed {$I-} (IOCHECKS) from MARIO.PAS and uncomment ReadConfig():

With undefined MENU it goes to proper VGA mode with black screen.
With defined MENU there's that original INT 6h loop again

Why is it so random and unpredictable?

[paule32]

in the Time of MS-DOS the GamePlay Video Interrupt Mode was

0x13 (hex)  with 320x200x256 (xpixel x ypixel with 256 Colors)

Code: ASM  [Select][+][-]

1. mov ax, 0x13  ; video mode
2. int 0x10      ; dos video interrupt

Your screen shot looks like over the limits, so the measure of the Screen Dimension may be lack ...

[_skrzyp_]

The game is already using 320x200@256, if you read the VGA256 unit you can already see the elaborate way of parametrized setting of VGA mode.

As I said in original post the game from upstream release binary works on the same environment(s). And yes, it has the same graphics mode.

For reference, I'm attaching the screenshot of original MARIO.EXE.

Actually, after all these posts and comments around, I should say out that loudly: I appreciate everyone's help here, but I'd prefer comments directly related and relevant to the current problem space instead of tossing random advice around.

Right now, the previous post explains the current state, including broken ReadConfig() proc (especially the DOS calls), graphical and input issues probably related to memory alignment (which I suppose might be the same case as ReadConfig). I still don't have any reliable way to debug the real-mode code with symbols loaded - considering adding a "logger" over parallel port or serial connection, but if the executable itself has issues with memory alignment it might lead me absolutely nowhere.

UPDATE: Added TPBUILD.BAT script for building sources with Turbo Pascal 7.x to the GitHub repository so you can double-check if the tree still builds with old TP. It does, and produces working executable on the same machine.

[Mr.Madguy]

I would help, but I'm on vacation now, i.e. no access to my computer. This program has lots of asm code. Including using procedures to store data. So first thing, I would do - disassemble binary and check, if some unnecessary code is injected into procedures or not.

P.S. The most suspicious part of ReadConfig - is GetConfigName. Plus there is some sort of game exe name CRC check at the end of it.

[_skrzyp_]

Yeah, I noticed that - DOS function calls in general make the program go crazy. But I sense it's coming from the same origin issue as garbled memory is.

ACTUALLY, we're quite lucky that the Mario even compiled.

If you load up latest 3.3.1-trunk which I'm using for these porting efforts and just put a simplest HELLO.PAS to it:

Code: Pascal  [Select][+][-]

1. program Hello;
2.  
3. begin
4.   WriteLn('Hello World!');
5. end.
6.  

and build it:

Code: [Select]

${FPC_LARGE}/bin/x86_64-linux/ppcross8086 \
  -XX -Tmsdos -WmLarge \
  -oOUT/HELLO.EXE \
  -Fu${FPC_LARGE}'/units/msdos/*' \
  -Fu${FPC_LARGE}'/units/i8086-msdos/*' \
SRC/HELLO.PAS

It gets compiled to executable, but hangs in a loop without printing anything and throws INT 6h all over the place.

So I wonder if there are actually bugs in the code. Of course there might be, sure, but hadn't thought about bugs in the compiler itself!

[Gustavo 'Gus' Carreno]

Hey _skrzyp_,

I'm now wondering if the Hello World issue is with FPC itself, or the DOSBox-X itself...

Also... Is it a trunk issue, or can you reproduce same hang on previous stable versions?

Asking for a friend

Cheers,
Gus

[_skrzyp_]

3.2-stable works properly with Hello World, of course.
But trying to build the Mario source there results in "Internal error" at some point.

Anyways, I am going to try building with actual HEAD from source, then start bisecting between 3.2 and HEAD maybe to track down the "malicious" commit.

UPDATE: HEAD build with make all TARGET=... and make crosszipinstall ... (as described in DOS docs on wiki) but it still fails :/

[Gustavo 'Gus' Carreno]

Hey _skrzyp_,

3.2-stable works properly with Hello World, of course.
But trying to build the Mario source there results in "Internal error" at some point.

That's good to know!! Thanks for having a go at testing it with 3.2-stable!!

Anyways, I am going to try building with actual HEAD from source, then start bisecting between 3.2 and HEAD maybe to track down the "malicious" commit.

You're my effing hero !!!
I'm loving your journey and I'm pleased you're sharing it with us plebs

Cheers,
Gus

[Mr.Madguy]

Yeah, I noticed that - DOS function calls in general make the program go crazy. But I sense it's coming from the same origin issue as garbled memory is.

ACTUALLY, we're quite lucky that the Mario even compiled.

If you load up latest 3.3.1-trunk which I'm using for these porting efforts and just put a simplest HELLO.PAS to it:

Code: Pascal  [Select][+][-]

1. program Hello;
2.  
3. begin
4.   WriteLn('Hello World!');
5. end.
6.  

and build it:

Code: [Select]

${FPC_LARGE}/bin/x86_64-linux/ppcross8086 \
  -XX -Tmsdos -WmLarge \
  -oOUT/HELLO.EXE \
  -Fu${FPC_LARGE}'/units/msdos/*' \
  -Fu${FPC_LARGE}'/units/i8086-msdos/*' \
SRC/HELLO.PAS

It gets compiled to executable, but hangs in a loop without printing anything and throws INT 6h all over the place.

So I wonder if there are actually bugs in the code. Of course there might be, sure, but hadn't thought about bugs in the compiler itself!

Do you use default DOSBox config? May be you use 8086 CPU? Try using default config. What default CPU setting FPC uses? May be it's 386 or something like that? Try setting CPU explicitly. Yeah. Debugging DOS applications is complex task. Because modern development tools generate debug info, that isn't compatible with old debug tools and new ones don't exist. So, debugging requires some advanced techniques, like debug log. Can you use some DOS debugger like TD or WD to see, what instruction causes Int 6 (invalid opcode)?

[Thaddy]

If it hangs in dosbox, try freedos instead (provided you have the hardware or use e.g. QEMU). Does it also hang?

[paule32]

INT 06 - INVALID OPCODE

This interrupt is generated when the CPU attempts to execute an invalid opcode (most protected-mode instructions are considered invalid in real mode) or a BOUND, LDS, LES, or LIDT instruction which specifies a register rather than a memory address
To go the Way to / back to DOS, You could check out the following Links:

DOS-Commands:
https://en.wikipedia.org/wiki/List_of_DOS_commands

Learning Assembly:
https://www.tutorialspoint.com/assembly_programming/assembly_introduction.htm


Some Assembly Notes:
https://www.philadelphia.edu.jo/academics/qhamarsheh/uploads/Lecture%2020%20----16-Bit%20MS-DOS%20Programming.pdf

A Assembly Example:
https://medium.com/ax1al/dos-assembly-101-4c3660957d25

DOS System Calls:
https://www.pcjs.org/documents/books/mspl13/msdos/encyclopedia/section5/

Interrupt List:
https://www.ctyme.com/rbrown.htm

DOS 2.0 Source Codes:
https://github.com/microsoft/MS-DOS/tree/main/v2.0/source