Recent

Author Topic: Free Pascal for GEMSOS, a fully segmented 16/32-bit x86 security kernel/apps  (Read 8423 times)

eclipse

  • New member
  • *
  • Posts: 5
Hello -

I work with Aesec, and the GEMSOS operating system from our subsidiary Gemini Computers.  GEMSOS is a high assurance security kernel running on the x86 architecture.  We have been using 16-bit and 32-bit versions of the MetaWare Professional Pascal compilers using Compact memory model.  GEMSOS uses a fully segmented memory system - segments are used to model / instantiate protected security objects in the the system.

We use the MetaWare compiler in a cross compiler configuration, hosting it on a Unix SysVr3 system.  Linux IBCS lets us run it on Linux, at least in 32-bit mode.

The kernel of the OS runs in 16-bit mode and exports a 32-bit interface.  Applications and services outside the security kernel run in 32-bit mode, with full 48-bit addressing and 32-bit offset support, so that segments are not limited to 64KB.  I think we impose a 2GB limit on segment sizes.

I'd like to migrate from the no-longer supported MetaWare compilers to something else. Free Pascal looks like a good candidate.

I see you're working on an i8086_msdos target support for the compiler (http://wiki.freepascal.org/DOS).  You support the compact memory model.

At present we use COFF, but would like to migrate to ELF, provided I can create a suitable ABI profile to meet our needs.  We use run-time intersegment linkage to allow far calls between segments using destination segment entry point symbols, which is why we can get by with Compact model near pointers for calls.  We provide our own intersegment linkage that hides the far calls from the developer.

I wonder if you believe it would be practical for me to extend that support to provide what amounts to an i386_gemsos (or i486_ or i586_) version, perhaps using gemsos as the system name in lieu of msdos to allow me to handle any binary interface issues I need to for backward compatibility with the MetaWare compilers - for example, they produce a SEG_12 (COFF) relocation entry for the 16-bit segment selector offset in static pointer storage (.data and .bss of COFF) to enable load/run-time dynamic linking of object files.

We use modules in MetaWare, which looks like it should map reasonably well to facilities you support.

We would like to improve our intersegment linkage to use a more natural set of tools - probably modeled on DLLs and ELF GOTs - for separate code segments.

I have already adapted (trivially) the GNU binutils gas and ld to recognize and deal with the SEG_12 relocation entries, and am able to routinely use gas and ld to create our COFF binaries.

So - my question for you has to do with whether you or anyone else has ever expressed interest in extending the DOS memory model support to support 48-bit pointers (16 bit segment selector + 32-bit offset), or does it sound like too large an effort to be worthwhile?

My understanding of the support we would require would include:

1) use 48-bit pointer storage for all data pointers, including any pointers stored in static (.data, .bss, etc.) areas, passed as parameters on the stack, stored as dynamic variables in stack frames, or returned as pointer function return values;
2) create a relocation reference in produced object file for the segment selector of static data storage locations, particularly those requiring run-time initialization
3) ability to specify the storage segment for static data declarations (MetaWare uses pragmas to do this, but other mechanisms would be fine).

Regards,
Ed Reed

Edit:  revised Subject for clarity
« Last Edit: January 29, 2015, 02:22:04 pm by eclipse »

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7592
Re: Free Pascal for GEMSOS, a fully segmented 32-bit x86 security kernel
« Reply #1 on: January 29, 2015, 12:09:15 pm »
I signaled Nikolay (the i8086) manager over this message.

But FPC uses different codegenerators per architecture. (so for x86 differing codegenerators for (16bit:) i8086, (32-bit:)i386 and 64-bit x86_64.

Afaik only the 16-bit supports segmentation.

Similarly there are also two dos ports, "msdos", the 16-bit one, and "go32v2" the 32-bit dos extender. 

As for binary format, that depends largely on the used and available binutils. 16-bit uses watcom tools, not GNU ones.

eclipse

  • New member
  • *
  • Posts: 5
Re: Free Pascal for GEMSOS, a fully segmented 32-bit x86 security kernel
« Reply #2 on: January 29, 2015, 02:20:45 pm »
Thanks - I'll watch for his reply.

Looking a bit at the code for the i8086, it looks like it presently has support for far proc memory models, but not yet for far data models - perhaps I can help out a bit there.  I'll keep studying in the mean time.

Thanks for flagging this for him.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7592
There is far, but only for heap/pointers, static datasegment still limited to 64kb like TP.

There is also huge where all pointers are normalized, but I don't know the impact on that on static datasegment.

There is no model (except maybe huge)  where static data segment can be larger than 64kb afaik.

nickysn

  • Newbie
  • Posts: 4
Hi,

The i8086 compiler (16-bit) has full support for near, far and huge pointers and you can use them in any memory model. Huge pointers can be normalized or not normalized - there are compiler directives, which turns normalization on or off, see:

http://wiki.freepascal.org/DOS#Huge_pointer_normalization

As for memory models, the following are supported and should work: tiny, small, medium, compact and large. Only the huge memory model is not implemented yet. Note that huge memory model has nothing to do with huge pointers (which are implemented and work). We're probably going to follow the Borland C definition of 'huge memory model', which basically means allowing static data to exceed 64kb (and pointers are still far by default, not huge).

The i386 compiler doesn't support far pointers. I've considered implementing them, but none of the platforms, currently supported by the i386 compiler even allow them (because they don't have a scratch segment register in their memory model, that can be used by the compiler to load the segment part from the 48-bit far pointer), so I decided it's not worth the trouble. It's certainly possible to implement them, especially thanks to the new high level code generator, introduced in fpc 3.x (which made implementing the far data memory models in the i8086 compiler surprisingly easy). Still, it's certainly not a trivial task and will require some decent amount of work, but it's doable.

eclipse

  • New member
  • *
  • Posts: 5
Hi,

The i8086 compiler (16-bit) has full support for near, far and huge pointers and you can use them in any memory model. Huge pointers can be normalized or not normalized - there are compiler directives, which turns normalization on or off, see:

http://wiki.freepascal.org/DOS#Huge_pointer_normalization

As for memory models, the following are supported and should work: tiny, small, medium, compact and large. Only the huge memory model is not implemented yet. Note that huge memory model has nothing to do with huge pointers (which are implemented and work). We're probably going to follow the Borland C definition of 'huge memory model', which basically means allowing static data to exceed 64kb (and pointers are still far by default, not huge).

The i386 compiler doesn't support far pointers. I've considered implementing them, but none of the platforms, currently supported by the i386 compiler even allow them (because they don't have a scratch segment register in their memory model, that can be used by the compiler to load the segment part from the 48-bit far pointer), so I decided it's not worth the trouble. It's certainly possible to implement them, especially thanks to the new high level code generator, introduced in fpc 3.x (which made implementing the far data memory models in the i8086 compiler surprisingly easy). Still, it's certainly not a trivial task and will require some decent amount of work, but it's doable.

Thank you for your reply - I'm uncertain at the moment just what the Huge memory model, and indeed, Huge Pointers are - but I'll read your link and follow where it leads me.

As I mentioned above, I need both 16-bit and 32-bit support, so both the i8086 for 16-bit and some variation for 32-bit (by which I mean, 32-bit offsets combined with 16-bit segment selectors in 48-bit pointers).

A simplifying assumption possible, given the degree of support we have in the version of MetaWare we're using, is perhaps to simply declare the size of a pointer to be 48-bits, and then use assembler routines we already have in hand to set the selector and offset portions of the pointers directly - we currently have no compiler support for doing that - all we rely on the compiler to do is to keep track of which section the stored pointer belongs in (.data, .stack, or .bss) and to place the SEG_12 (a COFF 16-bit absolute relocation entry) with the stored data location into the appropriate section in the object file.  Having more compiler support, and indeed, incorporating dynamic linking support, would be a wonderful long term goal, but isn't necessary for where I am, now.

Amusingly, it appears that MetaWare took the track of using the "unsigned" data type to represent 16-bit unsigned integers in 16-bit compiler mode, and 32-bit unsigned integers in 32-bit compiler mode, using "cardinal" to be always 16-bits.  I see fpc has followed a different scheme, which may cause me some murmuring as I convert my existing code - but perhaps a "MetaWare" compiler flavor would be useful, anyway.  Then I can call an fpc "unit" a MetaWare "package", and work out how to handle other "extensions" we make use of.

If you had to choose, base on what you know, which would be easier:  a) add 32-bit offset / 48-bit pointer support to i8086 to support Compact Mode (near code, far data), or b) restore segmentation support into i386?

Thanks,

eclipse

  • New member
  • *
  • Posts: 5
Okay - now I know what huge pointers are, and while they're interesting, they're not something any of my existing code uses, or likely will - that's fine, to each his own.

I realize the msdos i8086 compiler version uses NASM and Watcom linkers.  I'd prefer to continue to use the AT&T linker variation from binutils ... all my 16-bit assembler support code is in AT&T SysVr2 assembler, and linker tools assume ld. 

This, by itself, might force me into using the i386 as base for the 32-bit compiler I need, and either continuing to use my existing MetaWare and SysVr2 as and ld tools, or actually making my changes to the i386 base be able to do my 16-bit bidding, too.  Ugh.

Sorry if I'm thinking out loud, here.  It's what I do.  It has the advantage that if I go off onto a tangent, someone may notice and nudge me back ... or simply wave as I go off the road.

nickysn

  • Newbie
  • Posts: 4

If you had to choose, base on what you know, which would be easier:  a) add 32-bit offset / 48-bit pointer support to i8086 to support Compact Mode (near code, far data), or b) restore segmentation support into i386?


IMO, b) should be much easier, compared to a). Note that I didn't say easy, just much easier compared to a).

nickysn

  • Newbie
  • Posts: 4

Amusingly, it appears that MetaWare took the track of using the "unsigned" data type to represent 16-bit unsigned integers in 16-bit compiler mode, and 32-bit unsigned integers in 32-bit compiler mode, using "cardinal" to be always 16-bits.  I see fpc has followed a different scheme, which may cause me some murmuring as I convert my existing code - but perhaps a "MetaWare" compiler flavor would be useful, anyway.  Then I can call an fpc "unit" a MetaWare "package", and work out how to handle other "extensions" we make use of.


In FPC you can use SizeUInt for an unsigned type that's 16-bit on 16-bit platforms and 32-bit on 32-bit platforms. There's also PtrUInt, which is slightly different, because it follows the default pointer size (so on i8086, it changes with the memory model - it stays 16-bit in the near data memory models and it becomes 32-bit in the far data memory models). For an always 16-bit unsigned type you should use "word". Note that you can redefine the existing types. For example you can make an unit called "metaware", that does this:

unit metaware;

interface

type
  unsigned = SizeUInt;
  cardinal = word;

implementation
end.

And then add "metaware" this unit to the uses clause of all your programs and units.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7592
(and with a  -Fa on the cmdline, you can let the system load that unit before anything else but system:

  -Fa<x>[,y] (for a program) load units <x> and [y] before uses is parsed

)

eclipse

  • New member
  • *
  • Posts: 5
Thank you both for those suggestions - they're the sort I'll certainly need as I get under way.

I'll begin looking at the x386 compiler, next, per your suggestion.

Is there a particular developer forum or mailing list you think I should move my questions to, or is this as appropriate as any?

nickysn

  • Newbie
  • Posts: 4
Is there a particular developer forum or mailing list you think I should move my questions to, or is this as appropriate as any?

There's the fpc-devel mailing list:

http://www.freepascal.org/maillist.var

There's also the #fpc-dev irc channel on irc.freenode.net