gpStructureStorage Port

[Graeme]

I tried under FreeBSD 9.1 using FPC 2.7.1 - I must use 2.7.1 for FreeBSD, because there are lots of Threading and memory corruption issues in any released FPC version for FreeBSD.

Anyway, got your latest code and tried to compile the Tester project and got these errors.

/data/devel/tests/codelibrarian-code/Components/GpLists.pas(6423,22) Error: Incompatible types: got "TThreadID" expected "LongWord"
/data/devel/tests/codelibrarian-code/Components/GpLists.pas(6424,25) Error: Operator is not overloaded: "LongWord" = "TThreadID"


FLastThreadId is defined as a Cardinal, but GetCurrentThreadID under FreeBSD returns a TThreadID type which is a pointer to record structure. Slightly different to Linux or Windows.

Attached is a patch which will fix the compilation problem.

After this the Tester project compiled. When I ran "all tests", my system sat at 100% CPU load for a couple of minutes, then the application crashed out to the command prompt.

ps:
I would recommend you convert the Tester application to FPTest (Free Pascal Testing Framework). FPTest is a unit testing framework for exactly such needs. The Text Test Runner will give you progress output by default, and better results as to what worked, what doesn't and where it crashed. Test results can also be shown as XML or HTML output - see link below. FPTest also has a GUI Test Runner, but currently only for fpGUI. A LCL version is planned to.

FPTest code repository:
  https://github.com/graemeg/fptest

Some FPTest information and usage examples:
  http://wiki.freepascal.org/FPTest

Sample HTML test result output (actual test output for tiOPF project):
  http://opensoft.homeip.net/tiopf/unittests/freebsd64.html

[taazz]

Thank you graeme for the patch.

I'll download 2.7.1 and run some tests and see what I'll come up with.

[taazz]

Some more info, I have installed fpc 2.7.1 on windows32 and compiled the tester application, Storage Truncation test fails with 2.7.1 with a message of access denied I'm guessing it has something to do with the way fpc frees non referenced interfaced objects. I do not if it is possible to work around that problem probably a local procedure might solve it. The problem seems Freebsd specific I'm searching for vm with freebsd preinstalled to see if I can reproduce it. I'll keep you posted.

[taazz]

I have run more test on linux and created two executables for i386 one with gtk and one with QT as far as I have tested it it seems to work as expected. I haven't been ample to install freebsd and fpc 2.7.1 to test it on that platform as well so for now the only official supported fpc version is 2.6.2. Next step is a freeBSD VM and a 2.7.1 fpc installation any hints for a quick installation? Bigchimp FPCUP works with freebsd?

[BigChimp]

Bigchimp FPCUP works with freebsd?

Haven't tried it yet, don't think so. Shouldn't be too hard to fix it though.

[BigChimp]

Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)

If you want to compile, run & report problems, I'd be very grateful. If you don't - no problem either I'll get round to setting up a FreeBSD VM and doing it myself.

Things that may be wrong/tricky:
gnutar may not be available by default; the bunzip2 executable may have a different name and unzip may not be available by default either (so those should be installed in advance)

installercore.pas
@@ -180,6 +180,12 @@
+    {$IFDEF BSD} //Includes FreeBSD, NetBSD, OpenBSD
+    //todo: freebsd: check
+    FBunzip2 := 'bunzip2';
+    FTar := 'gnutar'; //check if available
+    FUnzip := 'unzip'; //unzip needed at least for FPC chm help
+    {$ENDIF BSD}

Check for valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD

@@ -296,6 +302,7 @@
OperationSucceeded := CheckExecutable(FBunzip2, '--help', '');

installerfpc.pas:
Extraction of a bootstrap compiler section. Basically copied from Linux section.
There's a chance the parameters for the exes are different in which case the code needs to be fixed.

@@ -478,6 +478,32 @@
+  {$IFDEF BSD} //Includes FreeBSD, OpenBSD, NetBSD
+  //todo: test parameters
+  //Extract bz2, overwriting without prompting
+  if ExecuteCommand(FBunzip2+' -d -f -q '+BootstrapArchive,FVerbose) <> 0 then
+    begin
+      infoln('Received non-zero exit code extracting bootstrap compiler. This will abort further processing.',eterror);
+      OperationSucceeded := False;
+    end
+    else
+    begin
+      ExtractedCompiler:=BootstrapArchive+'.out'; //default bzip2 output filename
+      OperationSucceeded := True; // Spelling it out can't hurt sometimes
+    end;
+  // Move compiler to proper directory; note bzip2 will append .out to file
+  if OperationSucceeded = True then
+  begin
+    infoln('Going to move ' + ExtractedCompiler + ' to ' + FBootstrapCompiler,etinfo);
+    OperationSucceeded:=MoveFile(ExtractedCompiler,FBootstrapCompiler);
+  end;
+  if OperationSucceeded then
+  begin
+    //Make executable
+    OperationSucceeded:=(fpChmod(FBootStrapCompiler, &700)=0); //rwx------
+    if OperationSucceeded=false then infoln('Bootstrap compiler: chmod failed for '+FBootstrapCompiler,etwarning);
+  end;
+  {$ENDIF BSD}

Thanks,
BigChimp

[marcov]

Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)

+    FTar := 'gnutar'; //check if available

The default tar is "tar". Since FreeBSD 8, it also supports some newer gnu commands like --strip   FreeBSD7 is EOL since 2012-02-28 (but there is a 2.6.2 for it)

Try to use it as much as possible. GNU tar, if available is called gtar, but I don't use it so I don't know exact versioning.

+    FUnzip := 'unzip'; //unzip needed at least for FPC chm help

I still plan to continue creating CHMs for 2.6.2 next weekend (I can generate them again, but the quality sucks), and I can also generate .tar.gz from now on (or .bz2)

valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD

bzip2 and bunzip2 are in base, so even minimal systems installed have it.

[marcov]

FLastThreadId is defined as a Cardinal, but GetCurrentThreadID under FreeBSD returns a TThreadID type which is a pointer to record structure. Slightly different to Linux or Windows.

Attached is a patch which will fix the compilation problem.

In general in such case it is best to change it by simply using tthreadid everywhere for threadids, and define dummy types for Delphi if needed, and not to start random typecasting.
 
E.g. in this case on 64-bit systems, cardinal is still 32-bit, and the upper 32-bit of FreebSD threadids will be truncated.

[JuhaManninen]

Sorry for the delay.

I finally updated my compiler to FPC 2.6.2. I run the test app Tester/testGpStructuredStorage.lpi.
In the very first test, Storage Creation, it throws a SIGSEGV. While debugging I don't see any reason for it. I think it is related to Interface but you wrote it works for you with FPC 2.6.2.
I have a 64-bit Linux system. Could there be a bug in FPC related to Interfaces on 64-bit systems? Sounds weird.

Code: [Select]

function TForm1.TestGSSCreation: boolean;
var
  storage: IGpStructuredStorage;
begin
  Result := false;
  Log('Testing storage creation');
  try
    DeleteFileUTF8(CStorageFile); { *Converted from DeleteFile*  }
    storage := CreateStorage;
    storage.Initialize(CStorageFile, fmCreate); <-- it crashes here. I cannot enter Initialize method in debugger.
...

Juha

[BigChimp]

Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)

+    FTar := 'gnutar'; //check if available

The default tar is "tar". Since FreeBSD 8, it also supports some newer gnu commands like --strip   FreeBSD7 is EOL since 2012-02-28 (but there is a 2.6.2 for it)

Try to use it as much as possible. GNU tar, if available is called gtar, but I don't use it so I don't know exact versioning.

Thanks, that's nice, will revert those changes again.

I still plan to continue creating CHMs for 2.6.2 next weekend (I can generate them again, but the quality sucks), and I can also generate .tar.gz from now on (or .bz2)

Thanks. Doesn't really affect fpcup... except perhaps the zip part...
Currently, fpcup downloads the chms from the Lazarus Sourceforge site:
http://sourceforge.net/projects/lazarus/files/Lazarus%20Documentation/Lazarus%201.0/fpc-lazarus-doc-chm-1.0.zip/download
Having .tar.gz or .tar.bz2 would be nice if zip isn't in base FreeBSD (I suspect it isn't). However, I check for zip being present and can kick out a warning to install a port/package.

That said ... having the 2.6.2 chms ready when Laz gets updated to use 2.6.2 would be good enough for me

valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD

bzip2 and bunzip2 are in base, so even minimal systems installed have it.

Great.

Looks like I'll have to download FreeBSD 9.1 soon and have a go...

[taazz]

In general in such case it is best to change it by simply using tthreadid everywhere for threadids, and define dummy types for Delphi if needed, and not to start random typecasting.
 
E.g. in this case on 64-bit systems, cardinal is still 32-bit, and the upper 32-bit of FreebSD threadids will be truncated.

That is why in my commits I changed this to ptruint instead of cardinal. As far as I understand this should be enough.

I'm going to start an freebsd vm tonight so more news tomorrow I guess.

[marcov]

That is why in my commits I changed this to ptruint instead of cardinal.

As far as I understand this should be enough.

I don't see a real problem, except that it is confusing, and  is not optimal. (64-bits systems with 32-bit threadid lug the extra 4 bytes around,
debuggers show 64-bit numbers with a lot of zeroes etc)

I really don't see the problem of changing it to tthreadid

[taazz]

I really don't see the problem of changing it to tthreadid

I don't either and I probably will change it at the next commit if the component in question survives the conversion which is unlikely since it is not used from gpstructuredstorage component and I do not want to carry with me dead code.

[taazz]

freebsd is a pain to in my behind. pkg_add does not work, it cant find anything online and I do not know how to add more repositories to it even if I did I don't any repositories to add .

Shitting here watching 3 consoles compiling my packages from source (ports works at least) is ahem narcotic. Narcolepsy is coming and I do not think I am ready for it yet.

[BigChimp]

I feel for you, taazz, looking at an x11 build screen myself

(Oh and assuming you meant "sitting here" unless you have very strange hobbies )