Recent

Author Topic: gpStructureStorage Port  (Read 21413 times)

Graeme

  • Hero Member
  • *****
  • Posts: 1431
    • Graeme on the web
Re: gpStructureStorage Port
« Reply #15 on: March 11, 2013, 01:32:48 pm »
I tried under FreeBSD 9.1 using FPC 2.7.1 - I must use 2.7.1 for FreeBSD, because there are lots of Threading and memory corruption issues in any released FPC version for FreeBSD.

Anyway, got your latest code and tried to compile the Tester project and got these errors.

/data/devel/tests/codelibrarian-code/Components/GpLists.pas(6423,22) Error: Incompatible types: got "TThreadID" expected "LongWord"
/data/devel/tests/codelibrarian-code/Components/GpLists.pas(6424,25) Error: Operator is not overloaded: "LongWord" = "TThreadID"


FLastThreadId is defined as a Cardinal, but GetCurrentThreadID under FreeBSD returns a TThreadID type which is a pointer to record structure. Slightly different to Linux or Windows.

Attached is a patch which will fix the compilation problem.

After this the Tester project compiled. When I ran "all tests", my system sat at 100% CPU load for a couple of minutes, then the application crashed out to the command prompt.

ps:
I would recommend you convert the Tester application to FPTest (Free Pascal Testing Framework). FPTest is a unit testing framework for exactly such needs. The Text Test Runner will give you progress output by default, and better results as to what worked, what doesn't and where it crashed. Test results can also be shown as XML or HTML output - see link below. FPTest also has a GUI Test Runner, but currently only for fpGUI. A LCL version is planned to.

FPTest code repository:
  https://github.com/graemeg/fptest

Some FPTest information and usage examples:
  http://wiki.freepascal.org/FPTest

Sample HTML test result output (actual test output for tiOPF project):
  http://opensoft.homeip.net/tiopf/unittests/freebsd64.html

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #16 on: March 11, 2013, 03:54:57 pm »
Thank you graeme for the patch.

I'll download 2.7.1 and run some tests and see what I'll come up with.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #17 on: March 11, 2013, 10:09:20 pm »
Some more info, I have installed fpc 2.7.1 on windows32 and compiled the tester application, Storage Truncation test fails with 2.7.1 with a message of access denied I'm guessing it has something to do with the way fpc frees non referenced interfaced objects. I do not if it is possible to work around that problem probably a local procedure might solve it. The problem seems Freebsd specific I'm searching for vm with freebsd preinstalled to see if I can reproduce it. I'll keep you posted.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #18 on: March 14, 2013, 09:28:36 pm »
I have run more test on linux and created two executables for i386 one with gtk and one with QT as far as I have tested it it seems to work as expected. I haven't been ample to install freebsd and fpc 2.7.1 to test it on that platform as well so for now the only official supported fpc version is 2.6.2. Next step is a freeBSD VM and a 2.7.1 fpc installation any hints for a quick installation? Bigchimp FPCUP works with freebsd?
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: gpStructureStorage Port
« Reply #19 on: March 15, 2013, 07:30:53 am »
Bigchimp FPCUP works with freebsd?
Haven't tried it yet, don't think so. Shouldn't be too hard to fix it though.
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: gpStructureStorage Port
« Reply #20 on: March 15, 2013, 10:14:21 am »
Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)

If you want to compile, run & report problems, I'd be very grateful. If you don't - no problem either ;) I'll get round to setting up a FreeBSD VM and doing it myself.

Things that may be wrong/tricky:
gnutar may not be available by default; the bunzip2 executable may have a different name and unzip may not be available by default either (so those should be installed in advance)
Quote
installercore.pas
@@ -180,6 +180,12 @@
+    {$IFDEF BSD} //Includes FreeBSD, NetBSD, OpenBSD
+    //todo: freebsd: check
+    FBunzip2 := 'bunzip2';
+    FTar := 'gnutar'; //check if available
+    FUnzip := 'unzip'; //unzip needed at least for FPC chm help
+    {$ENDIF BSD}

Check for valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD
Quote
@@ -296,6 +302,7 @@
OperationSucceeded := CheckExecutable(FBunzip2, '--help', '');

installerfpc.pas:
Extraction of a bootstrap compiler section. Basically copied from Linux section.
There's a chance the parameters for the exes are different in which case the code needs to be fixed.
Quote
@@ -478,6 +478,32 @@
+  {$IFDEF BSD} //Includes FreeBSD, OpenBSD, NetBSD
+  //todo: test parameters
+  //Extract bz2, overwriting without prompting
+  if ExecuteCommand(FBunzip2+' -d -f -q '+BootstrapArchive,FVerbose) <> 0 then
+    begin
+      infoln('Received non-zero exit code extracting bootstrap compiler. This will abort further processing.',eterror);
+      OperationSucceeded := False;
+    end
+    else
+    begin
+      ExtractedCompiler:=BootstrapArchive+'.out'; //default bzip2 output filename
+      OperationSucceeded := True; // Spelling it out can't hurt sometimes
+    end;
+  // Move compiler to proper directory; note bzip2 will append .out to file
+  if OperationSucceeded = True then
+  begin
+    infoln('Going to move ' + ExtractedCompiler + ' to ' + FBootstrapCompiler,etinfo);
+    OperationSucceeded:=MoveFile(ExtractedCompiler,FBootstrapCompiler);
+  end;
+  if OperationSucceeded then
+  begin
+    //Make executable
+    OperationSucceeded:=(fpChmod(FBootStrapCompiler, &700)=0); //rwx------
+    if OperationSucceeded=false then infoln('Bootstrap compiler: chmod failed for '+FBootstrapCompiler,etwarning);
+  end;
+  {$ENDIF BSD}

Thanks,
BigChimp
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9090
  • FPC developer.
Re: gpStructureStorage Port
« Reply #21 on: March 15, 2013, 12:19:37 pm »
Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)
Quote
+    FTar := 'gnutar'; //check if available

The default tar is "tar". Since FreeBSD 8, it also supports some newer gnu commands like --strip   FreeBSD7 is EOL since 2012-02-28 (but there is a 2.6.2 for it)

Try to use it as much as possible. GNU tar, if available is called gtar, but I don't use it so I don't know exact versioning.

Quote
+    FUnzip := 'unzip'; //unzip needed at least for FPC chm help

I still plan to continue creating CHMs for 2.6.2 next weekend (I can generate them again, but the quality sucks), and I can also generate .tar.gz from now on (or .bz2)

Quote
valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD

bzip2 and bunzip2 are in base, so even minimal systems installed have it.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9090
  • FPC developer.
Re: gpStructureStorage Port
« Reply #22 on: March 15, 2013, 12:27:17 pm »

FLastThreadId is defined as a Cardinal, but GetCurrentThreadID under FreeBSD returns a TThreadID type which is a pointer to record structure. Slightly different to Linux or Windows.

Attached is a patch which will fix the compilation problem.

In general in such case it is best to change it by simply using tthreadid everywhere for threadids, and define dummy types for Delphi if needed, and not to start random typecasting.
 
E.g. in this case on 64-bit systems, cardinal is still 32-bit, and the upper 32-bit of FreebSD threadids will be truncated.

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3977
  • I like bugs.
Re: gpStructureStorage Port
« Reply #23 on: March 15, 2013, 01:15:02 pm »
Sorry for the delay.

I finally updated my compiler to FPC 2.6.2. I run the test app Tester/testGpStructuredStorage.lpi.
In the very first test, Storage Creation, it throws a SIGSEGV. While debugging I don't see any reason for it. I think it is related to Interface but you wrote it works for you with FPC 2.6.2.
I have a 64-bit Linux system. Could there be a bug in FPC related to Interfaces on 64-bit systems? Sounds weird.

Code: [Select]
function TForm1.TestGSSCreation: boolean;
var
  storage: IGpStructuredStorage;
begin
  Result := false;
  Log('Testing storage creation');
  try
    DeleteFileUTF8(CStorageFile); { *Converted from DeleteFile*  }
    storage := CreateStorage;
    storage.Initialize(CStorageFile, fmCreate); <-- it crashes here. I cannot enter Initialize method in debugger.
...

Juha
Mostly Lazarus trunk and FPC 3.2 on Manjaro Linux 64-bit.

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: gpStructureStorage Port
« Reply #24 on: March 15, 2013, 01:23:26 pm »
Modified fpcup for FreeBSD, NetBSD - no testing though. (posted to fpcup thread as well)
Quote
+    FTar := 'gnutar'; //check if available

The default tar is "tar". Since FreeBSD 8, it also supports some newer gnu commands like --strip   FreeBSD7 is EOL since 2012-02-28 (but there is a 2.6.2 for it)

Try to use it as much as possible. GNU tar, if available is called gtar, but I don't use it so I don't know exact versioning.
Thanks, that's nice, will revert those changes again.

I still plan to continue creating CHMs for 2.6.2 next weekend (I can generate them again, but the quality sucks), and I can also generate .tar.gz from now on (or .bz2)
Thanks. Doesn't really affect fpcup... except perhaps the zip part...
Currently, fpcup downloads the chms from the Lazarus Sourceforge site:
http://sourceforge.net/projects/lazarus/files/Lazarus%20Documentation/Lazarus%201.0/fpc-lazarus-doc-chm-1.0.zip/download
Having .tar.gz or .tar.bz2 would be nice if zip isn't in base FreeBSD (I suspect it isn't). However, I check for zip being present and can kick out a warning to install a port/package.

That said ... having the 2.6.2 chms ready when Laz gets updated to use 2.6.2 would be good enough for me

Quote
valid bunzip2: hopefully bunzip2 --help works on FreeBSD and returns status code 0. If not, the code will need to be changed with an {$IFDEF BSD} or FREEBSD

bzip2 and bunzip2 are in base, so even minimal systems installed have it.
Great.

Looks like I'll have to download FreeBSD 9.1 soon and have a go...
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #25 on: March 15, 2013, 05:49:16 pm »
In general in such case it is best to change it by simply using tthreadid everywhere for threadids, and define dummy types for Delphi if needed, and not to start random typecasting.
 
E.g. in this case on 64-bit systems, cardinal is still 32-bit, and the upper 32-bit of FreebSD threadids will be truncated.

That is why in my commits I changed this to ptruint instead of cardinal. As far as I understand this should be enough.

I'm going to start an freebsd vm tonight so more news tomorrow I guess.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9090
  • FPC developer.
Re: gpStructureStorage Port
« Reply #26 on: March 15, 2013, 11:20:31 pm »

That is why in my commits I changed this to ptruint instead of cardinal.

As far as I understand this should be enough.

I don't see a real problem, except that it is confusing, and  is not optimal. (64-bits systems with 32-bit threadid lug the extra 4 bytes around,
debuggers show 64-bit numbers with a lot of zeroes etc)

I really don't see the problem of changing it to tthreadid

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #27 on: March 15, 2013, 11:45:24 pm »
I really don't see the problem of changing it to tthreadid

I don't either and I probably will change it at the next commit if the component in question survives the conversion which is unlikely since it is not used from gpstructuredstorage component and I do not want to carry with me dead code.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

taazz

  • Hero Member
  • *****
  • Posts: 5365
Re: gpStructureStorage Port
« Reply #28 on: March 16, 2013, 03:12:59 am »
freebsd is a pain to in my behind. pkg_add does not work, it cant find anything online and I do not know how to add more repositories to it even if I did I don't any repositories to add :'(.

Shitting here watching 3 consoles compiling my packages from source (ports works at least) is ahem narcotic. Narcolepsy is coming and I do not think I am ready for it yet.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: gpStructureStorage Port
« Reply #29 on: March 16, 2013, 07:37:28 am »
I feel for you, taazz, looking at an x11 build screen myself ;)

(Oh and assuming you meant "sitting here" unless you have very strange hobbies :) )
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

 

TinyPortal © 2005-2018