Recent

Author Topic: The Adventure of Porting PeaZip to Linux ARM  (Read 20508 times)

Arucard1983

  • New Member
  • *
  • Posts: 11
The Adventure of Porting PeaZip to Linux ARM
« on: December 14, 2013, 02:19:38 am »
I had working a few days ago to port the PeaZip software (written in Objective Pascal) to ARM architecture, beginning to a standard Linux ARM distro, and may be later to another platforms.
 (The recent Pelya's works to create a workable fake chrooted environment on Android that can be used to run standard Linux Applications without rooting the device, has give me another try to port the PeaZip).

After somes miss and trial issues, I could compile both the x86 and ARM versions of PeaZip.
However, the porting to ARM arch was not a trouble free procedure, since I had several issues to overcome.

The first major one, that I solved myself, was to rid the x86 assembly code in original source code. Luckly, all assembly code in units had an alternative written in Pascal, so I just need to change the source-code.
 The major hack I need to do was to change:
 {$ifdef PurePascal}
to
 {$ifndef PurePascal}

But an unit (TSC) was needed a real rewritten, since the PeaZip's author loves assembly, even to make time calculations. To avoid futher complications, I just assign a defaut value which compiles fine.

 After some struggle, I could build both the original x86 version of Lazarus, and the cross-compile ARM of same. Both the original and patched versions of PeaZip compiles fine in x86, but...  >:(
 
 In ARM mode, the linking fails due to a bug in GNU ld, spaming dozens of lines with the same error, nomatter the versions of binutils (2.23 or 2.24) or even compiled (2.24), it will give the same error:

/usr/local/bin/ld.bfd: BFD (GNU Binutils for Debian) 2.24 assertion fail ../../bfd/elf32-arm.c:12387

The most odd thing is the same error that was noticed in:
http://lists.freepascal.org/lists/fpc-devel/2013-May/032179.html

And despite the difference in number line, is the same issue on BFD_ASSERT that triggers the fault.

And that's I like to share, if someone had the same error, or how can fix this problem!
Since it's a linker problem, not Lazarus fault. I it's sad for my efforts to be in vain just by an error in the last step to building the application...  :(

Addendum:
If you want to build PeaZip by yourself, you need to edit several source files, and change the {$ifdef PurePascal} to {$ifndef PurePascal} to swap the x86 assembly version to Pascal one.

 But the TSC unit requires a real code change, so for now you could change the following functions in:

 {$ifdef FPC}

 function  _CheckCPUID: boolean;
 begin
  _CheckCPUID := false;
 end;

function  _CheckRDTSC: boolean;
 begin
  _CheckRDTSC := false;
 end;

procedure _RDTSC (var Ctr: TCtrRec)
begin
 {just x86 assembly code that can be removed without problem}
end;

{$endif}

I hope I could fix the ld problem...  :)

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #1 on: December 14, 2013, 03:32:21 am »
Hi Arucard1983 and welcome to Lazarus forum.  :)

Thank you for sharing your experience with us. Hopefully someone will help you fix your problem. Meanwhile:

Quote

The major hack I need to do was to change:
 {$ifdef PurePascal}
to
 {$ifndef PurePascal}

I'm afraid that PurePascal was *not* defined somewhere in the source code or project options. You didn't need to change $ifdef to $ifndef you needed to find where it was defined and remove it. define it, simply add -dPurePascal to Custom Options

Project Options (Ctrl+Shift+F11) -->
  Compiler Options -->
    Other -->
      Custom Options

« Last Edit: December 14, 2013, 04:38:44 am by engkin »

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #2 on: December 14, 2013, 07:38:03 am »
Quote
/usr/local/bin/ld.bfd: BFD (GNU Binutils for Debian) 2.24 assertion fail ../../bfd/elf32-arm.c:12387
This is an error in the linker itself, I think you could post a bugreport to GCC team. Despite there's also a chance the error happens in our side, assertion fail should NEVER happen.

Giorgio Tani

  • New Member
  • *
  • Posts: 28
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #3 on: December 14, 2013, 09:13:04 am »
Hi, thank you very much taking time to port PeaZip to ARM Linux.
ASM parts are from crypto library used in PeaZip, which is developed and maintained by Wolfgang Ehrhardt http://www.wolfgang-ehrhardt.de/ which may be more useful than me in supporting you with issues between Lazarus and internals of his library.
His library is a wide focused project meant to bring efficient math units, including cryptography, hashing, and random number generation, to Delphi/ObjectPascal, targeting the largest number possible of platforms (that's why pure Pascal replacements were introduced by the author of the library) and Pascal dialects.
Anyway, please don't hesitate to contact me on the application's support mail for any doubt or question.

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #4 on: December 15, 2013, 01:17:34 am »
 ;) Finally!!!!  :D

After some experiences, I just discover the issue with the GNU linker!
IN ARM machines, the ld linker cannot mix static and dynamic libraries without the -g option on gcc and fpc compilers!  ;D
Basically the BFD assert failure was a illegal try to link dynamic libraries (X11, GTK, ...) without debug symbols.
Using Project Options > Linking > Generate  debugging info to GDB...
and select automatic (-g), the linking process now works!!!  8)

 I will make some tests on my chrooted ARM machine that I build on my Linux computer (even a copy of ARM version of Lazarus was running... a bit slow... on this environment that I build for devkit proposes), before posting the final results.

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #5 on: December 15, 2013, 02:25:46 am »
The first tests shows some errors during execution:
The pea and pealauncher modules, somehow fail to detect the file or show segment fault.
The peazip file has somehow broken, but I expected due to some faulty port libraries.
Since I solved the linker issue, I will rebuild again from stratch the devkit environment to avoid the mistakes that I could make.  :D

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #6 on: December 16, 2013, 12:56:15 am »
PeaZip runs (almost) nicelly on Linux ARM...  :D
After somes adjustments on source code, the program now runs, extracts, compress and views PEA files. Also I could build the ARM version of some plug-ins (to handle RAR, TAR,... files) to test, and run fine so far.

A little screen-shot shows PeaZip running on my devkit computer (an chrooted virtual ARM machine on my Intel x86 PC), showing that this software can be built and run nativelly on ARM machines.
 Any Linux ARM PC with standard GTK2 libraries should run this program.
http://s27.postimg.org/t8guxyr03/Captura_de_ecra_de_2013_12_15_23_33_37.png

So far, this build was intended to ARMv7l machines (Raspberry Pi, for example, requires re-compiling to ARMv6 (armel)), like the Tegra CPU.

I also tested a little hack and run to see if the Pelya XSDL (An Debian emulator for non-rooted Android devices) can run directly the ARM port of PeaZip, but so far the main executable fails to run, because lacks some GTK libraries  :(
 (The PeaZip main window pop up, but crashes and closed before any rendering of elements). The pea and pealauncher somehow runs, but are useless without the main peazip executable.
But since the Pelya creates an image builder, probably I will try to check if with other custom build, PeaZip can run on XSDL emulator.

 I don't think that PeaZip source code can be easilly ported to a Standard Android application, even the Lazarus LCL was been ported to Android.

 

Giorgio Tani

  • New Member
  • *
  • Posts: 28
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #7 on: December 16, 2013, 10:07:02 am »
I'm glad hearing that!
Of course, while main executables alone can be useful for file management features and for .pea format, it needs ARM-p7zip to be available in res/7z path to handle mainstream archive formats.
I've seen various threads about that on p7zip forum on SourceForge, but I've not seen an ARM port in packages in download page, so I'm glad hearing it can be successfully build and tested.

I wonder what the missing GTK libs are on out of the box Android, so I could see if it is feasible to change classes of objects used in PeaZip GUI to make it run on those machines, or if Lazarus can bundle those missing libraries when those classes are used on a target which does not have the needed libraries out of the box.

I would like to know modifications needed to successfully target ARM architecture, so I can see if they can be implemented in the source code in order to make those attempt easier.

By the way, if you like, I'll be glad to link (or host, if you prefer) a package with the executables compiled ARM architecture.
« Last Edit: December 16, 2013, 10:10:22 am by Giorgio Tani »

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #8 on: December 16, 2013, 03:19:07 pm »
The XSDL issue was just a limitation of emulator itself: permission problems.
The Pelya's project is a work in progress, and without an workable Terminal in XFCE environment the permission settings cannot be adjusted, and any program that needs to write on configurations files will fail...
Using a testable chrooted environment I could debug the problem: if the XSDL was created a folder writable to any program, the PeaZip would run nicelly; since that's not the case, the program will fail. (pea and pealauncher since don't need a config file, will run).

The real solution is when Pelya add an workable deb package installer to fix any permission issues, without them, PeaZip will not run...

But with a Rapsberry Pi, just install Lazarus, compile and run without problems.

The source code changes are basically:
rmd160.pas, line 132:
....
implementation
{$ifndef PurePascal}
{$ifdef FPC}
  {$asmmode intel}
{$endif}
{$endif}
....

tsc.pas, line 162:
....
{$ifdef BIT64}
....
{$else}
{$ifdef PurePascal}

{---------------------------------------------------------------------------}
function _CheckCPUID: boolean;
  {-CPUID assumed to be supported for BIT64}
begin
  _CheckCPUID := false;
end;

{---------------------------------------------------------------------------}
function _CheckRDTSC: boolean;
  {-RDTSC assumed to be supported for BIT64}
begin
  _CheckRDTSC := false;
end;

{---------------------------------------------------------------------------}

procedure _RDTSC(var Ctr: TCtrRec);
  {-Read Time Stamp Counter}
begin
  int64(Ctr) := 0;
end;

{$else}
{---------------------------------------------------------------------------}
function _CheckCPUID: boolean; assembler;
  {-check if CPUID supported}
asm
  pushfd
  pushfd
  pop      eax
  mov      ecx,eax
  xor      eax,$200000
  push     eax
  popfd
  pushfd
  pop      eax
  popfd
  xor      eax,ecx
  setnz    al
end;


{---------------------------------------------------------------------------}
function _CheckRDTSC: boolean; assembler;
  {-check if RDTSC supported, check CPUID first!!}
asm
  push   ebx
  mov    eax,1
  db     $0f,$a2          {cpuid}
  test   dx,$10           {test RDTSC flag in Features}
  setnz  al
  pop    ebx
end;


{---------------------------------------------------------------------------}
procedure _RDTSC(var Ctr: TCtrRec);
  {-Read Time Stamp Counter}
begin
  asm
    push  ebx
    xor   eax, eax
    xor   ebx, ebx
    xor   ecx, ecx
    xor   edx, edx
    db    $0f,$a2    {cpuid}
    db    $0f,$31    {rdtsc}
    mov   ecx,[Ctr]
    mov   [ecx],eax
    mov   [ecx+4],edx
    xor   eax, eax
    db    $0f,$a2    {cpuid}
    pop   ebx
  end;
end;


{$endif}

{$endif}

Then define -dPurePascal in Options, and -g mode on linker, and PeaZip should now build and run on any Linux ARM machines (almost, since XSDL emulator for Android is not 100% compatible  ::))


For p7zip in ARM, there's already deb packages avaliable in Debian repository:
http://packages.debian.org/sid/p7zip

But for FreeArc I don't find any packaged.
Others formats need to be compiled from source code.

If you want a deb or tar package, it will take a bit. (I want to compile the FreeArc and UPX before try to build a package)

Giorgio Tani

  • New Member
  • *
  • Posts: 28
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #9 on: December 16, 2013, 03:55:19 pm »
In theory this problem could be mitigated, as I could make PeaZip not saving any configuration file - a sort of "private browsing mode", even if it would be not suitable for generic usage (not saving window position, sorting, custom options etc) but anyway interesting as alternative mode of use.
This could i.e. easily be implemented in next release both as a runtime option and as a switch to be set in res/altconf.txt to fix this behavior if needed on the target platform.

Please feel totally free about contributing the kind of package(s) or binaries you prefer and that is more compatible with your time availability, I'll be glad to add a PeaZip for Linux on ARM download page.
« Last Edit: December 16, 2013, 03:57:07 pm by Giorgio Tani »

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #10 on: December 16, 2013, 07:10:53 pm »
There's the tar.gz packages for the armv7l binary version of PeaZip and the patched source code. If anything fails, please reply to fix it!  :

Patched PeaZip 5.2.0 able to cross-compile in ARM:
http://www.datafilehost.com/d/e207aa40

Portable Version of PeaZip 5.2.0 with some third party plug-ins (FreeARC and UPX don't compile correctly in ARM  :(... but 7z is okay!  :D).
Requires a ARM computer with armv6l instruction set (armhf), other ARM systems should compile from sources:
http://www.datafilehost.com/d/82847525
GTK2 runtime is needed (if you install Gimp, then is fine).

Instalation Instructions:
Gunzip and Untar files to any unprotected folder in you ARM device (Raspberry Pi should work, since it suports the armhf debian packages), and then click peazip.

Potential Issues:
Peazip appears to freeze if the starting folder uses non-ASCII characters, if it happens, please check the Locales configuration to avoid this problem, or just kill PeaZip, edit the configuration file to use a non-problematic folder.
 After this fix, PeaZip should works nicelly.

Running on x86 machine:
Although there is a native x86 version, you can test the ARM version to check and debug potential problems.
 You can create a chrooted Debian or Ubuntu ARM system in less half hour, just follow this simple instructions
http://krosswindz.blogspot.pt/2012/02/setting-up-arm-chroot-on-ubuntu-oneiric.html

Afterall, I use a chrooted ARM system to test the ARM build of peaZip to diagnose issues before using a real ARM machine.  :)

Giorgio Tani

  • New Member
  • *
  • Posts: 28
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #11 on: December 16, 2013, 09:10:35 pm »
Nice, I can publish a page for Linux on ARM target with your notes, so more people/machines can test it; I can link your uploaded packages or host it if bandwidth is a problem.

About UPX, I found this binary package http://upx.sourceforge.net/download/upx-3.91-armeb_linux.tar.bz2 but I don't know exactly what instruction sets it does support.

For future build I may also suggest you to reduce the size of the executable files turning off debugging information in project options > linking, but in this test phase it may be better to have debugging information on.

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #12 on: December 17, 2013, 09:31:02 am »
@Arucard1983: awesome posts, thanks a lot for the effort and the clear descriptions!
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

Arucard1983

  • New Member
  • *
  • Posts: 11
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #13 on: December 17, 2013, 06:57:43 pm »
I upload the patched source code to my github repository:

https://github.com/Arucard1983/PeaZip-for-ARM

Giorgio Tani

  • New Member
  • *
  • Posts: 28
Re: The Adventure of Porting PeaZip to Linux ARM
« Reply #14 on: December 17, 2013, 09:57:16 pm »
Hi, I've integrated a page in PeaZip for Linux/BSD section,
http://peazip.sourceforge.net/peazip-linux-arm.html
pointing to the source repository on GitHub, the Portable package, and to this thread.
Please let me know if this is OK for you.

 

TinyPortal © 2005-2018