Recent

Author Topic: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)  (Read 12178 times)

evoshroom

  • Full Member
  • ***
  • Posts: 157
Here we go again.  This all started with me trying to find a replacement for TRichMemo that would be able to use cross-platform consistent code, complex text colorations, links and images.  Someone suggested THTMLView, which I tried, but it was unable to append text without fully re-parsing the whole document, leading to slowdown.  When I posted about this someone suggested to use TIpHtmlPanel instead.

TIpHtmlPanel worked very well for the task, with one exception.  It seems to use an unreasonable amount of memory for the nodes it uses for HTML, roughly 120k of ram per one line of text added in testing, which quickly adds up.  Additionally, if TIpHtmlPanel.SetHtmlFromStr() is used with a longer and longer string, this does not occur, which hints the nodes are at issue.  Finally, if TIpHtmlPanel.SetHtmlFromStr() is called after many nodes are in place, memory usage can dramatically decrease, which I assume is from clearing the nodes.  However, the nodes in some form are required to be used so the document does not have to be re-parsed for each line added, leading to slowdowns.

So, I have created a demo of the RichChat using TIpHtmlPanel in the hope someone can be of some help with this issue.  You can download it at:

http://www.mediamarksman.com/IpHTMLRichChatTest.zip

The demo consists of a functioning RichChat-like interface and buttons that let you add colored lines and lines with links.  Monitoring memory usage with Task Manager or Activity Monitor should show you the memory issue.

I'd be very grateful for any help anyone can offer.
« Last Edit: August 30, 2012, 12:25:27 am by evoshroom »

Phil

  • Hero Member
  • *****
  • Posts: 2737
Re: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)
« Reply #1 on: August 31, 2012, 05:10:12 am »
I'd be very grateful for any help anyone can offer.

I tested your app and confirm that the TIpHtmlPanel is not only extremely wasteful of memory, but also extremely slow. Click the Add 100 Lines button took about 10 seconds and bumped Real Memory use by about 140K / line added. Also, during this wait, the rotating ball came on and the CPU % pegged 100% in Activity Monitor.

As a test, I coded your example with FPC in Xcode using Cocoa's WebView. Here's what I got: Clicking the Add 100 Lines took about 1/2 sec and only bumped Real Memory by about 3K / line added. CPU % never exceeded 12% during the adds.

Other stats: Your Laz example's executable, stripped: Approx. 4MB. The Cocoa example's executable, stripped: < 200K.

I think you're just using the wrong tools. Maybe forget the cross-platform UI and focus on getting what you really want.

My example code attached.

Thanks.

-Phil

evoshroom

  • Full Member
  • ***
  • Posts: 157
Re: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)
« Reply #2 on: August 31, 2012, 08:23:32 am »

I tested your app and confirm that the TIpHtmlPanel is not only extremely wasteful of memory, but also extremely slow. Click the Add 100 Lines button took about 10 seconds and bumped Real Memory use by about 140K / line added. Also, during this wait, the rotating ball came on and the CPU % pegged 100% in Activity Monitor.

I think you're just using the wrong tools. Maybe forget the cross-platform UI and focus on getting what you really want.

Well, I suspect part of the reason that CPU was so high was the call to htmlChat.Update and the scrollbar moving command was in the loop.  It isn't really necessary to call these every loop.  I only provided a 100 line adder so the memory usage could be easily seen.  I don't ever use this functionality.  If you remove those and only call them once at the end, it should help a little.  After I did that it was still at 61% CPU usage and one second or so delay on my machine.  Additionally, you could throw a Application.ProcessMessages in there in the loop and your computer wouldn't beach-ball.

I'm glad you saw what I was seeing.  It's too bad you don't know what is causing the crazy memory usage either.

I was looking for something cross-platform because I didn't want to have to code something twice or three times any time I make some changes.  When I used to use TRichMemo I kept having to do this, as the control while working on all three major platforms, did not work consistently on them and individual code had to be created for each.  I recall it being annoying.  I also recall it being problematic because I tend to catch bugs best when testing my program on my platform.  With TRichMemo sometimes I wouldn't catch bugs as quickly as normal because Windows-only bugs would crop up because the code wasn't consistent across all platforms.

That said, I'm also saddened by the weird memory usage issues of TIpHtmlPanel, because other than that it pretty much is perfect for me.  Fonts using it even look nice on Windows, which has horrible font rendering most of the time in many default controls.  Also, executable size isn't all that relevant to anything and while it's great that WebView can add lines fast, my program will almost exclusively add one at a time. 

That said, I would consider using WebView if it can be used in Lazarus, but that presents its own issues.  First, I have no real idea how native Mac widgets which are part of Mac API's integrate with Lazarus.  If you were suggesting I redo what I'm doing in Xcode instead of Lazarus, that is not possible.  This chat component is part of a much larger cross-platform program written in Lazarus.  The chat of it is a lot like the demo, but the chat code is a page and the program is a novel.  If you know how to use WebView in Lazarus, let me know and I'll at least take a look.

However, even if I did so that only solves about 10% of my issue, because while I am on a Mac, 90% of my users are on Windows.  So, again, I would need another component for Windows that does the exact same thing including links, colored text, and maybe images.  Then, when I add some more advanced features to the chat later (hovering over certain key words that have programatic effects), I'd have to code it twice, or three times if I have Linux working by then.

So it's more work, but I'm open to it if it could work better.  The big two problems for me with going that route are:

1. How do I use WebView in Lazarus?
2. What do I use to do HTML RichChat on Windows then?
« Last Edit: August 31, 2012, 08:35:02 am by evoshroom »

Dibo

  • Hero Member
  • *****
  • Posts: 1048
Re: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)
« Reply #3 on: August 31, 2012, 11:48:10 am »
You can also use GeckoPort (firefox engine): http://wiki.freepascal.org/GeckoPort_version2
I had some problems with webview (need QT, libQT4Pas, webview etc), but GeckoPort need only xulrunner in the same directory where is your application

evoshroom

  • Full Member
  • ***
  • Posts: 157
Re: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)
« Reply #4 on: August 31, 2012, 01:39:11 pm »
You can also use GeckoPort (firefox engine): http://wiki.freepascal.org/GeckoPort_version2
I had some problems with webview (need QT, libQT4Pas, webview etc), but GeckoPort need only xulrunner in the same directory where is your application

I tried.

So, here is what I did.  First, there are two versions of GeckoPort, version1 and version2.  I first tried to install version2, but the package would not install.  It failed with:

Code: [Select]
/Developer/lazarus/components/geckoport/Components/GeckoChromeWindow.pas(177,32) Error: Incompatible types: got "LongInt" expected "Pointer"

function TGeckoChromeForm.GetNativeWindow : nativeWindow;
...
 {$IFDEF LCLCarbon}Result := THANDLE(TCarbonWindow(Handle).Window);{$ENDIF}
 //Carbon doesn't work but leave in so package compiles in Carbon IDE.

So then I tried version1.  Version1 installed just fine.  Then I went to install XULRunner.  I first tried version 15.0 of XULRunner, which I installed by putting XUL.Framework in /Library/Frameworks.  After that didn't work I tried installing the XULRunner DMG for version 2.0, which I inspected and just did the same thing, moved the XUL.Framework for 2.0 to /Library/Frameworks.  After doing this, if the GBrowser sample app is loaded it generates an error on loading the lpi:

Quote
Stream=TForm1: Root=:TForm1
Component Class: TGeckoBrowser
Resource "ID_GECKO_LOGO" not found
Stream position: 646

If the ChromeWin sample app is loaded it loads but executing it gives the error:

Quote
Project ChromeWin.app raised exeception class 'Exception' with message: Missing Gecko runtime library: /Library/Framesworks/XUL.framework/Versions/Current/libplc4.dylib

In file '/Developer/lazarus/components/geckoport/nsInit.pas' at line 1313.

But, that file is definitely there.  I also tried putting the framework in the basedir, the application bundle, and in ~/Library/Frameworks, but that didn't change anything.
« Last Edit: August 31, 2012, 01:43:30 pm by evoshroom »

evoshroom

  • Full Member
  • ***
  • Posts: 157
Re: TurboPower iPro TIpHtmlPanel Memory Usage Issue (Code Included)
« Reply #5 on: September 12, 2012, 06:58:17 am »
Seeing as how TurboPower iPro was the closest I've come to a full solution, I've developed a number of workarounds.  They aren't amazingly elegant, but they do the job.  However, I've also hit upon some small, but annoying, new bugs I could use some help with.

First the Improvements:
Taking off from the idea that it is the nodes which are inefficient, but not the HTML when set from a string, I sought to create a sort of hybrid where the new HTML is added as nodes, but replaced with a string version every so often.  The idea was then a full-reload would only be needed every 100 lines or so.  Ram would balloon up, but it would be kept in check by reloading a string form.  This is what you'll find implemented in the code below.  Essentially, I'm double-adding and double-keeping track of all the data, once in raw-string-HTML form in a TStringList and once as nodes and then swapping from nodes to strings every so often.  It won't win any awards for elegance, but it works.

It seems much more efficient than before, if not perfectly so.  I can get to 7000 lines of text using 40 meg of memory when starting from 15 megs (on a Mac, likely less on Windows).  Additionally, you can add 100 lines of text "instantly" or at least much, much faster than previously.  On the downside there is some slight lag when you hit large amounts of text, but it's not completely horrible and likely not hourglass/beachball stuff like before.  Additionally, I discovered that the "setfromhtml" seems to work much faster than one would expect.  It may, indeed, be even faster and simpler to only use the string to html for everything, however, for some reason I couldn't get the scrolling to work perfectly when trying this method out, so I left it as-is with the hybrid method which uses nodes.

Then the Bug:
The problem I'm having is a bug with TIpHtmlPanel and word-wrap.  The TurboPower iPro TIpHtmlPanel is meant to wordwrap and most of the time it does a great job.  When you resize it with text in it all the text wraps perfectly when you resize the form, as is proper.  However, there is a bug with long words without spaces (which includes long links without spaces).  If you enter a word without spaces that goes off the screen to the right, instead of hyphenating, truncating, splitting or allowing the word to hang off the screen while continuing to wordwrap, what the TIpHtmlPanel does is change the document wrap area to match the long word, which is off the screen.  Essentially, if you type a long word in it it kills wordwrap.

I looked through the component code, but couldn't find the part which governs this behavior.  I've added a new button to add a large string without spaces in the RichChatTest project code so you can see the bug in action (the link buttons will also show this behavior). Anyone have any ideas where the bug is in the TIpHtmlPanel code?

Finally the Code:
Here is the code for the updated and improved RichChat:

http://www.mediamarksman.com/RichChatTest2.zip
« Last Edit: September 12, 2012, 07:06:29 am by evoshroom »

 

TinyPortal © 2005-2018