Recent

Author Topic: [UPDATE] TOpenDialog - 100% CPU - Fan Revs Up  (Read 16970 times)

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #15 on: October 25, 2021, 10:34:32 pm »
Oh $deity... I'm reminded of some of the filesystem tuning hacks related to turning off atime updates etc.

There is of course an xkcd for this https://xkcd.com/2531/

I wonder: does this depend on the desktop environment? The CPU might not be going to the open dialog(ue) per se, but to desktop code which is monitoring file creation in the background (using fam) and is unhappy that something is trying to read the directory at the same time as it's being updated... does top show anything useful?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #16 on: October 26, 2021, 02:18:07 am »
Oh $deity... I'm reminded of some of the filesystem tuning hacks related to turning off atime updates etc.

There is of course an xkcd for this https://xkcd.com/2531/


Indeed turning off atime can speed up a file system that is handing a lot of files. In a previous life, it was a pretty standard thing to do where where each user had their own file system if they used lots of small files. Image processing, life sciences etc. And people almost never miss the information atime provided.

The degree of benefit did vary with the file system in use but as this was remote access, desktop never came into our equations so I cannot comment on that.

And, yes, there is a relevant XKCD, but there is a relevant XKCD for ANY situation.

Davo

Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

del

  • Sr. Member
  • ****
  • Posts: 258
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #17 on: October 27, 2021, 03:39:53 am »
The app is called "CustomOps". Right now I'm using ffmpeg to capture two webcams and save the streams into sequences of jpegs. If I navigate the open file dialog into either of the jpeg directories it will freeze. The first "top" is running a debug build from Lazarus. The second "top" is running the release build from the command line.

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #18 on: October 27, 2021, 04:09:22 am »
well, now that fiddling with FS has been mentioned, its quite safe and does not really take away anything you need, to remove atime.  There are other settings that may also help but I would be less enthusiastic if its not a specialist mount for just one purpose.  But, atime is easy and safe, Windows users for example don't have an atime available any way.

Edit, as root, /etc/fstab, your home dir will probably be like -
Code: Pascal  [Select][+][-]
  1. UUID=a748839c-84fb-40cc-9b5b-caa902ab00ad /home    ext4    defaults     0    2

Add 'noatime', ahead of 'defaults', no space between.  Remount, easiest way to do that is reboot if you can.
Code: Pascal  [Select][+][-]
  1. UUID=a748839c-84fb-40cc-9b5b-caa902ab00ad /home    ext4    noatime,defaults    0    2

It will make a difference, I just don't know how big a difference !

Davo
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

del

  • Sr. Member
  • ****
  • Posts: 258
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #19 on: October 27, 2021, 04:11:01 pm »
Thanks Davo. I did a "cat" of the file and got the results shown in the attached image. It looks like noatime is already set, but it's AFTER defaults. Do I need to do anything?

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #20 on: October 28, 2021, 01:01:53 am »
No, nothing more to do there I am afraid, that shot has already been fired.

There are some other FS fine tuning things but none of them are without cost nor really offer a lot of potential benefit.

Sorry !

Davo
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

del

  • Sr. Member
  • ****
  • Posts: 258
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #21 on: October 29, 2021, 02:01:35 pm »
No, nothing more to do there I am afraid, that shot has already been fired.

There are some other FS fine tuning things but none of them are without cost nor really offer a lot of potential benefit.

Sorry !

Davo
OK Davo - thanks for the suggestion. It's always great to learn new stuff anyway. The fact that the C++ / wxWidgets file dialog doesn't freeze means that the problem is solvable in software. It's a real corner case (large number of files with new files rapidly flowing in). I'll just accommodate the situation by avoiding the need to enter that directory during downloading.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #22 on: October 29, 2021, 02:15:51 pm »
I've just taken a quick look back through the thread: I don't think you've answered the question of what widget set the app is compiled against which implies that you've probably not looked at different ones.

If you're using GTK try Qt, and vice versa.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

del

  • Sr. Member
  • ****
  • Posts: 258
Re: TOpenDialog - 100% CPU - Fan Revs Up
« Reply #23 on: October 31, 2021, 04:54:46 am »
I've just taken a quick look back through the thread: I don't think you've answered the question of what widget set the app is compiled against which implies that you've probably not looked at different ones.

If you're using GTK try Qt, and vice versa.

MarkMLl
Going to QT fixed it. I built the libQt5Pas library (libQt5Pas.so -> libQt5Pas.so.1.2.6) in a non-root directory and put the path in the project options. After rebuilding the app and navigating to a directory with 46111 jpegs and two more being added each second the Open File Dialog did not get hung up. It wasn't snappy but that's because there were 46111 files in the directory. Thanks for the tip. Some of the visual stuff looks a little weird - (font sizes, etc) but I'll play around with that  :)

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #24 on: October 31, 2021, 07:58:09 am »
I am pretty sure you would have found libqt5pas1 and libqt5pas-dev in your distro's repository. But good practice ;-)

Davo
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

MarkMLl

  • Hero Member
  • *****
  • Posts: 6646
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #25 on: October 31, 2021, 09:18:00 am »
I am pretty sure you would have found libqt5pas1 and libqt5pas-dev in your distro's repository. But good practice ;-)

I agree, although I've seen Lazarus incautiously advance beyond what was supported by the version being shipped by Debian "Stable".

OP: I think you have enough detail there for a bug report. Even if it turns out to be a widget set rather than an LCL problem, it still needs to be got into the record.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

del

  • Sr. Member
  • ****
  • Posts: 258
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #26 on: October 31, 2021, 11:17:00 am »
I am pretty sure you would have found libqt5pas1 and libqt5pas-dev in your distro's repository. But good practice ;-)

Davo
Manjaro doesn't seem to have it. The only thing I could find that was close was in this directory:

/usr/lib/lazarus/lcl/interfaces/qt5/cbindings

the contents of which are:
qt5.pas  Qt5Pas.pro  README.TXT  src  TODO

So I just followed the instructions in README.TXT after I copied everything into a non-root directory. I wanted to avoid using "sudo" as much as possible. Also like you said it was probably better to build it on my machine just for "deeper compatibility" for the lack of a better term.


dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #27 on: October 31, 2021, 11:37:23 am »
Manjaro doesn't seem to have it. The only thing I could find that was close was in this directory:
What you have done is fine but if you want to let someone else use your software, you would need to tell them that it depends on libqt5pas, sadly that seems to have a number of different names in different repositories, libqt5pas, libqt5pas1 and surprisingly, at least one repo capitalises the 'P' (from memory).  On Manjaro, I think it uses pacman ? You might need to do some sort of wildcard search....

> Lazarus incautiously advance beyond what was supported by the version being shipped by Debian "Stable".

Not at present, the one you are thinking of was reverted. But there is a risk of it happening again, hard to see a way around it. Debian has a two year life cycle and they will only take 'release' code.

Davo
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #28 on: October 31, 2021, 11:38:39 am »
I think the OpenDialog (or a library it depends on) processes an OnChange event every time the contents of the directory change. Like the FindFile results getting invalidated.

del

  • Sr. Member
  • ****
  • Posts: 258
Re: [SOLVED] TOpenDialog - 100% CPU - Fan Revs Up
« Reply #29 on: October 31, 2021, 03:09:29 pm »
This also fixed the problem:

Project Options -> Compiler Options -> Additions and Overrides
LCLWidgetType:=gtk3

When I made the initial change to QT5 I forgot to notice what the default was. So I did a git diff on the .lpi file and it showed the QT5 as an addition:
Quote
+      <SharedMatrixOptions Count="1">
+        <Item1 ID="829919207870" Modes="Release" Type="IDEMacro" MacroName="LCLWidgetType" Value="qt5"/>
+      </SharedMatrixOptions>
But it didn't show any subtractions, i.e., it didn't show what got replaced by the QT5. So I can't tell you what the default was. Right now the test directory has over 51000 jpegs in it with two new ones streaming in per second, and the gtk3 Open File dialog is working fine.

 

TinyPortal © 2005-2018