Recent

Author Topic: Keeping track of project statistics - timings  (Read 16434 times)

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: Keeping track of project statistics - timings
« Reply #15 on: July 20, 2017, 02:58:54 pm »
IMHO it should be done a package.
It may be that it will go to Lazarus CCR. (but lets see what others say)
I agree it should an external package. but not CCR. CCR is where component go to die or at least that is how I see it.

The question is which info the package needs to get from the IDE, and if the units in IDEInf can provide this, or what hooks needs to be added there.
actually the info required are minimal focus changed to designer/editor/object inspector/debug windows etc, debugging started/ended (either normally or by resetting it), application.idle, focused, lost focus etc. and the package will have its own api that others can use.
« Last Edit: July 20, 2017, 03:03:59 pm by taazz »
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5699
    • wiki
Re: Keeping track of project statistics - timings
« Reply #16 on: July 20, 2017, 03:13:26 pm »
actually the info required are minimal focus changed to designer/editor/object inspector/debug windows etc, debugging started/ended (either normally or by resetting it), application.idle, focused, lost focus etc. and the package will have its own api that others can use.

SourceEditorIntf afaik has an event that can trigger on focus, and on any change (including scroll and caret) to detect activity (except: staring at code in disbelieve)

Not sure about the others.

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: Keeping track of project statistics - timings
« Reply #17 on: July 20, 2017, 04:03:24 pm »
actually the info required are minimal focus changed to designer/editor/object inspector/debug windows etc, debugging started/ended (either normally or by resetting it), application.idle, focused, lost focus etc. and the package will have its own api that others can use.

SourceEditorIntf afaik has an event that can trigger on focus, and on any change (including scroll and caret) to detect activity (except: staring at code in disbelieve)

Not sure about the others.
Taking advantage of the static linking nature of fpc the application.addXXXXXhandler can be used to support application changing from a package, assuming that the forms units and the application in there is used from the IDE it self. The Screen.OnActiveFormChange event can be used to monitor changes between forms inside the IDE. That leaves the start/stop debugging.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Keeping track of project statistics - timings
« Reply #18 on: July 20, 2017, 04:23:19 pm »
Thanks @Martin_fr, @Taazz for the feedback. I already started developing the package. Few details:
1. The package name is ProjectStatistic.lpk
2. It will be part of the Online Package Manager not CCR. Every major package from CCR is already ported to OPM
3. After install it can be accessed through Lazarus Menu-->Project-->Project Statistics(Ctrl + Shift + F12), just below project option(see attached image)
4. The timing is done with a worker thread
5. The statistics is loaded/saved from/to the *.lpi file
6. After a few short test, I was able to hook both the sourceeditor and the designer. The question is what happens when the main form is in focus(where the menu and component palette is) or the object inspector or any other window? In which timing category falls the main window?
7. Apparently it's possible to implement everything from the package itself, without changing the IDE source or IDEintf
8. What about Copyright infringement if any? Is this feature patented?
« Last Edit: July 20, 2017, 04:31:09 pm by GetMem »

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: Keeping track of project statistics - timings
« Reply #19 on: July 20, 2017, 04:46:15 pm »
Thanks @Martin_fr, @Taazz for the feedback. I already started developing the package. Few details:
1. The package name is ProjectStatistic.lpk
2. It will be part of the Online Package Manager not CCR. Every major package from CCR is already ported to OPM
3. After install it can be accessed through Lazarus Menu-->Project-->Project Statistics(Ctrl + Shift + F12), just below project option(see attached image)
4. The timing is done with a worker thread
5. The statistics is loaded/saved from/to the *.lpi file
erm no lpi is usually under source control. For now it should use its own file, based on the active project of course.
6. After a few short test, I was able to hook both the sourceeditor and the designer. The question is what happens when the main form is in focus(where the menu and component palette is) or the object inspector or any other window? In which timing category falls the main window?
both/none if the focused is changed to the menu/toolbar/component palette window then no action should be taken the existing 1st level windows keeps on time tracking. OBject inspector is probably safe to group it with the designer.
7. Apparently it's possible to implement everything from the package itself, without changing the IDE source or IDEintf
as long as it remains statically linked?
8. What about Copyright infringement if any? Is this feature patented?
This is better left for the team to answer. (that was quick!)
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3646
  • I like bugs.
Re: Keeping track of project statistics - timings
« Reply #20 on: July 20, 2017, 08:31:47 pm »
Oh yeah juhas request (more than a recommendation than request) to port a scripting IDE to lazarus
Sounds like a good idea although I have forgotten what it was about.

This timing statistics feature sounds good, too.
I personally miss a project metrics feature even more. It is also statistics but about the code already created instead of time.
Lines of code, amount of comments, maybe some code quality analysis, etc.
Codetools provide most tools to implement it I guess.
Anyway, it does not compete with the feature you are planning now. They are both usefull.

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Keeping track of project statistics - timings
« Reply #21 on: July 20, 2017, 08:59:09 pm »
@taazz
I sent a mail to devlist, let's see what they say about this feature.

@Juha
Quote
This timing statistics feature sounds good, too.
I personally miss a project metrics feature even more. It is also statistics but about the code already created instead of time.
Lines of code, amount of comments, maybe some code quality analysis, etc.
Codetools provide most tools to implement it I guess.
Anyway, it does not compete with the feature you are planning now. They are both usefull.
Good idea! I will implement the timing first, then we can add more and more feature. I also have to implement the OPM integration, but I need to build up some courage first, it's really complicated, time consuming stuff. :D

avra

  • Hero Member
  • *****
  • Posts: 1717
    • Additional info
Re: Keeping track of project statistics - timings
« Reply #22 on: July 20, 2017, 09:08:46 pm »
I look at project stats in a slightly different way. I find it useful if it can provide me enough data to bill a customer for contract work. It would also mean using separate XML, JSON, CSV or whatever instead of LPI file (of course also stored in project dir). It should be enabled per project (default off), and on each project exit it should simply add a new record with updated time for each stats category. This would allow using custom external tools for creating bills, creating internal report, and allow team work where each member would have his own stats file (excluded from version control system of course). I do not think that using a timer to detect idle work is a good idea. There should be no idle work when I open such a project. What if I look 10 minutes at some code trying to understand without using mouse and keyboard at all? It should be much simpler. If specific windows are opened then specific categories get their time updated. If Lazarus window is not focused then some 'Other' category gets it's time updated. If focused app name is logged then custom external tool (or a patch from interested party) could further refine stats for 'Other' category (if anyone would need such refinement).
Now please be gentle and don't shoot me (at least not in the head)  ;D
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: Keeping track of project statistics - timings
« Reply #23 on: July 20, 2017, 09:13:33 pm »
Oh yeah juhas request (more than a recommendation than request) to port a scripting IDE to lazarus
Sounds like a good idea although I have forgotten what it was about.
As it turns out I was a bit confused, but here is the thread that started me down the ide conversion road http://forum.lazarus.freepascal.org/index.php/topic,19225.msg139911.html#msg139911
Still on my list I'll eventually convert it and post it on SF. I just need to clean up my more argent tasks first.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

GetMem

  • Hero Member
  • *****
  • Posts: 3508
Re: Keeping track of project statistics - timings
« Reply #24 on: July 20, 2017, 09:43:12 pm »
@avra
Thanks for the feedback.

Quote
I look at project stats in a slightly different way. I find it useful if it can provide me enough data to bill a customer for contract work. It would also mean using separate XML, JSON, CSV or whatever instead of LPI file (of course also stored in project dir). It should be enabled per project (default off), and on each project exit it should simply add a new record with updated time for each stats category. This would allow using custom external tools for creating bills, creating internal report, and allow team work where each member would have his own stats file (excluded from version control system of course).
It will be the lps or some other external file, xml most likely. 

Quote
I do not think that using a timer to detect idle work is a good idea. There should be no idle work when I open such a project. What if I look 10 minutes at some code trying to understand without using mouse and keyboard at all? It should be much simpler. If specific windows are opened then specific categories get their time updated. If Lazarus window is not focused then some 'Other' category gets it's time updated. If focused app name is logged then custom external tool (or a patch from interested party) could further refine stats for 'Other' category (if anyone would need such refinement).
Now please be gentle and don't shoot me (at least not in the head)  ;D
@taazz already came up with a good idea. Start a timer on application idle, then after a predefined number of seconds stop the log. I'm still not sure about that categories though. Which window has focus it's not a problem, it will be easy to detect.

ertank

  • Full Member
  • ***
  • Posts: 244
Re: Keeping track of project statistics - timings
« Reply #25 on: July 20, 2017, 10:18:47 pm »
8. What about Copyright infringement if any? Is this feature patented?

I could not find anything in between Delphi Acknowledgements lines.
I could not find any patent or copyright for Castalia statistics which is taken by Embarcadero
See:
https://www.embarcadero.com/press-releases/embarcadero-acquires-castalia-and-usertility-from-twodesk-software
http://blog.marcocantu.com/blog/2014_september_free_castalia_xe7.html
http://blog.marcocantu.com/blog/2015-january-embarcadero-acquires-castalia-usertility.html

I would say it is a proprietary code for Delphi/C++ Builder IDE and we are dealing with Lazarus IDE here. So, I presume there is no copyright problem here.

avra

  • Hero Member
  • *****
  • Posts: 1717
    • Additional info
Re: Keeping track of project statistics - timings
« Reply #26 on: July 20, 2017, 10:26:24 pm »
It will be the lps or some other external file, xml most likely.
LPS is fine if you do not store history, but if you always add a record on each project exit then it can groooow. So I vote for separate file. It looks cleaner to me.

Quote
Start a timer on application idle, then after a predefined number of seconds stop the log.
Well if that number of seconds can be a zillion, in practice that would not differ from my idea to not allow idle time.  :)

Quote
I'm still not sure about that categories though. Which window has focus it's not a problem, it will be easy to detect.
This is one of possibilities for categories:

Category      Window/AppName         Time                       SumOfSeconds
============================================================================
Lazarus       Object Inpector        2012-04-23T18:25:43.511Z   2411
Lazarus       Form Designer          2012-04-23T18:25:43.511Z   825
Lazarus       Code Editor            2012-04-23T18:25:43.511Z   5432
Other         Photoshop              2012-04-23T18:25:43.511Z   1042
Lazarus       H2Pas                  2012-04-23T18:25:43.511Z   3119
Other         Notepad++              2012-04-23T18:25:43.511Z   753


In simple variant categories are not needed at all. Who needs them let him build external tool. In more complex variant you can have somewhere stored a list of Lazarus specific Window/AppName and flag them as 'Lazarus' category. Everything else can be 'Other' category.

Having an option not to save stats for windows which were active for less then some user chosen number of seconds would filter out irrelevant records.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3646
  • I like bugs.
Re: Keeping track of project statistics - timings
« Reply #27 on: July 20, 2017, 11:21:14 pm »
I could not find any patent or copyright for Castalia statistics which is taken by Embarcadero
Measuring time cannot be patented I hope. It would make many people's life difficult.

Quote
I would say it is a proprietary code for Delphi/C++ Builder IDE and we are dealing with Lazarus IDE here. So, I presume there is no copyright problem here.
Exactly.

ertank

  • Full Member
  • ***
  • Posts: 244
Re: Keeping track of project statistics - timings
« Reply #28 on: July 21, 2017, 01:05:13 am »
Also, I appreciate considering different operating systems. On my part, I am mostly using Lazarus on Raspberry Pi recently. It may not be so easy to collect "other software usage" statistics in there. On the other hand, I do really wonder what percentage will be compile timing in there  :)

avra

  • Hero Member
  • *****
  • Posts: 1717
    • Additional info
Re: Keeping track of project statistics - timings
« Reply #29 on: July 21, 2017, 10:23:56 am »
I am mostly using Lazarus on Raspberry Pi recently. It may not be so easy to collect "other software usage" statistics in there.
I do not see how would that differ to other Linuxes. Whenever Lazarus is not focused that can be 'Other' category, having or not having a subcategory with window title. That way it's not hard to filter out what info is not needed for internal or external stats presentation.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib