Recent

Author Topic: Usage of bug tracker  (Read 3843 times)

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3645
  • I like bugs.
Usage of bug tracker
« on: December 29, 2015, 03:29:29 pm »
Let's revise the proper usage of bug tracker.

A rule number one is to check for existing reports first. It is even documented here:
 http://wiki.freepascal.org/How_do_I_create_a_bug_report#Check_if_the_bug_is_not_already_reported

It is a good idea to first verify any problem by asking in forum or mailing list. Many problems turn out to be user errors or misunderstandings. Experienced users may also remember old bug reports which were difficult to find.

Build and compilation errors are often caused by a user's local configuration or environment. Such errors should not be reported. Report only compilation errors when they can be reproduced by others. Check by asking in forum or mailing list first.
The majority of such reports end up resolved as "Unable to reproduce" finally, after hanging around in the tracker for a year or so.

Crash-bug reports should always have a debugger backtrace with them. Especially the ones that happen only sometimes and are hard to reproduce need a backtrace always.

Questions should be asked in forum or mailing list, not in bug tracker. Fortunately this has improved because those issues are resolved immediately without answering the question.

Opening lots of reports for a sub-system which is in alpha-state or otherwise not maintained is bad. They will probably stay open for long and make finding important bugs difficult.
For example finding and prioritizing bugs to fix for a coming release becomes difficult.

But finally, reports with a patch that solves the issue are always welcome. I try to go through the reports again to make sure there are no ignored valid patches. There are > 1700 open issues for Lazarus project alone. Going through them takes time.


This post was inspired by the 5 duplicate reports that were opened within few hours for the same LazBuild compilation error. The original:
 http://bugs.freepascal.org/view.php?id=29274
Yes it took more than a day before I fixed it but still one report would be enough. The first report was easy to find because it was opened just few hours earlier.
For such clear errors just a post in the mailing list would be enough. Bug tracker is not really needed then.

ykot

  • Full Member
  • ***
  • Posts: 141
Re: Usage of bug tracker
« Reply #1 on: December 29, 2015, 05:40:49 pm »
I don't want to spoil your mood, but...

Build and compilation errors are often caused by a user's local configuration or environment. Such errors should not be reported. Report only compilation errors when they can be reproduced by others. Check by asking in forum or mailing list first.

1. How do you realistically expect from someone having problems with the build yet have sufficient knowledge to realize whether or not it can be reproduced by others?

2. In my own experience, errors during building of FPC and/or Lazarus from the trunk in 99.9% cases is caused by recent SVN commit, which carelessly broke one of few targets (perhaps, similarly to the issue you are mentioning). This can be quite frustrating for non-FPC developers, especially to figure out in which recent revision the problematic code was committed. Few breakages were, apparently, expected, but many simply look like a developer didn't test the patch on local machine (or did, but just one target or only with some default flags), submitted it to SVN and forgot about it. These issues sometimes get fixed really fast, but others can take few days, even when there IS a report for it on bugtracker.

Please let other FPC developers know that everyone's work is very important, valuable and greatly appreciated by the community. However, when you submit a patch to SVN, there could be a great number of people who will update and get their environment broken, if you don't test your patch locally first. Testing locally doesn't mean just running "make clean all", but actually building the whole trunk for at least Windows and Linux, both 32-bit and 64-bit, preferably also ARM. Scripts can be very helpful there too - you can just run a script, which will do the build for some common targets. On a typical today's desktop machine this wouldn't take more than 15 minutes, but will spare every other developer of frustration and time waste trying to figure out who broke what.

Crash-bug reports should always have a debugger backtrace with them. Especially the ones that happen only sometimes and are hard to reproduce need a backtrace always.
Ideally yes, but realistically, there are some targets and platforms, where getting backtrace can be very difficult (or impossible without proper tools). Embedded platforms come to mind. Also, not every user would have enough experience to get the backtrace. Nevertheless, I myself as a developer would really appreciate a bug report, even without backtrace - after all, this is better than nothing.

Some developers get angry when a customer submits bug report, but really this shouldn't be the case, especially with an open-source project. After all, a person cared enough for your product to spend time writing bug report, even if it's a duplicate (you can't expect every user to be a proficient bug-tracking software user too), not to mention that it could be a potentially real bug, fixing which will make the product even better - this kind of feedback is really valuable and should be appreciated. Sure, linking duplicate reports is also a time-waste, but in my opinion, it's a trade-off for an extra testing effort you get.

Opening lots of reports for a sub-system which is in alpha-state or otherwise not maintained is bad.

But...Why??! For example, "Alpha release" on Wikipedia is described as:
Quote
The alpha phase of the release life cycle is the first phase to begin software testing (alpha is the first letter of the Greek alphabet, used as the number 1). In this phase, developers generally test the software using white-box techniques. Additional validation is then performed using black-box or gray-box techniques, by another testing team. Moving to black-box testing inside the organization is known as alpha release.[2]

Alpha software can be unstable and could cause crashes or data loss. Alpha software may not contain all of the features that are planned for the final version. In general, external availability of alpha software is uncommon in proprietary software, while open source software often has publicly available alpha versions. The alpha phase usually ends with a feature freeze, indicating that no more features will be added to the software. At this time, the software is said to be feature complete.

So, it looks like alpha release is actually meant for testing. Considering that in an open-source community, users are testers themselves, why wouldn't you want people to report bugs on that? Sure, these can be left unattended for quite some time, but isn't a known issue better than unknown issue?

JuhaManninen

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3645
  • I like bugs.
Re: Usage of bug tracker
« Reply #2 on: December 29, 2015, 08:17:01 pm »
1. How do you realistically expect from someone having problems with the build yet have sufficient knowledge to realize whether or not it can be reproduced by others?

By asking in this forum for example. This is an active forum with knowledgeable people.

Quote
2. In my own experience, errors during building of FPC and/or Lazarus from the trunk in 99.9% cases is caused by recent SVN commit, which carelessly broke one of few targets (perhaps, similarly to the issue you are mentioning).

I think the less common platforms and architechtures are a problem indeed. When changing LCL code that affects widgetsets, there is a danger of breaking them. I typically change the IDE's code which is safer. LazBuild became buggy because I forgot (again) to test it. "make clean all" would have revealed it. Sorry about that.

Anyway, using trunk version is not recommended for production usage. IMO using trunk versions of both FPC and Lazarus is never recommended. Too many variables. It is like begging for trouble.

Quote
Ideally yes, but realistically, there are some targets and platforms, where getting backtrace can be very difficult (or impossible without proper tools). Embedded platforms come to mind. Also, not every user would have enough experience to get the backtrace. Nevertheless, I myself as a developer would really appreciate a bug report, even without backtrace - after all, this is better than nothing.

Maybe so, but fixing a bug which cannot be reproduced and which has no backtrace is very difficult.

Quote
Some developers get angry when a customer submits bug report, but really this shouldn't be the case, especially with an open-source project. After all, a person cared enough for your product to spend time writing bug report, even if it's a duplicate (you can't expect every user to be a proficient bug-tracking software user too), not to mention that it could be a potentially real bug, fixing which will make the product even better - this kind of feedback is really valuable and should be appreciated. Sure, linking duplicate reports is also a time-waste, but in my opinion, it's a trade-off for an extra testing effort you get.

Yes, maybe it is a necessary trade-off. The fact still is that finding the high priority issues is difficult when preparing for a new release for example. I know it because I tried to do it recently.

Quote
So, it looks like alpha release is actually meant for testing. Considering that in an open-source community, users are testers themselves, why wouldn't you want people to report bugs on that? Sure, these can be left unattended for quite some time, but isn't a known issue better than unknown issue?

No, alpha quality code can be tested by people who want to actually improve the code. Ok, it can be tested by other people, too, but reporting bugs makes no sense because there are so many of them and everybody knows it already.
If you want to improve such code then it is a different matter. You can use the bug tracker for uploading and explaining your patches then.

For example LCL-GTK3 is alpha. Only some of the controls work. The IDE cannot be built with it. This already implies thousands of bugs and missing or partially implemented features. Reporting them all would not benefit anybody. On the contrary, it would take time and energy from the person writing the reports and from the people trying to find relevant "real" bugs under this flood of useless reports.
The code must reach a certain level of maturity before bug reports make any sense!

Summa summarum: help by opening bug reports is ok, but fixing them (providing patches) is much better.

Bart

  • Hero Member
  • *****
  • Posts: 3538
    • Bart en Mariska's Webstek
Re: Usage of bug tracker
« Reply #3 on: December 29, 2015, 10:54:03 pm »
However, when you submit a patch to SVN, there could be a great number of people who will update and get their environment broken, if you don't test your patch locally first. Testing locally doesn't mean just running "make clean all", but actually building the whole trunk for at least Windows and Linux, both 32-bit and 64-bit, preferably also ARM. Scripts can be very helpful there too - you can just run a script, which will do the build for some common targets.

That's a rather unrealistic approach.
Most of my patches involve LCL and Components.
These are supposed to be WS independant.
Every now and then I break something, but most of the times I (or someone else) will fix it very quickly.

If every patch I (and others) propose has to be tested on all supported platforms, no work would be done at all.
I can test (and I do so) on Windows 32 and Linux 32. But even then, there are so many flavours.
I remember from the past I was driving Zeljan crazy with GTK2 bugs that only happened in my very old GTK2 version, and Zeljan just had to guess how to fix it, because he could not test.
You cannot realistically expect each and every committer to have dozens of virtual machines running (and still then, virtualized hardware may very well behave differently from actual hardware). It is not going to happen.

I accept that sometimes updating from svn breaks my Lazarus, but then either I find the cause and repair, or just go back to a revision that works.
This is the way progres is made.
And yes, sometimes it is annoying to see a patch committed and then have compiliation failing on a syntax error.

If you want the bleading edge, this is the price you must be willing to pay.
Nobody forces you to use trunk.

Bart

ykot

  • Full Member
  • ***
  • Posts: 141
Re: Usage of bug tracker
« Reply #4 on: December 29, 2015, 11:45:43 pm »
If every patch I (and others) propose has to be tested on all supported platforms, no work would be done at all.
I meant testing that the patch does not break the build. You can do this with a script, just to see if trunk compiles or not.

Anyway, using trunk version is not recommended for production usage. IMO using trunk versions of both FPC and Lazarus is never recommended. Too many variables. It is like begging for trouble.
This is pretty much an established proposition and nobody challenges that. Nevertheless, component developers, myself included, especially for new platforms such as singleboard/embedded, do require to use trunk; in fact, for "embedded" platforms this is pretty much a minimal requirement.

Maybe so, but fixing a bug which cannot be reproduced and which has no backtrace is very difficult.
No one challenges that, but that's a misquote. In my answer, please read the part that I've quoted and my answer. You asked to always include backtrace, no matter what, with a supporting secondary sentence starting "Especially the ones...". If your main point was in second sentence, whereas the first one was unimportant, sorry that I've misunderstood it, but IMHO that's not exactly a clear way to easily explain your point.

No, alpha quality code can be tested by people who want to actually improve the code. Ok, it can be tested by other people, too, but reporting bugs makes no sense because there are so many of them and everybody knows it already.
If you want to improve such code then it is a different matter. You can use the bug tracker for uploading and explaining your patches then.

For example LCL-GTK3 is alpha. Only some of the controls work. The IDE cannot be built with it. This already implies thousands of bugs and missing or partially implemented features. Reporting them all would not benefit anybody. On the contrary, it would take time and energy from the person writing the reports and from the people trying to find relevant "real" bugs under this flood of useless reports.
The code must reach a certain level of maturity before bug reports make any sense!
I'm sorry, but that doesn't sound like qualifying an alpha-release stage. That's more like a pre-alpha, or something like early development milestone. I totally agree on this, but again, the misunderstanding is that you've misused term "alpha release". Perhaps alpha release is meant something different in context of FPC/Lazarus than the rest of industry - this is okay too, each community/company/organization may have its own set of business rules; in such case, sorry, I didn't know about that.

Summa summarum: help by opening bug reports is ok, but fixing them (providing patches) is much better.
Totally agree with you.

That's a rather unrealistic approach.
Most of my patches involve LCL and Components.
These are supposed to be WS independant.
Every now and then I break something, but most of the times I (or someone else) will fix it very quickly.
I see you are taking this personally, please don't. To be fair,  most of the breakages that I've experienced myself were mainly caused by changes in FPC, not in Lazarus. I've actually rarely seen breakages in Lazarus trunk. In any case, in my opinion, trunk compilation breakages is a real issue - it's not too suitable for bug tracker, which is why I mentioned it here. As requested, I also included a "patch" to fix it, a suggestion to build trunk locally before committing. Nevertheless, if that's a "won't fix", no problem. :)

Nobody forces you to use trunk.
Please correct me if I'm wrong, but similarly, nobody is forcing you to work on that product, fix the bugs or answer to people on forums either, right? My point is, what you said is obvious, yet isn't exactly a nice thing to say. Sounds more like a STFU. :)

Bart

  • Hero Member
  • *****
  • Posts: 3538
    • Bart en Mariska's Webstek
Re: Usage of bug tracker
« Reply #5 on: December 30, 2015, 05:46:31 pm »
Please correct me if I'm wrong, but similarly, nobody is forcing you to work on that product, fix the bugs or answer to people on forums either, right?

STOP! This is getting too silly!

(Please click the link before responding).

Bart

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: Usage of bug tracker
« Reply #6 on: December 30, 2015, 09:33:59 pm »

Anyway, using trunk version is not recommended for production usage. IMO using trunk versions of both FPC and Lazarus is never recommended. Too many variables. It is like begging for trouble.
This is pretty much an established proposition and nobody challenges that. Nevertheless, component developers, myself included, especially for new platforms such as singleboard/embedded, do require to use trunk; in fact, for "embedded" platforms this is pretty much a minimal requirement.

   Stable FPC releases can be considered rare. The fact that usually Lazarus trunk can be built using FPC trunk is a significant feature.