Views

Quality/Bugtriage Guide

From MeeGo wiki

< Quality(Difference between revisions)

Latest revision as of 17:53, 27 June 2011

Note: This is a first and very rough draft. If you can add information please go ahead. -- User:Andre

This article is for anyone who wants to help triaging the incoming and existing bug reports in MeeGo Bugzilla. This article is not for bug reporters, but those people taking a look at the reports after they have been submitted.

Following the techniques listed and explained here will help important bugs get fixed, as well as optimize precious developer time.

Also see How To Report Bugs.

How can I help?

Create an account. The bugs.meego.com authentication system is integrated with the meego.com authentication system, so the same account is used for both sites. To create a MeeGo account visit the registration page. Soon after you will receive an email with a link that must be visited to confirm your account creation. You will have to change your password on meego.com before attempting to log into Bugzilla.

TO DO: Introduce Stock answers if wanted and agreed on -- User:Andre

Read the [Stock_answers Stock answers]. These stock responses are general examples of the kinds of comments that triagers should add to bug reports. There are even a surprising number of bug reports for which you can simply use these responses. Please always add a comment when changing the bug status to RESOLVED or NEEDINFO, so the user can comprehend your decision.
Find what should be changed. Read the Steps of Triaging below to determine what should be changed (if anything) and how.
Add yourself to the CC list of the reports you triage so you will receive updates and notifications on changes.
Ask someone to verify your changes. In #meego-bugs on irc.freenode.net mention the bug number you are looking at and the changes that you think should be made. (If no one is around, it is perfectly okay to leave a comment in the relevant bug with information that could be useful to others, e.g. "I think this bug report is a duplicate of bug 1234").

TO DO: editbugs permissions: Does this make any sense in the bugs.meego.com setup? -- User:Andre

As time goes by and you become more familiar with Bugzilla, please ask the Bugsquad members to provide editbugs permissions to you so you have gain much more power to change things and work efficiently in the bug database.

Potential tasks

Please take a look at some ideas here.

Steps of Triaging

This guide should make triaging a bug as easy and simple as possible. Remember, if in doubt, ask! Also, remember that you may not be able to find anything to change for some bugs since not all bugs are filed incorrectly or incompletely.

Is it part of MeeGo Bugzilla?

Some pieces of software that users think of as part of MeeGo might be 3rd party software and have their bug tracking systems elsewhere. If the bug relates to one of these components, please close the bug as INVALID and ask the reporter to report it to the corresponding tracking system.

Hardware problems and Hardware enhancement requests

MeeGo is a software platform, hence bug reports about hardware bugs are invalid in bugs.meego.com. Please ask the reporter to either contact the hardware manufactorer (in case of Nokia this would be www.nokia.com) or to contact the company who sold the product to them. In case of enhancement ideas for Nokia hardware there is http://conversations.nokia.com/design-by-community/.

Is it a valid, but wide and non-specific feature request?

Some reporters have great ideas and visions. They want to improve the User Experience in general or change behaviour of several applications at once. These ideas are very hard to handle in Bugzilla as they require extensive brainstorming, discussing and involvement of several people/teams. Such reports are better to be discussed on one of the MeeGo mailing list first .

Does it make sense?

Does it have enough information? Common sense rules here - "this sucks" or "this crashed" should be moved to NEEDINFO state with a polite message, as should other things that are too ambiguous to be useful. In many cases asking for a URL or attachment will help. If you need help making changes to bug fields, just comment describing what changes you want made, or ask for help from the #meego-bugs channel on Freenode IRC, or send an email to meego-qa@lists.meego.com.

Is the bug a duplicate?

Some bugs in Bugzilla have been reported before. Finding them is more of an art than a science :-) This is an important step, but don't spend too long over it; if a bug is filed twice, someone else will mark the duplicate later. Be creative, e.g. searching for "delet" will bring up all bugs that have "delete" or "deleting" in the report, and "remov" is another popular string with a similar meaning to "delet" that other people could have used to already report the same problem. If the bug is a duplicate, mark it as a duplicate in the resolution box at the bottom of bugzilla and add a polite comment explaining that this bug/request has been filed before. You don't need to carry on triaging it if it's a duplicate; press the "Commit" button and go onto your next bug!

Can you reproduce it?

If you can, please try to reproduce the problem. If you think the problem is not reproducable, write a comment indicating the exact software version you tested, the hardware that you used, the steps you took, the results you got, and why you think the bug WORKSFORME.

Is the version set correctly?

In general we always ask people to run the latest available software release, because nobody works on fixing ancient and obsolete releases anymore.

Image build date is available at /etc/meego-release, and you could refer Release_Engineering/Release_Versioning

Is it in the right place?

If you think the bug was filed against the wrong product, look for a better alternative in the drop-down list box (general information about the MeeGo architecture can be found here).

If the bug is about code which is buggy in its upstream form, please use the list of upstream bug trackers to find the correct bug tracker and then find/report it there. Once you have an upstream bug url, please add it to the bug by filling in <See Also>. If you can't add it to see also, add a comment to the bug.

What severity/priority should be set?

TO DO: Document restrictions and permissions -- User:Andre

Severity

Severity field describes the impact of a bug and is open to all bugzilla users. Bug triage team will adjust the severity if the original one set by bug reporter is not proper. Roughly, MeeGo bugzilla provides 5 levels of severities:

* critical --- Crashes, loss of data, negative impact to other components, etc
* major --- Major loss of functionality
* normal --- Regular issue, some loss of functionality under certain circumstance
* minor --- Minor loss of functionality, or issues with easy workaround available
* enhancement --- Request for enhancement

TO DO: Please explain enhancement severity vs MeeGo Features classification -- User:Andre

Priority

Priority field describes the importance and order in which a bug should be fixed. The priority field is restricted to group_of_some_people so it cannot be set by reporters or changed by average users. Triage team has the privilege to assign a proper priority based on bug severity and user imapct described by bug reporter. Roughly, MeeGo Bugs are with 3 levels of priorities:

* High --- SHOW STOPPER, must fix shortly in upcoming milestone and definitely for final product
* Medium --- fix before product release, however for upcoming milestones, we can document the issue with possible work-around for visible defects
* Low --- Lowest priority, harder to reproduce or are component specific in a different non-MeeGo environment.

Field Priority may keep "Undecided" for the first triage if triage team does not get enough information to assign a proper value. For example, lack of reproduce steps, image date for issue verification, user impact, etc. In this scenario, triage team will set bug status to "NEEDINFO" and add comments about what kind of information is needed for triaging. The Priority of the very bug is kept "Undecided" until information is feed and bug re-triaged.

Crasher bugs

Should have Severity=Critical.

TO DO: Introduce Stock answers if wanted and agreed on -- User:Andre

Set the status to NEEDINFO if they don't have a stack trace, and ask the reporter to get a decent stack trace (see Stock_answers for more information). If there is no information about how it is triggered you should ask the original reporter how to reproduce the bug.

Something not working as expected

Severity=Low or Medium depending on how much it breaks the application.

Incorrect strings, typos, etc.

Severity=Low, Priority=Normal or High depending on how user-visible it is.

Feature requests

If it really is a new feature, and not an improvement to something already in the product, set Severity=Enhancement. If you're unsure about your Severity triage, add a comment in the bug explaining why you triaged it as you did. You can also add yourself to the CC field, so if someone retriages it differently you can find out why.

Is the assignee and QA contact correctly?

MeeGo bugzilla provides a default assignee and QA contact for each new filed bug. However, due to the complexity of each component and different supported platforms, the default one may not always be the proper one. Triage team also checks whether these two field are filled with proper person, to ensure that bugs will be owned and followed smoothly in their life cycle. However, if the issue is a system level one, probably cross components, then it is difficult for triage team to assign a proper owner. In this scenario, simply add keyword "need-sysdebug" and then it will be performed in-depth triage during SysDebug. See more details about SysDebug at http://wiki.meego.com/Quality/SysDebug.

Does it have the correct keywords?

Get familiar with the existing keywords and add those that apply to the report.

Is the described problem obviously easy to fix?

For reported issues that are obviously easy to fix, please add the "EasyFix" keyword which marks bugs that can easily be fixed by MeeGo community members and contributors. This is especially meant for beginners as an entry point to start contributing to MeeGo.

Has it been in NEEDINFO state for more than four weeks?

As decided by the QA team, reports that are in NEEDINFO state and have not been changed for four weeks can be closed. This means that more information had been requested from the reporter but has not been provided. These reports should be closed as WORKSFORME if you are not able to reproduce, or as INVALID if there is simply not enough information to be a useful report. Please always explain your decision by adding a friendly comment and in the latter case, ask the reporter to provide the information asked for, if s/he can, and reopen the bug.

Example comment: "Unfortunately, without more information, as requested in one of the last comments, we are unable to do anything more about this report. Until more information is available, I'm resolving the report as WORKSFORME/INVALID. Please feel free to reopen this bug if you can provide the information asked for, or if you can still reproduce this in a recent version."

Specific areas

Is the bug a release/update blocker?

When approaching the upcoming MeeGo release or update release, MeeGo bug fixing will be controlled through Flags like "MeeGo_1.0_Update_Release_Blocker", "MeeGo_1.1_Release_Blocker", etc. During these period, triage team will also propose potential blocker issues for Management team to judge. This is to push show stopper issues be fixed timely before formal release.

TO DO: Unclear / intransparent process how and who ("Management team") decides. See these postings: 1 2 --Andre 17:53, 27 June 2011 (UTC)

Acknowledgment

Kudos to timeless, the Mozilla Triage Team and the GNOME Bugsquad. This guide is based on the maemo.org guide.

Retrieved from "http://wiki.meego.com/Quality/Bugtriage_Guide"

Category: QA

@@ Line 3: / Line 3: @@
 |}
-This article is for anyone who wants to help triaging the incoming and existing bug reports in !MeeGo Bugzilla. This article is not for bug reporters, but those people taking a look at the reports after they have been submitted.
+This article is for anyone who wants to help triaging the incoming and existing bug reports in MeeGo Bugzilla. This article is not for bug reporters, but those people taking a look at the reports after they have been submitted.
 Following the techniques listed and explained here will help important bugs get fixed, as well as optimize precious developer time.
-{|
+Also see [[Quality/How To Report Bugs|How To Report Bugs]].
-| style="background: #ffcc00; color: black" | '''TO DO:''' Sync prefixes (move from "Bugzilla" to "Quality" and update link -- [[User:Andre]]
-|}
-Also see [http://wiki.meego.com/Quality/How_To_Report_Bugs How To Report Bugs].
 == How can I help? ==
@@ Line 29: / Line 25: @@
 | style="background: #ffcc00; color: black" | '''TO DO:''' editbugs permissions: Does this make any sense in the bugs.meego.com setup? -- [[User:Andre]]
 |}
-As time goes by and you become more familiar with Bugzilla, please ask the [http://wiki.meego.com/Quality/Bugtriage#MeeGo_Bug_Triage_Team Bugsquad members] to provide [http://www.bugzilla.org/docs/3.6/en/html/useradmin.html#modifyusers editbugs permissions] to you so you have gain much more power to change things and work efficiently in the bug database.
+As time goes by and you become more familiar with Bugzilla, please ask the [[Quality/Bugtriage#MeeGo_Bug_Triage_Team|Bugsquad members]] to provide [http://www.bugzilla.org/docs/3.6/en/html/useradmin.html#modifyusers editbugs permissions] to you so you have gain much more power to change things and work efficiently in the bug database.
 === Potential tasks ===
-Please take a look at [http://wiki.meego.com/Quality/Bugtriage_Tasks some ideas here].
+Please take a look at [[Quality/Bugtriage_Tasks|some ideas here]].
 == Steps of Triaging ==
@@ Line 60: / Line 56: @@
 === Is the bug a duplicate? ===
-Some bugs in Bugzilla have been reported before. Finding them is more of an art than a science :-) This is an important step, but don't spend too long over it; if a bug is filed twice, someone else will mark the duplicate later. Be creative, e.g. searching for "delet" will bring up all bugs that have "delete" or "deleting" in the report.
+Some bugs in Bugzilla have been reported before. Finding them is more of an art than a science :-) This is an important step, but don't spend too long over it; if a bug is filed twice, someone else will mark the duplicate later. Be creative, e.g. searching for "delet" will bring up all bugs that have "delete" or "deleting" in the report, and "remov" is another popular string with a similar meaning to "delet" that other people could have used to already report the same problem.
 If the bug is a duplicate, mark it as a duplicate in the resolution box at the bottom of bugzilla and add a polite comment explaining that this bug/request has been filed before. You don't need to carry on triaging it if it's a duplicate; press the "Commit" button and go onto your next bug!
 === Can you reproduce it? ===
@@ Line 70: / Line 66: @@
 In general we always ask people to run the latest available software release, because nobody works on fixing ancient and obsolete releases anymore.
+Image build date is available at /etc/meego-release, and you could refer [[Release_Engineering/Release_Versioning]]
 === Is it in the right place? ===
 If you think the bug was filed against the wrong product, look for a better alternative in the drop-down list box (general information about the MeeGo architecture can be found [http://meego.com/developers/meego-architecture here]).
+If the bug is about code which is buggy in its upstream form, please use the [[Quality/UpstreamBugTrackers|list of upstream bug trackers]] to find the correct bug tracker and then find/report it there. Once you have an upstream bug url, please add it to the bug by filling in <See Also>. If you can't add it to see also, add a comment to the bug.
 === What severity/priority should be set? ===
 {|
-| style="background: #ffcc00; color: black" | '''TO DO:''' Document restrictions and permissions -- [[User:Andre]]
+| style="background: #ffcc00; color: black" | '''TO DO:''' [https://bugs.meego.com/show_bug.cgi?id=12254 Document restrictions and permissions] -- [[User:Andre]]
 |}
-The priority field is restricted to '''group_of_some_people''' so it cannot be set by reporters or changed by average users.
+==== Severity ====
+Severity field describes the impact of a bug and is open to all bugzilla users. Bug triage team will adjust the severity if the original one set by bug reporter is not proper. Roughly, MeeGo bugzilla provides 5 levels of severities:
+ * ''critical'' --- Crashes, loss of data, negative impact to other components, etc
+ * ''major'' --- Major loss of functionality
+ * ''normal'' --- Regular issue, some loss of functionality under certain circumstance
+ * ''minor'' --- Minor loss of functionality, or issues with easy workaround available
+ * ''enhancement'' --- Request for enhancement
+{|
+| style="background: #ffcc00; color: black" | '''TO DO:''' [https://bugs.meego.com/show_bug.cgi?id=12252 Please explain ''enhancement'' severity vs ''MeeGo Features'' classification] -- [[User:Andre]]
+|}
-Finding the correct severity takes some experience, and you will get a feel for it after triaging several bugs. Here are some guidelines for the most commonly reported bugs. You should read the severity and priority guidelines at least once.
+==== Priority ====
+Priority field describes the importance and order in which a bug should be fixed. The priority field is restricted to '''group_of_some_people''' so it cannot be set by reporters or changed by average users. Triage team has the privilege to assign a proper priority based on bug severity and user imapct described by bug reporter. Roughly, MeeGo Bugs are with 3 levels of priorities:
+ * ''High'' --- SHOW STOPPER, must fix shortly in upcoming milestone and definitely for final product
+ * ''Medium'' --- fix before product release, however for upcoming milestones, we can document the issue with possible work-around for visible defects
+ * ''Low'' --- Lowest priority, harder to reproduce or are component specific in a different non-MeeGo environment.
-Please note that ''only'' changing severity and/or priority ''slightly'' may do more fuss than good. It causes an extra email to the maintainers and consumes their precious time and attention. If other changes are done as well then there is only little additional impact when changing severity and/or priority.
+Field Priority may keep "Undecided" for the first triage if triage team does not get enough information to assign a proper value. For example, lack of reproduce steps, image date for issue verification, user impact, etc. In this scenario, triage team will set bug status to "NEEDINFO" and add comments about what kind of information is needed for triaging. The Priority of the very bug is kept "Undecided" until information is feed and bug re-triaged.
 * '''Crasher bugs'''
 Should have Severity=Critical.
 {|
 | style="background: #ffcc00; color: black" | '''TO DO:''' Introduce [http://wiki.maemo.org/Bugs:Stock_answers Stock answers] if wanted and agreed on -- [[User:Andre]]
@@ Line 99: / Line 113: @@
 If it really is a new feature, and not an improvement to something already in the product, set Severity=Enhancement.
 If you're unsure about your Severity triage, add a comment in the bug explaining why you triaged it as you did. You can also add yourself to the CC field, so if someone retriages it differently you can find out why.
+=== Is the assignee and QA contact correctly? ===
+MeeGo bugzilla provides a default assignee and QA contact for each new filed bug. However, due to the complexity of each component and different supported platforms, the default one may not always be the proper one. Triage team also checks whether these two field are filled with proper person, to ensure that bugs will be owned and followed smoothly in their life cycle. However, if the issue is a system level one, probably cross components, then it is difficult for triage team to assign a proper owner. In this scenario, simply add keyword "need-sysdebug" and then it will be performed in-depth triage during SysDebug. See more details about SysDebug at http://wiki.meego.com/Quality/SysDebug.
+=== Does it have the correct keywords? ===
+Get familiar with the [https://bugs.meego.com/describekeywords.cgi existing keywords] and add those that apply to the report.
+==== Is the described problem obviously easy to fix? ====
+For reported issues that are obviously easy to fix, please add the "EasyFix" [https://bugs.meego.com/describekeywords.cgi keyword] which marks bugs that can easily be fixed by MeeGo community members and contributors. This is especially meant for beginners as an entry point to start contributing to MeeGo.
+=== Has it been in NEEDINFO state for more than four weeks? ===
+As [https://bugs.meego.com/show_bug.cgi?id=12257 decided] by the QA team, [https://bugs.meego.com/buglist.cgi?bug_status=NEEDINFO&chfieldfrom=-29d&chfieldto=Now reports that are in NEEDINFO state and have not been changed for four weeks] can be closed. This means that more information had been requested from the reporter but has not been provided. These reports should be closed as WORKSFORME if you are not able to reproduce, or as INVALID if there is simply not enough information to be a useful report. Please always explain your decision by adding a friendly comment and in the latter case, ask the reporter to provide the information asked for, if s/he can, and reopen the bug.
+Example comment: "Unfortunately, without more information, as requested in one of the last comments, we are unable to do anything more about this report. Until more information is available, I'm resolving the report as WORKSFORME/INVALID. Please feel free to reopen this bug if you can provide the information asked for, or if you can still reproduce this in a recent version."
 == Specific areas ==
+=== Is the bug a release/update blocker? ===
+When approaching the upcoming MeeGo release or update release, MeeGo bug fixing will be controlled through Flags like "MeeGo_1.0_Update_Release_Blocker", "MeeGo_1.1_Release_Blocker", etc. During these period, triage team will also propose potential blocker issues for Management team to judge. This is to push show stopper issues be fixed timely before formal release.
 {|
-| style="background: #ffcc00; color: black" | '''TO DO: Anything needed here?'''
+| style="background: #ffcc00; color: black" | '''TO DO: Unclear / intransparent process how and who ("Management team") decides. See these postings: [http://lists.meego.com/pipermail/meego-releases/2011-June/001431.html 1] [http://lists.meego.com/pipermail/meego-releases/2011-June/001543.html 2] --[[User:Andre|Andre]] 17:53, 27 June 2011 (UTC)'''
 |}
 == Acknowledgment ==
 Kudos to [http://browser.garage.maemo.org/news/8/ timeless], the [http://wiki.mozilla.org/MozillaQualityAssurance:Triage Mozilla Triage Team] and the [http://live.gnome.org/Bugsquad/TriageGuide GNOME Bugsquad]. This guide is based on the [http://wiki.maemo.org/Bugs:Triage_guide maemo.org guide].
+[[Category:QA]]