Meego Wiki
Views

Quality/How To Report Bugs

From MeeGo wiki
< Quality(Difference between revisions)
Jump to: navigation, search
(Bug Life Cycle)
(1.2.0/1.2.1 Bug Tracking Process)
 
(6 intermediate revisions not shown)
Line 1: Line 1:
-
=Getting started=
+
==Getting started: Creating an Account==
All you need is an account to be able to file and comment on reports. The http://bugs.meego.com authentication system is integrated with the http://meego.com authentication system, so the same account is used  for both sites. To create a MeeGo account visit the [http://meego.com/user/register registration page]. Soon after you will receive an email with a link that must be visited to confirm your account creation. You will have to change your password on meego.com before attempting to log into Bugzilla.
All you need is an account to be able to file and comment on reports. The http://bugs.meego.com authentication system is integrated with the http://meego.com authentication system, so the same account is used  for both sites. To create a MeeGo account visit the [http://meego.com/user/register registration page]. Soon after you will receive an email with a link that must be visited to confirm your account creation. You will have to change your password on meego.com before attempting to log into Bugzilla.
-
=Bugzilla Fields=
+
==Reporting an issue==
-
==MeeGo Release==
+
-
MeeGo Release field defines in which MeeGo version a bug has been found.
+
* (Optional:) [https://bugs.meego.com/query.cgi Search] Bugzilla, to see whether your issue has already been reported.
-
* How to identify the used MeeGo version?
+
* Go to https://bugs.meego.com/enter_bug.cgi and choose the appropriate classification and product in which the bug happens. If you are unsure, just guess.
-
** MeeGo version: enter in terminal: <code>cat /etc/meego-release</code>
+
* Note: For feature/enhancement requests, please ''always'' use the classification "MeeGo Features".
-
** Kernel version: enter in terminal: <code>uname -a</code>
+
* On the next page:
 +
** Choose the '''Component''' in which the bug happens.
 +
** Choose the '''MeeGo Release''' in which the problem was found. To find out:
 +
*** MeeGo version: enter in terminal: <code>cat /etc/meego-release</code>
 +
*** Kernel version: enter in terminal: <code>uname -a</code>
 +
** Choose the '''Severity''' according to [https://bugs.meego.com/page.cgi?id=fields.html#bug_severity the definition].
 +
** Choose the type of '''Hardware''' (like Netbook, Nettop, Notebook, Handset, Automotive, TV).
 +
** If you know, choose the hardware '''Architecture''' (For example: N900 is ARM, most Netbooks are IA). Else leave it untouched. In case you test on a N900 please add the keyword ''N900''.
 +
** In case you have time, check whether some existing '''[https://bugs.meego.com/describekeywords.cgi keywords]''' apply to your issue, and add them.
 +
** Enter a '''Summary''' describing the bug. A good summary should be straightforward and precise, to clearly identify the issue. This also helps avoiding duplicate reports.
 +
** In the '''Description''' field, please try to fill in ''all'' data requested in the template, as text.
 +
*** List the exact steps to reproduce the bug, step by step, click by click, so it is easy to reproduce by testers. For example, instead of paraphrasing commands by saying: "suspend the system or put system to S3" we prefer to see the exact command like "echo mem >/sys/power/state" as steps to reproduce.
 +
*** Describe the current outcome and the expected outcome. Try to avoid writing "See screenshot" or "See attached file" but describe it briefly in words so it can be found when querying for existing bug reports (this helps avoiding duplicate reports).
 +
** (Optional:) Add an '''Attachment''' such as a testcase, or a screenshot in case it makes sense.
 +
** Click '''Commit'''.
 +
* You are done - Thanks for your report!
-
Version names are defined according to the [http://wiki.meego.com/Release_Engineering/Plans MeeGo release plan]. If a bug was reported already for an older MeeGo version make sure to [[Quality/defects#Bug Cleanup after Project Release|handle this bugs as described here]].
+
== Good practices and Guidelines ==
-
==Target Milestone==
+
* Each bug report is for only one issue. If you find several issues, please separate them into several reports.
 +
* DO NOT reopen if the same issue is found again after more than 2 weeks. Instead, create a new bug report with “[REG]” as a prefix in the summary.
 +
* Do search before you file a bug - Try to avoid filing duplicates by [http://bugs.meego.com/query.cgi taking a look] whether your issue has already been filed before.
 +
* Keep it clean:
 +
** Don't start endless debates on topics not directly related to the scope of a specific bug report, for example release engineering or marketing. We have [[Community communication|forums and mailing lists]] for that.
 +
** Avoid quoting complete previous comments by stripping unneeded lines.
 +
** Avoid answering above the quoted text.
 +
* The admins Rule: Ultimately, the admins reserve the right to block your account. See our [[Community guidelines|process for dealing with violations]] for more details.
-
When bugs are in OPEN or RESOLVED status, it is used by the bug reporter to propose which milestone this bug is expected to be fixed.  
+
== 1.2.0/1.2.1 Bug Tracking Process ==
 +
In order to align with [http://wiki.meego.com/Release_Engineering/Plans/1.2 MeeGo 1.2 release plan ], QA add a new value "1.2.0" in "MeeGo Release" field to track 1.2.0 update bugs. So for MeeGo 1.2.0 update and MeeGo 1.2.1 release, we will have 2 field: "1.2.0" and "1.2" to track related bugs:
 +
* MeeGo Release = 1.2.0: if new bug is found in MeeGo 1.2 final release or 1.2.0 update, new bug should be reported to 1.2.0;
 +
* MeeGo Release = 1.2: if new bug is found in MeeGo 1.2.1 release or 1.2.1 update, such as 1.2.0.9X image, new bug should be reported to 1.2.1;
 +
* For current existed bug in 1.2, we will keep the MeeGo release version unchanged. If we found one existed bug in 1.2 need to be fixed in 1.2.0 update, QA will clone one bug to 1.2.0 version;
 +
* MeeGo 1.2.0 update team will only focus on MeeGo Release = "1.2.0" bug fix and verification.
-
* --- Undecided
+
==See Also==
-
* 1.0 Propose bug is expected to be fixed in 1.0
+
* [[Quality/Bugzilla_Fields|Fields in a bug report]]
-
* 1.1 Propose bug is expected to be fixed in 1.1
+
* [[Quality/Bug_Life_Cycle_and_Handling|Bug Life Cycle and Report Handling/Follow-up Guidelines]]
 +
* [[Quality/Bugtriage|MeeGo Bug Triage]]
-
==Importance==
+
[[Category:QA]]
-
===Priority===
+
-
 
+
-
Priority field describes the importance and order in which a bug should be fixed. It helps developers to prioritize their work. By default the priority is set as "Undecided" when reporting a new bug. The bug reporter could propose a priority in a comment. A dedicated bug triage team should set the initial priority for the bug while processing it as early as possible. Final priority setting would be decided by bug triage process. Priorities range from High (most important) to Low (least important).
+
-
* High: Bug fixing is on-going, or is planned within 2 weeks, no later than the up-coming milestone. Reproducible crash issues, major function loss, issues greatly impact user experience or issues which block other key features to work would fall in to this category.
+
-
* Medium: Bug fixing is planned before project release, but can't start before HIGH priority items are cleaned up.
+
-
* Low: Bug fixing is not planned for the up-coming project release. Will re-evaluate the importance in next release.
+
-
 
+
-
===Severity===
+
-
 
+
-
The "Severity" field describes the impact of a bug, and the options include:
+
-
 
+
-
* Critical: crashes, other components are affected
+
-
* Major: major loss of own function
+
-
* Normal: regular issue, some loss of functionality under specific circumstances
+
-
* Trival:cosmetic problem like misspelled words or misaligned text
+
-
* Enhancement: request for enhancement
+
-
 
+
-
==Platform==
+
-
===Hardware===
+
-
 
+
-
Please select the UX (like Netbook, Nettop, Notebook, Handset, Automotive, TV) that you identified the bug in.
+
-
For bugs which apply to all UX (like bugs for Middleware/Core components), please select "All".
+
-
 
+
-
===Architecture===
+
-
 
+
-
Please select the architecture ([http://en.wikipedia.org/wiki/Intel_Atom IA], [http://en.wikipedia.org/wiki/ARM_architecture ARM]) that you identified the bug in. For example for N900 Base OS layer bugs this should be set to ARM. For most Netbooks available IA is the right choice.
+
-
For bugs which apply for multiple architectures, like bugs for middleware or applications, please select "All".
+
-
 
+
-
= Bug Life Cycle =
+
-
[[file:Bug-life-cycle-meego.PNG]]
+
-
# The reporter submits the bug for a specific component, then the bug would be assigned to a default owner for each component, showing in field "Assign To". The status is '''NEW''' initially and priority is '''Undecided'''.
+
-
# Bug triage team will follow the responsibilities defined in bug triage process to process bugs: ensure bug component is correct set, enough information has been collected, bug priority is set etc.
+
-
# The component owner should follow up when getting a new bug notification:
+
-
#* If the owner accepts the bug and assigns to himself/herself, he/she should change the status as '''ASSIGNED'''.
+
-
#* The owner may think that it's a wontfix bug. Bug owner should change the status to '''RESOLVED''' with resolution '''INVALID/WONTFIX/WORKSFORME''' with proper comments.
+
-
#* If the owner thinks that it is a bug of the other component, he/she needs to add a justification comment.
+
-
#* If the owner disposes the bug to other individual. The new owner follow the same process.
+
-
#* If the owner thinks that the bug information is incomplete, he/she could change "status" as '''NEEDINFO''' to ask more info from bug reporter.
+
-
#* The owner does bug analysis and finds this bug comes from upstream. Owner can post a new bug in upstream bugzilla (or ask QA help to do this) and change status to '''WAITING FOR UPSTREAM'''. Be sure to add the upstream bugzilla link in moblin bugzilla comment area. When upstream bug is fixed, firstly, integrate the bug fix in distribution image, then mark status to be '''RELEASED - FIXED'''. 
+
-
# The owner (bug assignee) fixed bug and should change the bug status to '''RESOLVED - FIXED'''. Assignee should add comments to explain the resolution and provide necessary infomation for distro engineer to get the fix and integrate it into distribution.
+
-
# Then distro engineer needs to integrate the fix into distribution and change the status to '''RELEASED - FIXED''' after that.
+
-
# '''QA''' or '''Bug Submitter''' verifies bug fixing with the "how to re-produce" instructions in the original bug report when the bug is marked as "RELEASED - FIXED".
+
-
#* Bug is fixed: QA or reporter should change the bug status to '''VERIFIED''' and add a comment telling which image (e.g. netbook-200901221904) the bug has been fixed in. Don't need to mark it as "CLOSED".
+
-
#* Bug is not fixed: QA or bug submitter should change the bug status to '''REOPENED''' and add a comment telling which image you are using and the bug is still there.
+
-
 
+
-
= Bug Reporting/Follow-up Guideline =
+
-
 
+
-
1. Each bug report is for only one issue. If you find several issues in one test, please separate them into several bugs.
+
-
 
+
-
2. Write a precise summary (bug title). A good summary should be straightforward and precise, to clearly identify the issue. This also helps avoiding duplicate reports.
+
-
 
+
-
3. List the exact steps to reproduce the bug (bash commands, results after executing the commands etc, instead of only paraphrasing). For example, instead of paraphrasing commands by saying:
+
-
      "suspend the system or put system to S3"
+
-
we prefer to see the following exact commands as steps to reproduce the bug:
+
-
      echo mem >/sys/power/state
+
-
 
+
-
4. Describe the current outcome and the expected outcome. Try to avoid writing "See screenshot" or "See attached file" but describe it briefly in words so it can be found when querying for existing bug reports (this helps avoiding duplicate reports).
+
-
 
+
-
5. Reproducibility. For bugs which are not 100% reproducible, please provide an estimate of the probability of reproduction.
+
-
 
+
-
6. Impact to system or user. Provide a description of the actual impact to the system or user.
+
-
 
+
-
7. DO NOT reopen if same defect found again after more than 2 weeks. New bug report with “[REG]” as a prefix in the summary is mandatory.
+
-
 
+
-
8. NEEDINFO bugs should be assigned back to original bug reporter and NEEDINFO status should be removed after requested missing data has been provided in the report. NEEDINFO status should only be used for requests against the original bug reporter.
+
-
 
+
-
9. For "Target Milestone", when reports are in OPEN or RESOLVED status, it means to propose the bug to get fixed for a particular release; When verifying bugs, set the correct milestone where the bug is fixed.
+
-
 
+
-
= MeeGo update bug fix acceptance criteria =
+
-
Generally, the approved update release bugs must be fixed and verified in MeeGo Trunk firstly.
+
-
 
+
-
1. With Bugzilla, an approved release update (eg 1.1) bug must have a MeeGo bug entry in the latest release (eg 1.2) so QA can verify the latest (eg 1.2) Trunk build has the same bug fix on a target vertical platform.
+
-
 
+
-
2. With Bugzilla, an approved release update (eg 1.1) bug should have a bug in the latest release (eg 1.2) as a dependency (or blocker bug) to ensure that the associated bug in the latest release is verified first.
+
-
 
+
-
3. Update release engineer should _not_ accept development patch submission for update release unless QA verified in Trunk build and mark the Bugzilla entry accordingly.
+
-
 
+
-
Of course, there are some exceptions that don't apply this guideline especially for those bugs in old releases are not applicable to latest release. For example:
+
-
 
+
-
1. Bug fixes in old releases' components where the latest release is using a different version;
+
-
 
+
-
2. Components used in old releases that are no longer used in trunk;
+
-
 
+
-
3. Security fixes that don't apply to the version in trunk, etc.
+
-
 
+
-
= Guidelines =
+
-
We like to have fun but we also like the communications to run smoothly. To that end here are some guidelines for participating in  [https://bugs.meego.com Bugzilla].
+
-
 
+
-
==Do be patient with others==
+
-
We have both techies and non-techies here and we both know we don't exactly speak the same language at all times. In case of missing information kindly tell reporters how to provide it, and/or point to https://bugs.meego.com/page.cgi?id=bug-writing.html in case of very vague reports (like "Can't send mail, plz help!!").
+
-
 
+
-
==Do search before you file a bug==
+
-
Try to avoid filing duplicates by [http://bugs.meego.com/query.cgi taking a look] whether your issue has already been filed before.
+
-
 
+
-
==Keep it clean==
+
-
* Don't start endless debates on topics not directly related to the scope of a specific bug report, for example release engineering or marketing. We have [[Community communication|forums and mailing lists]] for that.
+
-
* Avoid quoting complete previous comments by stripping unneeded lines.
+
-
* Avoid answering above the quoted text.
+
-
 
+
-
==The admins Rule==
+
-
Ultimately, the admins reserve the right to block your account. See our [[Community guidelines|process for dealing with violations]] for more details.
+
-
 
+
-
= How to get involved =
+
-
As a non-coder you can get involved by [http://bugs.meego.com/enter_bug.cgi reporting bugs] and [http://wiki.meego.com/Quality/Bugtriage triaging bug reports].
+

Latest revision as of 07:57, 10 June 2011

Contents

Getting started: Creating an Account

All you need is an account to be able to file and comment on reports. The http://bugs.meego.com authentication system is integrated with the http://meego.com authentication system, so the same account is used for both sites. To create a MeeGo account visit the registration page. Soon after you will receive an email with a link that must be visited to confirm your account creation. You will have to change your password on meego.com before attempting to log into Bugzilla.

Reporting an issue

  • (Optional:) Search Bugzilla, to see whether your issue has already been reported.
  • Go to https://bugs.meego.com/enter_bug.cgi and choose the appropriate classification and product in which the bug happens. If you are unsure, just guess.
  • Note: For feature/enhancement requests, please always use the classification "MeeGo Features".
  • On the next page:
    • Choose the Component in which the bug happens.
    • Choose the MeeGo Release in which the problem was found. To find out:
      • MeeGo version: enter in terminal: cat /etc/meego-release
      • Kernel version: enter in terminal: uname -a
    • Choose the Severity according to the definition.
    • Choose the type of Hardware (like Netbook, Nettop, Notebook, Handset, Automotive, TV).
    • If you know, choose the hardware Architecture (For example: N900 is ARM, most Netbooks are IA). Else leave it untouched. In case you test on a N900 please add the keyword N900.
    • In case you have time, check whether some existing keywords apply to your issue, and add them.
    • Enter a Summary describing the bug. A good summary should be straightforward and precise, to clearly identify the issue. This also helps avoiding duplicate reports.
    • In the Description field, please try to fill in all data requested in the template, as text.
      • List the exact steps to reproduce the bug, step by step, click by click, so it is easy to reproduce by testers. For example, instead of paraphrasing commands by saying: "suspend the system or put system to S3" we prefer to see the exact command like "echo mem >/sys/power/state" as steps to reproduce.
      • Describe the current outcome and the expected outcome. Try to avoid writing "See screenshot" or "See attached file" but describe it briefly in words so it can be found when querying for existing bug reports (this helps avoiding duplicate reports).
    • (Optional:) Add an Attachment such as a testcase, or a screenshot in case it makes sense.
    • Click Commit.
  • You are done - Thanks for your report!

Good practices and Guidelines

  • Each bug report is for only one issue. If you find several issues, please separate them into several reports.
  • DO NOT reopen if the same issue is found again after more than 2 weeks. Instead, create a new bug report with “[REG]” as a prefix in the summary.
  • Do search before you file a bug - Try to avoid filing duplicates by taking a look whether your issue has already been filed before.
  • Keep it clean:
    • Don't start endless debates on topics not directly related to the scope of a specific bug report, for example release engineering or marketing. We have forums and mailing lists for that.
    • Avoid quoting complete previous comments by stripping unneeded lines.
    • Avoid answering above the quoted text.
  • The admins Rule: Ultimately, the admins reserve the right to block your account. See our process for dealing with violations for more details.

1.2.0/1.2.1 Bug Tracking Process

In order to align with MeeGo 1.2 release plan , QA add a new value "1.2.0" in "MeeGo Release" field to track 1.2.0 update bugs. So for MeeGo 1.2.0 update and MeeGo 1.2.1 release, we will have 2 field: "1.2.0" and "1.2" to track related bugs:

  • MeeGo Release = 1.2.0: if new bug is found in MeeGo 1.2 final release or 1.2.0 update, new bug should be reported to 1.2.0;
  • MeeGo Release = 1.2: if new bug is found in MeeGo 1.2.1 release or 1.2.1 update, such as 1.2.0.9X image, new bug should be reported to 1.2.1;
  • For current existed bug in 1.2, we will keep the MeeGo release version unchanged. If we found one existed bug in 1.2 need to be fixed in 1.2.0 update, QA will clone one bug to 1.2.0 version;
  • MeeGo 1.2.0 update team will only focus on MeeGo Release = "1.2.0" bug fix and verification.

See Also

Personal tools