Views

Quality/How To Report Bugs

From MeeGo wiki

(Difference between revisions)

Revision as of 08:32, 14 December 2010

1 Getting started
2 Bugzilla Fields
3 Bug Life Cycle
4 Bug Reporting/Follow-up Guideline
5 MeeGo update bug fix acceptance criteria
6 Guidelines
7 How to get involved

Getting started

All you need is an account to be able to file and comment on reports. The http://bugs.meego.com authentication system is integrated with the http://meego.com authentication system, so the same account is used for both sites. To create a MeeGo account visit the registration page. Soon after you will receive an email with a link that must be visited to confirm your account creation. You will have to change your password on meego.com before attempting to log into Bugzilla.

Bugzilla Fields

MeeGo Release

MeeGo Release field defines in which MeeGo version a bug has been found.

How to identify the used MeeGo version?
- MeeGo version: enter in terminal: cat /etc/meego-release
- Kernel version: enter in terminal: uname -a

Version names are defined according to the MeeGo release plan. If a bug was reported already for an older MeeGo version make sure to handle this bugs as described here.

Target Milestone

When bugs are in OPEN or RESOLVED status, it is used by the bug reporter to propose which milestone this bug is expected to be fixed.

--- Undecided
1.0 Propose bug is expected to be fixed in 1.0
1.1 Propose bug is expected to be fixed in 1.1

Importance

Priority

Priority field describes the importance and order in which a bug should be fixed. It helps developers to prioritize their work. By default the priority is set as "Undecided" when reporting a new bug. The bug reporter could propose a priority in a comment. A dedicated bug triage team should set the initial priority for the bug while processing it as early as possible. Final priority setting would be decided by bug triage process. Priorities range from High (most important) to Low (least important).

High: Bug fixing is on-going, or is planned within 2 weeks, no later than the up-coming milestone. Reproducible crash issues, major function loss, issues greatly impact user experience or issues which block other key features to work would fall in to this category.
Medium: Bug fixing is planned before project release, but can't start before HIGH priority items are cleaned up.
Low: Bug fixing is not planned for the up-coming project release. Will re-evaluate the importance in next release.

Severity

The "Severity" field describes the impact of a bug, and the options include:

Critical: crashes, other components are affected
Major: major loss of own function
Normal: regular issue, some loss of functionality under specific circumstances
Trival:cosmetic problem like misspelled words or misaligned text
Enhancement: request for enhancement

Platform

Hardware

Please select the UX (like Netbook, Nettop, Notebook, Handset, Automotive, TV) that you identified the bug in. For bugs which apply to all UX (like bugs for Middleware/Core components), please select "All".

Architecture

Please select the architecture (IA, ARM) that you identified the bug in. For example for N900 Base OS layer bugs this should be set to ARM. For most Netbooks available IA is the right choice. For bugs which apply for multiple architectures, like bugs for middleware or applications, please select "All".

Bug Life Cycle

The reporter submits the bug for a specific component, then the bug would be assigned to a default owner for each component, showing in field "Assign To". The status is NEW initially and priority is Undecided.
Bug triage team will follow the responsibilities defined in bug triage process to process bugs: ensure bug component is correct set, enough information has been collected, bug priority is set etc.
The component owner should follow up when getting a new bug notification:
- If the owner accepts the bug and assigns to himself/herself, he/she should change the status as ASSIGNED.
- The owner may think that it's a wontfix bug. Bug owner should change the status to RESOLVED with resolution INVALID/WONTFIX/WORKSFORME with proper comments.
- If the owner thinks that it is a bug of the other component, he/she needs to add a justification comment.
- If the owner disposes the bug to other individual. The new owner follow the same process.
- If the owner thinks that the bug information is incomplete, he/she could change "status" as NEEDINFO to ask more info from bug reporter.
- The owner does bug analysis and finds this bug comes from upstream. Owner can post a new bug in upstream bugzilla (or ask QA help to do this) and change status to WAITING FOR UPSTREAM. Be sure to add the upstream bugzilla link in moblin bugzilla comment area. When upstream bug is fixed, firstly, integrate the bug fix in distribution image, then mark status to be RELEASED - FIXED.
The owner (bug assignee) fixed bug and should change the bug status to RESOLVED - FIXED. Assignee should add comments to explain the resolution and provide necessary infomation for distro engineer to get the fix and integrate it into distribution.
Then distro engineer needs to integrate the fix into distribution and change the status to RELEASED - FIXED after that.
QA or Bug Submitter verifies bug fixing with the "how to re-produce" instructions in the original bug report when the bug is marked as "RELEASED - FIXED".
- Bug is fixed: QA or reporter should change the bug status to VERIFIED and add a comment telling which image (e.g. netbook-200901221904) the bug has been fixed in. Don't need to mark it as "CLOSED".
- Bug is not fixed: QA or bug submitter should change the bug status to REOPENED and add a comment telling which image you are using and the bug is still there.

Bug Reporting/Follow-up Guideline

1. Each bug report is for only one issue. If you find several issues in one test, please separate them into several bugs.

2. Write a precise summary (bug title). A good summary should be straightforward and precise, to clearly identify the issue. This also helps avoiding duplicate reports.

3. List the exact steps to reproduce the bug (bash commands, results after executing the commands etc, instead of only paraphrasing). For example, instead of paraphrasing commands by saying:

     "suspend the system or put system to S3"

we prefer to see the following exact commands as steps to reproduce the bug:

     echo mem >/sys/power/state

4. Describe the current outcome and the expected outcome. Try to avoid writing "See screenshot" or "See attached file" but describe it briefly in words so it can be found when querying for existing bug reports (this helps avoiding duplicate reports).

5. Reproducibility. For bugs which are not 100% reproducible, please provide an estimate of the probability of reproduction.

6. Impact to system or user. Provide a description of the actual impact to the system or user.

7. DO NOT reopen if same defect found again after more than 2 weeks. New bug report with “[REG]” as a prefix in the summary is mandatory.

8. NEEDINFO bugs should be assigned back to original bug reporter and NEEDINFO status should be removed after requested missing data has been provided in the report. NEEDINFO status should only be used for requests against the original bug reporter.

9. For "Target Milestone", when reports are in OPEN or RESOLVED status, it means to propose the bug to get fixed for a particular release; When verifying bugs, set the correct milestone where the bug is fixed.

MeeGo update bug fix acceptance criteria

Generally, the approved update release bugs must be fixed and verified in MeeGo Trunk firstly.

1. With Bugzilla, an approved release update (eg 1.1) bug must have a MeeGo bug entry in the latest release (eg 1.2) so QA can verify the latest (eg 1.2) Trunk build has the same bug fix on a target vertical platform.

2. With Bugzilla, an approved release update (eg 1.1) bug should have a bug in the latest release (eg 1.2) as a dependency (or blocker bug) to ensure that the associated bug in the latest release is verified first.

3. Update release engineer should _not_ accept development patch submission for update release unless QA verified in Trunk build and mark the Bugzilla entry accordingly.

Of course, there are some exceptions that don't apply this guideline especially for those bugs in old releases are not applicable to latest release. For example:

1. Bug fixes in old releases' components where the latest release is using a different version;

2. Components used in old releases that are no longer used in trunk;

3. Security fixes that don't apply to the version in trunk, etc.

Guidelines

We like to have fun but we also like the communications to run smoothly. To that end here are some guidelines for participating in Bugzilla.

Do be patient with others

We have both techies and non-techies here and we both know we don't exactly speak the same language at all times. In case of missing information kindly tell reporters how to provide it, and/or point to https://bugs.meego.com/page.cgi?id=bug-writing.html in case of very vague reports (like "Can't send mail, plz help!!").

Do search before you file a bug

Try to avoid filing duplicates by taking a look whether your issue has already been filed before.

Keep it clean

Don't start endless debates on topics not directly related to the scope of a specific bug report, for example release engineering or marketing. We have forums and mailing lists for that.
Avoid quoting complete previous comments by stripping unneeded lines.
Avoid answering above the quoted text.

The admins Rule

Ultimately, the admins reserve the right to block your account. See our process for dealing with violations for more details.

@@ Line 51: / Line 51: @@
 = Bug Life Cycle =
 [[file:Bug-life-cycle-meego.PNG]]
+# The reporter submits the bug for a specific component, then the bug would be assigned to a default owner for each component, showing in field "Assign To". The status is '''NEW''' initially and priority is '''Undecided'''.
+# Bug triage team will follow the responsibilities defined in bug triage process to process bugs: ensure bug component is correct set, enough information has been collected, bug priority is set etc.
+# The component owner should follow up when getting a new bug notification:
+#* If the owner accepts the bug and assigns to himself/herself, he/she should change the status as '''ASSIGNED'''.
+#* The owner may think that it's a wontfix bug. Bug owner should change the status to '''RESOLVED''' with resolution '''INVALID/WONTFIX/WORKSFORME''' with proper comments.
+#* If the owner thinks that it is a bug of the other component, he/she needs to add a justification comment.
+#* If the owner disposes the bug to other individual. The new owner follow the same process.
+#* If the owner thinks that the bug information is incomplete, he/she could change "status" as '''NEEDINFO''' to ask more info from bug reporter.
+#* The owner does bug analysis and finds this bug comes from upstream. Owner can post a new bug in upstream bugzilla (or ask QA help to do this) and change status to '''WAITING FOR UPSTREAM'''. Be sure to add the upstream bugzilla link in moblin bugzilla comment area. When upstream bug is fixed, firstly, integrate the bug fix in distribution image, then mark status to be '''RELEASED - FIXED'''.
+# The owner (bug assignee) fixed bug and should change the bug status to '''RESOLVED - FIXED'''. Assignee should add comments to explain the resolution and provide necessary infomation for distro engineer to get the fix and integrate it into distribution.
+# Then distro engineer needs to integrate the fix into distribution and change the status to '''RELEASED - FIXED''' after that.
+# '''QA''' or '''Bug Submitter''' verifies bug fixing with the "how to re-produce" instructions in the original bug report when the bug is marked as "RELEASED - FIXED".
+#* Bug is fixed: QA or reporter should change the bug status to '''VERIFIED''' and add a comment telling which image (e.g. netbook-200901221904) the bug has been fixed in. Don't need to mark it as "CLOSED".
+#* Bug is not fixed: QA or bug submitter should change the bug status to '''REOPENED''' and add a comment telling which image you are using and the bug is still there.
 = Bug Reporting/Follow-up Guideline =