Shuangeeer (Talk | contribs) (→The Process) |
|||
| Line 5: | Line 5: | ||
Current testability analysis provides no real value. Marking a feature as testable does not mean that an adequate test can be written from the description. The process of marking a feature as testable will now require feedback from the feature owner. This will introduce an extra step to the process. The process now looks like this: | Current testability analysis provides no real value. Marking a feature as testable does not mean that an adequate test can be written from the description. The process of marking a feature as testable will now require feedback from the feature owner. This will introduce an extra step to the process. The process now looks like this: | ||
# The feature owner creates a [FEA] bug and describes a feature | # The feature owner creates a [FEA] bug and describes a feature | ||
| - | # The QA contact comments on the feature with a testability analysis and a theoretical test description. | + | # The QA contact comments on the feature with a testability analysis and a theoretical test description/check points. |
| - | ## | + | ## If the feature description is with enough information and QA contact knows well about how to test it, QA comments with check points of test cases and set testability to Yes. Testability review ends unless there is new change in the feature. |
| - | ## | + | ## If the feature description is not clear or ambiguous but QA still knows some ways to cover the feature, QA comments with check points from his/her understanding, and ask feature owner to review. Go to 3. |
| - | # Feature owner comments on the theoretical test. | + | ## If the feature description is limited and difficult for QA to identify the feature, QA contact comments asking more information and marking NOT testable. Either the feature description is improved by PM or QA's questions are answered by feature owner, the testability should be set back to ---, thus QA contact review it again(step 1&2 above). If no feedback from feature owner or PM, QA keeps Testability as No. |
| + | ## For MASTER feature, QA contact set Testability field to N/A. Testability review ends unless there is new change in the feature. | ||
| + | # Feature owner comments on the theoretical test, go to 4 | ||
| + | ## Feature owner is encouraged to provided best known methods to validate the feature. | ||
| + | ## Feedback or suggestion from feature owner that may help QA improve coverage or quality of QA's test cases are mostly welcomed. | ||
# If the test is adequate, the feature is marked as testable (set testability field to 'Yes'), if it is not, go back to 2. | # If the test is adequate, the feature is marked as testable (set testability field to 'Yes'), if it is not, go back to 2. | ||
| + | ## After feature owner agrees with QA's check points, QA contact set Testability to Yes. | ||
| + | ## If there is no feedback from feature owner for more than 2 weeks(or more?), QA contact considers his/her check points is adequate to cover the feature, thus mark testable. | ||
| + | |||
=== Testability Analysis === | === Testability Analysis === | ||
Contents |
In the past, a QA contact marked a feature with a yes/no statement and a checklist. This does not add any useful information to the process and provides little useful feedback for the feature owner. The objective of this document is to lay down guidelines for feature testability comments that will allow for discussion about test development with the feature owner.
Current testability analysis provides no real value. Marking a feature as testable does not mean that an adequate test can be written from the description. The process of marking a feature as testable will now require feedback from the feature owner. This will introduce an extra step to the process. The process now looks like this:
The testability checklist can be found here: Feature Testability Checklist
The QA Contact should analyze the feature description and compare it to the requirements of the Testability Checklist. A yes/no verdict should be supplied (in the comment), and, (especially if the verdict is no) a justification should be provided for the verdict. If the feature cannot be tested, suggesting a method to rectify the situation is strongly suggested.
In general, the person who would know how best to test a feature is the feature developer themselves. The suggested way to get their input about the feature and have the information easily accessible is to suggest a test while commenting on the testability.
The suggested test can consist of a name and/or a test description.
By suggesting a test to the feature developer, the barriers for participating in the test development process are lowered. While the test developer may not have time or be inclined to describe a test on their own, if a test is suggested, they can easily comment on the suggested test. This makes it easier to catch a misinterpretation of the feature in an early stage, and can allow for better, more complete, tests.
A feature should only be marked as Testable after the feature owner has confirmed that the test is adequate to test his/her feature.
By suggesting a test (especially with a test case name), feature verification will be easier as all the information required for feature verification is already present in the feature bugzilla.
This template is suggested for use when commenting features:
Testability statement as per: http://wiki.meego.com/Quality/TestabilityChecklist 1. Can I write a test based on the information in this description? Yes/No. Comment? 2. Are relevant technology parameters available (Sample rates, Bit Rates, supported picture sizes, etc.) described? Yes/No. Comment? 3. For Core OS: Is API Under test defined clearly? Yes/No. Comment? Summary: Testability is OK/not OK. Required from assignee to fix Testability: List here if Testability is NOT OK QA Proposed test as per: http://wiki.meego.com/Quality/Testability-commenting Test would ... If ... the test would pass. Feature Assignee: Please confirm if this test is adequate to verify that the feature works. The feature has been marked as not testable. Testability will be set to 'Yes' after the Feature Assignee has commented on the test.
By allowing a little more time for testability studies, feature owners/developers can be provided with a clear view on the testing of their feature. This can help avoid misunderstandings with regard to feature functions and can make feature verification easier.