(→Testability Checklist) |
|||
| (9 intermediate revisions not shown) | |||
| Line 1: | Line 1: | ||
| - | For a previous version of the Testability Checklist, see: | + | For a previous version of the Testability Checklist, see: [[Quality/Plans/Testability checklist MeeGo 1.1|Testability checklist MeeGo 1.1]] |
== Introduction == | == Introduction == | ||
| - | + | Describing user interaction is a simple way to describe features, thus, they should be simple to write and to understand. By describing user interaction with the features in Bugzilla, features can contain sufficient information to make test development possible without increasing the workload of the feature owner (as much as the [[Quality/Plans/Testability checklist MeeGo 1.1|Testability checklist MeeGo 1.1]]). The testability checklist used for MeeGo 1.1 contained items that were related to the process of feature creation which should be self evident and not be included in testability checklist. | |
== Describing the Feature == | == Describing the Feature == | ||
| - | The most effective way to create testable features from QA point of view is to | + | The most effective way to create testable features from QA point of view is to describe how the user interacts with the feature. The objective of writing such descriptions is to explain the feature to the extent that a test to verify the feature can be written. To that end, after writing a description, ask yourself this question: |
| - | '''Can someone write a test using the information in this | + | '''Can someone write a test using the information in this description?''' |
| - | If the answer is 'yes' then the | + | If the answer is 'yes' then the description is complete, if the answer is 'no', then it may be worth it to consider rewriting the it or (if this seems like it will not help) add extra information (see the extra information section). |
| - | Some of the use case may not be fully applicable for Core OS QA but in most of the | + | Some of the use case may not be fully applicable for Core OS QA but in most of the use cases can be divided to smaller particles and can be considered as enablers for the main use case. E.g. Browsing use case can be verified using these smaller particles (enablers) as follows: |
* Scan and connect to WLAN Access Point and conduct data transfer | * Scan and connect to WLAN Access Point and conduct data transfer | ||
* Download specific file over WLAN with performance measurement (Throughput) | * Download specific file over WLAN with performance measurement (Throughput) | ||
| - | == | + | == Structuring the Description == |
| - | There is a good wikipedia article on the concept of the user | + | There is a good wikipedia article on the concept of the describing a feature from a user's point of view [http://en.wikipedia.org/wiki/User_story here.] While there are some differences from the article because MeeGo is not a small project that is being implemented by one team. |
| - | Three basic forms for a | + | Three basic forms for a description are: |
* I as a <user type/role> want to <perform action> and <result/goal> | * I as a <user type/role> want to <perform action> and <result/goal> | ||
* "As a <role>, I want <goal/desire> so that <benefit>" (from wikipedia) | * "As a <role>, I want <goal/desire> so that <benefit>" (from wikipedia) | ||
| Line 28: | Line 28: | ||
=== Dissected Example === | === Dissected Example === | ||
| - | The following feature can be found [http://bugs.meego.com/show_bug.cgi?id=6724 here] and can be considered a good example of a | + | The following feature can be found [http://bugs.meego.com/show_bug.cgi?id=6724 here] and can be considered a good example of a feature description: |
''User is browsing multimedia content in browser and a system event triggers a sound, e.g. battery low or alarm. System sound is mixed in with the browser audio.'' | ''User is browsing multimedia content in browser and a system event triggers a sound, e.g. battery low or alarm. System sound is mixed in with the browser audio.'' | ||
| Line 39: | Line 39: | ||
== Extra Information == | == Extra Information == | ||
| - | The basic | + | The basic description from a user's point of view does not usually include the provision for extra information (things that the user would not see). This is usually not a problem for a single, coherent, project. In a project with as many components (from as many sources) as MeeGo, extra information may be required. |
| - | NOTE: Adding extra information is not a requirement, if your feature is adequately described by the | + | NOTE: Adding extra information is not a requirement, if your feature is adequately described by the description itself, then, stop there. |
Some suggestions on what extra information to add: | Some suggestions on what extra information to add: | ||
| Line 52: | Line 52: | ||
'''Mandatory:''' | '''Mandatory:''' | ||
| - | * '''Can I write a test based on the information in this | + | * '''Can I write a test based on the information in this description?''' <br /> |
** Is the goal defined clearly? <br /> | ** Is the goal defined clearly? <br /> | ||
** Is the procedure required to reach the goal clearly defined? | ** Is the procedure required to reach the goal clearly defined? | ||
| - | ** Is the | + | ** Is the description short enough? |
** Is there any further information that is easy to put down and may help a test developer? | ** Is there any further information that is easy to put down and may help a test developer? | ||
* '''Are relevant technology parameters available (Sample rates, Bit Rates, supported picture sizes, etc.) described?''' | * '''Are relevant technology parameters available (Sample rates, Bit Rates, supported picture sizes, etc.) described?''' | ||
* '''For Core OS: Is API Under test defined clearly?'''<br /> | * '''For Core OS: Is API Under test defined clearly?'''<br /> | ||
| - | '''Optional:''' | + | '''Optional (and highly recommended):''' |
| - | * Is there | + | * Is there a reference to API documentation of API(s) implementing features? Location where more information is available? <br /> |
* Is there '''Performance''' aspects or requirements (response time, latency, raw throughput) that should be verified concerning this feature? <br /> | * Is there '''Performance''' aspects or requirements (response time, latency, raw throughput) that should be verified concerning this feature? <br /> | ||
* Is there '''Reliability''' aspects or requirements (expected use rate per day, how long feature should survive in stress situation) that should be verified? | * Is there '''Reliability''' aspects or requirements (expected use rate per day, how long feature should survive in stress situation) that should be verified? | ||
| Line 67: | Line 67: | ||
* Is there significant interactions (e.g. local music playback while browsing) with other features which should be taken into account when designing test cases concerning this feature? | * Is there significant interactions (e.g. local music playback while browsing) with other features which should be taken into account when designing test cases concerning this feature? | ||
| - | == | + | == Master Features == |
| - | + | Master features are features that are made up of other (sub-)features. As such, if all the sub-features are implemented, the master feature can be considered to be implemented. The same applies for testability: If all the sub-features are testable, then the master feature is testable. | |
| - | - | + | If a master feature has all if its sub-features set to 'testable', then the master feature can be marked as 'testable'. If any of the sub-features are marked as 'not testable', the master feature should be marked as 'not testable'. |
| + | |||
| + | == Conclusion == | ||
| + | By creating descriptions at user level for each feature, the feature owner can avoid having to write a [[Quality/Plans/Testability checklist MeeGo 1.1|Testability checklist MeeGo 1.1]] as was required for MeeGo 1.1. Adequately written descriptions should make life easier for both the product owner and the test developer. | ||
For a previous version of the Testability Checklist, see: Testability checklist MeeGo 1.1
Contents |
Describing user interaction is a simple way to describe features, thus, they should be simple to write and to understand. By describing user interaction with the features in Bugzilla, features can contain sufficient information to make test development possible without increasing the workload of the feature owner (as much as the Testability checklist MeeGo 1.1). The testability checklist used for MeeGo 1.1 contained items that were related to the process of feature creation which should be self evident and not be included in testability checklist.
The most effective way to create testable features from QA point of view is to describe how the user interacts with the feature. The objective of writing such descriptions is to explain the feature to the extent that a test to verify the feature can be written. To that end, after writing a description, ask yourself this question:
Can someone write a test using the information in this description?
If the answer is 'yes' then the description is complete, if the answer is 'no', then it may be worth it to consider rewriting the it or (if this seems like it will not help) add extra information (see the extra information section).
Some of the use case may not be fully applicable for Core OS QA but in most of the use cases can be divided to smaller particles and can be considered as enablers for the main use case. E.g. Browsing use case can be verified using these smaller particles (enablers) as follows:
There is a good wikipedia article on the concept of the describing a feature from a user's point of view here. While there are some differences from the article because MeeGo is not a small project that is being implemented by one team.
Three basic forms for a description are:
It is important to define the goal clearly. This allows the success criteria of the test to be clearly defined.
It is also important to define the set of actions leading up to the goal clearly. For example, if the user needs to tilt the device from horizontal to vertical for an action to occur, this needs to be described.
The following feature can be found here and can be considered a good example of a feature description:
User is browsing multimedia content in browser and a system event triggers a sound, e.g. battery low or alarm. System sound is mixed in with the browser audio.
We can see in this example:
From the above description, a test can be written.
The basic description from a user's point of view does not usually include the provision for extra information (things that the user would not see). This is usually not a problem for a single, coherent, project. In a project with as many components (from as many sources) as MeeGo, extra information may be required.
NOTE: Adding extra information is not a requirement, if your feature is adequately described by the description itself, then, stop there.
Some suggestions on what extra information to add:
Even though Testability Checklist is tool for QA teams, persons creating features in Bugzilla should have a thorough look to this in order to make feature flow fluently. Testability Checklist contains mandatory parts (to fulfill the criteria to have Testability marked as YES) and optional part which is recommended for one creating features to consider seriously if there is such information available that will help QA teams to conduct their work at appropriate level, leading well tested MeeGo features and Releases.
Mandatory:
Optional (and highly recommended):
Master features are features that are made up of other (sub-)features. As such, if all the sub-features are implemented, the master feature can be considered to be implemented. The same applies for testability: If all the sub-features are testable, then the master feature is testable.
If a master feature has all if its sub-features set to 'testable', then the master feature can be marked as 'testable'. If any of the sub-features are marked as 'not testable', the master feature should be marked as 'not testable'.
By creating descriptions at user level for each feature, the feature owner can avoid having to write a Testability checklist MeeGo 1.1 as was required for MeeGo 1.1. Adequately written descriptions should make life easier for both the product owner and the test developer.