Quality/Test case template

2011-02-18T06:58:04Z

Jkunnari: /* Fields in Test Case Template */

== Test Case Template ==
Purpose of test case template is to provide sufficient amount of information about test cases for the users. Intention is to keep test case template as simple as possible. Test case template does not contain any information about test execution schedule, intervals how often it will be executed or so. This is defined in test execution management tool(s) what ever they might be. The same template should be used despite the test case concerning UX or Core.

[[File:TCTemplate.JPG]]

== Fields in Test Case Template ==

Template contains following fields:

* '''TC Title:''' (Mandatory) ''<Descriptive title for test case>'' '''Shall be used by QA team'''
* '''TC ID:''' (Optional) '' <Test case unique identifier>'' '''Could be used by QA team'''
** Can consist of characters, numbers or if test case name is unique this can be used as well
* '''Requirement:''' (Optional) ''<Requirement reference>'' '''Should be used by QA team'''
** Field to have feature or bug ID's as comma separated list. This enables the linkage between tests and bugs.meego.com or any other requirement system items.
* '''Type:''' (Optional) ''<Type of test case>'' '''Shall be used by QA team''' ''update description''
** Values: please refer to http://wiki.meego.com/Quality/Test_areas_and_types
* '''Domain:''' (Optional) <Vertical> '''Shall be used by QA team'''
** Domains are defined in http://meego.com/developers/meego-architecture/meego-architecture-domain-view
* '''Feature:''' (Optional) '''Shall be used by QA team if Requirement is empty - otherwise Could be used'''
** Free text describing the functionality tested.
* '''Subfeature:''' (Optional) '''Could be used by QA team'''
** Can be used when test case designer wants to identify more detailed level what is tested.
* '''Component:''' (Optional) ''<Bugzilla Component>'' '''Shall be used by QA team'''
* '''Execution Type:''' (Optional) ''<Manual, Auto>'' '''Should be used by QA team'''
* '''Test Case State:''' (Optional) ''<Design/Ready/Approved>'' '''Could be used by QA team'''
* '''Description:'''
** Purpose (Optional): <Test case purpose here. What is tried to achieve with this specific TC> '''Shall be used by QA team'''
** Method (Optional): <Short description of TC method here, e.g., how measurement is done and what the result describes. Could be a link to another document. This field is optional> '''Could be used by QA team'''
** References (Optional) :<Link to web, wiki or similar place where more detailed information is available (if any) for this TC> '''Could be used by QA team'''
** Pre/Post-conditions (Optional): <Specific description of things or settings needed prior to this TC run. E.g. memory card in device and audio clip at path /home/user/audioclips> '''Could be used by QA team'''
** Run instructions (Optional): <Run instructions specific for this TC. Describe instructions by steps (n) – refer to PASS / FAIL criteria for this step.> '''Shall be used by QA team'''
** Pass/Fail criteria (Optional): <What are the criterion that TC shall fulfil in order to consider TC as passed. Describe PASS / FAIL criteria for each run step.> '''Shall be used by QA team'''
** Test Environment (Optional): <Link to a document describing manual or automatic test environment. No test cases having dependencies to corporate network or other proprietary SW / HW> '''Could be used by QA team'''
** Required test data (Optional): <Description of specific test data / parameters for this TC with unambiguous file references> '''Could be used by QA team'''
** Change history (Optional): <Description of modifications done to this TC, with name of author. Change history of last 5 changes at least> '''Could be used by QA team'''

=== Example 1, Core Test Case for eMMC ===

* '''TC Title:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''TC ID:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:'''
* '''Subfeature:''' copy
* '''Component:''' Kernel
* '''Execution Type:''' Auto
* '''Test Case State:''' Approved
* '''Description:'''
** Purpose: This case tests file can be copied from eMMC to eMMC
** Method: Test case copies file from eMMC to eMMC, file block size 1kB, block count 1000
** References: None
** Pre/Post-conditions: Test asset and MIN test frame Work is installed to the device. Memory card must be mounted in device. Test must be executed as root.
** Run instructions: Start 'min' in the shell of the device. Select a case, observe device, and check the test result from min and the log files. Or use command line format 'min -c -t <test case name>'.
** Pass/Fail criteria: Test case returns passed if operation was succesfull. Result is logged into /var/log/tests/."
** Test Environment: Handset device
** Required test data: None
** Change history:
*** 11 May 2010. (v.0.1.8) Added steps to test.xml. Tommi Toropainen
*** 21 Apr 2010. (v.0.1.7) Added new steps to OTS execution to get results on Test reporting tools, Removed tmp filesystem tests from OTS. Tommi Toropainen
*** 20 Apr 2010. (v.0.1.6) Changed filesystem partitions in configure file, Added Security file system test case. Tommi Toropainen
*** 15 Mar 2010. (v.0.1.5) Changed mass storage cases to use 1,6GB files. Tommi Toropainen
*** 1 Mar 2010. (0.1.4) added unmounting before formatting (do not format target or source by default) Tommi Toropainen

=== Example 2, UX Test Case for Browser ===

* '''TC Title:''' Watching flash video via browser by using WLAN
* '''TC ID:''' PJ114257
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:''' Mozilla Fennec Browser
* '''Subfeature:''' copy
* '''Component:'''
* '''Execution Type:''' Manual
* '''Test Case State:''' Design
* '''Description:'''
** Purpose: Purpose of this test case is to verify that it is possible to watch flash video via browser by using WLAN.
** Method: Test case is run manually
** References: www.youtube.com
** Pre/Post-conditions: WLAN Access Point is available. Browser application is opened
** Run instructions:
:: 1) Open webpage where flash videos can be watched (youtube.com)
Expected: Web page is displayed with flash videos
:: 2) Start watching some flash video on the web page.
Expected: Flash video can be watched with proper audio and video.
:: 3) Stop watching flash video.
Expected: Flash video playback should be stopped and web page with videos should be displayed.
:: 4) Close the browser application.
Expected: Browser application should be closed successfully and home page is displayed.
** Pass/Fail criteria: It is possible to watch flash videos via browser by using WLAN
** Test Environment: Open WLAN network
** Required test data: None
** Change history:
*** 9 Jun 2010. (v.0.1) Test case skeleton created, basic information included. Petri Jylhä
*** 6 Sep 2010. Test Case should contain simple and meaningful steps and coverage should be good enough. Added expected result for each step. Reena

Quality/Test case template

2011-02-18T06:54:16Z

Jkunnari: /* Fields in Test Case Template */

== Test Case Template ==
Purpose of test case template is to provide sufficient amount of information about test cases for the users. Intention is to keep test case template as simple as possible. Test case template does not contain any information about test execution schedule, intervals how often it will be executed or so. This is defined in test execution management tool(s) what ever they might be. The same template should be used despite the test case concerning UX or Core.

[[File:TCTemplate.JPG]]

== Fields in Test Case Template ==

Template contains following fields:

* '''TC Title:''' (Mandatory) ''<Descriptive title for test case>'' '''Shall be used by QA team'''
* '''TC ID:''' (Optional) '' <Test case unique identifier>'' '''Could be used by QA team'''
** Can consist of characters, numbers or if test case name is unique this can be used as well
* '''Requirement:''' (Optional) ''<Requirement reference>'' '''Should be used by QA team'''
** Field to have feature or bug ID's as comma separated list. This enables the linkage between tests and bugs.meego.com or any other requirement system items.
* '''Type:''' (Optional) ''<Type of test case>'' '''Shall be used by QA team''' ''update description''
** Values: Certification, Internationalisation, Localisation, System Integration, Variant Testing, 3PO Testing, Services Testing, Functional, Latency, Response time, Frame rate measurement, Technical Benchmark, Benchmark, Memory consumption measurement, Throughput measurement, Load measurement, Power management, Low resource, Recovery, Iterative, Long-lasting, FIT (Feature Interaction Testing),User eXperience testing, Mobility testing, Long Period testing (LPT)
* '''Domain:''' (Optional) <Vertical> '''Shall be used by QA team'''
** Domains are defined in http://meego.com/developers/meego-architecture/meego-architecture-domain-view
* '''Feature:''' (Optional) ''<Bugzilla Feature>'' '''Shall be used by QA team if Requirement is empty - otherwise Could be used'''
* '''Subfeature:''' (Optional) '''Could be used by QA team'''
** Can be used when test case designer wants to identify more detailed level what is tested.
* '''Component:''' (Optional) ''<Bugzilla Component>'' '''Shall be used by QA team'''
* '''Execution Type:''' (Optional) ''<Manual, Auto>'' '''Should be used by QA team'''
* '''Test Case State:''' (Optional) ''<Design/Ready/Approved>'' '''Could be used by QA team'''
* '''Description:'''
** Purpose (Optional): <Test case purpose here. What is tried to achieve with this specific TC> '''Shall be used by QA team'''
** Method (Optional): <Short description of TC method here, e.g., how measurement is done and what the result describes. Could be a link to another document. This field is optional> '''Could be used by QA team'''
** References (Optional) :<Link to web, wiki or similar place where more detailed information is available (if any) for this TC> '''Could be used by QA team'''
** Pre/Post-conditions (Optional): <Specific description of things or settings needed prior to this TC run. E.g. memory card in device and audio clip at path /home/user/audioclips> '''Could be used by QA team'''
** Run instructions (Optional): <Run instructions specific for this TC. Describe instructions by steps (n) – refer to PASS / FAIL criteria for this step.> '''Shall be used by QA team'''
** Pass/Fail criteria (Optional): <What are the criterion that TC shall fulfil in order to consider TC as passed. Describe PASS / FAIL criteria for each run step.> '''Shall be used by QA team'''
** Test Environment (Optional): <Link to a document describing manual or automatic test environment. No test cases having dependencies to corporate network or other proprietary SW / HW> '''Could be used by QA team'''
** Required test data (Optional): <Description of specific test data / parameters for this TC with unambiguous file references> '''Could be used by QA team'''
** Change history (Optional): <Description of modifications done to this TC, with name of author. Change history of last 5 changes at least> '''Could be used by QA team'''

=== Example 1, Core Test Case for eMMC ===

* '''TC Title:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''TC ID:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:'''
* '''Subfeature:''' copy
* '''Component:''' Kernel
* '''Execution Type:''' Auto
* '''Test Case State:''' Approved
* '''Description:'''
** Purpose: This case tests file can be copied from eMMC to eMMC
** Method: Test case copies file from eMMC to eMMC, file block size 1kB, block count 1000
** References: None
** Pre/Post-conditions: Test asset and MIN test frame Work is installed to the device. Memory card must be mounted in device. Test must be executed as root.
** Run instructions: Start 'min' in the shell of the device. Select a case, observe device, and check the test result from min and the log files. Or use command line format 'min -c -t <test case name>'.
** Pass/Fail criteria: Test case returns passed if operation was succesfull. Result is logged into /var/log/tests/."
** Test Environment: Handset device
** Required test data: None
** Change history:
*** 11 May 2010. (v.0.1.8) Added steps to test.xml. Tommi Toropainen
*** 21 Apr 2010. (v.0.1.7) Added new steps to OTS execution to get results on Test reporting tools, Removed tmp filesystem tests from OTS. Tommi Toropainen
*** 20 Apr 2010. (v.0.1.6) Changed filesystem partitions in configure file, Added Security file system test case. Tommi Toropainen
*** 15 Mar 2010. (v.0.1.5) Changed mass storage cases to use 1,6GB files. Tommi Toropainen
*** 1 Mar 2010. (0.1.4) added unmounting before formatting (do not format target or source by default) Tommi Toropainen

=== Example 2, UX Test Case for Browser ===

* '''TC Title:''' Watching flash video via browser by using WLAN
* '''TC ID:''' PJ114257
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:''' Mozilla Fennec Browser
* '''Subfeature:''' copy
* '''Component:'''
* '''Execution Type:''' Manual
* '''Test Case State:''' Design
* '''Description:'''
** Purpose: Purpose of this test case is to verify that it is possible to watch flash video via browser by using WLAN.
** Method: Test case is run manually
** References: www.youtube.com
** Pre/Post-conditions: WLAN Access Point is available. Browser application is opened
** Run instructions:
:: 1) Open webpage where flash videos can be watched (youtube.com)
Expected: Web page is displayed with flash videos
:: 2) Start watching some flash video on the web page.
Expected: Flash video can be watched with proper audio and video.
:: 3) Stop watching flash video.
Expected: Flash video playback should be stopped and web page with videos should be displayed.
:: 4) Close the browser application.
Expected: Browser application should be closed successfully and home page is displayed.
** Pass/Fail criteria: It is possible to watch flash videos via browser by using WLAN
** Test Environment: Open WLAN network
** Required test data: None
** Change history:
*** 9 Jun 2010. (v.0.1) Test case skeleton created, basic information included. Petri Jylhä
*** 6 Sep 2010. Test Case should contain simple and meaningful steps and coverage should be good enough. Added expected result for each step. Reena

Quality/Test case template

2011-02-18T06:45:47Z

Jkunnari: /* Fields in Test Case Template */

== Test Case Template ==
Purpose of test case template is to provide sufficient amount of information about test cases for the users. Intention is to keep test case template as simple as possible. Test case template does not contain any information about test execution schedule, intervals how often it will be executed or so. This is defined in test execution management tool(s) what ever they might be. The same template should be used despite the test case concerning UX or Core.

[[File:TCTemplate.JPG]]

== Fields in Test Case Template ==

Template contains following fields:

* '''TC Title:''' (Mandatory) ''<Descriptive title for test case>'' '''Shall be used by QA team'''
* '''TC ID:''' (Optional) '' <Test case unique identifier>'' '''Could be used by QA team'''
** Can consist of characters, numbers or if test case name is unique this can be used as well
* '''Requirement:''' (Optional) ''<Requirement reference>'' '''Should be used by QA team''' ''update description''
* '''Type:''' (Optional) ''<Type of test case>'' '''Shall be used by QA team''' ''update description''
** Values: Certification, Internationalisation, Localisation, System Integration, Variant Testing, 3PO Testing, Services Testing, Functional, Latency, Response time, Frame rate measurement, Technical Benchmark, Benchmark, Memory consumption measurement, Throughput measurement, Load measurement, Power management, Low resource, Recovery, Iterative, Long-lasting, FIT (Feature Interaction Testing),User eXperience testing, Mobility testing, Long Period testing (LPT)
* '''Domain:''' (Optional) <Vertical> '''Shall be used by QA team'''
** Values: Common, Handset, Netbook, In-Vehicle, Connected TV, Media Phone ''update description''
* '''Feature:''' (Optional) ''<Bugzilla Feature>'' '''Shall be used by QA team if Requirement is empty - otherwise Could be used'''
* '''Subfeature:''' (Optional) '''Could be used by QA team'''
** Can be used when test case designer wants to identify more detailed level what is tested.
* '''Component:''' (Optional) ''<Bugzilla Component>'' '''Shall be used by QA team'''
* '''Execution Type:''' (Optional) ''<Manual, Auto>'' '''Should be used by QA team'''
* '''Test Case State:''' (Optional) ''<Design/Ready/Approved>'' '''Could be used by QA team'''
* '''Description:'''
** Purpose (Optional): <Test case purpose here. What is tried to achieve with this specific TC> '''Shall be used by QA team'''
** Method (Optional): <Short description of TC method here, e.g., how measurement is done and what the result describes. Could be a link to another document. This field is optional> '''Could be used by QA team'''
** References (Optional) :<Link to web, wiki or similar place where more detailed information is available (if any) for this TC> '''Could be used by QA team'''
** Pre/Post-conditions (Optional): <Specific description of things or settings needed prior to this TC run. E.g. memory card in device and audio clip at path /home/user/audioclips> '''Could be used by QA team'''
** Run instructions (Optional): <Run instructions specific for this TC. Describe instructions by steps (n) – refer to PASS / FAIL criteria for this step.> '''Shall be used by QA team'''
** Pass/Fail criteria (Optional): <What are the criterion that TC shall fulfil in order to consider TC as passed. Describe PASS / FAIL criteria for each run step.> '''Shall be used by QA team'''
** Test Environment (Optional): <Link to a document describing manual or automatic test environment. No test cases having dependencies to corporate network or other proprietary SW / HW> '''Could be used by QA team'''
** Required test data (Optional): <Description of specific test data / parameters for this TC with unambiguous file references> '''Could be used by QA team'''
** Change history (Optional): <Description of modifications done to this TC, with name of author. Change history of last 5 changes at least> '''Could be used by QA team'''

=== Example 1, Core Test Case for eMMC ===

* '''TC Title:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''TC ID:''' DATAFLOW-FS-Copy-File-eMMC-to-MMC
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:'''
* '''Subfeature:''' copy
* '''Component:''' Kernel
* '''Execution Type:''' Auto
* '''Test Case State:''' Approved
* '''Description:'''
** Purpose: This case tests file can be copied from eMMC to eMMC
** Method: Test case copies file from eMMC to eMMC, file block size 1kB, block count 1000
** References: None
** Pre/Post-conditions: Test asset and MIN test frame Work is installed to the device. Memory card must be mounted in device. Test must be executed as root.
** Run instructions: Start 'min' in the shell of the device. Select a case, observe device, and check the test result from min and the log files. Or use command line format 'min -c -t <test case name>'.
** Pass/Fail criteria: Test case returns passed if operation was succesfull. Result is logged into /var/log/tests/."
** Test Environment: Handset device
** Required test data: None
** Change history:
*** 11 May 2010. (v.0.1.8) Added steps to test.xml. Tommi Toropainen
*** 21 Apr 2010. (v.0.1.7) Added new steps to OTS execution to get results on Test reporting tools, Removed tmp filesystem tests from OTS. Tommi Toropainen
*** 20 Apr 2010. (v.0.1.6) Changed filesystem partitions in configure file, Added Security file system test case. Tommi Toropainen
*** 15 Mar 2010. (v.0.1.5) Changed mass storage cases to use 1,6GB files. Tommi Toropainen
*** 1 Mar 2010. (0.1.4) added unmounting before formatting (do not format target or source by default) Tommi Toropainen

=== Example 2, UX Test Case for Browser ===

* '''TC Title:''' Watching flash video via browser by using WLAN
* '''TC ID:''' PJ114257
* '''Requirement'''
* '''Type:''' Functional
* '''Domain:''' Handset
* '''Feature:''' Mozilla Fennec Browser
* '''Subfeature:''' copy
* '''Component:'''
* '''Execution Type:''' Manual
* '''Test Case State:''' Design
* '''Description:'''
** Purpose: Purpose of this test case is to verify that it is possible to watch flash video via browser by using WLAN.
** Method: Test case is run manually
** References: www.youtube.com
** Pre/Post-conditions: WLAN Access Point is available. Browser application is opened
** Run instructions:
:: 1) Open webpage where flash videos can be watched (youtube.com)
Expected: Web page is displayed with flash videos
:: 2) Start watching some flash video on the web page.
Expected: Flash video can be watched with proper audio and video.
:: 3) Stop watching flash video.
Expected: Flash video playback should be stopped and web page with videos should be displayed.
:: 4) Close the browser application.
Expected: Browser application should be closed successfully and home page is displayed.
** Pass/Fail criteria: It is possible to watch flash videos via browser by using WLAN
** Test Environment: Open WLAN network
** Required test data: None
** Change history:
*** 9 Jun 2010. (v.0.1) Test case skeleton created, basic information included. Petri Jylhä
*** 6 Sep 2010. Test Case should contain simple and meaningful steps and coverage should be good enough. Added expected result for each step. Reena

Quality/Plans/Test-plan-template

2011-02-17T06:50:30Z

Jkunnari: /* Traceability of Features */

Proposal - please contribute by editing this page or posting your comments on [[Talk:TestPlanTemplate|discussion area]] for this page.

Note that this is template giving the frame from actual test plan.
The intent is to list areas/things which one need to consider when planning needed QA activities for certain MeeGo Release (e.g. 1.2, 1.3).

== Introduction ==

=== Purpose ===

The purpose of the Test Plan is to introduce the testing strategy, scope, sub-plans and testing activities done or supported by QA team for MeeGo X.X release.

The intended readers for this document are people interested about the reasoning behind the actual activities done by QA team.

<please write more>

=== Objectives ===

The Test Plan is intended to provide the vehicle for QA team to confirm the functionality and completeness of the MeeGo X.X release. It is expected that the satisfaction of the complete series of criteria contained in this plan will signify successful functionality of the integrated release.

<please list more objectives>

=== Goal ===

The goal is to deliver a product with no open bugs with a severity level of critical and a minimal number of open bugs with severity level major.

<please add other goals>

=== Document Overview ===

==== Test Cases ====

During the the release phase aiming to final release, test cases will be executed. The test cases are described in test documents which are in gitorius at meego.gitorious.org/meego-quality-assurance. The actual test case code might be (depending on the license type and test level) part of the source package. Each test case also contains test specific criteria which decide upon test case success or failure.

Test cases shall fulfill the definitions given in [[Quality/Test_case_template|Test case template]]

<add other sources and definitions used>

==== Test Case Result ====

Verdict shall be assigned for a test case after execution according [[Quality/Glossary#Test_case_verdict|test case verdict instructions]].

==== Traceability of Features ====

For each feature listed bugs.meego.com, in principle, one or more test cases should exist (see detailed test plans). This information should be defined in the test case according instructions given in [[Quality/Test_case_template|Test Case Template]]

==== Test Case Documentation ====

Each test case described in the detailed test plan contains the fields according [[Quality/Test_case_template|Test Case Template]] and [http://meego.gitorious.org/meego-quality-assurance/test-definition test-definition]

== Overall test strategy ==

=== Introduction ===

Components can be tested by a combination of component (unit, module and component integration testing put together), integration and system tests.

For a system test the component is typically tested by launching the application in the test environment and by executing functionality of the component using the User Interface, and by verifying the outcome against an 'expected result'.
In cases where manual verification is required the expected outcome is specified in the test case.

Where possible all or parts of the test steps are automated, but in all cases there will be a manually executed 'Acceptance Test' for each GUI component in which the module will, at least, be verified against a standard checklist.

For an Integration test the component is typically tested by launching the component (if it has a GUI), or by launching other applications that make use of the tested component, inside the test environment and by executing functionality of the component using the User Interface. Where applicable, parts of the system may be replaced by stubs to partially insulate the component from its environment.

Unit tests are so named because they each test one unit of code. This type of tests are usually written by developers as they work on code (white-box style), to ensure that the specific function is working as expected. Whether a module of code has hundreds of unit tests or only five is irrelevant. One function might have multiple tests, to catch corner cases or other branches in the code. A test suite of unit tests should never cross process boundaries in a program, let alone network connections. Doing so introduces delays that make tests run slowly and discourage developers from running the whole suite.

Module test tests a module of code similar way than unit tests - the difference is that test are generated black-box style (without reference to the internal structure of the module).

Introducing dependencies on external modules or data also turns unit tests into component integration tests. If one module misbehaves in a chain of interrelated modules, it is not so immediately clear where to look for the cause of the failure. Tests are checking ready made part of the component with simulators (stubs, test drivers) for missing external dependencies.

=== Priorization ===

Tests are written on a highest priority first basis, and secondly on broad coverage.

<add other priorization rules used by vertical or project>

=== Coverage ===

At this stage we are aiming towards feature and functionality coverage.

<put additional coverage targets here>

=== Configurations ===

MeeGo X.X is tested in a number of configurations, both in reference hardwares and a QEMU environment. The public configurations used for this release are YY.

<update data according the information given by product management>

=== Test Execution ===

All automated tests are executed in a automated environment ([http://ots.meego.com/ OTS]), and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

<update your rules and environments here>

=== Test Reporting ===

Reporting for individual test sessions is done in [http://qa-reports.meego.com/ qa-reports]. Verticals might have additional reporting practices defined.

=== Test Tools ===

Additional tools such as Valgrind may be used to do a deep analysis of test details/failures and tools such as <please contribute> are used in the documentation process. These tools which will provide additional data used to analyse the quality of the product but are not necessarily used in the Go/No-Go decision making process.

<add your own tool portfolio here>

=== Testing Quality Characteristics ===

Check out the [[Quality/Test_areas_and_types|Test areas and types]].

==== Functionality ====

The basic functionality checklist can be categorized as below:
* Installation and un-installation
* Startup and exiting
* Screen Layout and Navigation
* Mobility and connectivity
* Security
* Touch interaction
* UI style
* Accessibility and clarity
* Help and documentation
* UI Controls
[[Quality/Test_areas_and_types#Functionality_quality_characteristics|Test areas and types for functionality]] has more details/ideas.

<list your goals for functional testing here>

==== Performance ====

The performance target is to produce a quality product with a performance that is competitive in the market. This is going to be achieved by first looking at the different areas that affect the performance of MeeGo X.X release, on actual reference hardwares, and then trying to measure these values to find out where performance needs to be improved.
The focus areas might be:
* Binary sizes
* Functional Performance
* Application Startup times
* Startup times
* Memory usage
[[Quality/Test_areas_and_types#Performance_quality_characteristics|Test areas and types for performance]] has more details/ideas.

Our goal to have a fast and responsive system having following targets:
* <list your measurable targets for performance aspects of the system here>

These are our initial targets only. Once we have reached these targets we will continue to improve and create a new set of targets to aim for.

<define your approach in more details here>

==== Reliability ====

Our goal is to have reliable system having following targets:
* <list your measurable targets for reliability aspects of the system here>
[[Quality/Test_areas_and_types#Reliability_quality_characteristics|Test areas and types for reliability]] has more details/ideas.

==== Usability ====

The usability check list can be categorized as below:
* Task success rates and Error prevention
* Help users recognize, diagnose, and recover from errors
* Ease of use
* Flexibility and efficiency of use
* Consistency and standards
* Time on task
* Task path, task flow and sequence
* Aesthetic and minimalist design
* User preference
* User control and freedom
[[Quality/Test_areas_and_types#Usability_quality_characteristics|Test areas and types for usability]] has more details/ideas.

== General Environment Setup ==

MeeGo can be tested on reference hardwares as well as in a QEMU environment. Both cases have value and may provide data that would be hard or impossible to acquire 'on the other side'.

=== Basic Environment Setup ===

Once MeeGo is installed in the QEMU environment or installed onto a reference hardware, no additional environment setup should be required that isn't already required by MeeGo itself. It is important though that the environment is equal for each testrun, i.e. the same background processes are running for each test run, and that only background processes are running that are required.

Certain aspects of MeeGo can only be tested if associated hardware modules are available on the reference hardware/desktop machine on which the test is executed. For instance:
* Bluetooth
* Wireless
* TV
* Radio
* Telephony
* etc.
Please refer to the manuals that come with these items to setup correctly.

<put reference to your test environment here>

=== Testing MeeGo in a QEMU environment ===

To test MeeGo in a QEMU environment, MeeGo needs to be built for the desktop which is accomplished by <put reference/link to the latest instructions over here>

=== Testing MeeGo on a reference hardware ===

To test MeeGo on a reference hardware, MeeGo needs to be built for that specific reference hardware, which is accomplished by <put reference/link to the latest instructions over here>

== Detailed Test Plans ==

<Structure taken here is following components and verticals in bugs.meego.com. This is provided as the illustrative example how subplans can be organized. Align this according your project setup>

{| border="1"
!
! Core OS
! Handheld UX
! Netbook UX
|-
|Bluetooth
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Contacts
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Content framework
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Graphics subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Media subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Photo viewer
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Qt
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Virtual keyboard
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Web browser
|<please add reference here>
|<please add reference here>
|<please add reference here>
|}

Quality/TestDesignProcessAndGuideline

2011-02-17T06:32:06Z

Jkunnari: /* Implement Test Case(s) */

== Introduction ==

Purpose of this document is to introduce process and guideline for creating high quality test cases for common usage in MeeGo quality assurance teams.
* Common:
** Common test case template to be used in UX and Core.
*** [[Quality/Test_case_template|Test case template]]
* For UX:
** Test cases are designed and stored to Test Link test asset management tool.
* For Core:
** Test cases are following test packaging instructions available at: [[Quality/QA-tools/Test packaging|Test packaging]]
** Instances of test cases are available in Test Link test asset management tool
* This document describes overall test design process and guideline for MeeGo QA

== Overall Test Design Process ==

[[File:TDPaG.jpg]]

== Features for Meego ==
* Defined by Product Management and/or relevant stakeholder(s)
* Located in Bugzilla
* Additional material that would be useful
** UI documents / UI design briefs / Visual references

== Analysis for Testability ==
* Done by Quality Assurance Owner
* Checklist to be used can be found:
** [[Quality_Assurance_Team|Quality Assurance Team]] or
** [[Quality/Plans/Testability-checklist|Testability Checklist]]
* Feedback of feature testability is given directly as comment to Feature comment-field in Bugzilla. Also separate meeting between QA owner and relevant Product Manager can be arranged on need basis.

== Test Case Title(s) ==
* Filled by Quality Assurance Owner
* Define TC title(s) and create coverage analysis matrix
** TC title length should be less than 30
** Use ‘_’ to connect words
** Title should be describing well purpose of test case

== Informal Review ==
* Arranged by QA owner
* Wiki, Email and IRC to be used
* Participants: Feature owner, QA owner, QA CC-owner, Developer and community members
* Minutes are stored to <exact place to be defined later>
* Target is to ensure that designed TC title’s are matching to given feature in high level, target test case coverage is achievable and share information between relevant parties

== Implement Test Case(s) ==
* Done by QA owner to Test Link (UX) or as test code and test.xml created accordingly. Test case instance in Test Link to be available in Test Link
* Test Case Template and its guidelines must be followed when filing Test Case Description
** Link: [[Quality/Test_case_template|Test Case Template]]
* Note! Test Case Template must be available also directly in Test Link database for manual TC’s
* Design Steps
** Each test step must have description and expected results defined.
** Each clause should start with a capital letter and end with full stop mark.
** One test step should contain only one activity.
** Measurement start and end points should be clearly defined in test steps.

== Technical Review ==
* Arranged by QA owner
* Wiki, Email and IRC to be used
* Participants: QA Owners, QA CC-owner, Developer, Test Engineer and community members
* Test Case(s) including design steps are reviewed from Test Link, Source Code review in need basis.
* TC status is marked as Proposal
* Target is to finalize and approve full test cases used in test execution and share information between QA owners and Test Engineers

== Test Case(s) ==
* Test cases in place in Test Link tool and test-xml’s available
* TC status is marked as Approved
* Test code available in agreed place (as development project in MeeGo.Com)
* Test data available in public domain <link to be added>

== Other Notes ==
* Test Plan and Scope creation/definition is excluded from this process
* Model of Coverage analysis matrix: <Link to be added here when available>

Quality/Plans/Test-plan-template

2011-02-17T06:30:41Z

Jkunnari: /* Test Case Documentation */

Proposal - please contribute by editing this page or posting your comments on [[Talk:TestPlanTemplate|discussion area]] for this page.

Note that this is template giving the frame from actual test plan.
The intent is to list areas/things which one need to consider when planning needed QA activities for certain MeeGo Release (e.g. 1.2, 1.3).

== Introduction ==

=== Purpose ===

The purpose of the Test Plan is to introduce the testing strategy, scope, sub-plans and testing activities done or supported by QA team for MeeGo X.X release.

The intended readers for this document are people interested about the reasoning behind the actual activities done by QA team.

<please write more>

=== Objectives ===

The Test Plan is intended to provide the vehicle for QA team to confirm the functionality and completeness of the MeeGo X.X release. It is expected that the satisfaction of the complete series of criteria contained in this plan will signify successful functionality of the integrated release.

<please list more objectives>

=== Goal ===

The goal is to deliver a product with no open bugs with a severity level of critical and a minimal number of open bugs with severity level major.

<please add other goals>

=== Document Overview ===

==== Test Cases ====

During the the release phase aiming to final release, test cases will be executed. The test cases are described in test documents which are in gitorius at meego.gitorious.org/meego-quality-assurance. The actual test case code might be (depending on the license type and test level) part of the source package. Each test case also contains test specific criteria which decide upon test case success or failure.

Test cases shall fulfill the definitions given in [[Quality/Test_case_template|Test case template]]

<add other sources and definitions used>

==== Test Case Result ====

Verdict shall be assigned for a test case after execution according [[Quality/Glossary#Test_case_verdict|test case verdict instructions]].

==== Traceability of Features ====

For each feature listed bugs.meego.com, in principle, one or more test cases should exist (see detailed test plans). This information should be defined in the test case according instructions given in [[TestCaseTemplate|Test case template]]

==== Test Case Documentation ====

Each test case described in the detailed test plan contains the fields according [[Quality/Test_case_template|Test Case Template]] and [http://meego.gitorious.org/meego-quality-assurance/test-definition test-definition]

== Overall test strategy ==

=== Introduction ===

Components can be tested by a combination of component (unit, module and component integration testing put together), integration and system tests.

For a system test the component is typically tested by launching the application in the test environment and by executing functionality of the component using the User Interface, and by verifying the outcome against an 'expected result'.
In cases where manual verification is required the expected outcome is specified in the test case.

Where possible all or parts of the test steps are automated, but in all cases there will be a manually executed 'Acceptance Test' for each GUI component in which the module will, at least, be verified against a standard checklist.

For an Integration test the component is typically tested by launching the component (if it has a GUI), or by launching other applications that make use of the tested component, inside the test environment and by executing functionality of the component using the User Interface. Where applicable, parts of the system may be replaced by stubs to partially insulate the component from its environment.

Unit tests are so named because they each test one unit of code. This type of tests are usually written by developers as they work on code (white-box style), to ensure that the specific function is working as expected. Whether a module of code has hundreds of unit tests or only five is irrelevant. One function might have multiple tests, to catch corner cases or other branches in the code. A test suite of unit tests should never cross process boundaries in a program, let alone network connections. Doing so introduces delays that make tests run slowly and discourage developers from running the whole suite.

Module test tests a module of code similar way than unit tests - the difference is that test are generated black-box style (without reference to the internal structure of the module).

Introducing dependencies on external modules or data also turns unit tests into component integration tests. If one module misbehaves in a chain of interrelated modules, it is not so immediately clear where to look for the cause of the failure. Tests are checking ready made part of the component with simulators (stubs, test drivers) for missing external dependencies.

=== Priorization ===

Tests are written on a highest priority first basis, and secondly on broad coverage.

<add other priorization rules used by vertical or project>

=== Coverage ===

At this stage we are aiming towards feature and functionality coverage.

<put additional coverage targets here>

=== Configurations ===

MeeGo X.X is tested in a number of configurations, both in reference hardwares and a QEMU environment. The public configurations used for this release are YY.

<update data according the information given by product management>

=== Test Execution ===

All automated tests are executed in a automated environment ([http://ots.meego.com/ OTS]), and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

<update your rules and environments here>

=== Test Reporting ===

Reporting for individual test sessions is done in [http://qa-reports.meego.com/ qa-reports]. Verticals might have additional reporting practices defined.

=== Test Tools ===

Additional tools such as Valgrind may be used to do a deep analysis of test details/failures and tools such as <please contribute> are used in the documentation process. These tools which will provide additional data used to analyse the quality of the product but are not necessarily used in the Go/No-Go decision making process.

<add your own tool portfolio here>

=== Testing Quality Characteristics ===

Check out the [[Quality/Test_areas_and_types|Test areas and types]].

==== Functionality ====

The basic functionality checklist can be categorized as below:
* Installation and un-installation
* Startup and exiting
* Screen Layout and Navigation
* Mobility and connectivity
* Security
* Touch interaction
* UI style
* Accessibility and clarity
* Help and documentation
* UI Controls
[[Quality/Test_areas_and_types#Functionality_quality_characteristics|Test areas and types for functionality]] has more details/ideas.

<list your goals for functional testing here>

==== Performance ====

The performance target is to produce a quality product with a performance that is competitive in the market. This is going to be achieved by first looking at the different areas that affect the performance of MeeGo X.X release, on actual reference hardwares, and then trying to measure these values to find out where performance needs to be improved.
The focus areas might be:
* Binary sizes
* Functional Performance
* Application Startup times
* Startup times
* Memory usage
[[Quality/Test_areas_and_types#Performance_quality_characteristics|Test areas and types for performance]] has more details/ideas.

Our goal to have a fast and responsive system having following targets:
* <list your measurable targets for performance aspects of the system here>

These are our initial targets only. Once we have reached these targets we will continue to improve and create a new set of targets to aim for.

<define your approach in more details here>

==== Reliability ====

Our goal is to have reliable system having following targets:
* <list your measurable targets for reliability aspects of the system here>
[[Quality/Test_areas_and_types#Reliability_quality_characteristics|Test areas and types for reliability]] has more details/ideas.

==== Usability ====

The usability check list can be categorized as below:
* Task success rates and Error prevention
* Help users recognize, diagnose, and recover from errors
* Ease of use
* Flexibility and efficiency of use
* Consistency and standards
* Time on task
* Task path, task flow and sequence
* Aesthetic and minimalist design
* User preference
* User control and freedom
[[Quality/Test_areas_and_types#Usability_quality_characteristics|Test areas and types for usability]] has more details/ideas.

== General Environment Setup ==

MeeGo can be tested on reference hardwares as well as in a QEMU environment. Both cases have value and may provide data that would be hard or impossible to acquire 'on the other side'.

=== Basic Environment Setup ===

Once MeeGo is installed in the QEMU environment or installed onto a reference hardware, no additional environment setup should be required that isn't already required by MeeGo itself. It is important though that the environment is equal for each testrun, i.e. the same background processes are running for each test run, and that only background processes are running that are required.

Certain aspects of MeeGo can only be tested if associated hardware modules are available on the reference hardware/desktop machine on which the test is executed. For instance:
* Bluetooth
* Wireless
* TV
* Radio
* Telephony
* etc.
Please refer to the manuals that come with these items to setup correctly.

<put reference to your test environment here>

=== Testing MeeGo in a QEMU environment ===

To test MeeGo in a QEMU environment, MeeGo needs to be built for the desktop which is accomplished by <put reference/link to the latest instructions over here>

=== Testing MeeGo on a reference hardware ===

To test MeeGo on a reference hardware, MeeGo needs to be built for that specific reference hardware, which is accomplished by <put reference/link to the latest instructions over here>

== Detailed Test Plans ==

<Structure taken here is following components and verticals in bugs.meego.com. This is provided as the illustrative example how subplans can be organized. Align this according your project setup>

{| border="1"
!
! Core OS
! Handheld UX
! Netbook UX
|-
|Bluetooth
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Contacts
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Content framework
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Graphics subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Media subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Photo viewer
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Qt
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Virtual keyboard
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Web browser
|<please add reference here>
|<please add reference here>
|<please add reference here>
|}

Quality/Plans/Handset UX test plan

2011-02-17T06:25:31Z

Jkunnari: /* Test Cases */

= MeeGo HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo HandSet UX release. A series of component and system test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA Handset UX team.

=== Objectives ===

Objectives in MeeGo HandSet UX software testing is to validate the functionality of entire MeeGo HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo release HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set
* Each test component will be documented in component test plan. Test plans will cover all testing aspects for specific component/feature(s).

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)
* System Test Plans are created as separate test plans containing both Functional and Non-Functional System Testing aspects

=== Testability ===

Testability of MeeGo HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo UX HandSet Features are stored in Bugzilla)
* [[Quality/TestabilityChecklist]]
* [[Quality/HandsetTestabilityStatus]]

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Common Test Case Template is used when designing test cases. Test cases are released publicly in MeeGo Gitorious under Handset UX Tests part.

* Overall test design process and guideline from features to actual test cases can be found [[Quality/TestDesignProcessAndGuideline]]

Relevant Links
* [[Quality/Test_case_template|Test Case Template]]
* http://gitorious.org/qa-tools/
* http://meego.gitorious.org/meego-quality-assurance/handset-ux-tests

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo HandSet UX Testing will cover the MeeGo HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* List of exact features not to be tested can be found from Featurezilla @ Bugzilla. One must use Testability query there to have full list identified.
** [http://bugs.meego.com/report.cgi?x_axis_field=cf_testability&y_axis_field=component&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&classification=MeeGo+Features&product=MeeGo+Handset+Features&version=1.2&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&deadlinefrom=&deadlineto=&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Testability in Featurezilla (1.2)]

=== Configurations ===

MeeGo HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* [http://meego.com/devices/handset/handset-supported-hardware Supported Hardware]

=== Test Sets, Definitions and Priorization ===

Test sets are formed to Test Management Tool by using specific field inside the tool. Test sets that are formed are
* Acceptance/Sanity Test
** Acceptance/Sanity test set is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in acceptance/sanity test reflects the most important and basic functionalities of the distribution.
** Acceptance and Sanity test sets are relatively stable and will be run on daily basis.
* Feature Test
** Key Feature Test Set is used to verify MeeGo Handset UX most critical key use cases functionalities with well selected basic feature test cases.
** Basic Feature Test Set is verifying MeeGo HandSet UX delivered features with basic feature test cases. Test set is always static to show overall feature functionalities progress and maturity of the entire MeeGo distribution. Based on test results QA is able to identify components with working features to enable extended feature testing and system testing.
** Extended Feature Test Set is used to verify delivery of features forming full functionality of entirely component. After component is fully integrated all component related test cases will be executed for selected weekly release and report out all the bugs against component and it’s features. Extended feature test set will be run again in the upcoming milestone or when significant changes are applied to component and it’s features.
* System Functional
** System Functional Test Set is targeting to evaluate delivered functionalities from system perspective. Test cases are not testing UI or Application itself, instead test cases are testing how whole system is working and interacting with Consumer (end user). Test cases are covering most critical interaction and negative scenarios that consumers will encounter in their daily usage.
* System Non-Functional
** System Performance Test Sets target is to evaluate overall system performance by executing well-selected set of cases from different test areas - for example response and reaction times, use times and frame rate measurements. Test set gives a quick view of system performance from end-user point of view.
** System Reliability Test Sets target is to provide an overview to system reliability by executing iterative tests that focus on the most important and most used end-user features of MeeGo.

Quality Assurance Owners are setting priorities for Test Cases to form these Test Sets to be used for test execution.

More detailed information: [[Quality/TestSetGuideline]]

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool
* Main focus in test automation will be in acceptance, sanity and regression testing automisation
* Automated scripts are released in Gitorius: http://gitorious.org/qa-tools/ under Handset UX Tests part

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

* [[Quality/TestSetGuideline]]

=== Test Reporting ===

All MeeGo HandSet UX test results are stored to one place.

* MeeGo Test Repository for HandSet
** [[Quality/HandsetTestReport]]

Use Test Report Templates can be found: [[TestReportTemplateCollection]]

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to [[Release_Engineering/Release_Timeline]]

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

=== Component Test Plans ===

{| border="1"
!| Component
!| QA Owner
!| QA CC-owner
!| Detailed test plan
|-
| Applets || Cathy Li || Mika Ikonen || [[Quality/MeeGo1.2_Handset_Applets_Test_Plan|MeeGo 1.2 Handset Applets Test Plan]]
|-
| Short Message Service || Mika Ikonen || Lili || [[Quality/MeeGo1.2HandSetUXTestPlanforShortMessageService|MeeGo 1.2 Handset SMS Test Plan]]
|-
| Dialer || Mika Ikonen || Lili || [[Quality/MeeGo1.2HandSetUXTestPlanforDialer|MeeGo 1.2 Handset Dialer Test Plan]]
|-
| Media Applications || Jessica Ji || Anssi Takku || [[Quality/Plans/Meego1.2_media_test_plan|MeeGo1.2 Handset Media Applications Test Plan]]
|-
| Mozilla Fennec Browser || Anssi Takku || Qin Mu || [[Quality/MeeGo1.2HandSetUXTestPlanforMozillaFennecBrowser|MeeGo 1.2 Handset Mozilla Fennec Browser Test Plan]]
|-
| Contacts|| Dayu Yang || Mika Ikonen || [[Quality/Meego_contacts_test_plan_v12|MeeGo1.2 Handset Contacts Test Plan]]
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || Mika Ikonen || [[Quality/MeeGo1.2_Handset_CoreUX_TestPlan|MeeGo1.2 Handset Core UX Test Plan]]
|-
| Social Networking || Cathy Li || Mika Ikonen || [[Quality/MeeGo1.2_Handset_UX_Social_Networking_TestPlan|MeeGo1.2 Handset Social Networking Test Plan]]
|-
| Compositing Window Manager|| N.N. || N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || N.N. || No requirement yet.
|-
| Virtual Keyboard || Yi Fu || Anssi Takku || [[Quality/Plans/Meego1.2_vkb_test_plan/|VKB Test Plan]]
|-
| Sync client || Qin Mu || N.N. || [[Quality/Meego_Handset_SyncUI_TestPlan_v1.2|SyncUI Test Plan]]
|-
| Email ||Yi Fu || Mika Ikonen || [[Quality/Plans/Meego1.2_email_test_plan|Email Test Plan]]
|-
| Calendar || Dayu Yang || Anssi Takku || [[Quality/Meego_Handset_Calendar_TestPlan_v12|MeeGo1.2 Handset Calendar Test Plan]]
|-
| Instant Messaging || Mika Ikonen || Yi Fu || [[Quality/MeeGo1.2HandSetUXTestPlanforInstantMessaging|MeeGo 1.2 Handset Instant Messaging test Plan]]
|-
| Connectivity UI || Mika Ikonen || N.N. || Currently only one requirement for Internalisation available - NO testplan needed
|-
| Settings || Dayu Yang || Anssi Takku || [[Quality/Meego_settings_test_plan_v12|MeeGo1.2 Handset Settings Test Plan]]
|-
| UI Infrastructure || Mika Ikonen || N.N || [[Quality/Plans/Handset_UX_test_planUI_infrastructure|MeeGo 1.2 Handset UI Infrastructure Test Plan]]
|}

=== System Test Plans ===

{| border="1"
!| System Test Plans
!| QA Owner
!| Detailed test plan
|-
| System Functional Test Plan || Mika Ikonen || [[Quality/MeeGo1.2HandSetUXTestPlanforSyFuTe|MeeGo 1.2 Handset System Functional Testing Test Plan]]
|-
| System Non-Functional Test Plan || Anssi Takku || [[Quality/MeeGo1.2HandSetUXTestPlanforSystemNFT|MeeGo 1.2 Handset System NFT Test Plan]]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: [[Quality]]
* Feature Testability checklist: [[Quality/TestabilityChecklist]]
* Testability Status Report: [[Quality/HandsetTestabilityStatus]]
* Test Case Design Progress Follow-up: [[Quality/HandsetTestSuite]]
* Test Result Reports: [[Quality/HandsetTestReport]]
* Test Set Guideline: [[Quality/TestSetGuideline]]
* Test Design Process and Guideline: [[Quality/TestDesignProcessAndGuideline]]
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: [[Quality/HandSetUXRamp-Up1.1]]

Quality/Plans/MeeGo Core Test Plan

2011-02-17T06:24:58Z

Jkunnari: /* Test cases */

'''Proposal For MeeGo 1.2. This plan is under review. actively revising and updating'''
= MeeGo Core Overall Test Plan =

== Introduction ==
This is overall test plan for MeeGo Core of '''MeeGo open source project''', which defines test scope, test strategy, test configurations as well as test execution cycle for MeeGo Core. It will give readers an overview of validation activities for MeeGo Core of MeeGo open source releases. A series of component test plans will also be linked in this overall test plan to cover the detailed test approaches and test design for ingredients of MeeGo Core stack. It will be the joint effort from MeeGo community.

This plan describes MeeGo Core OS QA for MeeGo 1.2 and onwards. Test plan is revised for each MeeGo Release and thus this test plan should be considered as evolving document. New testing methods, practicalities and approaches are described in revisions.

== Objectives ==
Objectives in MeeGo Core OS QA is to validate the functionality of entire MeeGo Core OS software delivery by performing hourly, daily and weekly testing for weekly releases. More information about testing cycle and test types can be found later in this document. Target is to ensure that:

* New features introduced in specific Core OS weekly release are working as specified as a part of system.
* Stakeholders and community get visibility to release quality.
* Validate that relevant bugs are fixed in software release.

Weekly testing is cumulative in terms of test coverage. Week to week test cases included in test run will vary and new test cases are introduced. Thus test case coverage increases constantly. Increase is dependent on release content (how many new features there was released in specific weekly release).

For these activities MeeGo Core OS QA follows iteration cycle and process described in [[Release_Engineering/Process|Release Engineering’s Process]].

As addition to fast cycle testing more thorough testing (Full Pass) is done for MeeGo Releases. Full Pass testing is massive test execution for entire test asset available at the moment. With full pass all features are re-verified and regression is measured. This activity is taking place after MeeGo Release Feature Complete. Target is to ensure that:

* Delivered features for MeeGo Core OS are working as specified as a part of system.
* MeeGo Core OS is performing well and is reliable.
* Program maturity statement can be given.

In order to verify features of MeeGo Core OS requires exhaustive documentation of feature under test. insufficient documentation has negative impact to test asset quality as stated in [[Quality/TestSuite/MCTS/MCTS_Development_Guideline|MCTS Development Guidelines]].

For these activities MeeGo Core OS QA follows release cycle described in [[Release_Engineering/Release_Timeline|Release Engineering’s release timeline]].

== Test strategy and approach ==

=== Introduction ===

The overall objective of MeeGo Core QA is to ensure that MeeGo middleware and OS Base provide stable hardware and usage model independent application services and APIs for building the vertical specific user experience. Each core component has different quality risk regarding to MeeGo integration. For example, some core component is mature in upstream and MeeGo do the integration without heavy new feature development; some component is contributed and open-sourced from proprietary component with heavy development. Considering most of MeeGo Core components will be adopted by multiple vertical usages and run on a number of MeeGo devices, Test execution efficiency shall taken into account when creating the test cases. Given this, there are following strategy considerations:

*Unified test suite (MeeGo Core Test Suite) for MeeGo Core validation. It's open-sourced and could start with the first batch of test cases contributed from community. MeeGo Core Test Suite (MCTS) docs can be found at MeeGo Quality page, and the code in Gitorious.
*Test suites from different sources are utilized as much as possible (e.g. Qt Mobility API test cases)
*If test cases are available in upstream project they will be used in order to keep maintainability.
*Appropriate test coverage in defined (density) categories. The test coverage of each component is based on the quality risk regarding to MeeGo integration.
*Test cases are automated s much as possible and able to run under MeeGo automated testing framework (OTS).
*MeeGo Core test cases are independent from vertical specific UX’s

MeeGo Core OS is verified with test assets available in MeeGo GIT in different projects and other open source projects. One can divide used test cases to two different types:

*Component tests. Unit/module like test cases verifying API’s of specific component.
*System tests. E2E type system level test cases verifying MeeGo Core OS stack as whole.

Typical characteristics for these component test cases are that they verify specific component or library with extensive set of parameters. Often they are missing E2E approach where entire stack is exercised. Test cases may not necessarily leave from component under test.

Typical characteristics for these system test cases are that they are based on use cases or user stories and often testing entire stack from top most interfaces provided by MeeGo Core OS Middleware and exercises HW peripheral beneath SW stack. These types of test suites are the most efficient ones for measuring and providing visibility to maturity of MeeGo Core OS.

MeeGo Core OS QA uses mainly test framework and other testing tools provided by [[Quality/QA-tools|MeeGo QA Tools Team]].

In order to ensure that MeeGo is competitive SW platform MeeGo Core OS QA is executing Performance test cases and driving performance improvements to MeeGo Core OS stack. Majority of the performance test cases are measuring raw performance of the system and do not necessarily measure end user experience. End user experience (response time measurements) is measured by Handset UX QA. For more detailed information of End User Experience testing see [[Quality#MeeGo_Handset_Testing|Handset UX Test Plan]].

In order to ensure reliability of MeeGo, MeeGo Core OS QA is executing Reliability test cases and driving reliability improvements to MeeGo Core OS stack. As addition to conventional test types such as Long-lasting and iterative, also Feature Interaction Testing is done as part of reliability testing. Feature Interaction Testing is based on user scenarios.

Test cases are following test type definition Aligned with ISO/IEC 9126-1 Software Quality Model and ISTQB Advanced Level Syllabus. Test types are defined in [[Quality/Test_areas_and_types|test areas]].

=== Development teams and MeeGo Core OS QA ===

It is a fact that Quality Assurance cannot create quality of the release by doing exhaustive testing. Quality is build in development phase by developers contributing to MeeGo.

Developers has significant role also in QA.Here are QA recommendations for developers contributing to MeeGo:

* Developer contributing to MeeGo release content shall verify their deliveries prior to commit.
* Test results shall be posted to appropriate MeeGo mailing list and results shall be stored to punblicly available location (link to be added to MeeGo Core testing results).
* Provide test asset with the component including detailed documentation.
* Propose sanity test cases for the component To QA Contact in order to include them to Sanity and to weekly testing.
* Maintain test asset.

At the end of the day, developer is responsible of Quality of the his/hers delivery.

== Quality requirements for MCTS ==

Even though there are certain problematic when testing code with a code it is very efficient of testing while:

*Fast test execution time & feedback
*Enables high automation rate (less resource hungry)
*Error debugging is fast (focused errors with appropriate log files and root cause analysis can be provided)
*Test coverage can be defined easily.

In order to take advantage of items described above test asset shall follow demanding quality standards. Test asset producing lots of false positives and negatives confuses community, provides wrong information about release quality and sends developers to wild goose hunt. This shall never happen. To ensure this MCTS code will follow quality requirements described [[Quality/TestSuite/MCTS/MCTS_Development_Guideline|MCTS Development Guidelines]].

== Feature test and System test ==
QA target is to validate MeeGo distribution

*Feature functionality
*System functionality (Interaction and negative scenarios)
*System performance (Data Throughput, Latencies, Framerate, etc.)
*System reliability

=== Feature Testing ===
*In this context known also as Component Testing.
*Target is to verify feature provided by one or more component with specific test cases designed for this purpose. Found 3G Device
*Every component (formed by features) basic functionality is tested in weekly testing and after feature complete phase as full pass testing.

=== System Testing ===
*Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Bluetooth, send 1MB file L2Cap PC.
*Target is to test system testing (performance). Test case example: NFT-BT-L2CAP_Send_10MB-THRO-MS (target: 1.9Mbit/s)
*Target is to test system testing (reliability). Test case example: NFT-BT-L2CAP_Send_2h-LOLA-MS (target: Data transfer for 120mins without performance decrease)
*Not tested NFT types: Performance (Memory Consumption and Power Management) and Reliability (Aging and Long Period)

== Testability ==
Testability of MeeGo 1.2 Core OS features is ensured.

* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Defined QA Owner checks features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.
* QA does not does not accept that features without proper information to ensure testability are assigned to development.

Testability can be seen as main key for the success of QA. In order to ensure high quality QA, testability percentage of the MeeGo 1.2. features shall be 90% or higher.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.2 Core OS Features stored in Bugzilla)
* [[Quality/Plans/Testability-checklist|Testability Checklist]]

== Test cases ==
Well defined test cases are the key to success in MeeGo Core OS Testing. While the objective of testing is assist developers in creating software that functions correctly, quite often testing falls into the trap of attempting to demonstrate that the software works. This shall be avoided. In test case development following should be considered:

*Do not attempt to find work-arounds for problems in the API under test: if A + B = C and that is not the obtained result, then the test must fail
*Tests should be written against the specification/documentation of software instead of against the implementation as it is the implementation itself that is under test.
*Always attempt to 'break' the software under test: Do not only test things that are known to be working, testing the unknown areas is as important as testing the commonly used functions.
*The final objective of testing is to provide information/evidence to developers, therefore, the more (and more detailed) information that the test can provide, the better.

MeeGo Core OS Test Design follows spirit of [[Quality/TestDesignProcessAndGuideline|MeeGo QA Common Test Design Process and Guidelines]]. Specifics being:
*Test Cases are designed by test asset developers based on QA Owners input.
*QA Owners input is based on existing features and which have been approved from testability point of view.
*Test Cases are described according to [[Quality/QA-tools/Test plan|test definition]] in tests.xml files
*Tests.xml files and are stored to MeeGo Gitorious.
*[[Quality/Test_case_template|Test Case Template]] is used when designing test cases.

== Features to Be Tested ==

Overall the MeeGo Core Testing will cover the '''MeeGo OS Middlewares layer''' and '''MeeGo OS Base layer''' in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

Specific features to be tested will be aligned with the features under '''MeeGo Core OS Features''' product in [http://bugs.meego.com MeeGo Featurezilla]

== What will not be tested ==
Following feature category won't be covered in MeeGo Core validation for MeeGo open source releases.
* Any proprietary component which is not included in the open source MeeGo release
* Certification testing like: Ofono, Bluetooth, USB, WLAN, and similar.
* Any UI or application related testing.

=== Test Coverage Definition ===
In order to use resources efficiently and control risk level on component maturity. Testing is done in different levels as follows:

* '''Thorough'''
** Full testing service for API
** Target is to have 100% function/method coverage provided by API
** Parameter coverage defined by using testing techniques as follows:
*** All Pairs (Pair wise)
*** Equivalence Partitioning
*** Boundary Value Analysis
*** Subjective test case selection (according to Expert opinion or similar)

* '''Average'''
** Between Thorough and Light
** Full API coverage (according to rules for Thorough) for new functionality.
** Integration level testing for legacy features.

* '''Light'''
** Integration level testing
** Upstream Middleware components shall be tested with limited amount of test cases (sub set) ensuring that integration of component to MeeGo is successful and component is working as expected

* '''Not Covered'''
** No Middleware testing, covered indirectly by UX Testing
** There will be API’s not feasible to cover with test cases at middleware level but more efficiently tested indirectly by UX testing

=== Test Coverage Measurement ===
In order to understand how well certain component is covered with test cases there shall be test coverage measurement done. This is directly linking to risk level of specific component. Test coverage is based on Function/Method coverage per API.

[[Quality/TestSuite/MCTS/MCTS_API_analysis|MCTS API analysis]] describes methods to be used for test coverage measurement.

=== Component Test Plans and Coverage ===
Each component will have detailed test plan for the specific component. Test Plans are available in [[../../TestSuite/MCTS#Test_Suite_Status_For_Middleware |MeeGo Core Test Suite]] for details (each component name is link to test plan).

Test plan shall follow structure described below: :

* 1 Introduction
* 2 Test Coverage
** 2.1 Software Coverage
** 2.2 Hardware Coverage
** 2.3 Items not Covered
* 3 Running the tests
** 3.1 Executing tests
** 3.2 Test case arguments
** 3.3 Test cases
** 3.4 Configuration file
* 4 Module Design and Architecture
** 4.1 Architecture diagram
* 5 References
* 6 Change History

See [[Quality/TestSuite/Video_Playback_Driver_Test_Specification|Video playback driver test plan]] for reference.

=== Test Cycle ===
MeeGo Core will be tested from the following different test execution levels. Testing Gear Box is as follows.

[[File:Coreosqawkcycle.PNG]]

==== Hourly Testing ====
Testing is done against Trunk:Testing. It will run a portion of fully automated test cases for core components and aims to provide quick acceptance testing to support incremental packages integration. It will be conducted under OTS (Open Test System).

==== Sanity (Daily) Testing ====
Testing is done against Trunk and also for weekly release prior to release announcement to provide visibility to release quality and to ensure that last fixes does not cause regression to release. Release Engineering includes links to test reports in release announcement. Sanity testing is static set of test cases which is modified on need basis. Thus Sanity test set may contain test cases for functionality which are not introduced yet. These test cases are marked as N/A with comment that feature not integrated yet. Sanity testing consists of:

* Set of fully automated test cases for release content developed under Bugzilla components
* Set of fully automated system level test cases verifying functionality from top most interfaces provided by MeeGo Core OS.

Daily Sanity testing aims to identify the regressions as early as possible and provide easy to understand visibility to SW maturity. This testing is answering to questions like:

* Is it possible to scan and connect to WLAN Access Point and conduct data transfer as a enabler for browsing?
* How fast specific file is downloaded over WLAN as a enabler for good user experience for browsing.

While test cycle needs to be fast, reliability is not reasonable to measure in daily testing.

==== Weekly Testing ====
Weekly Testing is a test cycle against the weekly preview images released from Release Engineering. Test frequency is once a week, which aligns with distribution's weekly image release cadence.
Weekly testing is incremental testing and target for weekly testing is to:

* Test the new features early
* Bug verification in order to track critical/major bugs got fixed in timely fashion
* Measure regression

'''New features''' are verified as soon as they are ready for testing. QA Owners follows release engineering’s release plans and feature status in Featurezilla. When feature is turned to Released sate in Featurezilla, test cases mapped to this feature are taken as part of next weekly testing execution. If test cases for specific feature are passing, Feature shall be marked as verified in Featurezilla.

'''Regression''' test cases are chosen amongst test cases designed for newly verified feature and are included in next weekly testing round.
Regression is a set of tests to verify that changes from the last build (code enhancements, bug fixes) don’t introduce new issues to the previous working code as well as new features work as expected. This cycle include the tests for previous major bug fixes and areas of the code that will be affected by new implementation.
The regression test will be taken in following test cycle:

* In the test cycle for milestone release, before the release criteria is satisfied, it is possible that there are several release candidates which includes bug fixes for previous release candidates. These bug-fixes need be tested
* In the intermediate regular release, only the modified features need to be heavily tested.

'''Bug verification''' on weekly basis to make sure the bug fixes be verified within one week after bug fixing

Performance or reliability test cases by default are not included in weekly testing. Performance ro Reliability test cases are included only if bug fix has been provided against performance reliability related Bugzilla item or there are other suspicious changes in release content which may have a effect to performance or reliability

==== Full Pass (Milestone) Testing ====
In the formal test cycle for milestone test, after a new build passed the weekly test, QA will start the full validation testing for it, following test will be involved:

* Component Functional Test – which will follow the Bugzilla component test plan to run all tests.
* System functional Test – which are defined in the individual Bugzilla component test plan.
* System Non-functional Test – which are defined in the individual Bugzilla component test plan.

Purpose of the Full Pass is to measure release maturity in detailed level. In Full Pass testing entire test asset is executed for all the features released and previously marked as verified. Thus visibility and detailed information about release maturity is gained. Target is to have two Full Pass testing cycles during release life cycle. First Pull Pass test round starts at feature complete and last round ends few days prior to release date.

Between these two rounds failing cases and related bugs are followed closely in weekly testing. If there are very good grounds Full Pass testing can be executed more than twice during release cycle life cycle.

=== Test Report ===
* Send Test report to mailing list
** Weekly MeeGo Core Test Report (Bug verification, new feature exploration...)
** Distribution Milestone Full Pass Test report
* Archive reports at [[Quality/CoreTestReports|MeeGo Quality]] wiki

== Environmental Needs ==
=== Hardware Platforms ===
MeeGo Core OS is tested with numerous reference devices. The public reference configurations used are:
* Aava DV1/DV2
* N900
* MRST CDK
* Netbook

=== Test enviromental requirements ===

Each test suite shall contain README file describing test environment in detailed level. In a complex cases specific test environment description can be provided with reference in README file.

Test environment description includes everything needed to run the test or tests. This included hardware and software configuration of the device under test as well as any equipment (and its configuration) outside the device itself.

== Risks ==

One main expectation from program management is that QA is able to pinpoint risks concerning Quality Risks. For MeeGo 1.2 QA sees following risks:

{|cellspacing="0" border="1"
!|Risk #
!|Description
!|impact
!|Severity
!|Mitigation actions

|-
|| 1 || Insufficient documentation and/or low quality feature descriptions not meeting that testability requirements || Bad quality test asset not verifying feature in detailed level. || '''High''' || Significant effort put to feature testability in early phase of release life cycle. Review test assets with feature owner and assignees.
|-
|| 2 || Insufficient amount of system level test cases to show functionality of the release || Unclear release quality statement. Community confused and developers hunting errors which do not exist|| '''High''' || Significant effort to system level test case development. Each functionality (main success scenario and it's variants tested at least with one set of parameters.
|-
|| 3 || Changed components are verified in weekly testing but if dependencies are not understood impact to indirectly affected component not verified. || Surprises in release quality || Medium || Developers to address dependencies in detailed level showing significant dependencies
|}

== References ==
* MCTS (MeeGo Core Test Suite) code git tree: http://gitorious.org/qa-tools (to be moved meego.gitorious.org)
* MCTS docs http://wiki.meego.com/Quality/TestSuite/MCTS
* MeeGo Architecture: http://meego.com/developers/meego-architecture
* MeeGo Core test reports archive: http://wiki.meego.com/Quality/CoreTestReports
* MeeGo bugzilla: http://bugs.meego.com
* MeeGo 1.2 release planning: http://wiki.meego.com/Core_OS_Program_Dashboard#1.2_Release
* MeeGo test areas and types: http://wiki.meego.com/Quality/Test_areas_and_types
* Release Engineering Process: http://wiki.meego.com/Release_Engineering/Process
* Release Engineering/Release Timeline: http://wiki.meego.com/Release_Engineering/Release_Timeline
* MCTS Development Guideline: http://wiki.meego.com/Quality/TestSuite/MCTS/MCTS_Development_Guideline
* QA tools team: http://wiki.meego.com/index.php?title=Quality/TestPlan/MeeGo_Core_Test_Plan&action=edit&section=28
* MeeGo Handset Testing: http://wiki.meego.com/Quality#MeeGo_Handset_Testing
* Testability Checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Test Design Process And Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* Test Packaging: http://wiki.meego.com/Test_Packaging/Test_Plan
* Common Test Plan Template: http://wiki.meego.com/TestCaseTemplate
* MeeGo Architecture: http://meego.com/developers/meego-architecture
* MCTS API analysis: http://wiki.meego.com/Quality/TestSuite/MCTS/MCTS_API_analysis
* Video Playback Driver Test Specification: http://wiki.meego.com/Quality/TestSuite/Video_Playback_Driver_Test_Specification

Quality/Plans/1.1 Handset UX TestPlan

2011-02-17T06:23:49Z

Jkunnari: /* Test Cases */

= MeeGo 1.1 HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo 1.1 HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo 1.1 HandSet UX release. A series of component test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA.

=== Objectives ===

Objectives in MeeGo 1.1 HandSet UX software testing is to validate the functionality of entire MeeGo 1.1 HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo 1.1 HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo 1.1 HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review component test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance (response time)
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)

* Not tested NFT types: Performance (Throughput, Framerate, Load, Memory Consumption and Power Management) and Reliability (Endurance, Aging, Long Period and Low Resource)

* Each test component will be documented in component test plan. Test plan will cover all testing aspects for specific component/feature(s).

=== Testability ===

Testability of MeeGo 1.1 HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.1 UX HandSet Features are stored in Bugzilla)
* [[Quality/TestabilityChecklist]]
* [[Quality/HandsetTestabilityStatus]]

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are stored to TestLink tool. Common Test Case Template is used when designing test cases.

* Overall test design process and guideline from features to actual test cases can be found [[Quality/TestDesignProcessAndGuideline]]

Relevant Links
* <Link to Test Link tool>
* [[Quality/Test_case_template]]

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo 1.1 HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo 1.1 HandSet UX Testing will cover the MeeGo 1.1 HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* <List of exact features not to be tested can be added after needed query is implemented to Featurezilla @ Bugzilla>

=== Configurations ===

MeeGo 1.1 HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* N900
* AAVA
* MRST CDK

=== Test Sets and Priorization ===

Test sets are formed to TestLink tool by using specific field inside the tool. Test sets that are formed are
* Sanity Test Set
* Regression Test Set
* Feature Test Suite (Basic and Extended)
* System Functional Test Suite (Interactions and negative user scenarios)
* System Non-Functional Test Suite (Performance and Reliability) - Note! Excluded from 1.1 and targeted to 1.2 release.
* Milestone Test Suite

Quality Assurance Owners are setting priorities for Test Cases to form these Test Suites to be used for test execution.

When test suites are in place in public Test Link -tool, then every test suite is reviewed and approved with respective persons.

More detailed information: [[Quality/TestSetGuideline]]

Note! During MeeGo 1.1 HandSet UX Timeframe QA will not form System Non-Functional Test Suites. Those will be targeted for 1.2 release.

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

* [[Quality/TestSetGuideline]]

=== Test Reporting ===

All MeeGo 1.1 UX HandSet test results are stored to one place.

* MeeGo 1.1 Test Repository for HandSet
** [[Quality/HandsetTestReport]]

Use Test Report Templates can be found: [[TestReportTemplateCollection]]

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to [[Release_Engineering/Release_Timeline]]

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

{| border="1"
!| Application
!| QA Owner
!| Detailed test plan
|-
| Short Message Service || Mika Ikonen || [[Quality/MeeGo1.1HandSetUXTestPlanforShortMessageService|MeeGo 1.1 HandSet UX SMS Test Plan]]
|-
| Dialer || Mika Ikonen || [[Quality/MeeGo1.1HandSetUXTestPlanforDialer|MeeGo 1.1 HandSet UX Dialer Test Plan]]
|-
| Media Applications || Jessica Ji || [[Quality/MeeGo1.1_Handset_Media_TestPlan|MeeGo 1.1 Handset UX Media Test Plan]]
|-
| Mozilla Fennec Browser || Petri Jylha || [[Quality/MeeGo1.1HandSetUXTestPlanforMozillaFennecBrowser|MeeGo 1.1 HandSet UX Mozilla Fennec Browser Test Plan]]
|-
| Contacts|| Dayu Yang || [[Quality/Meego_contacts_test_plan|MeeGo 1.1 Handset Contacts Test Plan]]>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || [[Quality/MeeGo1.1_Handset_CoreUX_TestPlan|MeeGo 1.1 HandSet Core UX Test Plan]]
|-
| Social Networking || Cathy Li || [[Quality/MeeGo1.1_Handset_UX_Social_Networking_TestPlan|MeeGo 1.1 HandSet Social Networking Test Plan]]
|-
| Compositing Window Manager|| N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || [[Quality/1.1HandsetUXVkbTestPlan|MeeGo 1.1 HandSet UX Virtual Keyboard Test Plan]]
|-
| PIM Management (Clock and Sync client) || Qin Mu || [[Quality/MeeGo1.1_HandSet_UX_PIM_TestPlan|MeeGo 1.1 HandSet UX PIM Test Plan]]
|-
| Email ||Yi Fu || [[Quality/1.1HandsetUXEmailTestPlan|MeeGo 1.1 HandSet UX Email application Test Plan]]
|-
| Calendar || Dayu Yang || [[Quality/Meego_Handset_Calendar_TestPlan|MeeGo 1.1 Handset Calendar Test Plan]]
|-
| Instant Messaging || Mika Ikonen || [[Quality/MeeGo1.1HandSetUXTestPlanforInstantMessaging|MeeGo 1.1 HandSet UX Instant Messaging Test Plan]]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: [[Quality]]
* Feature Testability checklist: [[Quality/TestabilityChecklist]]
* Testability Status Report: [[Quality/HandsetTestabilityStatus]]
* Test Case Design Progress Follow-up: [[Quality/HandsetTestSuite]]
* Test Result Reports: [[Quality/HandsetTestReport]]
* Test Set Guideline: [[Quality/TestSetGuideline]]
* Test Design Process and Guideline: [[Quality/TestDesignProcessAndGuideline]]
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: [[Quality/HandSetUXRamp-Up1.1]]

Quality/TestDesignProcessAndGuideline

2011-02-17T06:22:17Z

Jkunnari: /* Introduction */

== Introduction ==

Purpose of this document is to introduce process and guideline for creating high quality test cases for common usage in MeeGo quality assurance teams.
* Common:
** Common test case template to be used in UX and Core.
*** [[Quality/Test_case_template|Test case template]]
* For UX:
** Test cases are designed and stored to Test Link test asset management tool.
* For Core:
** Test cases are following test packaging instructions available at: [[Quality/QA-tools/Test packaging|Test packaging]]
** Instances of test cases are available in Test Link test asset management tool
* This document describes overall test design process and guideline for MeeGo QA

== Overall Test Design Process ==

[[File:TDPaG.jpg]]

== Features for Meego ==
* Defined by Product Management and/or relevant stakeholder(s)
* Located in Bugzilla
* Additional material that would be useful
** UI documents / UI design briefs / Visual references

== Analysis for Testability ==
* Done by Quality Assurance Owner
* Checklist to be used can be found:
** [[Quality_Assurance_Team|Quality Assurance Team]] or
** [[Quality/Plans/Testability-checklist|Testability Checklist]]
* Feedback of feature testability is given directly as comment to Feature comment-field in Bugzilla. Also separate meeting between QA owner and relevant Product Manager can be arranged on need basis.

== Test Case Title(s) ==
* Filled by Quality Assurance Owner
* Define TC title(s) and create coverage analysis matrix
** TC title length should be less than 30
** Use ‘_’ to connect words
** Title should be describing well purpose of test case

== Informal Review ==
* Arranged by QA owner
* Wiki, Email and IRC to be used
* Participants: Feature owner, QA owner, QA CC-owner, Developer and community members
* Minutes are stored to <exact place to be defined later>
* Target is to ensure that designed TC title’s are matching to given feature in high level, target test case coverage is achievable and share information between relevant parties

== Implement Test Case(s) ==
* Done by QA owner to Test Link (UX) or as test code and test.xml created accordingly. Test case instance in Test Link to be available in Test Link
* Test Case Template and its guidelines must be followed when filing Test Case Description
** Link: [[TestCaseTemplate|Test Case Template]]
* Note! Test Case Template must be available also directly in Test Link database for manual TC’s
* Design Steps
** Each test step must have description and expected results defined.
** Each clause should start with a capital letter and end with full stop mark.
** One test step should contain only one activity.
** Measurement start and end points should be clearly defined in test steps.

== Technical Review ==
* Arranged by QA owner
* Wiki, Email and IRC to be used
* Participants: QA Owners, QA CC-owner, Developer, Test Engineer and community members
* Test Case(s) including design steps are reviewed from Test Link, Source Code review in need basis.
* TC status is marked as Proposal
* Target is to finalize and approve full test cases used in test execution and share information between QA owners and Test Engineers

== Test Case(s) ==
* Test cases in place in Test Link tool and test-xml’s available
* TC status is marked as Approved
* Test code available in agreed place (as development project in MeeGo.Com)
* Test data available in public domain <link to be added>

== Other Notes ==
* Test Plan and Scope creation/definition is excluded from this process
* Model of Coverage analysis matrix: <Link to be added here when available>

Quality/Plans/Test-plan-template

2011-02-17T06:21:34Z

Jkunnari: /* Test Cases */

Proposal - please contribute by editing this page or posting your comments on [[Talk:TestPlanTemplate|discussion area]] for this page.

Note that this is template giving the frame from actual test plan.
The intent is to list areas/things which one need to consider when planning needed QA activities for certain MeeGo Release (e.g. 1.2, 1.3).

== Introduction ==

=== Purpose ===

The purpose of the Test Plan is to introduce the testing strategy, scope, sub-plans and testing activities done or supported by QA team for MeeGo X.X release.

The intended readers for this document are people interested about the reasoning behind the actual activities done by QA team.

<please write more>

=== Objectives ===

The Test Plan is intended to provide the vehicle for QA team to confirm the functionality and completeness of the MeeGo X.X release. It is expected that the satisfaction of the complete series of criteria contained in this plan will signify successful functionality of the integrated release.

<please list more objectives>

=== Goal ===

The goal is to deliver a product with no open bugs with a severity level of critical and a minimal number of open bugs with severity level major.

<please add other goals>

=== Document Overview ===

==== Test Cases ====

During the the release phase aiming to final release, test cases will be executed. The test cases are described in test documents which are in gitorius at meego.gitorious.org/meego-quality-assurance. The actual test case code might be (depending on the license type and test level) part of the source package. Each test case also contains test specific criteria which decide upon test case success or failure.

Test cases shall fulfill the definitions given in [[Quality/Test_case_template|Test case template]]

<add other sources and definitions used>

==== Test Case Result ====

Verdict shall be assigned for a test case after execution according [[Quality/Glossary#Test_case_verdict|test case verdict instructions]].

==== Traceability of Features ====

For each feature listed bugs.meego.com, in principle, one or more test cases should exist (see detailed test plans). This information should be defined in the test case according instructions given in [[TestCaseTemplate|Test case template]]

==== Test Case Documentation ====

Each test case described in the detailed test plan contains the fields according [[TestCaseTemplate|Test case template]] and [http://meego.gitorious.org/meego-quality-assurance/test-definition test-definition]

== Overall test strategy ==

=== Introduction ===

Components can be tested by a combination of component (unit, module and component integration testing put together), integration and system tests.

For a system test the component is typically tested by launching the application in the test environment and by executing functionality of the component using the User Interface, and by verifying the outcome against an 'expected result'.
In cases where manual verification is required the expected outcome is specified in the test case.

Where possible all or parts of the test steps are automated, but in all cases there will be a manually executed 'Acceptance Test' for each GUI component in which the module will, at least, be verified against a standard checklist.

For an Integration test the component is typically tested by launching the component (if it has a GUI), or by launching other applications that make use of the tested component, inside the test environment and by executing functionality of the component using the User Interface. Where applicable, parts of the system may be replaced by stubs to partially insulate the component from its environment.

Unit tests are so named because they each test one unit of code. This type of tests are usually written by developers as they work on code (white-box style), to ensure that the specific function is working as expected. Whether a module of code has hundreds of unit tests or only five is irrelevant. One function might have multiple tests, to catch corner cases or other branches in the code. A test suite of unit tests should never cross process boundaries in a program, let alone network connections. Doing so introduces delays that make tests run slowly and discourage developers from running the whole suite.

Module test tests a module of code similar way than unit tests - the difference is that test are generated black-box style (without reference to the internal structure of the module).

Introducing dependencies on external modules or data also turns unit tests into component integration tests. If one module misbehaves in a chain of interrelated modules, it is not so immediately clear where to look for the cause of the failure. Tests are checking ready made part of the component with simulators (stubs, test drivers) for missing external dependencies.

=== Priorization ===

Tests are written on a highest priority first basis, and secondly on broad coverage.

<add other priorization rules used by vertical or project>

=== Coverage ===

At this stage we are aiming towards feature and functionality coverage.

<put additional coverage targets here>

=== Configurations ===

MeeGo X.X is tested in a number of configurations, both in reference hardwares and a QEMU environment. The public configurations used for this release are YY.

<update data according the information given by product management>

=== Test Execution ===

All automated tests are executed in a automated environment ([http://ots.meego.com/ OTS]), and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

<update your rules and environments here>

=== Test Reporting ===

Reporting for individual test sessions is done in [http://qa-reports.meego.com/ qa-reports]. Verticals might have additional reporting practices defined.

=== Test Tools ===

Additional tools such as Valgrind may be used to do a deep analysis of test details/failures and tools such as <please contribute> are used in the documentation process. These tools which will provide additional data used to analyse the quality of the product but are not necessarily used in the Go/No-Go decision making process.

<add your own tool portfolio here>

=== Testing Quality Characteristics ===

Check out the [[Quality/Test_areas_and_types|Test areas and types]].

==== Functionality ====

The basic functionality checklist can be categorized as below:
* Installation and un-installation
* Startup and exiting
* Screen Layout and Navigation
* Mobility and connectivity
* Security
* Touch interaction
* UI style
* Accessibility and clarity
* Help and documentation
* UI Controls
[[Quality/Test_areas_and_types#Functionality_quality_characteristics|Test areas and types for functionality]] has more details/ideas.

<list your goals for functional testing here>

==== Performance ====

The performance target is to produce a quality product with a performance that is competitive in the market. This is going to be achieved by first looking at the different areas that affect the performance of MeeGo X.X release, on actual reference hardwares, and then trying to measure these values to find out where performance needs to be improved.
The focus areas might be:
* Binary sizes
* Functional Performance
* Application Startup times
* Startup times
* Memory usage
[[Quality/Test_areas_and_types#Performance_quality_characteristics|Test areas and types for performance]] has more details/ideas.

Our goal to have a fast and responsive system having following targets:
* <list your measurable targets for performance aspects of the system here>

These are our initial targets only. Once we have reached these targets we will continue to improve and create a new set of targets to aim for.

<define your approach in more details here>

==== Reliability ====

Our goal is to have reliable system having following targets:
* <list your measurable targets for reliability aspects of the system here>
[[Quality/Test_areas_and_types#Reliability_quality_characteristics|Test areas and types for reliability]] has more details/ideas.

==== Usability ====

The usability check list can be categorized as below:
* Task success rates and Error prevention
* Help users recognize, diagnose, and recover from errors
* Ease of use
* Flexibility and efficiency of use
* Consistency and standards
* Time on task
* Task path, task flow and sequence
* Aesthetic and minimalist design
* User preference
* User control and freedom
[[Quality/Test_areas_and_types#Usability_quality_characteristics|Test areas and types for usability]] has more details/ideas.

== General Environment Setup ==

MeeGo can be tested on reference hardwares as well as in a QEMU environment. Both cases have value and may provide data that would be hard or impossible to acquire 'on the other side'.

=== Basic Environment Setup ===

Once MeeGo is installed in the QEMU environment or installed onto a reference hardware, no additional environment setup should be required that isn't already required by MeeGo itself. It is important though that the environment is equal for each testrun, i.e. the same background processes are running for each test run, and that only background processes are running that are required.

Certain aspects of MeeGo can only be tested if associated hardware modules are available on the reference hardware/desktop machine on which the test is executed. For instance:
* Bluetooth
* Wireless
* TV
* Radio
* Telephony
* etc.
Please refer to the manuals that come with these items to setup correctly.

<put reference to your test environment here>

=== Testing MeeGo in a QEMU environment ===

To test MeeGo in a QEMU environment, MeeGo needs to be built for the desktop which is accomplished by <put reference/link to the latest instructions over here>

=== Testing MeeGo on a reference hardware ===

To test MeeGo on a reference hardware, MeeGo needs to be built for that specific reference hardware, which is accomplished by <put reference/link to the latest instructions over here>

== Detailed Test Plans ==

<Structure taken here is following components and verticals in bugs.meego.com. This is provided as the illustrative example how subplans can be organized. Align this according your project setup>

{| border="1"
!
! Core OS
! Handheld UX
! Netbook UX
|-
|Bluetooth
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Contacts
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Content framework
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Graphics subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Media subsystem
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Photo viewer
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Qt
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Virtual keyboard
|<please add reference here>
|<please add reference here>
|<please add reference here>
|-
|Web browser
|<please add reference here>
|<please add reference here>
|<please add reference here>
|}

Quality

2011-02-17T06:20:26Z

Jkunnari:

= Quality =

WORK IN PROGRESS - This page is for MeeGo Quality Assurance material.
* ''You can join to'' '''[http://lists.meego.com/listinfo/meego-qa MeeGo-qa Mailing List]''' ''or follow daily activities through irc://irc.freenode.net/#meego-qa.''
* ''Meetings will be held weekly on Tuesdays 07:00 UTC in #meego-meeting on irc.freenode.net.''

== Projects ==

For all test reports see [http://qa-reports.meego.com qa-reports]

{| border = 1 valign="top"
! Release
! Core OS
! Handset UX
! Netbook UX
! SDK
! In Vehicle Infotainment
|-
| Release 1.2
|
* [[/Plans/MeeGo Core Test Plan | MeeGo Core Test Plan]]
* [[/TestSuite/MCTS | MeeGo Core Test Suite (MCTS)]]
* [http://qa-reports.meego.com/1.2/Core Core Test Reports]
|
* [[Quality/Plans/Handset UX test plan|MeeGo HandSet UX Test Plan]] - Generic HandSet UX QA test plan. Valid from MeeGo HandSet UX 1.2 release and onwards.
** [[Quality/Plans/Handset UX ramp-up 1.2|QA Ramp-Up Schedule for 1.2]]
* [[Quality/HandsetUXQATestFrequency#Handset_UX_QA_Testing_Frequency Handset Testing Frequency]]
* [[Quality/Reports/Handset-testability-status|Testability Status Report]]
* [[Quality/TestSuite/handset-test-suite|Test Suite]]
* [http://qa-reports.meego.com/1.2/Handset Handset Test Reports]
** [[Quality/Reports/MeeGo1.2|MeeGo 1.2 Test Reports]]
* [[Quality/HandsetQualityMetrics|Quality Metrics]]
|
* Test Suites & Utilities
** [[Quality/Netbook_Test_Suite_and_Utilities|Test Suites and Utilities]] - Here are some available test suites and utilities. Welcome download and contribute.
* Test Plan
* Test Suite
* [http://qa-reports.meego.com/1.2/Netbook Netbook Test Reports]
|
* [[SDKTestPlan|Test Plan]] describes the test strategy, test approach, checkpoints and etc of MeeGo SDK
* Test Suite
* [[/SDKTestReports | Test Report]]
* [http://qa-reports.meego.com/1.2/SDK SDK Test Reports]
|
* Test Plan
* Test Suite
* [http://qa-reports.meego.com/1.2/IVI IVI Test Reports]
|-
| Release 1.1
|
* [[/Reports/core-testability-status | Core OS Testability Status]]
* [[/Reports/Core-test-reports | Core Testing Reports]]
* [[/Defects/Core | Core Defects]]
|
* [[Quality/Plans/1.1 Handset UX TestPlan | MeeGo HandSet UX Test Plan]] (for 1.1)
** [[Quality/Plans/Handset UX ramp-up 1.1|QA Ramp-Up Schedule]]
* [[Quality/HandsetTestReport/MeeGo1.1|MeeGo 1.1 Test Reports]]
|
* MeeGo 1.1
** Test Plan
** Test Suite
** [[1.1NetbookTestReport|Test Report]]
|
|
|-
| Release 1.0
|
|
|
* MeeGo 1.0 Software Update Testing
** [[1.0SWUpdateTestPlan|Test Plan]] describes the test strategy, test approach, checkpoints and etc of software update testing for MeeGo 1.0 Netbook
*** You are encouraged to refer to the [[1.0SWUpdateTestPlan|Test Plan]] and follow [[TestReportTemplateCollection|Test Report Template]]
** [[Quality/1.0_sw_update_test_reports|Test Report Archive]]
|
|
|}

== QA tools ==

Quality assurance tools are developed to ensure MeeGo SW quality. They are developed and maintained by QA tools team.
* [[Quality/QA-tools|Meet the team and find our tools]]
* [[Quality/QA tools development|Participate in development activities]]. You are most welcome!

The following links provide some basic information on QA tools and their usage.
* [[Quality/QA-tools/How_to_set_up_repositories|Setting up the repositories for installing tools]] (please note that not all tools are in these repositories yet)
* [[Quality/QA-tools/Test packaging|Test packaging]]: Test packaging is the mechanism to wrap any tests in rpm packages to automate execution.
* [[Quality/QA-tools/Autotest-guide|Autotest-Guide]]: A guide for setting up automated testing environment. It includes instructions how to automate test execution and image installations.

== Procedures and best practices ==

* [[Quality/Plans/Testability-checklist|Feature Testability Checklist]]
** The basic intent of the feature review is to make sure that all features/requirements defined for MeeGo release are '''testable'''. The target is to map test cases in Distro testing against requirements, so we can determine whether features are '''done'''. This checklist / guideline should be used when reviewing MeeGo's features. Review comments will be given based on this checklist.
* [[Quality/Testability-commenting|Testability Commenting Guide]]
** This is a guide for QA contacts about how to comment on feature testability so as to provide as much information as possible to test developers.
*[[Quality/TestSuite/MCTS/MCTS_Development_Guideline | MeeGo Core Test Suites Development Guidelines]] 
*[[MeeGo Core Test Suites Packaging draft]] 
* [[Quality/Plans/Test-plan-template|Test plan template]] -- PROPOSAL, please contribute
* [[CompTestPlanTemplate|Component Test Plan Template]] -- WORK IN PROGRESS, please contribute
* [[TestReportTemplateCollection|Test Report Template]] -- WORK IN PROGRESS, pls. contribute
* [[Quality/Test_case_template|Test Case Template]] -- WORK IN PROGRESS, pls. contribute
* [[Quality/TestDesignProcessAndGuideline| Test Design Process and Guideline]] -- WORK IN PROGRESS, please contribute
* [[Quality/TestSetGuideline| Test Set Guideline]]
* [[Quality/Test management overview|Test management overview]]

== Defect tracking ==
* [[/defects | Defects Management]]
* [[/Bug_Life_Cycle_and_Handling|Bug Life Cycle and Report Handling/Follow-up Guidelines]]
* [[/Bugzilla_Fields|Fields in a bug report]]
* [[/Bugtriage | MeeGo Bug Triage]]
* [[/How_To_Report_Bugs|How to report bugs]]
* [[/Bug_Access_Restrictions|Bug Report Access Restrictions]]
* [[MeeGoBugzilla_Customization|Bugzilla customized features]]

== Compliance Program ==
[[Quality/Compliance | MeeGo Compliance Program]] - the compliance spec (initially)

== Meetings ==
Meetings will be held weekly on Tuesdays 07:00 UTC in #meego-meeting on irc.freenode.net.
* [[/meetings | Meetings ]]

== See also ==

* [[/Glossary | QA Glossary ]]
* [[/Test_areas_and_types | Short descriptions for Test Areas/Types]]
* [[Quality/Plans/Quality-considerations|Quality Considerations / Testing Quality Characteristics]]
** Some ideas around testing quality characteristics hopefully helping you identify what kind of things could be checked from applications - can be used as a frame to define test considerations – as a set of logical test cases - for applications targeting to run on MeeGo.

TestCaseTemplate

2011-02-17T06:18:26Z

Jkunnari: moved TestCaseTemplate to Quality/Test case template: page moved to correct place.

#REDIRECT [[Quality/Test case template]]