Proposal - please contribute by editing this page or posting your comments on discussion area for this page.
Note that this is template giving the frame from actual test plan. The intent is to list areas/things which one need to consider when planning needed QA activities for certain MeeGo Release (e.g. 1.2, 1.3).
Contents |
The purpose of the Test Plan is to define a strategy, scope and testing activities done or supported by SDK QA team for MeeGo 1.3 release involving the HW-accelerated MeeGo Emulator for Mac & Windows host, that is developed and maintained by Intel. This plan describes the QA approach of testing the VMM module.
Objectives in VMM QA test plan is to validate the functionality and stability of VMM module running on diffrent host OS'es. The OS'es that we are focusing on are Mac OS and Windows. The target is to ensure that:
The goal is to deliver a product with no open bugs with a severity level of critical and a minimal number of open bugs with severity level major.
During the the release phase aiming to final release, test cases will be executed. Test cases should be run from Test link and they should cover the main Objectives described above. Each test case also contains test specific criteria which decide upon test case success or failure.
Verdict shall be assigned for a test case after execution according test case verdict instructions.
For each feature listed in bugs.meego.com, in principle, one or more test cases should exist. This information should be defined in the test case according instructions given in Test Case Template
Each test case described in the detailed test plan contains the fields according Test Case Template and test-definition
The overall objective of VMM QA is to ensure the stability and performance of the VMM module that will be used in Windows and Mac OS hosts. For example the image under QEMU is not affected by the VMM module, the Host is stable when VMM service is running and so on. Different test types will be done, including:
Tests will be ran on QEMU to ensure that it's components were not affected by the VMM. The tests performed will be the basic tests that are performed on a QEMU weekly image. They should cover the overall behavior of the QEMU.
After ensuring that basic tests are passed, the VMM module will be tested and monitored by checking it's stability under various conditions (eg. VMM does not crash/ restart if the Host is under stress) or check the amount of memory it's using when idle or when playing a movie.
The priorities are as follows:
| Host\Target | VMM |
|---|---|
| Mac OS | P1 |
| Windows XP | P2 |
| Windows 7 | TBD |
At this stage we are aiming towards feature and functionality coverage.
VMM will tested in a number of computer configurations that include Mac and Windows.
The tests will be executed manually, until some of the testing process will be automated. The automation tests will vary according to the Host OS that they will be run.
Reporting for individual test sessions is done in qa-reports. Verticals might have additional reporting practices defined.
QA will focus to automate tests as much as possible. On benchmarks, various tools will be used including OpenArena, Sunspider, Phoronix Test Suite. They will give data helping us to compare the performance of the emulator when VMM is used.
Check out the Test areas and types.
The basic functionality checklist can be categorized as below:
Test areas and types for functionality has more details/ideas.
The performance target is to produce a quality product with a performance that is competitive in the market. This is going to be achieved by first looking at the different areas that affect the performance of the emulator and then trying to measure these values to find out where performance needs to be improved. The focus areas might be:
Our goal to have a fast, responsive and stable emulator using VMM on Mac and Windows.
Our goal is to have a reliable(stable) VMM module in the following targets:
The VMM should be available on any machine that runs Windows or Mac OS.