Meego Wiki
Views

Quality/1.2NetbookTestPlan

From MeeGo wiki
< Quality(Difference between revisions)
Jump to: navigation, search
(Created page with "= MeeGo 1.0/1.1/1.2 Netbook Software Update Test Plan = == Test Summary == MeeGo 1.0/1.1/1.2 Netbook Software update testing is to validate the functionality of entire MeeGo 1.0...")
(→Referrences)
 
(11 intermediate revisions not shown)
Line 26: Line 26:
=== Test Cycle ===
=== Test Cycle ===
-
* Weekly Update Repo
+
* Weekly Sanity Testing
** Generally, update repo will be delivered each 1-2 weeks based on bug fix progress;
** Generally, update repo will be delivered each 1-2 weeks based on bug fix progress;
** QA will do sanity testing when new update repo is ready and cover:
** QA will do sanity testing when new update repo is ready and cover:
-
** 1) UX component sanity test case;
+
*** UX component sanity test case;
-
** 2) Core component sanity test case;
+
*** Core component sanity test case;
-
** 3) User data preserving testing to ensure no user data is lost after update;
+
*** User data preserving testing to ensure no user data is lost after update;
-
** 4) Bug verification and regression testing;
+
*** Bug verification and regression testing;
* Monthly System Use Testing
* Monthly System Use Testing
** QA will complete one round of system use testing to evaludate whole update system quality for update release final repo;
** QA will complete one round of system use testing to evaludate whole update system quality for update release final repo;
** QA will complete one round of power and performance testing to evaludate update system performance quality based on requirement/change;
** QA will complete one round of power and performance testing to evaludate update system performance quality based on requirement/change;
-
 
-
=== Test Cadence ===
 
-
 
-
=== Test Coverage ===
 
== Test Design ==
== Test Design ==
Line 218: Line 214:
* P1 Platform
* P1 Platform
** Pinetrail Netbooks:
** Pinetrail Netbooks:
 +
*** Lenovo S10-3
*** HP Mini
*** HP Mini
*** Asus EeePC 1005PE
*** Asus EeePC 1005PE
Line 225: Line 222:
=== Test Repos ===
=== Test Repos ===
 +
==== MeeGo 1.0 Update ====
* MeeGo 1.0 Release Repo
* MeeGo 1.0 Release Repo
-
** http://repo.meego.com/MeeGo/releases/1.0/core/
+
** http://repo.meego.com/MeeGo/releases/1.0/netbook/images/
-
** http://repo.meego.com/MeeGo/releases/1.0/extra/
+
-
** http://repo.meego.com/MeeGo/releases/1.0/netbook
+
* MeeGo 1.0 Update Repo
* MeeGo 1.0 Update Repo
** Formal Repo:
** Formal Repo:
-
*** http://repo.meego.com/MeeGo/updates/1.0/core/
+
*** http://repo.meego.com/MeeGo/updates/1.0/
-
*** http://repo.meego.com/MeeGo/updates/1.0/extra/
+
-
*** http://repo.meego.com/MeeGo/updates/1.0/netbook/
+
** Test Repo:
** Test Repo:
*** http://repo.meego.com/MeeGo/builds/1.0/latest/
*** http://repo.meego.com/MeeGo/builds/1.0/latest/
 +
==== MeeGo 1.1 Update ====
 +
* MeeGo 1.1 Release Repo
 +
** http://repo.meego.com/MeeGo/releases/1.1/netbook/images/meego-netbook-ia32/
 +
* MeeGo 1.1 Update Repo
 +
** Formal Repo:
 +
*** http://repo.meego.com/MeeGo/updates/1.1/
 +
** Test Repo:
 +
*** http://repo.meego.com/MeeGo/builds/1.1/latest/
 +
==== MeeGo 1.2 Update ====
 +
* MeeGo 1.2 Release Repo
 +
** http://repo.meego.com/MeeGo/releases/1.2.0/images/meego-netbook-ia32/
 +
* MeeGo 1.2 Update Repo
 +
** Formal Repo:
 +
*** http://repo.meego.com/MeeGo/updates/1.2.0/
 +
** Test Repo:
 +
*** http://repo.meego.com/MeeGo/builds/1.2.0/latest/
== QA Contact ==
== QA Contact ==
* daniel.tao@intel.com
* daniel.tao@intel.com
-
== Developers ==
+
== Release Engineer ==
-
* peter.j.zhu@intel.com "Release Engineer for MeeGo v1.0 Netbook Update"
+
* ning.w.wang@intel.com/yang.jie@intel.com: "Release Engineer for MeeGo v1.0/1.1/1.2 Netbook Update"
== Test Report ==
== Test Report ==
 +
=== MeeGo 1.0 Update Test Report===
 +
* Sanity Test Report
 +
** http://qa-reports.meego.com/1.0/Netbook/Sanity/Pinetrail
 +
* System Use Test Report
 +
** http://qa-reports.meego.com/1.0/Netbook/System%20Functional/Pinetrail
 +
 +
=== MeeGo 1.1 Update Test Report===
 +
* Sanity Test Report
 +
** http://qa-reports.meego.com/1.1/Netbook/Sanity/Pinetrail
 +
* System Use Test Report
 +
** http://qa-reports.meego.com/1.1/Netbook/System%20Functional/Pinetrail
 +
 +
=== MeeGo 1.2 Update Test Report===
 +
* Sanity Test Report
 +
** http://qa-reports.meego.com/1.2/Netbook/Sanity/Pinetrail
 +
* System Use Test Report
 +
** http://qa-reports.meego.com/1.2/Netbook/System%20Functional/Pinetrail
== Referrences ==
== Referrences ==
 +
* 1.2 Release Plan: http://wiki.meego.com/Release_Engineering/Plans/1.2
 +
* Software Update Process: http://wiki.meego.com/Release_Engineering/Software_update_process

Latest revision as of 02:57, 17 June 2011

Contents

MeeGo 1.0/1.1/1.2 Netbook Software Update Test Plan

Test Summary

MeeGo 1.0/1.1/1.2 Netbook Software update testing is to validate the functionality of entire MeeGo 1.0/1.1/1.2 system updated distribution, to assure that the health of the distribution after system update, validate whether relative bugs are fixed by update patch and report regression issues for software update. Meanwhile, we will also ensure that user preserving data will not be removed or changed after software update.

Feature to be Tested

  • MeeGo 1.0/1.1/1.2 final release can be successfully upgraded to MeeGo 1.0/1.1/1.2 update release, including incremental update and service pack update
  • New features or bug fixings introduced by updated packages
  • Functionality and reliability in resulting upgraded system function as the same as the fresh installed MeeGo 1.0/1.1/1.2 update release.
  • Performance of MeeGo 1.0/1.1/1.2 update release is as good as 1.0/1.1/1.2 final release at least.
  • All system and user settings and application-specific data should be preserved after the upgrade.

Test Strategy and Approach

Test Approach

  • Install/Update system in various ways as below:
    • Upgrade from 1.0/1.1/1.2 final release
    • Upgrade from previous service packs
  • Validate the system with below criteria:
    • Software update can be previewed and easily completed via both application and command line (zyper).
    • New functions of latest build works
    • Unchanged functions are not broken
    • User data are preserved during system upgrading, and keep effects in the upgraded system.
    • No regression on performance and reliability of whole system
  • Validate from end-users' perspective
    • System Use Testing

Test Cycle

  • Weekly Sanity Testing
    • Generally, update repo will be delivered each 1-2 weeks based on bug fix progress;
    • QA will do sanity testing when new update repo is ready and cover:
      • UX component sanity test case;
      • Core component sanity test case;
      • User data preserving testing to ensure no user data is lost after update;
      • Bug verification and regression testing;
  • Monthly System Use Testing
    • QA will complete one round of system use testing to evaludate whole update system quality for update release final repo;
    • QA will complete one round of power and performance testing to evaludate update system performance quality based on requirement/change;

Test Design

Sanity Test Checkpoints

Checkpoints Details
Image update
  • Packages can be updated by system update app
System startup & shutdown (Goal: same with 1.0 final release at least)
  • Boot from local disk (14.0s for the system with HDD)
  • Shutdown (Goal: 9.1s)
Suspend & resume (Goal: same with 1.0 final release at least)
  • Suspend via closing lid and cmd (Goal: 2.1s)
  • Resume (Goal: 3.6s)
Powertop Log
  • Powertop log analysis, Cn state, avg resident, Wakeup times
Stability
  • Shutdown and Reboot system 10 times without failure or crash
  • Use multiple applications (>8, media player, browser, IM, email, and so on) based on scopes defined in other checkpoints
Network/connectivity - ethernet, wifi connection basic check
  • Check Ethernet connection
  • Wifi scan & associate (check from carrick)
UI shell - dropdown panels basic check , zones switch, application launcher to launch key applications
  • Dropdown panel: click icons on drop down panel to check corresponding panels could launched(Will not test the functionality of panels to avoid to many tests)
  • Zones: switch in zone panel; drag application to another zone; switch by alt+tab
  • App-launcher: check if the UI is correct; check if key applications (browser, media, about MeeGo, settings) could be launched
Myzone - web service, event, favorite apps basic check
  • Check if UI is correct
  • Check if web services could be fetched
  • Check if new dates&tasks could be displayed
  • Check browser history and recent docs
Internet Applications -- Multi-tabs websites surfing and flash plugin check
  • Open 2 tabs to surf 2 websites
  • Check if flash plugin could basically work
Multimedia -- audio/video/photo playback, and play views switch
  • Check if the UI is correct (top bar, side bar, play queue)
  • Check if sample audio/video/photo could playback successfully
  • Check if thumbnail/index of sample picture/video works
Device Panel
  • Check if the device panel could basically work
Peripheral devices (touchpad, USB, SD, LCD)
  • Touchpad
  • USB keyboard/mouse
  • External monitor
  • USB stick/SD/LCD
Basic linux command such as Zypper install/ debuginfo-install
  • Zypper install/default repo configuration

User Data Preserving Checkpoints

Checkpoints Details
Hardware related settings .
  • Check Keyboard, Trackpad and Mouse setting have no change
  • Check Printer setting has no change
  • Check Sound setting has no change
  • Check Power and Brightness setting have no change
  • Check Display setting has no change
  • Check Proxy setting has no change
Customized Layout Settings
  • Check Wallpaper has no change
  • Check Myzone has no change
  • Check Systray setting has no change
  • Check Fonts has no change
  • Check Toolbar setting has no change
User profiles
  • Check User Name, Password has no change
  • Check Date/Time setting has no change
  • Check Web Accounts setting has no change
  • Check Email Settings has no change
  • Check IM Settings has no change
  • Check Language setting has no change
MyZone panel
  • Check Tasks/Appointments setting have no change
  • Check Favorite Apps setting has no change
  • Check RecentDocs/WebService Update have no change
People Panel
  • Check buddy list and history have no change
Status Panel
  • Check Cached data has no change
Application Panel
  • Check Favorite Apps has no change
  • Check Installed Apps have no change
Internet Panel
  • Check Favorite pages have no change
Time and Date panel
  • Check Time/Appointments/Tasks have no change
Bluetooth panel
  • Check Bluetooth devices list has no change
Media
  • Check Music in play queue/play list have no change
  • Check Import files have no change
  • Check Shuffle/repeat setting have no change
  • Check Music metadata has no change
Web Browser
  • Check Web-browser Preferences have no change
  • Check Web-browser Bookmarks have no change
  • Check Web browser History have no change
  • Check Thumbnails in web-browser New Tab have no change
  • Check the passwords typed by user in web have no change
Email
  • Check Inbox mails have no change
  • Check Mail accounts have no change
Calendar/tasks/contacts
  • Check Calendar have no change
  • Check Tasks have no change
  • Check Contacts info have no change
Network
  • Check Network/connectivity setting- ethernet, wifi have no change
File system
  • Check user created file/folder has no change

Test Environment

Hardware Platform

  • P1 Platform
    • Pinetrail Netbooks:
      • Lenovo S10-3
      • HP Mini
      • Asus EeePC 1005PE
  • P2 Platform
    • Calpella Notebook
    • Diamondville Nettops

Test Repos

MeeGo 1.0 Update

MeeGo 1.1 Update

MeeGo 1.2 Update

QA Contact

  • daniel.tao@intel.com

Release Engineer

  • ning.w.wang@intel.com/yang.jie@intel.com: "Release Engineer for MeeGo v1.0/1.1/1.2 Netbook Update"

Test Report

MeeGo 1.0 Update Test Report

MeeGo 1.1 Update Test Report

MeeGo 1.2 Update Test Report

Referrences

Retrieved from "http://wiki.meego.com/Quality/1.2NetbookTestPlan"
Personal tools