Meego Wiki
Views

Quality/1.2.0 IVI Update Test Plan

From MeeGo wiki
< Quality(Difference between revisions)
Jump to: navigation, search
(Created page with "= MeeGo 1.2.0 IVI Software Update Test Plan = == Test Objective == MeeGo 1.2.0 IVI Software update testing is to validate the functionality of entire MeeGo 1.2.0 system updated ...")
(→Test Report)
 
(5 intermediate revisions not shown)
Line 38: Line 38:
== Test Design ==
== Test Design ==
-
=== Sanity Test Checkpoints ===
+
=== Checkpoints ===
 +
Note: '''checkpoint''' in bold means it is also a sanity test point.
 +
 
{|cellspacing="0" border="1"
{|cellspacing="0" border="1"
-
!|Checkpoints
+
!|Category
-
!|Details                 
+
!|Checkpoints                 
|-
|-
-
|Installation
+
|System Overall
|
|
-
* Installer/Installer-shell launch and launch time
+
* [''Startup''] Liveboot/'''boot from local disk'''
-
* Installation in text mode and system startup(installer removal)
+
* [''Installation''] Installer/Installer-shell launch and launch time
-
* Installation in GUI mode and system startup(installer removal)
+
* [''Installation''] '''Installation in text mode and system start up(installer removal)'''
 +
* [''Installation''] Installation in GUI mode and system start up(installer removal)
 +
* [''Peripherals''] USB/SD auto-mounter/read/write
 +
* [''AppManager''] '''Zypper install'''/update/debuginfo-install/source-install default repo configuration
 +
* [''AppManager''] pkcon install/update
 +
* [''Settings''] date&time settings
 +
* [''Settings''] system build ID
 +
* [''Pnp''] '''Cold boot time'''
 +
* [''Pnp''] Powertop log analysis, Cn state, avg resident, Wakeup times
 +
* [''Stability''] Shutdown and Reboot system 10 times without failure or crash
 +
* [''Stability''] Use multiple applications
|-
|-
-
|System startup & shutdown
+
|Hardware Adaptation
|
|
-
* Boot from local disk(X and console)
+
* ML7213 drivers
-
* Cold boot time and shutdown time
+
* EG20T drivers
-
* Startx from console
+
* '''touch screen'''
 +
* '''Ethernet'''
 +
* Wifi
 +
* '''snd card'''
 +
* onboard MMC
 +
* '''removable MMC/USB'''
 +
* '''USB HID'''/hub/LAN/BT dangle
 +
* '''dmesg'''
|-
|-
-
|Powertop Log
+
|Graphics Driver(EMGD)
|
|
-
* Powertop log analysis, Cn state, avg resident, Wakeup times
+
* '''System Startup with EMGD enabled'''
 +
* startx from console
 +
* Switch between X/console
 +
* '''Desktop rendering'''
 +
* GUI installer rendering
 +
* Check basic 3D application glxgears can be run successfully
 +
* '''Single(LVDS) display'''
|-
|-
-
|Stability
+
|Connectivity/Communication
|
|
-
* Shutdown and Reboot system 10 times without failure or crash
+
* '''Ethernet connection'''/disconnection/file transfer(proxy)
-
* Use multiple applications
+
-
|-
+
-
|XFCE desktop
+
-
|
+
-
* Desktop render
+
-
* X/console switch
+
-
* XFCE menu view
+
-
* app launch/close/switch
+
-
|-
+
-
|Connectivity -- Multi-tabs websites surfing and flash plugin check
+
-
|
+
-
* Ethernet connection/disconnection/file transfer(proxy)
+
* Wifi scan/connection/disconnection
* Wifi scan/connection/disconnection
-
* Bluetooth pairing/connection(A2DP, HFP)
+
* Bluetooth pairing/connection(A2DP, '''HFP''')
|-
|-
|Multimedia  
|Multimedia  
Line 81: Line 94:
* aplay/arecord
* aplay/arecord
* paplay/parec
* paplay/parec
-
* gstreamer play audio/video
+
* '''gstreamer play audio/video'''
|-
|-
-
|Peripheral devices 
+
|XFCE desktop
|
|
-
* Touch Screen
+
* '''Desktop render'''
-
* USB keyboard/mouse
+
* XFCE menu view
-
* USB stick/SD
+
* '''app launch/close/switch'''
-
|-
+
-
|Basic linux command such as Zypper install/ debuginfo-install 
+
-
|
+
-
* Zypper install/update/debuginfo-install/source-install default repo configuration
+
-
* pkcon install/update
+
|-
|-
|}
|}
Line 99: Line 107:
=== Hardware ===
=== Hardware ===
* Crossville/Lapis
* Crossville/Lapis
 +
** CRB = Brookville + ICM(Intel® Atom™ CPU E6XX Series + LAPIS ML7213 IOH)
 +
** BIOS: AMI 4.6.3.7/0ABXJ 0.03
 +
** Display: 7’’ LVDS
 +
** Hard drive: SATA SSD 8G
* Crown Bay
* Crown Bay
 +
** Crown Bay CRB = Shell Bay Fab2(Intel® EG20T IOH) + Little Bay REV D(Intel® Atom™ CPU E6XX Series)
 +
** BIOS: AMI 4.6.3.7, Project 0ABTN 0.20
 +
** Display = 13’’ LVDS
 +
** Hard drive: SATA HD 160G
 +
** wifi card: Intel wifi link 5100
=== Test Repos ===
=== Test Repos ===
Line 116: Line 133:
* Sanity Test Report
* Sanity Test Report
** http://qa-reports.meego.com/1.2/IVI/Sanity/
** http://qa-reports.meego.com/1.2/IVI/Sanity/
-
* System Use Test Report
+
* Milestone Test Report
** http://qa-reports.meego.com/1.2/IVI/Milestone/
** http://qa-reports.meego.com/1.2/IVI/Milestone/
== Referrences ==
== Referrences ==
* Software Update Process: http://wiki.meego.com/Release_Engineering/Software_update_process
* Software Update Process: http://wiki.meego.com/Release_Engineering/Software_update_process

Latest revision as of 06:26, 19 October 2011

Contents

MeeGo 1.2.0 IVI Software Update Test Plan

Test Objective

MeeGo 1.2.0 IVI Software update testing is to validate the functionality of entire MeeGo 1.2.0 system updated distribution, to assure that

  • the health of distribution built based on 1.2.0 release repo and update repo(1.2.0 update release build)
  • the health of 1.2.0 release distribution after system update
  • validate whether relative bugs are fixed by update patch
  • explore regression issues when system is with updated software

Feature to be Tested

  • MeeGo 1.2.0 IVI update release(XFCE desktop, BAT drivers/middleware)
  • MeeGo 1.2.0 IVI final release can be successfully upgraded to update release
  • New features or bug fixings introduced by updated packages
  • Functionality and reliability in resulting upgraded system function
  • Performance of MeeGo 1.2.0 update release

Test Strategy and Approach

Test Approach

  • Install/Update system in various ways as below:
    • Install 1.2.0 update release build directly to get latest software stuff
    • system Upgrade from 1.2.0 final release
  • Validate the system with below criteria:
    • Software update can be previewed and easily completed via command line (zypper, pkcon).
    • New functions of latest build works
    • Unchanged functions are not broken
    • No regression on performance and reliability of whole system

Test Cycle

  • Sanity Testing
    • Generally, update repo will be delivered each 1-2 weeks based on bug fix progress;
    • QA will do sanity testing when new update repo/build is ready and cover:
      • XFCE sanity testing
      • Middleware components sanity testing
      • IVI drivers sanity testing
      • Bug verification and regression testing
  • System Function Testing
    • QA will conduct one round of system function testing to evaluate whole update system quality for update release final repo

Test Design

Checkpoints

Note: checkpoint in bold means it is also a sanity test point.

Category Checkpoints
System Overall
  • [Startup] Liveboot/boot from local disk
  • [Installation] Installer/Installer-shell launch and launch time
  • [Installation] Installation in text mode and system start up(installer removal)
  • [Installation] Installation in GUI mode and system start up(installer removal)
  • [Peripherals] USB/SD auto-mounter/read/write
  • [AppManager] Zypper install/update/debuginfo-install/source-install default repo configuration
  • [AppManager] pkcon install/update
  • [Settings] date&time settings
  • [Settings] system build ID
  • [Pnp] Cold boot time
  • [Pnp] Powertop log analysis, Cn state, avg resident, Wakeup times
  • [Stability] Shutdown and Reboot system 10 times without failure or crash
  • [Stability] Use multiple applications
Hardware Adaptation
  • ML7213 drivers
  • EG20T drivers
  • touch screen
  • Ethernet
  • Wifi
  • snd card
  • onboard MMC
  • removable MMC/USB
  • USB HID/hub/LAN/BT dangle
  • dmesg
Graphics Driver(EMGD)
  • System Startup with EMGD enabled
  • startx from console
  • Switch between X/console
  • Desktop rendering
  • GUI installer rendering
  • Check basic 3D application glxgears can be run successfully
  • Single(LVDS) display
Connectivity/Communication
  • Ethernet connection/disconnection/file transfer(proxy)
  • Wifi scan/connection/disconnection
  • Bluetooth pairing/connection(A2DP, HFP)
Multimedia
  • aplay/arecord
  • paplay/parec
  • gstreamer play audio/video
XFCE desktop
  • Desktop render
  • XFCE menu view
  • app launch/close/switch

Test Environment

Hardware

  • Crossville/Lapis
    • CRB = Brookville + ICM(Intel® Atom™ CPU E6XX Series + LAPIS ML7213 IOH)
    • BIOS: AMI 4.6.3.7/0ABXJ 0.03
    • Display: 7’’ LVDS
    • Hard drive: SATA SSD 8G
  • Crown Bay
    • Crown Bay CRB = Shell Bay Fab2(Intel® EG20T IOH) + Little Bay REV D(Intel® Atom™ CPU E6XX Series)
    • BIOS: AMI 4.6.3.7, Project 0ABTN 0.20
    • Display = 13’’ LVDS
    • Hard drive: SATA HD 160G
    • wifi card: Intel wifi link 5100

Test Repos

QA Contact

Test Report

Referrences

Retrieved from "http://wiki.meego.com/Quality/1.2.0_IVI_Update_Test_Plan"
Personal tools