Quality/Plans/Meego1.2 media test plan

2010-12-03T07:27:54Z

Jessicaji: /* Feature to be Tested */

=Media Application Test Plan for MeeGo HandSet 1.2=
==Component Summary==
Media subsystem plays an important role in Handset vertical. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application. Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, QML,Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.2 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
The features listed here are all from MeeGo handset 1.2 media components feature zilla, media test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as tool bar, i18/l10n etc.
* Detail feature list please see [http://wiki.meego.com/Quality/Plans/Meego1.2_media_features Meego1.2_media_features]
* Feature summary please see below:

{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio Player
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features Audio_Feature]
|-
|Photo Viewer
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features Photo_Features]
|-
|Video Player
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features Video_Features]
|-
|Camera
|*Basic operation with Camera *Shutter Key *Mode select *Capture or stop capture *Pre-capture & Post-capture *Auto focus *Flash *Zoom *Erase image and store image *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Camera%20application&product=MeeGo%20Handset%20Features&query_format=advanced&order=bug_id&query_based_on= Camera_Features]
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility,v4l2, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
* only support public codec list, we don't release any license codec.

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video/Camera application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N/L10n testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.
* Test case leverage
more design is same with tablet vertical. we will leverage to use more test cases to avoid overlap.

==Test Design==
====Audio====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Audio Playback control
|
* Audio play/pause
* Audio mute/unmute
* Back to previous song
* Skip to next song
* Repeat playback
* Shuffle playback
|-
|artist/album/song Navigation$Playback
|
* Select given artist and browse the all albums within it
* Select given album and browse the all songs within it
* Playback all songs within one artist
* Playback all songs within one album
* Playback one song
|-
|Alphabetic scrolling
|
* Scrolling to a given character head in My Music view
* Scrolling to a givne character head in playlist view
|-
|Playlist manipulation
|
* Create one new playlist
* Delete one playlist
* Add all songs within one artist to a given playlist
* Add all songs wihtin one Album to a given playlist
* Add one song to a given playlist
* Playback a given playlist
|-
|Audio Search
|
* Search Artists/Albums/Songs by a given string
* All results/Artists/Albums/Songs tab view switch
* Playback the selected item from search result
* Add the selected item from search result into a given playlist
|-
|Play Queue manipulation
|
* Switch to PlayQueue view
* Add a given song/album/artist to PlayQueue
* Check PlayQueue status display like show current song title in bottom bar
|-
|gesture
|
* Flick action on lists causes the list to be scrolled up/down.
* Tapping on a letter name causes the artist list to jump to artist name starting with selected letter
|-
|Notifications
|
* Song title/album art and artist name are visible in notification area
* Pause and mute button are also visible in notification
|-
|Volume control
|
* Audio play in foreground with volume control
* Audio play in background with volume control
|-
|Usability testing
|
* navigate the photos and listening music
* navigate the browsers and listening music
* audio volume control in foreground and background
|-
|stress testing
|
* Playback the audio for a long time
* Browse great number of audio files
* Change the playback state randomly
|-
|Negative testing
|
* Playback non-audio files
* Playback un-supported files
|-
|I18N/L10n testing
|
* Switch language between Simplified Chinese and English
|-
|}

====Photo====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|View Function
|
* Photo thumbnail
* View a given photo
* Photo slide show
* Photo navigation (Previous, Next)
|-
|Format
|
* JPEG
* BMP
* PNG
* GIF
|-
|Album Generation
|
* Organize photo album taken in similar time
* Organize photo album taken in similar place
|-
|Search
|
* Search a photo by a given string
* Search an album by a given string
* Search result view switch between photo and album
|-
|Contextual Menu
|
* Slide show
* Tag
* Copy
* Rotate CW
* Rotate CCW
* Delete
|-
|Orientation
|
* Portrait
* Landscape
|-
|Operation
|
* save
* delete
* rotate
* set as wallpaper
* set as avatar
|-
|Info
|
* review image detail with metadata
|-
|Sharing
|
* share with sms
* share with email
* share with bluetooth
* share with flickr
* share with facebook
* share with picasa
|-
|Resize
|
* Photo panning and scrolling
* Photo zoom
|-
|Stress
|
* Slide show the photo files for a long time
* Browse great number of photo files
* view large photo file
|-
|Negative
|
* View non-picture files
* View un-supported picture files
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}

====Video====
In MeeGo 1.2 release, video player test points includes:
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Play Control
|
* play
* pause
* seek
* fastforward
* backforward
|-
|Info
|
* total time line
* reserved time line
|-
|Alphabetic scrolling
|
* sort
* gesture
|-
|contextual menu
|
* play
* delete
* favorite
|-
|Search
|
* by name
* by words
* by artist
|-
|Streaming
|
* by SDP file with domain name
* by SDP file with ip address
* by rtsp session
|-
|Stress
|
* Playback a long video file
* Browse great number of video files
* Change the playback state randomly
|-
|Negative
|
* Playback non-video files
* Playback un-supported video files
|-
|I18N/L10N
|
** Switch language between Simplified Chinese and English
|-
|}

====Camera====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Capture
|
* Start camera via app menu/lens cover/shutter key
* Close camera via app menu/lens cover
|-
|Mode select
|
** capture photo
** capture video
|-
|Pre-capture & Post-capture
|
* pre-capture
* post-capture
|-
|Autofocus
|
* Autofocus indicator
* Autofocus performance
|-
|Flash
|
* Flash on
* Flash off
* Flash auto
|-
|zoom
|
* zoom in
* zoom out
|-
|file operation
|
* Erase image
* store image
|-
|self timer
|
* Self timer
* burst mode
|-
|Stress
|
* Capture a long video file
* Capture one high definition video file
* Capture one high definition photo file
* burst mode capture for long time
* Change the capture mode randomly
|-
|Negative
|
* no power while capture
* under sun light environment while capture
* far away with target while capture
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}
===Collaboration===
Mainly for the conditions that the application will meet, mainly introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
| Virtual Keyboard
|
* Input strings via virtual keyboard
* Short cut key to operate browser
| .
|-
|2
| Device management
| file privisioning or device sync
| .
|-
|3
|Web browser
|
* try to progressive download one video or audio file
* try to launch video or audio player from web browser
| .
|-
|4
| telephony
| how to handle audio/video playback while one phone is comming
| .
|-
|5
| Location
|
* all media applications need to support landscape and portrait mode
* photo exif data will have location info. and group with place
| .
|-
|6
| webservice
|
* through SMS share some video/photo/audio with others
* through email share
* through bluetooth share
* through popular website share
| .
|-
|7
| Notification
|
* handle notification while playing music
* handle notification while playing video
* handle notification while playing photo
* handle notification while capture picture by camera application
| .
|-
|8
| Instant Message
|
* browser with one photo, set with avatar
| .
|-
|}

===Condition===
Mainly for the conditions that the application will meet, maily introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
|Touch framework supported devices
|Test devices need support touch framework
|.
|-
|2
|Wired network
|
Connects to wired network:
* Using proxy
* Without proxy
|.
|-
|3
|Wireless network
|
Connects to wireless network:
* Using proxy
* Without proxy
| .
|-
|4
|Landscape and portrait views
|
Change screen orientation to get different views:
* Landscape view with proper window size
* Portrait view with proper windows size
| .
|-
|5
|Power state
|
The state may include:
*idle
*sleep
*resume
| .
|-
|6
|Touch and gesture
|
* MeeGo Touch Framework gestures
* Additional gestures support in media applications:
**Pinch to zoom in and out, scaling the whole page.
**Flick back and forward to go back and forward with previous/next track;
**Scroll to page up and down
**Finger scrolling to move up and down
**Long press to trigger context menu
**Double tap to switch zoom level
**Two-fingers long press to select text
| .
|-
|7
|Other Devices
|
* speaker
* headset
* bluetooth headset
| .
|-
|8
|Virtual Keyboard
|
* Virtual keyboard interaction
* hardware keyboard interaction
| .
|-
|}

==Test Environment==
===Hardware Platforms===
* NCDK and ICDK

===Network===
* Internal network uses proxy to access internet
* Access internet directly without any proxy
* wireless
===Tools===
* gst-launch
* paplay
* aplay

==QA Contact==
[mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio Player
** [mailto:forrest.zhao@intel.com Forrest Zhao] and [mailto:jia-cheng.hu@intel.com Jason Hu]
* Photo viewer
** [mailto:ziv.chang@intel.com Ziv Chang]
* Camera
** pending

==References==
* Application Test Design Method ---TBD
* MeeGo UI design wireframes

Quality/Plans/Meego1.2 media features

2010-12-03T07:26:06Z

Jessicaji: Created page with "{| {{table}} | align="center" style="background:#f0f0f0;"|'''ID''' | align="center" style="background:#f0f0f0;"|'''Summary''' | align="center" style="background:#f0f0f0;"|'''Comp…"

{| {{table}}
| align="center" style="background:#f0f0f0;"|'''ID'''
| align="center" style="background:#f0f0f0;"|'''Summary'''
| align="center" style="background:#f0f0f0;"|'''Comp'''
|-
| [http://bugs.meego.com/show_bug.cgi?id=4220 4220]||[FEA] Music Player - Main screen||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4221 4221]||[FEA] Music Player - Top toolbar||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4222 4222]||[FEA] Music Player - Play controls||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4223 4223]||[FEA] Music Player - My Music||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4224 4224]||[FEA] Music Player - My Music - actions||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4225 4225]||[FEA] Music Player - My Music - Artist name context menu||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4226 4226]||[FEA] Music Player - My Music - Artist view||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4227 4227]||[FEA] Music Player - My Music - Artist view - actions||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4228 4228]||[FEA] Music Player - My Music - Artist view - context menu||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4229 4229]||[FEA] Music Player - My Music - Album view||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4230 4230]||[FEA] Music Player - My Music - Album view - actions||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4231 4231]||[FEA] Music Player - My Music - Album view - context menu||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4232 4232]||[FEA] Music Player - My Music - gestures||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4233 4233]||[FEA] Music Player - Now playing||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4234 4234]||[FEA] Music Player - Search||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4235 4235]||[FEA] Music Player - Search Results Screen||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4236 4236]||[FEA] Music Player - Search Results - actions||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4237 4237]||[FEA] Music Player - Search Results - context menu||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4238 4238]||[FEA] Music Player - Playlist select pop up||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4239 4239]||[FEA] Music Player - Playlist view||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4240 4240]||[FEA] Music Player - Playlist view - context menu||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4241 4241]||[FEA] Music Player - New Playlist||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4242 4242]||[FEA] Music Player - New Playlist||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4243 4243]||[FEA] Music Player - Music Player notifications||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4244 4244]||[FEA] Music Player - Alternate Browsing Hierarchies||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4245 4245]||[FEA] Music Player - Landscape and Portrait orientation||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=4246 4246]||[FEA] Music player application||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=6713 6713]||[FEA] Policy - Playback volume in foreground||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=6714 6714]||[FEA] Policy - Playback volume once in background||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=9248 9248]||[FEA] [I18N]The Audio Player will be internationalized according to the dependencies listed in Core Master 5068.||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=9265 9265]||[FEA] [L10N] Audio Player shall be translated into the POR languages as listed in Core Master #5068.||Audio
|-
| [http://bugs.meego.com/show_bug.cgi?id=2631 2631]||[FEA] Image viewing application||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=2672 2672]||[FEA] Images application image handling functionality||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=2673 2673]||[FEA] Images sharing from the application||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3089 3089]||[FEA] Images open an image||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3090 3090]||[FEA] Images saving||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3091 3091]||[FEA] Images delete||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3092 3092]||[FEA] Images rotate||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3093 3093]||[FEA] Image details information||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3094 3094]||[FEA] Images set as wall paper||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3095 3095]||[FEA] Images set as avatar||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3096 3096]||[FEA] Images panning & scrolling||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=3097 3097]||[FEA] Images zooming||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4247 4247]||[FEA] Photo viewer - View a slide show||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4248 4248]||[FEA] Photo viewer - View a selected picture||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4249 4249]||[FEA] Photo viewer - View a list of pictures||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4250 4250]||[FEA] Photo viewer - Email integration||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4251 4251]||[FEA] Photo viewer - photo Viewer main Screen||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4252 4252]||[FEA] Photo viewer - Photo Viewer main Screen Actions||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4253 4253]||[FEA] Photo viewer - Photo Viewer main Screen Actions - View photo||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4254 4254]||[FEA] Photo viewer - Photo Viewer main Screen Actions - View Albums||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4255 4255]||[FEA] Photo viewer - Photo Viewer - View Albums context menu||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4256 4256]||[FEA] Photo viewer - Photo Viewer - Photo thumbnail list context menu||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4257 4257]||[FEA] Photo viewer - Photo Viewer - Breadcrumb||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4258 4258]||[FEA] Photo viewer - Photo Viewer Actions - Search||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4259 4259]||[FEA] Photo viewer - Photo Viewer Search - matching methods||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4260 4260]||[FEA] Photo viewer - Photo Viewer - Photo View Context menu||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4261 4261]||[FEA] Photo viewer - Photo Viewer - Photo View Context Menu - Tag||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4262 4262]||[FEA] Photo viewer - Photo Viewer - Portrait and landscape orientations||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4263 4263]||[FEA] Photo viewer - Autogenerated albums principles||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4264 4264]||[FEA] Photo viewer - Autogenerated albums - division per day||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4265 4265]||[FEA] Photo viewer - Autogenerated albums - division within a day||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4266 4266]||[FEA] Photo viewer - Autogenerated albums - combination over a day||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4267 4267]||[FEA] Photo viewer - Autogenerated albums - automatic splits||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4268 4268]||[FEA] Photo viewer - Photo Viewer - Album view - Share action||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4269 4269]||[FEA] Photo viewer - Photo View Album View - Share action details||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4270 4270]||[FEA] Photo viewer - Photo Viewer - View Photo Zoom/Pan Gestures||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4271 4271]||[FEA] Photo viewer - Photo Viewer - View Photo browsing gestures||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4272 4272]||[FEA] Photo viewer - Photo Viewer Main Screen - Slide Show action||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=9257 9257]||[FEA] [I18N]The Photo Viewer app shall be internationalized according to the dependencies listed in Core Master 5068.||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=9274 9274]||[FEA] [L10N] Photo Viewer shall be translated into the POR languages as listed in Core Master #5068.||Photo
|-
| [http://bugs.meego.com/show_bug.cgi?id=4273 4273]||[FEA] Video Player - main screen||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4274 4274]||[FEA] Video Player - Main Screen - Actions||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4275 4275]||[FEA] Video Player - Main screen - Context menu||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4276 4276]||[FEA] Video Player - Video Playback||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4277 4277]||[FEA] Video Player -Video Playback actions||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4278 4278]||[FEA] Video Player - Search||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=4279 4279]||[FEA] Video player application||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=7974 7974]||[FEA] Streaming: File size of audiovisual contents supported||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=7980 7980]||[FEA] Streaming: SDP: support of IP addresses||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=7981 7981]||[FEA] Streaming: SDP: support of IP addresses||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=7982 7982]||[FEA] Streaming: RTSP URL: support of IP addresses||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=8507 8507]||[FEA] Support portrait mode in Multimedia applications||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=9261 9261]||[FEA] [I18N]The Video Player shall be internationalized according to the dependencies listed in Core Master 5068.||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=9278 9278]||[FEA] [L10N] Video Player shall be translated into the POR languages as listed in Core Master #5068.||Video
|-
| [http://bugs.meego.com/show_bug.cgi?id=5438 5438]||[FEA] Start camera via app menu||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5439 5439]||[FEA] Start camera via lens cover||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5440 5440]||[FEA] Start camera via shutter key||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5441 5441]||[FEA] Close camera via SW||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5442 5442]||[FEA] Close camera via lens cover||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5443 5443]||[FEA] Pre capture: Mode select||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5444 5444]||[FEA] Still format||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5445 5445]||[FEA] Video format||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5446 5446]||[FEA] Capture: Grab shot||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5447 5447]||[FEA] Capture: Stop video capture||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5448 5448]||[FEA] Fixed focus||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5449 5449]||[FEA] Automatic review after capture||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5450 5450]||[FEA] Pre-capture: go to review via on-screen controls||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5451 5451]||[FEA] Post-capture: to capture via shutter key||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5452 5452]||[FEA] Post-capture: to capture via SW key||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5453 5453]||[FEA] Autofocus||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5454 5454]||[FEA] Focus indicator||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5455 5455]||[FEA] Simple touch focus||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5456 5456]||[FEA] Flash||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5457 5457]||[FEA] Flash settings||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5458 5458]||[FEA] Transitions||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5459 5459]||[FEA] Standby||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5460 5460]||[FEA] Zoom||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=5461 5461]||[FEA] Camera application||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=8157 8157]||[FEA] Erase Image||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=8158 8158]||[FEA] Store Image||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=8159 8159]||[FEA] Self Timer||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=8160 8160]||[FEA] Burst Mode Still Capture middleware||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=8390 8390]||[FEA] Burst Mode Still Capture application||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=9250 9250]||[FEA] [I18N] The Camera app shall be internationalized according to the dependencies listed in Core Master 5068.||Camera
|-
| [http://bugs.meego.com/show_bug.cgi?id=9267 9267]||[FEA] [L10N] Camera shall be translated into the POR languages as listed in Core Master #5068.||Camera
|}

Quality/Plans/Meego1.2 media test plan

2010-12-03T07:24:24Z

Jessicaji: /* Test Design */

=Media Application Test Plan for MeeGo HandSet 1.2=
==Component Summary==
Media subsystem plays an important role in Handset vertical. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application. Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, QML,Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.2 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
The features listed here are all from MeeGo handset 1.2 media components feature zilla, media test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as tool bar, i18/l10n etc.
* Detail feature list please see [http://moblin.intel.com/wiki/Meego1.2_media_features Meego1.2_media_features]
* Feature summary please see below:

{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio Player
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features Audio_Feature]
|-
|Photo Viewer
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features Photo_Features]
|-
|Video Player
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features Video_Features]
|-
|Camera
|*Basic operation with Camera *Shutter Key *Mode select *Capture or stop capture *Pre-capture & Post-capture *Auto focus *Flash *Zoom *Erase image and store image *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Camera%20application&product=MeeGo%20Handset%20Features&query_format=advanced&order=bug_id&query_based_on= Camera_Features]
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility,v4l2, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
* only support public codec list, we don't release any license codec.

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video/Camera application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N/L10n testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.
* Test case leverage
more design is same with tablet vertical. we will leverage to use more test cases to avoid overlap.

==Test Design==
====Audio====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Audio Playback control
|
* Audio play/pause
* Audio mute/unmute
* Back to previous song
* Skip to next song
* Repeat playback
* Shuffle playback
|-
|artist/album/song Navigation$Playback
|
* Select given artist and browse the all albums within it
* Select given album and browse the all songs within it
* Playback all songs within one artist
* Playback all songs within one album
* Playback one song
|-
|Alphabetic scrolling
|
* Scrolling to a given character head in My Music view
* Scrolling to a givne character head in playlist view
|-
|Playlist manipulation
|
* Create one new playlist
* Delete one playlist
* Add all songs within one artist to a given playlist
* Add all songs wihtin one Album to a given playlist
* Add one song to a given playlist
* Playback a given playlist
|-
|Audio Search
|
* Search Artists/Albums/Songs by a given string
* All results/Artists/Albums/Songs tab view switch
* Playback the selected item from search result
* Add the selected item from search result into a given playlist
|-
|Play Queue manipulation
|
* Switch to PlayQueue view
* Add a given song/album/artist to PlayQueue
* Check PlayQueue status display like show current song title in bottom bar
|-
|gesture
|
* Flick action on lists causes the list to be scrolled up/down.
* Tapping on a letter name causes the artist list to jump to artist name starting with selected letter
|-
|Notifications
|
* Song title/album art and artist name are visible in notification area
* Pause and mute button are also visible in notification
|-
|Volume control
|
* Audio play in foreground with volume control
* Audio play in background with volume control
|-
|Usability testing
|
* navigate the photos and listening music
* navigate the browsers and listening music
* audio volume control in foreground and background
|-
|stress testing
|
* Playback the audio for a long time
* Browse great number of audio files
* Change the playback state randomly
|-
|Negative testing
|
* Playback non-audio files
* Playback un-supported files
|-
|I18N/L10n testing
|
* Switch language between Simplified Chinese and English
|-
|}

====Photo====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|View Function
|
* Photo thumbnail
* View a given photo
* Photo slide show
* Photo navigation (Previous, Next)
|-
|Format
|
* JPEG
* BMP
* PNG
* GIF
|-
|Album Generation
|
* Organize photo album taken in similar time
* Organize photo album taken in similar place
|-
|Search
|
* Search a photo by a given string
* Search an album by a given string
* Search result view switch between photo and album
|-
|Contextual Menu
|
* Slide show
* Tag
* Copy
* Rotate CW
* Rotate CCW
* Delete
|-
|Orientation
|
* Portrait
* Landscape
|-
|Operation
|
* save
* delete
* rotate
* set as wallpaper
* set as avatar
|-
|Info
|
* review image detail with metadata
|-
|Sharing
|
* share with sms
* share with email
* share with bluetooth
* share with flickr
* share with facebook
* share with picasa
|-
|Resize
|
* Photo panning and scrolling
* Photo zoom
|-
|Stress
|
* Slide show the photo files for a long time
* Browse great number of photo files
* view large photo file
|-
|Negative
|
* View non-picture files
* View un-supported picture files
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}

====Video====
In MeeGo 1.2 release, video player test points includes:
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Play Control
|
* play
* pause
* seek
* fastforward
* backforward
|-
|Info
|
* total time line
* reserved time line
|-
|Alphabetic scrolling
|
* sort
* gesture
|-
|contextual menu
|
* play
* delete
* favorite
|-
|Search
|
* by name
* by words
* by artist
|-
|Streaming
|
* by SDP file with domain name
* by SDP file with ip address
* by rtsp session
|-
|Stress
|
* Playback a long video file
* Browse great number of video files
* Change the playback state randomly
|-
|Negative
|
* Playback non-video files
* Playback un-supported video files
|-
|I18N/L10N
|
** Switch language between Simplified Chinese and English
|-
|}

====Camera====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Capture
|
* Start camera via app menu/lens cover/shutter key
* Close camera via app menu/lens cover
|-
|Mode select
|
** capture photo
** capture video
|-
|Pre-capture & Post-capture
|
* pre-capture
* post-capture
|-
|Autofocus
|
* Autofocus indicator
* Autofocus performance
|-
|Flash
|
* Flash on
* Flash off
* Flash auto
|-
|zoom
|
* zoom in
* zoom out
|-
|file operation
|
* Erase image
* store image
|-
|self timer
|
* Self timer
* burst mode
|-
|Stress
|
* Capture a long video file
* Capture one high definition video file
* Capture one high definition photo file
* burst mode capture for long time
* Change the capture mode randomly
|-
|Negative
|
* no power while capture
* under sun light environment while capture
* far away with target while capture
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}
===Collaboration===
Mainly for the conditions that the application will meet, mainly introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
| Virtual Keyboard
|
* Input strings via virtual keyboard
* Short cut key to operate browser
| .
|-
|2
| Device management
| file privisioning or device sync
| .
|-
|3
|Web browser
|
* try to progressive download one video or audio file
* try to launch video or audio player from web browser
| .
|-
|4
| telephony
| how to handle audio/video playback while one phone is comming
| .
|-
|5
| Location
|
* all media applications need to support landscape and portrait mode
* photo exif data will have location info. and group with place
| .
|-
|6
| webservice
|
* through SMS share some video/photo/audio with others
* through email share
* through bluetooth share
* through popular website share
| .
|-
|7
| Notification
|
* handle notification while playing music
* handle notification while playing video
* handle notification while playing photo
* handle notification while capture picture by camera application
| .
|-
|8
| Instant Message
|
* browser with one photo, set with avatar
| .
|-
|}

===Condition===
Mainly for the conditions that the application will meet, maily introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
|Touch framework supported devices
|Test devices need support touch framework
|.
|-
|2
|Wired network
|
Connects to wired network:
* Using proxy
* Without proxy
|.
|-
|3
|Wireless network
|
Connects to wireless network:
* Using proxy
* Without proxy
| .
|-
|4
|Landscape and portrait views
|
Change screen orientation to get different views:
* Landscape view with proper window size
* Portrait view with proper windows size
| .
|-
|5
|Power state
|
The state may include:
*idle
*sleep
*resume
| .
|-
|6
|Touch and gesture
|
* MeeGo Touch Framework gestures
* Additional gestures support in media applications:
**Pinch to zoom in and out, scaling the whole page.
**Flick back and forward to go back and forward with previous/next track;
**Scroll to page up and down
**Finger scrolling to move up and down
**Long press to trigger context menu
**Double tap to switch zoom level
**Two-fingers long press to select text
| .
|-
|7
|Other Devices
|
* speaker
* headset
* bluetooth headset
| .
|-
|8
|Virtual Keyboard
|
* Virtual keyboard interaction
* hardware keyboard interaction
| .
|-
|}

==Test Environment==
===Hardware Platforms===
* NCDK and ICDK

===Network===
* Internal network uses proxy to access internet
* Access internet directly without any proxy
* wireless
===Tools===
* gst-launch
* paplay
* aplay

==QA Contact==
[mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio Player
** [mailto:forrest.zhao@intel.com Forrest Zhao] and [mailto:jia-cheng.hu@intel.com Jason Hu]
* Photo viewer
** [mailto:ziv.chang@intel.com Ziv Chang]
* Camera
** pending

==References==
* Application Test Design Method ---TBD
* MeeGo UI design wireframes

Quality/Plans/Meego1.2 media test plan

2010-12-03T07:23:49Z

Jessicaji: /* References */

=Media Application Test Plan for MeeGo HandSet 1.2=
==Component Summary==
Media subsystem plays an important role in Handset vertical. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application. Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, QML,Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.2 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
The features listed here are all from MeeGo handset 1.2 media components feature zilla, media test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as tool bar, i18/l10n etc.
* Detail feature list please see [http://moblin.intel.com/wiki/Meego1.2_media_features Meego1.2_media_features]
* Feature summary please see below:

{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio Player
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features Audio_Feature]
|-
|Photo Viewer
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features Photo_Features]
|-
|Video Player
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features Video_Features]
|-
|Camera
|*Basic operation with Camera *Shutter Key *Mode select *Capture or stop capture *Pre-capture & Post-capture *Auto focus *Flash *Zoom *Erase image and store image *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Camera%20application&product=MeeGo%20Handset%20Features&query_format=advanced&order=bug_id&query_based_on= Camera_Features]
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility,v4l2, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
* only support public codec list, we don't release any license codec.

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video/Camera application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N/L10n testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.
* Test case leverage
more design is same with tablet vertical. we will leverage to use more test cases to avoid overlap.

==Test Design==
All media application test design will follow below guide line. '''[http://moblin.intel.com/wiki/MeeGo_QA/Tablet/1.1/TestDesignGuideline Test Design Guideline ] and [http://moblin.intel.com/wiki/UX_Application_Test_Design_Method test method]:'''
====Audio====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Audio Playback control
|
* Audio play/pause
* Audio mute/unmute
* Back to previous song
* Skip to next song
* Repeat playback
* Shuffle playback
|-
|artist/album/song Navigation$Playback
|
* Select given artist and browse the all albums within it
* Select given album and browse the all songs within it
* Playback all songs within one artist
* Playback all songs within one album
* Playback one song
|-
|Alphabetic scrolling
|
* Scrolling to a given character head in My Music view
* Scrolling to a givne character head in playlist view
|-
|Playlist manipulation
|
* Create one new playlist
* Delete one playlist
* Add all songs within one artist to a given playlist
* Add all songs wihtin one Album to a given playlist
* Add one song to a given playlist
* Playback a given playlist
|-
|Audio Search
|
* Search Artists/Albums/Songs by a given string
* All results/Artists/Albums/Songs tab view switch
* Playback the selected item from search result
* Add the selected item from search result into a given playlist
|-
|Play Queue manipulation
|
* Switch to PlayQueue view
* Add a given song/album/artist to PlayQueue
* Check PlayQueue status display like show current song title in bottom bar
|-
|gesture
|
* Flick action on lists causes the list to be scrolled up/down.
* Tapping on a letter name causes the artist list to jump to artist name starting with selected letter
|-
|Notifications
|
* Song title/album art and artist name are visible in notification area
* Pause and mute button are also visible in notification
|-
|Volume control
|
* Audio play in foreground with volume control
* Audio play in background with volume control
|-
|Usability testing
|
* navigate the photos and listening music
* navigate the browsers and listening music
* audio volume control in foreground and background
|-
|stress testing
|
* Playback the audio for a long time
* Browse great number of audio files
* Change the playback state randomly
|-
|Negative testing
|
* Playback non-audio files
* Playback un-supported files
|-
|I18N/L10n testing
|
* Switch language between Simplified Chinese and English
|-
|}

====Photo====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|View Function
|
* Photo thumbnail
* View a given photo
* Photo slide show
* Photo navigation (Previous, Next)
|-
|Format
|
* JPEG
* BMP
* PNG
* GIF
|-
|Album Generation
|
* Organize photo album taken in similar time
* Organize photo album taken in similar place
|-
|Search
|
* Search a photo by a given string
* Search an album by a given string
* Search result view switch between photo and album
|-
|Contextual Menu
|
* Slide show
* Tag
* Copy
* Rotate CW
* Rotate CCW
* Delete
|-
|Orientation
|
* Portrait
* Landscape
|-
|Operation
|
* save
* delete
* rotate
* set as wallpaper
* set as avatar
|-
|Info
|
* review image detail with metadata
|-
|Sharing
|
* share with sms
* share with email
* share with bluetooth
* share with flickr
* share with facebook
* share with picasa
|-
|Resize
|
* Photo panning and scrolling
* Photo zoom
|-
|Stress
|
* Slide show the photo files for a long time
* Browse great number of photo files
* view large photo file
|-
|Negative
|
* View non-picture files
* View un-supported picture files
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}

====Video====
In MeeGo 1.2 release, video player test points includes:
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Play Control
|
* play
* pause
* seek
* fastforward
* backforward
|-
|Info
|
* total time line
* reserved time line
|-
|Alphabetic scrolling
|
* sort
* gesture
|-
|contextual menu
|
* play
* delete
* favorite
|-
|Search
|
* by name
* by words
* by artist
|-
|Streaming
|
* by SDP file with domain name
* by SDP file with ip address
* by rtsp session
|-
|Stress
|
* Playback a long video file
* Browse great number of video files
* Change the playback state randomly
|-
|Negative
|
* Playback non-video files
* Playback un-supported video files
|-
|I18N/L10N
|
** Switch language between Simplified Chinese and English
|-
|}

====Camera====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Capture
|
* Start camera via app menu/lens cover/shutter key
* Close camera via app menu/lens cover
|-
|Mode select
|
** capture photo
** capture video
|-
|Pre-capture & Post-capture
|
* pre-capture
* post-capture
|-
|Autofocus
|
* Autofocus indicator
* Autofocus performance
|-
|Flash
|
* Flash on
* Flash off
* Flash auto
|-
|zoom
|
* zoom in
* zoom out
|-
|file operation
|
* Erase image
* store image
|-
|self timer
|
* Self timer
* burst mode
|-
|Stress
|
* Capture a long video file
* Capture one high definition video file
* Capture one high definition photo file
* burst mode capture for long time
* Change the capture mode randomly
|-
|Negative
|
* no power while capture
* under sun light environment while capture
* far away with target while capture
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}
===Collaboration===
Mainly for the conditions that the application will meet, mainly introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
| Virtual Keyboard
|
* Input strings via virtual keyboard
* Short cut key to operate browser
| .
|-
|2
| Device management
| file privisioning or device sync
| .
|-
|3
|Web browser
|
* try to progressive download one video or audio file
* try to launch video or audio player from web browser
| .
|-
|4
| telephony
| how to handle audio/video playback while one phone is comming
| .
|-
|5
| Location
|
* all media applications need to support landscape and portrait mode
* photo exif data will have location info. and group with place
| .
|-
|6
| webservice
|
* through SMS share some video/photo/audio with others
* through email share
* through bluetooth share
* through popular website share
| .
|-
|7
| Notification
|
* handle notification while playing music
* handle notification while playing video
* handle notification while playing photo
* handle notification while capture picture by camera application
| .
|-
|8
| Instant Message
|
* browser with one photo, set with avatar
| .
|-
|}

===Condition===
Mainly for the conditions that the application will meet, maily introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
|Touch framework supported devices
|Test devices need support touch framework
|.
|-
|2
|Wired network
|
Connects to wired network:
* Using proxy
* Without proxy
|.
|-
|3
|Wireless network
|
Connects to wireless network:
* Using proxy
* Without proxy
| .
|-
|4
|Landscape and portrait views
|
Change screen orientation to get different views:
* Landscape view with proper window size
* Portrait view with proper windows size
| .
|-
|5
|Power state
|
The state may include:
*idle
*sleep
*resume
| .
|-
|6
|Touch and gesture
|
* MeeGo Touch Framework gestures
* Additional gestures support in media applications:
**Pinch to zoom in and out, scaling the whole page.
**Flick back and forward to go back and forward with previous/next track;
**Scroll to page up and down
**Finger scrolling to move up and down
**Long press to trigger context menu
**Double tap to switch zoom level
**Two-fingers long press to select text
| .
|-
|7
|Other Devices
|
* speaker
* headset
* bluetooth headset
| .
|-
|8
|Virtual Keyboard
|
* Virtual keyboard interaction
* hardware keyboard interaction
| .
|-
|}

==Test Environment==
===Hardware Platforms===
* NCDK and ICDK

===Network===
* Internal network uses proxy to access internet
* Access internet directly without any proxy
* wireless
===Tools===
* gst-launch
* paplay
* aplay

==QA Contact==
[mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio Player
** [mailto:forrest.zhao@intel.com Forrest Zhao] and [mailto:jia-cheng.hu@intel.com Jason Hu]
* Photo viewer
** [mailto:ziv.chang@intel.com Ziv Chang]
* Camera
** pending

==References==
* Application Test Design Method ---TBD
* MeeGo UI design wireframes

Quality/Plans/Handset UX test plan

2010-12-03T07:22:02Z

Jessicaji: /* Component Test Plans */

= MeeGo HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo HandSet UX release. A series of component and system test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA Handset UX team.

=== Objectives ===

Objectives in MeeGo HandSet UX software testing is to validate the functionality of entire MeeGo HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo release HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set
* Each test component will be documented in component test plan. Test plans will cover all testing aspects for specific component/feature(s).

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)
* System Test Plans are created as separate test plans containing both Functional and Non-Functional System Testing aspects

=== Testability ===

Testability of MeeGo HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are internally stored to TestLink tool. Common Test Case Template is used when designing test cases. Test cases are released publicly in MeeGo Gitorious under Handset UX Tests part.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* http://wiki.meego.com/TestCaseTemplate
* http://gitorious.org/qa-tools/

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo HandSet UX Testing will cover the MeeGo HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* List of exact features not to be tested can be found from Featurezilla @ Bugzilla. One must use Testability query there to have full list identified.
** [http://bugs.meego.com/report.cgi?x_axis_field=cf_testability&y_axis_field=component&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&classification=MeeGo+Features&product=MeeGo+Handset+Features&version=1.2&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&deadlinefrom=&deadlineto=&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Testability in Featurezilla (1.2)]

=== Configurations ===

MeeGo HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* [http://meego.com/devices/handset/handset-supported-hardware Supported Hardware]

=== Test Sets, Definitions and Priorization ===

Test sets are formed to Test Management Tool by using specific field inside the tool. Test sets that are formed are
* Acceptance/Sanity Test
** Acceptance/Sanity test set is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in acceptance/sanity test reflects the most important and basic functionalities of the distribution.
** Acceptance and Sanity test sets are relatively stable and will be run on daily basis.
* Feature Test
** Key Feature Test Set is used to verify MeeGo Handset UX most critical key use cases functionalities with well selected basic feature test cases.
** Basic Feature Test Set is verifying MeeGo HandSet UX delivered features with basic feature test cases. Test set is always static to show overall feature functionalities progress and maturity of the entire MeeGo distribution. Based on test results QA is able to identify components with working features to enable extended feature testing and system testing.
** Extended Feature Test Set is used to verify delivery of features forming full functionality of entirely component. After component is fully integrated all component related test cases will be executed for selected weekly release and report out all the bugs against component and it’s features. Extended feature test set will be run again in the upcoming milestone or when significant changes are applied to component and it’s features.
* System Functional
** System Functional Test Set is targeting to evaluate delivered functionalities from system perspective. Test cases are not testing UI or Application itself, instead test cases are testing how whole system is working and interacting with Consumer (end user). Test cases are covering most critical interaction and negative scenarios that consumers will encounter in their daily usage.
* System Non-Functional
** System Performance Test Sets target is to evaluate overall system performance by executing well-selected set of cases from different test areas - for example response and reaction times, use times and frame rate measurements. Test set gives a quick view of system performance from end-user point of view.
** System Reliability Test Sets target is to provide an overview to system reliability by executing iterative tests that focus on the most important and most used end-user features of MeeGo.

Quality Assurance Owners are setting priorities for Test Cases to form these Test Sets to be used for test execution.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool
* Main focus in test automation will be in acceptance, sanity and regression testing automisation
* Automated scripts are released in Gitorius: http://gitorious.org/qa-tools/ under Handset UX Tests part

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo HandSet UX test results are stored to one place.

* MeeGo Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

=== Component Test Plans ===

{| border="1"
!| Component
!| QA Owner
!| QA CC-owner
!| Detailed test plan
|-
| Applets || Cathy Li || Mika Ikonen || <link to detailed test plan>
|-
| Short Message Service || Mika Ikonen || Lili || <link to detailed test plan>
|-
| Dialer || Mika Ikonen || Lili || <link to detailed test plan>
|-
| Media Applications || Jessica Ji || Anssi Takku || [http://wiki.meego.com/Quality/Plans/Meego1.2_media_test_plan MeeGo1.2 Handset Media Applications Test Plan]
|-
| Mozilla Fennec Browser || Anssi Takku || Qin Mu || <link to detailed test plan>
|-
| Contacts|| Dayu Yang || Mika Ikonen || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.2_Handset_CoreUX_TestPlan MeeGo1.2 Handset Core UX Test Plan]
|-
| Social Networking || Cathy Li || Mika Ikonen || <link to detailed test plan>
|-
| Compositing Window Manager|| N.N. || N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || Anssi Takku || <link to detailed test plan>
|-
| Sync client || Qin Mu || N.N. || <link to detailed test plan>
|-
| Email ||Yi Fu || Mika Ikonen || <link to detailed test plan>
|-
| Calendar || Dayu Yang || Anssi Takku || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || Yi Fu || <link to detailed test plan>
|-
| Connectivity UI || Mika Ikonen || N.N. || <link to detailed test plan>
|-
| Settings || Dayu Yang || Anssi Takku || <link to detailed test plan>
|-
| UI Infrastructure || Mika Ikonen || N.N || <link to detailed test plan>

|}

=== System Test Plans ===

{| border="1"
!| System Test Plans
!| QA Owner
!| Detailed test plan
|-
| System Functional Test Plan || Mika Ikonen || <link to detailed test plan>
|-
| System Non-Functional Test Plan || Anssi Takku || <link to detailed test plan>
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/Plans/Handset UX test plan

2010-12-03T07:21:33Z

Jessicaji: /* Component Test Plans */

= MeeGo HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo HandSet UX release. A series of component and system test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA Handset UX team.

=== Objectives ===

Objectives in MeeGo HandSet UX software testing is to validate the functionality of entire MeeGo HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo release HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set
* Each test component will be documented in component test plan. Test plans will cover all testing aspects for specific component/feature(s).

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)
* System Test Plans are created as separate test plans containing both Functional and Non-Functional System Testing aspects

=== Testability ===

Testability of MeeGo HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are internally stored to TestLink tool. Common Test Case Template is used when designing test cases. Test cases are released publicly in MeeGo Gitorious under Handset UX Tests part.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* http://wiki.meego.com/TestCaseTemplate
* http://gitorious.org/qa-tools/

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo HandSet UX Testing will cover the MeeGo HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* List of exact features not to be tested can be found from Featurezilla @ Bugzilla. One must use Testability query there to have full list identified.
** [http://bugs.meego.com/report.cgi?x_axis_field=cf_testability&y_axis_field=component&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&classification=MeeGo+Features&product=MeeGo+Handset+Features&version=1.2&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&deadlinefrom=&deadlineto=&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Testability in Featurezilla (1.2)]

=== Configurations ===

MeeGo HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* [http://meego.com/devices/handset/handset-supported-hardware Supported Hardware]

=== Test Sets, Definitions and Priorization ===

Test sets are formed to Test Management Tool by using specific field inside the tool. Test sets that are formed are
* Acceptance/Sanity Test
** Acceptance/Sanity test set is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in acceptance/sanity test reflects the most important and basic functionalities of the distribution.
** Acceptance and Sanity test sets are relatively stable and will be run on daily basis.
* Feature Test
** Key Feature Test Set is used to verify MeeGo Handset UX most critical key use cases functionalities with well selected basic feature test cases.
** Basic Feature Test Set is verifying MeeGo HandSet UX delivered features with basic feature test cases. Test set is always static to show overall feature functionalities progress and maturity of the entire MeeGo distribution. Based on test results QA is able to identify components with working features to enable extended feature testing and system testing.
** Extended Feature Test Set is used to verify delivery of features forming full functionality of entirely component. After component is fully integrated all component related test cases will be executed for selected weekly release and report out all the bugs against component and it’s features. Extended feature test set will be run again in the upcoming milestone or when significant changes are applied to component and it’s features.
* System Functional
** System Functional Test Set is targeting to evaluate delivered functionalities from system perspective. Test cases are not testing UI or Application itself, instead test cases are testing how whole system is working and interacting with Consumer (end user). Test cases are covering most critical interaction and negative scenarios that consumers will encounter in their daily usage.
* System Non-Functional
** System Performance Test Sets target is to evaluate overall system performance by executing well-selected set of cases from different test areas - for example response and reaction times, use times and frame rate measurements. Test set gives a quick view of system performance from end-user point of view.
** System Reliability Test Sets target is to provide an overview to system reliability by executing iterative tests that focus on the most important and most used end-user features of MeeGo.

Quality Assurance Owners are setting priorities for Test Cases to form these Test Sets to be used for test execution.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool
* Main focus in test automation will be in acceptance, sanity and regression testing automisation
* Automated scripts are released in Gitorius: http://gitorious.org/qa-tools/ under Handset UX Tests part

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo HandSet UX test results are stored to one place.

* MeeGo Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

=== Component Test Plans ===

{| border="1"
!| Component
!| QA Owner
!| QA CC-owner
!| Detailed test plan
|-
| Applets || Cathy Li || Mika Ikonen || <link to detailed test plan>
|-
| Short Message Service || Mika Ikonen || Lili || <link to detailed test plan>
|-
| Dialer || Mika Ikonen || Lili || <link to detailed test plan>
|-
| Media Applications || Jessica Ji || Anssi Takku || [http://wiki.meego.com/Quality/Plans/Meego1.2_media_test_plan MeeGo1.2 Handset Media Test Plan]
|-
| Mozilla Fennec Browser || Anssi Takku || Qin Mu || <link to detailed test plan>
|-
| Contacts|| Dayu Yang || Mika Ikonen || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.2_Handset_CoreUX_TestPlan MeeGo1.2 Handset Core UX Test Plan]
|-
| Social Networking || Cathy Li || Mika Ikonen || <link to detailed test plan>
|-
| Compositing Window Manager|| N.N. || N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || Anssi Takku || <link to detailed test plan>
|-
| Sync client || Qin Mu || N.N. || <link to detailed test plan>
|-
| Email ||Yi Fu || Mika Ikonen || <link to detailed test plan>
|-
| Calendar || Dayu Yang || Anssi Takku || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || Yi Fu || <link to detailed test plan>
|-
| Connectivity UI || Mika Ikonen || N.N. || <link to detailed test plan>
|-
| Settings || Dayu Yang || Anssi Takku || <link to detailed test plan>
|-
| UI Infrastructure || Mika Ikonen || N.N || <link to detailed test plan>

|}

=== System Test Plans ===

{| border="1"
!| System Test Plans
!| QA Owner
!| Detailed test plan
|-
| System Functional Test Plan || Mika Ikonen || <link to detailed test plan>
|-
| System Non-Functional Test Plan || Anssi Takku || <link to detailed test plan>
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/Plans/Meego1.2 media test plan

2010-12-03T07:17:09Z

Jessicaji: Created page with "=Media Application Test Plan for MeeGo HandSet 1.2= ==Component Summary== Media subsystem plays an important role in Handset vertical. The media software stack in MeeGo is divide…"

=Media Application Test Plan for MeeGo HandSet 1.2=
==Component Summary==
Media subsystem plays an important role in Handset vertical. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application. Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, QML,Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.2 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
The features listed here are all from MeeGo handset 1.2 media components feature zilla, media test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as tool bar, i18/l10n etc.
* Detail feature list please see [http://moblin.intel.com/wiki/Meego1.2_media_features Meego1.2_media_features]
* Feature summary please see below:

{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio Player
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features Audio_Feature]
|-
|Photo Viewer
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features Photo_Features]
|-
|Video Player
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features Video_Features]
|-
|Camera
|*Basic operation with Camera *Shutter Key *Mode select *Capture or stop capture *Pre-capture & Post-capture *Auto focus *Flash *Zoom *Erase image and store image *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: feature sets: [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Camera%20application&product=MeeGo%20Handset%20Features&query_format=advanced&order=bug_id&query_based_on= Camera_Features]
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility,v4l2, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
* only support public codec list, we don't release any license codec.

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video/Camera application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N/L10n testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.
* Test case leverage
more design is same with tablet vertical. we will leverage to use more test cases to avoid overlap.

==Test Design==
All media application test design will follow below guide line. '''[http://moblin.intel.com/wiki/MeeGo_QA/Tablet/1.1/TestDesignGuideline Test Design Guideline ] and [http://moblin.intel.com/wiki/UX_Application_Test_Design_Method test method]:'''
====Audio====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Audio Playback control
|
* Audio play/pause
* Audio mute/unmute
* Back to previous song
* Skip to next song
* Repeat playback
* Shuffle playback
|-
|artist/album/song Navigation$Playback
|
* Select given artist and browse the all albums within it
* Select given album and browse the all songs within it
* Playback all songs within one artist
* Playback all songs within one album
* Playback one song
|-
|Alphabetic scrolling
|
* Scrolling to a given character head in My Music view
* Scrolling to a givne character head in playlist view
|-
|Playlist manipulation
|
* Create one new playlist
* Delete one playlist
* Add all songs within one artist to a given playlist
* Add all songs wihtin one Album to a given playlist
* Add one song to a given playlist
* Playback a given playlist
|-
|Audio Search
|
* Search Artists/Albums/Songs by a given string
* All results/Artists/Albums/Songs tab view switch
* Playback the selected item from search result
* Add the selected item from search result into a given playlist
|-
|Play Queue manipulation
|
* Switch to PlayQueue view
* Add a given song/album/artist to PlayQueue
* Check PlayQueue status display like show current song title in bottom bar
|-
|gesture
|
* Flick action on lists causes the list to be scrolled up/down.
* Tapping on a letter name causes the artist list to jump to artist name starting with selected letter
|-
|Notifications
|
* Song title/album art and artist name are visible in notification area
* Pause and mute button are also visible in notification
|-
|Volume control
|
* Audio play in foreground with volume control
* Audio play in background with volume control
|-
|Usability testing
|
* navigate the photos and listening music
* navigate the browsers and listening music
* audio volume control in foreground and background
|-
|stress testing
|
* Playback the audio for a long time
* Browse great number of audio files
* Change the playback state randomly
|-
|Negative testing
|
* Playback non-audio files
* Playback un-supported files
|-
|I18N/L10n testing
|
* Switch language between Simplified Chinese and English
|-
|}

====Photo====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|View Function
|
* Photo thumbnail
* View a given photo
* Photo slide show
* Photo navigation (Previous, Next)
|-
|Format
|
* JPEG
* BMP
* PNG
* GIF
|-
|Album Generation
|
* Organize photo album taken in similar time
* Organize photo album taken in similar place
|-
|Search
|
* Search a photo by a given string
* Search an album by a given string
* Search result view switch between photo and album
|-
|Contextual Menu
|
* Slide show
* Tag
* Copy
* Rotate CW
* Rotate CCW
* Delete
|-
|Orientation
|
* Portrait
* Landscape
|-
|Operation
|
* save
* delete
* rotate
* set as wallpaper
* set as avatar
|-
|Info
|
* review image detail with metadata
|-
|Sharing
|
* share with sms
* share with email
* share with bluetooth
* share with flickr
* share with facebook
* share with picasa
|-
|Resize
|
* Photo panning and scrolling
* Photo zoom
|-
|Stress
|
* Slide show the photo files for a long time
* Browse great number of photo files
* view large photo file
|-
|Negative
|
* View non-picture files
* View un-supported picture files
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}

====Video====
In MeeGo 1.2 release, video player test points includes:
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Play Control
|
* play
* pause
* seek
* fastforward
* backforward
|-
|Info
|
* total time line
* reserved time line
|-
|Alphabetic scrolling
|
* sort
* gesture
|-
|contextual menu
|
* play
* delete
* favorite
|-
|Search
|
* by name
* by words
* by artist
|-
|Streaming
|
* by SDP file with domain name
* by SDP file with ip address
* by rtsp session
|-
|Stress
|
* Playback a long video file
* Browse great number of video files
* Change the playback state randomly
|-
|Negative
|
* Playback non-video files
* Playback un-supported video files
|-
|I18N/L10N
|
** Switch language between Simplified Chinese and English
|-
|}

====Camera====
{|cellspacing="0" border="1"
!|Feature Set
!|Test Point
|-
|Capture
|
* Start camera via app menu/lens cover/shutter key
* Close camera via app menu/lens cover
|-
|Mode select
|
** capture photo
** capture video
|-
|Pre-capture & Post-capture
|
* pre-capture
* post-capture
|-
|Autofocus
|
* Autofocus indicator
* Autofocus performance
|-
|Flash
|
* Flash on
* Flash off
* Flash auto
|-
|zoom
|
* zoom in
* zoom out
|-
|file operation
|
* Erase image
* store image
|-
|self timer
|
* Self timer
* burst mode
|-
|Stress
|
* Capture a long video file
* Capture one high definition video file
* Capture one high definition photo file
* burst mode capture for long time
* Change the capture mode randomly
|-
|Negative
|
* no power while capture
* under sun light environment while capture
* far away with target while capture
|-
|I18N/L10N
|
* Switch language between Simplified Chinese and English
|-
|}
===Collaboration===
Mainly for the conditions that the application will meet, mainly introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
| Virtual Keyboard
|
* Input strings via virtual keyboard
* Short cut key to operate browser
| .
|-
|2
| Device management
| file privisioning or device sync
| .
|-
|3
|Web browser
|
* try to progressive download one video or audio file
* try to launch video or audio player from web browser
| .
|-
|4
| telephony
| how to handle audio/video playback while one phone is comming
| .
|-
|5
| Location
|
* all media applications need to support landscape and portrait mode
* photo exif data will have location info. and group with place
| .
|-
|6
| webservice
|
* through SMS share some video/photo/audio with others
* through email share
* through bluetooth share
* through popular website share
| .
|-
|7
| Notification
|
* handle notification while playing music
* handle notification while playing video
* handle notification while playing photo
* handle notification while capture picture by camera application
| .
|-
|8
| Instant Message
|
* browser with one photo, set with avatar
| .
|-
|}

===Condition===
Mainly for the conditions that the application will meet, maily introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.

{|cellspacing="0" border="1"
!|Item ID
!|Name
!|Details
!|Comments
|-
|1
|Touch framework supported devices
|Test devices need support touch framework
|.
|-
|2
|Wired network
|
Connects to wired network:
* Using proxy
* Without proxy
|.
|-
|3
|Wireless network
|
Connects to wireless network:
* Using proxy
* Without proxy
| .
|-
|4
|Landscape and portrait views
|
Change screen orientation to get different views:
* Landscape view with proper window size
* Portrait view with proper windows size
| .
|-
|5
|Power state
|
The state may include:
*idle
*sleep
*resume
| .
|-
|6
|Touch and gesture
|
* MeeGo Touch Framework gestures
* Additional gestures support in media applications:
**Pinch to zoom in and out, scaling the whole page.
**Flick back and forward to go back and forward with previous/next track;
**Scroll to page up and down
**Finger scrolling to move up and down
**Long press to trigger context menu
**Double tap to switch zoom level
**Two-fingers long press to select text
| .
|-
|7
|Other Devices
|
* speaker
* headset
* bluetooth headset
| .
|-
|8
|Virtual Keyboard
|
* Virtual keyboard interaction
* hardware keyboard interaction
| .
|-
|}

==Test Environment==
===Hardware Platforms===
* NCDK and ICDK

===Network===
* Internal network uses proxy to access internet
* Access internet directly without any proxy
* wireless
===Tools===
* gst-launch
* paplay
* aplay

==QA Contact==
[mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio Player
** [mailto:forrest.zhao@intel.com Forrest Zhao] and [mailto:jia-cheng.hu@intel.com Jason Hu]
* Photo viewer
** [mailto:ziv.chang@intel.com Ziv Chang]
* Camera
** pending

==References==
* Application Test Design Method
http://moblin.intel.com/wiki/UX_Application_Test_Design_Method
* MeeGo UI design wireframes

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-12T02:39:13Z

Jessicaji: /* Handset-Ux-Test-Releases */

== Handset-Ux-Test-Releases==
=== Handset-Ux-Test-Release 1.1.0.1 ===

Target date: November 12th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change schema to align with testrunner tool
|-
|| mhts-videoplayer-tests|| change schema to align with testrunner tool
|-
|| mhts-sms-tests || change schema to align with testrunner tool
|-
|| mhts-contacts-tests || change schema to align with testrunner tool
|-
|| mhts-vkb-tests || change schema to align with testrunner tool
|-
|| mhts-coreux-tests || change schema to align with testrunner tool
|-
|| mhts-browser-tests || change schema to align with testrunner tool
|-
|| mhts-dialer-tests || change schema to align with testrunner tool
|-
|| meego-handsetux-sanity-tests || change schema to align with testrunner tool
|}
=== Handset-Ux-Test-Release Initial version ===

Target date: October 20th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || Initial version
|-
|| mhts-videoplayer-tests|| Initial version
|-
|| mhts-sms-tests || Initial version
|-
|| mhts-contacts-tests || Initial version
|-
|| mhts-vkb-tests || Initial version
|-
|| mhts-coreux-tests || Initial version
|-
|| mhts-browser-tests || Initial version
|-
|| mhts-dialer-tests || Initial version
|}

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-12T02:12:35Z

Jessicaji: /* Handset-Ux-Test-Releases */

== Handset-Ux-Test-Releases==
=== Handset-Ux-Test-Release 1.1.0.3 ===

Target date: November 12th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change schema to align with testrunner tool
|-
|| mhts-videoplayer-tests|| change schema to align with testrunner tool
|-
|| mhts-sms-tests || change schema to align with testrunner tool
|-
|| mhts-contacts-tests || change schema to align with testrunner tool
|-
|| mhts-vkb-tests || change schema to align with testrunner tool
|-
|| mhts-coreux-tests || change schema to align with testrunner tool
|-
|| mhts-browser-tests || change schema to align with testrunner tool
|-
|| mhts-dialer-tests || change schema to align with testrunner tool
|-
|| meego-handsetux-sanity-tests || change schema to align with testrunner tool
|}
=== Handset-Ux-Test-Release 1.1.0.1 ===

Target date: November 10th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change title
|-
|| mhts-videoplayer-tests|| change title
|-
|| mhts-sms-tests || change title
|-
|| mhts-contacts-tests || change title
|-
|| mhts-vkb-tests || change title
|-
|| mhts-coreux-tests || change title
|-
|| mhts-browser-tests || change title
|-
|| mhts-dialer-tests || change title
|-
|| meego-handsetux-sanity-tests || add sanity testing
|}

=== Handset-Ux-Test-Release Initial version ===

Target date: October 20th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || Initial version
|-
|| mhts-videoplayer-tests|| Initial version
|-
|| mhts-sms-tests || Initial version
|-
|| mhts-contacts-tests || Initial version
|-
|| mhts-vkb-tests || Initial version
|-
|| mhts-coreux-tests || Initial version
|-
|| mhts-browser-tests || Initial version
|-
|| mhts-dialer-tests || Initial version
|}

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-11T02:41:04Z

Jessicaji: /* Handset-Ux-Test-Release Initial version */

== Handset-Ux-Test-Releases==

=== Handset-Ux-Test-Release 1.1.0.1 ===

Target date: November 10th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change title
|-
|| mhts-videoplayer-tests|| change title
|-
|| mhts-sms-tests || change title
|-
|| mhts-contacts-tests || change title
|-
|| mhts-vkb-tests || change title
|-
|| mhts-coreux-tests || change title
|-
|| mhts-browser-tests || change title
|-
|| mhts-dialer-tests || change title
|-
|| meego-handsetux-sanity-tests || add sanity testing
|}

=== Handset-Ux-Test-Release Initial version ===

Target date: October 20th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || Initial version
|-
|| mhts-videoplayer-tests|| Initial version
|-
|| mhts-sms-tests || Initial version
|-
|| mhts-contacts-tests || Initial version
|-
|| mhts-vkb-tests || Initial version
|-
|| mhts-coreux-tests || Initial version
|-
|| mhts-browser-tests || Initial version
|-
|| mhts-dialer-tests || Initial version
|}

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-11T02:40:53Z

Jessicaji: /* Handset-Ux-Test-Release 1.1.0.1 */

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-11T02:40:27Z

Jessicaji:

== Handset-Ux-Test-Releases==

=== Handset-Ux-Test-Release 1.1.0.1 ===

Target date: November 10th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change title
|-
|| mhts-videoplayer-tests|| change title
|-
|| mhts-sms-tests || change title
|-
|| mhts-contacts-tests || change title
|-
|| mhts-vkb-tests || change title
|-
|| mhts-coreux-tests || change title
|-
|| mhts-browser-tests || change title
|-
|| mcts-dialer-tests || change title
|-
|| meego-handsetux-sanity-tests || add sanity testing
|}

=== Handset-Ux-Test-Release Initial version ===

Target date: October 20th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || Initial version
|-
|| mhts-videoplayer-tests|| Initial version
|-
|| mhts-sms-tests || Initial version
|-
|| mhts-contacts-tests || Initial version
|-
|| mhts-vkb-tests || Initial version
|-
|| mhts-coreux-tests || Initial version
|-
|| mhts-browser-tests || Initial version
|-
|| mcts-dialer-tests || Initial version
|}

Quality/TestSuite/handset-test-suite/handset-ux-test Releases

2010-11-11T02:38:34Z

Jessicaji: Created page with "== Handset-Ux-Test-Releases== === Handset-Ux-Test-Release 1.1.0.1 === Target date: November 10th, 2010 Status: '''Complete''' Released content: {|cellspacing="0" bor…"

== Handset-Ux-Test-Releases==

=== Handset-Ux-Test-Release 1.1.0.1 ===

Target date: November 10th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || change title
|-
|| mhts-videoplayer-tests|| change title
|-
|| mhts-sms-tests || change title
|-
|| mhts-contacts-tests || change title
|-
|| mhts-vkb-tests || change title
|-
|| mhts-coreux-tests || change title
|-
|| mhts-browser-tests || change title
|-
|| mcts-dialer-tests || change title
|}

=== Handset-Ux-Test-Release Initial version ===

Target date: October 20th, 2010 
Status: '''Complete'''

Released content:

{|cellspacing="0" border="1"
!|Component
!|Changes

|-
|| mhts-audioplayer-tests || Initial version
|-
|| mhts-videoplayer-tests|| Initial version
|-
|| mhts-sms-tests || Initial version
|-
|| mhts-contacts-tests || Initial version
|-
|| mhts-vkb-tests || Initial version
|-
|| mhts-coreux-tests || Initial version
|-
|| mhts-browser-tests || Initial version
|-
|| mcts-dialer-tests || Initial version
|}

Quality/TestSuite/handset-test-suite

2010-09-16T06:46:24Z

Jessicaji: /* Test Suite Design Progress */

== Test Suite Design Progress ==
Follow-up table for HandSet UX 1.1 Test Case creation progress.

* Planned Test: test cases are planned to be developed
* Actual Test - Designed: actual designed test cases
* Actual Test - Ready: test cases that are ready to run currently

{| border="1"
!
! Planned Test
! Actual Test - Designed
! Actual Test - Ready
! Automation %
! Test Suite Due
! Cross Review
! Test Suite Maintainer
! QA CCed
! Comments
|-
|Photo Viewer
|87
|9
|78
|0
|TBD
|Not Started
|Jessica Ji (Intel)
|Kalle Kyllönen (Nokia)
|To be updated
|-
|Audio Player
|110
|7
|103
|0
|TBD
|Not Started
|Jessica Ji (Intel)
|Juha Paukku (Nokia)
|To be updated
|-
|Video Player
|68
|7
|61
|0
|TBD
|Not Started
|Jessica Ji (Intel)
|Kalle Kyllönen (Nokia)
|To be updated
|-
|Contacts / Address Book
|149
|33
|116
|0
|End Of August
|Not Started
|Dayu Yang (Intel)
|Kalle Kyllönen (Nokia)
|To be updated
|-
|Email
|89
|29
|60
|0
|End Of August
|Not Started
|Yi Fu (Intel)
|Mika Ikonen (Nokia)
|To be updated
|-
|Core UX
|134
|16
|118
|0
|End Of August except systemUI
|Not Started
|Cathy Li (Intel)
|Juha Paukku (Nokia)
|No feature request or UI design doc for system UI. System UI test cases are based on real implementation in current stage.
|-
|Settings
|50
|26
|6
|0
|TBD
|Not Started
|Dayu Yang (Intel)
|Juha Paukku (Nokia)
|We have got the feature request in WW35, test cases are under development now
|-
|VKB
|50
|35
|15
|0
|End Of August
|Not Started
|Yi Fu (Intel)
|Juha Paukku (Nokia)
|To be updated
|-
|Calendar
|87
|0
|87
|0
|End Of August
|Not Started
|Dayu Yang (Intel)
|Juha Paukku (Nokia)
|To be updated
|-
|Social Networking
|38
|0
|38
|0
|TBD
|Not Started
|Cathy Li (Intel)
|Juha Paukku (Nokia)
|No feature request or UI design doc for social networking.
|-
|Data Sync
|30
|30
|0
|0
|TBD
|Not Started
|Qin Mu (Intel)
|Mika Ikonen (Nokia)
|To be updated
|-
|System functional
|207
|107
|100
|0
|TBD
|Not Started
|Qin Mu (Intel)
|To be updated
|To be updated
|-
|Dialer
|60
|60
|20
|0
|End Of August
|Not Started
|Mika Ikonen (Nokia)
|Lily (Intel)
|To be updated
|-
|SMS
|40
|40
|15
|0
|End Of August
|Not Started
|Mika Ikonen (Nokia)
|Lily (Intel)
|To be updated
|-
|Browser
|100
|100
|50
|0
|End Of August
|Not Started
|Petri Jylhä (Nokia)
|Qin Mu (Intel)
|To be updated
|-
|Networking
|50
|50
|30
|0
|End Of August
|Not Started
|Petri Jylhä (Nokia)
|Dayu Yang (Intel)
|To be updated
|-
|Instant Messaging
|15
|21
|27
|0
|End Of August
|Not Started
|Mika Ikonen (Nokia)
|Yi Fu (Intel)
|To be updated
|-
|VoIP
|N/A
|N/A
|N/A
|N/A
|TBD
|Not Started
|Mika Ikonen (Nokia)
|Yi Fu (Intel)
|VoIP has been removed from 1.1 content
|}

Quality/MeeGo1.1 Handset Media TestPlan

2010-09-13T03:17:41Z

Jessicaji: /* Feature to be Tested */

=Media Application Test Plan=
==Component Summary==
Media subsystem plays an important role in Handset usage model. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application (camera application don't cover in MeeGo 1.1 release). Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.1 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio Player
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 27 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features
|-
|Photo Viewer
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features
|-
|Video Player
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
See Bug 150 - pulseaudio not work on CDK & Aava
* Video playback is not working also:
See Bug 5712 - Video playback is not working

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.

==Test Design==
====Audio====
* Audio playback
** Audio play/pause
** Audio mute/unmute
** Back to previous song
** Skip to next song
** Repeat playback
** Shuffle playback
* Artist/Album/Song Navigation & Playback
** Select given artist and browse the all albums within it
** Select given album and browse the all songs within it
** Playback all songs within one artist
** Playback all songs within one album
** Playback one song
* Alphabetic scrolling
** Scrolling to a given character head in My Music view
** Scrolling to a givne character head in playlist view
* Playlist manipulation
** Create one new playlist
** Delete one playlist
** Add all songs within one artist to a given playlist
** Add all songs wihtin one Album to a given playlist
** Add one song to a given playlist
** Playback a given playlist
* Audio search
** Search Artists/Albums/Songs by a given string
** All results/Artists/Albums/Songs tab view switch
** Playback the selected item from search result
** Add the selected item from search result into a given playlist
* Play Queue manipulation
** Switch to PlayQueue view
** Add a given song/album/artist to PlayQueue
** Check PlayQueue status display like show current song title in bottom bar
* Usability testing
** navigate the photos and listening music
** navigate the browsers and listening music
* Stability testing
** Playback the audio for a long time
** Browse great number of audio files
** Change the playback state randomly
* Negative testing
** Playback non-audio files
** Playback un-supported files
* I18N testing
** Switch language between Simplified Chinese and English

====Photo====
* Basic photo viewer functionality check
** Photo thumbnail
** View a given photo
** Photo slide show
** Photo navigation (Previous, Next)
* Photo format support (It should support common used free codec photo formats)
** JPEG
** BMP
** PNG
** GIF
* Photo album organization
** Organize photo album taken in similar time
** Organize photo album taken in similar place
* Photo search
** Search a photo by a given string
** Search an album by a given string
** Search result view switch between photo and album
* Photo mainipulation via popup menu
** Slide show
** Tag
** Copy
** Rotate CW
** Rotate CCW
** Delete
* Photo share
** email
** sms
** Facebook
** Picasa
** Bluetooth
** Flickr
* Photo Orientation
** Portrait
** Landscape
* Stability testing
** Slide show the photo files for a long time
** Browse great number of photo files
** view large photo file
* Negative testing
** View non-picture files
** View un-supported picture files
* I18N testing
** Switch language between Simplified Chinese and English

====Video====
In MeeGo 1.1 release, video player test points includes:
* Basic video playback check
* Video control bar functionality check
* Alphabetic scrolling
* Video file mainipulation via popup menu
* Video files filtering
* Video file search
* Video snapshot browsing
* Snapshot view switch between "My Movies", "Films" and "TV Shows"
* Stability testing
** Playback a long video file
** Browse great number of video files
** Change the playback state randomly
* Negative testing
** Playback non-video files
** Playback un-supported video files
* I18N testing
** Switch language between Simplified Chinese and English

==Test Environment==
===Hardware Platforms===
* Aava Koski EV2(Qual Sample) with Maxim PMIC (Firmware:2.46.4)
* Moorestown CDK with NEC PMIC (Firmware: IFWI 2.43, Kboot 0.10)

==QA Contact==
[mailto:shuang.wan@intel.com Wan Shuang]
 [mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio
** Audio Player [mailto:forrest.zhao@intel.com Forrest Zhao]
* Photo viewer
** [mailto:long.bu@intel.com Bu Long], [mailto:ziv.chang@intel.com Ziv Chang]

Quality/Plans/1.1 Handset UX TestPlan

2010-09-13T01:32:28Z

Jessicaji: /* Detailed Test Plans */

= MeeGo 1.1 HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo 1.1 HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo 1.1 HandSet UX release. A series of component test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA.

=== Objectives ===

Objectives in MeeGo 1.1 HandSet UX software testing is to validate the functionality of entire MeeGo 1.1 HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo 1.1 HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo 1.1 HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review component test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance (response time)
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)

* Not tested NFT types: Performance (Throughput, Framerate, Load, Memory Consumption and Power Management) and Reliability (Endurance, Aging, Long Period and Low Resource)

* Each test component will be documented in component test plan. Test plan will cover all testing aspects for specific component/feature(s).

=== Testability ===

Testability of MeeGo 1.1 HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.1 UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are stored to TestLink tool. Common Test Case Template is used when designing test cases.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* <Link to Test Link tool>
* http://wiki.meego.com/TestCaseTemplate

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo 1.1 HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo 1.1 HandSet UX Testing will cover the MeeGo 1.1 HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* <List of exact features not to be tested can be added after needed query is implemented to Featurezilla @ Bugzilla>

=== Configurations ===

MeeGo 1.1 HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* N900
* AAVA
* MRST CDK

=== Test Sets and Priorization ===

Test sets are formed to TestLink tool by using specific field inside the tool. Test sets that are formed are
* Sanity Test Set
* Regression Test Set
* Feature Test Suite (Basic and Extended)
* System Functional Test Suite (Interactions and negative user scenarios)
* System Non-Functional Test Suite (Performance and Reliability) - Note! Excluded from 1.1 and targeted to 1.2 release.
* Milestone Test Suite

Quality Assurance Owners are setting priorities for Test Cases to form these Test Suites to be used for test execution.

When test suites are in place in public Test Link -tool, then every test suite is reviewed and approved with respective persons.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

Note! During MeeGo 1.1 HandSet UX Timeframe QA will not form System Non-Functional Test Suites. Those will be targeted for 1.2 release.

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo 1.1 UX HandSet test results are stored to one place.

* MeeGo 1.1 Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

{| border="1"
!| Application
!| QA Owner
!| Detailed test plan
|-
| Short Message Service || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforShortMessageService MeeGo 1.1 HandSet UX SMS Test Plan]
|-
| Dialer || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforDialer MeeGo 1.1 HandSet UX Dialer Test Plan]
|-
| Media Applications || Jessica Ji || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_Media_TestPlan MeeGo 1.1 Handset UX Media Test Plan]
|-
| Mozilla Fennec Browser || Petri Jylha || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforMozillaFennecBrowser MeeGo 1.1 HandSet UX Mozilla Fennec Browser Test Plan]
|-
| Contacts|| Dayu Yang || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_CoreUX_TestPlan MeeGo 1.1 HandSet Core UX Test Plan]
|-
| Social Networking || Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_UX_Social_Networking_TestPlan MeeGo 1.1 HandSet Social Networking Test Plan]
|-
| Compositing Window Manager|| N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || [http://wiki.meego.com/Quality/1.1HandsetUXVkbTestPlan MeeGo 1.1 HandSet UX Virtual Keyboard Test Plan]
|-
| Clock || Qin Mu || <link to detailed test plan>
|-
| Email ||Yi Fu || <link to detailed test plan>
|-
| Calendar || Dayu Yang || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforInstantMessaging MeeGo 1.1 HandSet UX Instant Messaging Test Plan]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/Plans/1.1 Handset UX TestPlan

2010-09-13T01:30:22Z

Jessicaji: /* Detailed Test Plans */

= MeeGo 1.1 HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo 1.1 HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo 1.1 HandSet UX release. A series of component test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA.

=== Objectives ===

Objectives in MeeGo 1.1 HandSet UX software testing is to validate the functionality of entire MeeGo 1.1 HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo 1.1 HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo 1.1 HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review component test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance (response time)
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)

* Not tested NFT types: Performance (Throughput, Framerate, Load, Memory Consumption and Power Management) and Reliability (Endurance, Aging, Long Period and Low Resource)

* Each test component will be documented in component test plan. Test plan will cover all testing aspects for specific component/feature(s).

=== Testability ===

Testability of MeeGo 1.1 HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.1 UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are stored to TestLink tool. Common Test Case Template is used when designing test cases.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* <Link to Test Link tool>
* http://wiki.meego.com/TestCaseTemplate

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo 1.1 HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo 1.1 HandSet UX Testing will cover the MeeGo 1.1 HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* <List of exact features not to be tested can be added after needed query is implemented to Featurezilla @ Bugzilla>

=== Configurations ===

MeeGo 1.1 HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* N900
* AAVA
* MRST CDK

=== Test Sets and Priorization ===

Test sets are formed to TestLink tool by using specific field inside the tool. Test sets that are formed are
* Sanity Test Set
* Regression Test Set
* Feature Test Suite (Basic and Extended)
* System Functional Test Suite (Interactions and negative user scenarios)
* System Non-Functional Test Suite (Performance and Reliability) - Note! Excluded from 1.1 and targeted to 1.2 release.
* Milestone Test Suite

Quality Assurance Owners are setting priorities for Test Cases to form these Test Suites to be used for test execution.

When test suites are in place in public Test Link -tool, then every test suite is reviewed and approved with respective persons.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

Note! During MeeGo 1.1 HandSet UX Timeframe QA will not form System Non-Functional Test Suites. Those will be targeted for 1.2 release.

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo 1.1 UX HandSet test results are stored to one place.

* MeeGo 1.1 Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

{| border="1"
!| Application
!| QA Owner
!| Detailed test plan
|-
| Short Message Service || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforShortMessageService MeeGo 1.1 HandSet UX SMS Test Plan]
|-
| Dialer || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforDialer MeeGo 1.1 HandSet UX Dialer Test Plan]
|-
| Media Applications || Jessica Ji || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_Media_TestPla n MeeGo]
|-
| Mozilla Fennec Browser || Petri Jylha || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforMozillaFennecBrowser MeeGo 1.1 HandSet UX Mozilla Fennec Browser Test Plan]
|-
| Contacts|| Dayu Yang || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_CoreUX_TestPlan MeeGo 1.1 HandSet Core UX Test Plan]
|-
| Social Networking || Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_UX_Social_Networking_TestPlan MeeGo 1.1 HandSet Social Networking Test Plan]
|-
| Compositing Window Manager|| N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || [http://wiki.meego.com/Quality/1.1HandsetUXVkbTestPlan MeeGo 1.1 HandSet UX Virtual Keyboard Test Plan]
|-
| Clock || Qin Mu || <link to detailed test plan>
|-
| Email ||Yi Fu || <link to detailed test plan>
|-
| Calendar || Dayu Yang || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforInstantMessaging MeeGo 1.1 HandSet UX Instant Messaging Test Plan]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/MeeGo1.1 Handset Media TestPlan

2010-09-13T01:26:31Z

Jessicaji: Created page with "=Media Application Test Plan= ==Component Summary== Media subsystem plays an important role in Handset usage model. The media software stack in MeeGo is divided into 3 layers bri…"

=Media Application Test Plan=
==Component Summary==
Media subsystem plays an important role in Handset usage model. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application (camera application don't cover in MeeGo 1.1 release). Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.1 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 27 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features
|-
|Photo
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features
|-
|Video
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
See Bug 150 - pulseaudio not work on CDK & Aava
* Video playback is not working also:
See Bug 5712 - Video playback is not working

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.

==Test Design==
====Audio====
* Audio playback
** Audio play/pause
** Audio mute/unmute
** Back to previous song
** Skip to next song
** Repeat playback
** Shuffle playback
* Artist/Album/Song Navigation & Playback
** Select given artist and browse the all albums within it
** Select given album and browse the all songs within it
** Playback all songs within one artist
** Playback all songs within one album
** Playback one song
* Alphabetic scrolling
** Scrolling to a given character head in My Music view
** Scrolling to a givne character head in playlist view
* Playlist manipulation
** Create one new playlist
** Delete one playlist
** Add all songs within one artist to a given playlist
** Add all songs wihtin one Album to a given playlist
** Add one song to a given playlist
** Playback a given playlist
* Audio search
** Search Artists/Albums/Songs by a given string
** All results/Artists/Albums/Songs tab view switch
** Playback the selected item from search result
** Add the selected item from search result into a given playlist
* Play Queue manipulation
** Switch to PlayQueue view
** Add a given song/album/artist to PlayQueue
** Check PlayQueue status display like show current song title in bottom bar
* Usability testing
** navigate the photos and listening music
** navigate the browsers and listening music
* Stability testing
** Playback the audio for a long time
** Browse great number of audio files
** Change the playback state randomly
* Negative testing
** Playback non-audio files
** Playback un-supported files
* I18N testing
** Switch language between Simplified Chinese and English

====Photo====
* Basic photo viewer functionality check
** Photo thumbnail
** View a given photo
** Photo slide show
** Photo navigation (Previous, Next)
* Photo format support (It should support common used free codec photo formats)
** JPEG
** BMP
** PNG
** GIF
* Photo album organization
** Organize photo album taken in similar time
** Organize photo album taken in similar place
* Photo search
** Search a photo by a given string
** Search an album by a given string
** Search result view switch between photo and album
* Photo mainipulation via popup menu
** Slide show
** Tag
** Copy
** Rotate CW
** Rotate CCW
** Delete
* Photo share
** email
** sms
** Facebook
** Picasa
** Bluetooth
** Flickr
* Photo Orientation
** Portrait
** Landscape
* Stability testing
** Slide show the photo files for a long time
** Browse great number of photo files
** view large photo file
* Negative testing
** View non-picture files
** View un-supported picture files
* I18N testing
** Switch language between Simplified Chinese and English

====Video====
In MeeGo 1.1 release, video player test points includes:
* Basic video playback check
* Video control bar functionality check
* Alphabetic scrolling
* Video file mainipulation via popup menu
* Video files filtering
* Video file search
* Video snapshot browsing
* Snapshot view switch between "My Movies", "Films" and "TV Shows"
* Stability testing
** Playback a long video file
** Browse great number of video files
** Change the playback state randomly
* Negative testing
** Playback non-video files
** Playback un-supported video files
* I18N testing
** Switch language between Simplified Chinese and English

==Test Environment==
===Hardware Platforms===
* Aava Koski EV2(Qual Sample) with Maxim PMIC (Firmware:2.46.4)
* Moorestown CDK with NEC PMIC (Firmware: IFWI 2.43, Kboot 0.10)

==QA Contact==
[mailto:shuang.wan@intel.com Wan Shuang]
 [mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio
** Audio Player [mailto:forrest.zhao@intel.com Forrest Zhao]
* Photo viewer
** [mailto:long.bu@intel.com Bu Long], [mailto:ziv.chang@intel.com Ziv Chang]

Quality/Plans/1.1 Handset UX TestPlan

2010-09-10T08:46:09Z

Jessicaji: /* Detailed Test Plans */

= MeeGo 1.1 HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo 1.1 HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo 1.1 HandSet UX release. A series of component test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA.

=== Objectives ===

Objectives in MeeGo 1.1 HandSet UX software testing is to validate the functionality of entire MeeGo 1.1 HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo 1.1 HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo 1.1 HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review component test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance (response time)
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)

* Not tested NFT types: Performance (Throughput, Framerate, Load, Memory Consumption and Power Management) and Reliability (Endurance, Aging, Long Period and Low Resource)

* Each test component will be documented in component test plan. Test plan will cover all testing aspects for specific component/feature(s).

=== Testability ===

Testability of MeeGo 1.1 HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.1 UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are stored to TestLink tool. Common Test Case Template is used when designing test cases.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* <Link to Test Link tool>
* http://wiki.meego.com/TestCaseTemplate

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo 1.1 HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo 1.1 HandSet UX Testing will cover the MeeGo 1.1 HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* <List of exact features not to be tested can be added after needed query is implemented to Featurezilla @ Bugzilla>

=== Configurations ===

MeeGo 1.1 HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* N900
* AAVA
* MRST CDK

=== Test Sets and Priorization ===

Test sets are formed to TestLink tool by using specific field inside the tool. Test sets that are formed are
* Sanity Test Set
* Regression Test Set
* Feature Test Suite (Basic and Extended)
* System Functional Test Suite (Interactions and negative user scenarios)
* System Non-Functional Test Suite (Performance and Reliability) - Note! Excluded from 1.1 and targeted to 1.2 release.
* Milestone Test Suite

Quality Assurance Owners are setting priorities for Test Cases to form these Test Suites to be used for test execution.

When test suites are in place in public Test Link -tool, then every test suite is reviewed and approved with respective persons.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

Note! During MeeGo 1.1 HandSet UX Timeframe QA will not form System Non-Functional Test Suites. Those will be targeted for 1.2 release.

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo 1.1 UX HandSet test results are stored to one place.

* MeeGo 1.1 Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

{| border="1"
!| Application
!| QA Owner
!| Detailed test plan
|-
| Short Message Service || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforShortMessageService MeeGo 1.1 HandSet UX SMS Test Plan]
|-
| Dialer || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforDialer MeeGo 1.1 HandSet UX Dialer Test Plan]
|-
| Media Applications || Jessica Ji || <link to detailed test plan>
|-
| Mozilla Fennec Browser || Petri Jylha || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforMozillaFennecBrowser MeeGo 1.1 HandSet UX Mozilla Fennec Browser Test Plan]
|-
| Contacts|| Dayu Yang || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_CoreUX_TestPlan MeeGo 1.1 HandSet Core UX Test Plan]
|-
| Social Networking || Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_UX_Social_Networking_TestPlan MeeGo 1.1 HandSet Social Networking Test Plan]
|-
| Compositing Window Manager|| N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || [http://wiki.meego.com/Quality/1.1HandsetUXVkbTestPlan MeeGo 1.1 HandSet UX Virtual Keyboard Test Plan]
|-
| Clock || Qin Mu || <link to detailed test plan>
|-
| Email ||Yi Fu || <link to detailed test plan>
|-
| Calendar || Dayu Yang || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforInstantMessaging MeeGo 1.1 HandSet UX Instant Messaging Test Plan]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/Plans/1.1 Handset UX TestPlan

2010-09-10T08:42:35Z

Jessicaji: /* Detailed Test Plans */

= MeeGo 1.1 HandSet UX Test Plan =

== Introduction ==

This is overall test plan for MeeGo 1.1 HandSet UX of MeeGo open source project, which defines overall Quality Assurance procedure of validation activities done for MeeGo 1.1 HandSet UX release. A series of component test plans will also be linked in this overall test plan to cover detailed test approaches. This will be joint effort from MeeGo QA.

=== Objectives ===

Objectives in MeeGo 1.1 HandSet UX software testing is to validate the functionality of entire MeeGo 1.1 HandSet UX software delivery by performing daily and weekly testing for software releases. Target is to ensure that

* Planned and delivered features for MeeGo 1.1 HandSet UX are working as specified as a part of system.
* Validate that relevant bugs are fixed in software release.
* Program maturity statement can be and is given.

=== Goal ===

The goal is to deliver software release with no open bugs with a priority level of high and a minimal number of open bugs with priority level medium.

== Test Strategy and Approach ==

=== Introduction ===

Application is launched from Graphical User Interface and features are used inside application to see that how those are working inside application. Also in system testing applications are used simultaneously to see how applications are interacting as part of system.

Overall procedure in Quality Assurance for MeeGo 1.1 HandSet UX is as following

* Firstly decompose features to component, each will be associated with one component test plan
* Ensure testability of planned features forming component
* Write test design in component test plan
* Define and store (to Test Link) test cases for features
* Connect test cases to features in test management tool
* Prioritize test cases to form test sets
* Review component test plan and test cases
* Automate test cases and add tests to fully automated test infrastructure
* Execute test cases in test sets for software releases following test execution and feature releasing plan
* Report test results and raise relevant bugs
* Provide maturity statement for main releases based on received test results

=== Feature Test and System Test ===

QA target is to validate MeeGo distribution
* Feature functionality
* System functionality (Interaction and negative scenarios)
* System performance (response time)
* System reliability

Following chart illuminates scope and relationship of feature and system testing.

==== Feature Testing ====
* Target is to test full functionality of specified feature forming component (e.g. Dialer) following the features' definition in featurezilla. Test case example: Make a phone call
* Every component (formed by features) basic functionality is tested in feature test set

==== System Testing ====
* Target is to test basic (functional) system testing of several components/features simultaneously. Test case example: Listening music while receiving incoming call
* Target is to test system testing (performance). Test case example: Open dialer application (Target value 0.1 sec)
* Target is to test system testing (reliability). Test case example: Make 200 calls (Target 199 pass, 1 fail)

* Not tested NFT types: Performance (Throughput, Framerate, Load, Memory Consumption and Power Management) and Reliability (Endurance, Aging, Long Period and Low Resource)

* Each test component will be documented in component test plan. Test plan will cover all testing aspects for specific component/feature(s).

=== Testability ===

Testability of MeeGo 1.1 HandSet UX features are ensured at first.
* Features are defined by Product Management and relevant stakeholders to Bugzilla tool.
* Selected Quality Assurance Owners are checking those features through from Bugzilla against defined Testability Checklist and adding comment to feature in Bugzilla that can feature be used as QA input and it is possible validate in software release with relevant test case(s). Also more information is requested from Feature owner if it is seen insufficient from QA perspective.

Relevant Links
* http://bugs.meego.com/ (MeeGo 1.1 UX HandSet Features are stored in Bugzilla)
* http://wiki.meego.com/Quality/TestabilityChecklist
* http://wiki.meego.com/Quality/HandsetTestabilityStatus

=== Test Cases ===

Test Cases are designed by QA owners based on existing features and which have been approved from testability point of view. Test Cases itself are stored to TestLink tool. Common Test Case Template is used when designing test cases.

* Overall test design process and guideline from features to actual test cases can be found http://wiki.meego.com/Quality/TestDesignProcessAndGuideline

Relevant Links
* <Link to Test Link tool>
* http://wiki.meego.com/TestCaseTemplate

=== Coverage ===

When features forming components are analysed and test cases are designed based on those also coverage matrix will be created for each component. From coverage matrix it can be seen that what is feature coverage i.e. planned test cases vs. maximum amount of test cases to cover every user scenarios from component/feature.

Relevant Links
* <Coverage Matrix Template>
* <Coverage Matrix for MeeGo 1.1 HandSet UX component/features>

==== Features to be Tested ====
* Overall the MeeGo 1.1 HandSet UX Testing will cover the MeeGo 1.1 HandSet UX layer in [http://meego.com/developers/meego-architecture MeeGo Architecture]:
[[File:MeeGoArch.png]]

* Specific components/features to be tested will be aligned with the features under MeeGo HandSet Features product in [http://bugs.meego.com MeeGo Featurezilla @ Bugzilla]

==== Features not to be Tested ====
* <List of exact features not to be tested can be added after needed query is implemented to Featurezilla @ Bugzilla>

=== Configurations ===

MeeGo 1.1 HandSet UX is tested in a number of reference devices. The public reference configurations used for this release are
* N900
* AAVA
* MRST CDK

=== Test Sets and Priorization ===

Test sets are formed to TestLink tool by using specific field inside the tool. Test sets that are formed are
* Sanity Test Set
* Regression Test Set
* Feature Test Suite (Basic and Extended)
* System Functional Test Suite (Interactions and negative user scenarios)
* System Non-Functional Test Suite (Performance and Reliability) - Note! Excluded from 1.1 and targeted to 1.2 release.
* Milestone Test Suite

Quality Assurance Owners are setting priorities for Test Cases to form these Test Suites to be used for test execution.

When test suites are in place in public Test Link -tool, then every test suite is reviewed and approved with respective persons.

More detailed information: http://wiki.meego.com/Quality/TestSetGuideline

Note! During MeeGo 1.1 HandSet UX Timeframe QA will not form System Non-Functional Test Suites. Those will be targeted for 1.2 release.

=== Test Automation ===
* Testability driver has been selected as Handset UX automation tool

=== Requirement Coverage Visibility ===

* All relevant features are taken from featurezilla @bugzilla and inserted as testing requirements to Test Link-tool requirement interleaf
* Test cases which have been designed against features are then connected under features to show feature coverage

[[File:Feature_TC_Mapping.png]]

* Target is also to be able to show latest test execution status against features

=== Test Execution ===

All automated tests are executed in a MeeGo QA automated environment, and typically test results are available for each build.

Manual tests are executed regularly, but certainly before each release.

In general, MeeGo will be tested from the following different test execution levels.

*http://wiki.meego.com/Quality/TestSetGuideline

=== Test Reporting ===

All MeeGo 1.1 UX HandSet test results are stored to one place.

* MeeGo 1.1 Test Repository for HandSet
** http://wiki.meego.com/Quality/HandsetTestReport

Use Test Report Templates can be found: http://wiki.meego.com/TestReportTemplateCollection

=== Milestone Criteria ===

* There will be entry and exit criteria defined for each main milestone (Developer Preview, Feature Complete, Release Candidate and Project Release).
* All materials currently related to milestone quality criteria are stored to http://wiki.meego.com/Release_Engineering/Release_Timeline

== Network Environment ==
* Networking environment needed to conduct testing
** LAN
** WiFi network
** Internet
** 3G network

== Detailed Test Plans ==
To categorize the production requirements and identify the production functionality that will be tested, the product will be broken down to series of requirement set that QA owners are responsible for the validating.

{| border="1"
!| Application
!| QA Owner
!| Detailed test plan
|-
| Short Message Service || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforShortMessageService MeeGo 1.1 HandSet UX SMS Test Plan]
|-
| Dialer || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforDialer MeeGo 1.1 HandSet UX Dialer Test Plan]
|-
| Media Applications || Shuang Wan || <link to detailed test plan>
|-
| Mozilla Fennec Browser || Petri Jylha || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforMozillaFennecBrowser MeeGo 1.1 HandSet UX Mozilla Fennec Browser Test Plan]
|-
| Contacts|| Dayu Yang || <link to detailed test plan>
|-
| Core UX (Home, Theme, System UI)|| Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_CoreUX_TestPlan MeeGo 1.1 HandSet Core UX Test Plan]
|-
| Social Networking || Cathy Li || [http://wiki.meego.com/Quality/MeeGo1.1_Handset_UX_Social_Networking_TestPlan MeeGo 1.1 HandSet Social Networking Test Plan]
|-
| Compositing Window Manager|| N.N. || <link to detailed test plan>
|-
| Application install/uninstall || N.N. || <link to detailed test plan>
|-
| Virtual Keyboard || Yi Fu || [http://wiki.meego.com/Quality/1.1HandsetUXVkbTestPlan MeeGo 1.1 HandSet UX Virtual Keyboard Test Plan]
|-
| Clock || Qin Mu || <link to detailed test plan>
|-
| Email ||Yi Fu || <link to detailed test plan>
|-
| Calendar || Dayu Yang || <link to detailed test plan>
|-
| Instant Messaging || Mika Ikonen || [http://wiki.meego.com/Quality/MeeGo1.1HandSetUXTestPlanforInstantMessaging MeeGo 1.1 HandSet UX Instant Messaging Test Plan]
|}

== Dependency and Constraints ==
* Features' testability is a big dependency for test case design.
* Features' integration time line is another dependency for test case design. If features are integrated late, a lot of test cases' debug will be blocked.

== References ==
* QA main wiki: http://wiki.meego.com/Quality
* Feature Testability checklist: http://wiki.meego.com/Quality/TestabilityChecklist
* Testability Status Report: http://wiki.meego.com/Quality/HandsetTestabilityStatus
* Test Case Design Progress Follow-up: http://wiki.meego.com/Quality/HandsetTestSuite
* Test Result Reports: http://wiki.meego.com/Quality/HandsetTestReport
* Test Set Guideline: http://wiki.meego.com/Quality/TestSetGuideline
* Test Design Process and Guideline: http://wiki.meego.com/Quality/TestDesignProcessAndGuideline
* MeeGo Architecture http://meego.com/developers/meego-architecture
* MeeGo Bugzilla: http://bugs.meego.com/
* HandSet UX QA Ramp-Up follow up: http://wiki.meego.com/Quality/HandSetUXRamp-Up1.1

Quality/Media application test plan

2010-09-10T08:32:37Z

=Media Application Test Plan=
==Component Summary==
Media subsystem plays an important role in Handset usage model. The media software stack in MeeGo is divided into 3 layers briefly. Upper level applications include Music/Photo/Video/camera application (camera application don't cover in MeeGo 1.1 release). Middleware libraries include audio server(Pulseaudio), audio library(ALSA), audio/video decoder library (Gstreamer), Gstreamer plugin, QT-mobility, Tracker etc. Low level includes kernel modules or drivers such as Intel SST audio driver, V4l2 etc.
 

Our testing for handset 1.1 releases focus on upper layer application. And take advantage of shell command or script to validate the middleware or driver status just for bug root cause purpose.

The relevant media format & codecs tested is focusing on the free license file format & codecs. For example: ogg format, theora video, vorbis audio, bmp, jpg, png, gif, wav etc.

==Feature to be Tested==
{|cellspacing="0" border="1"
!|Feature Name
!|Feature Description
!|Vertical
!|Priority
!|Comments
|-
|Audio
|*Audio playback *Audio play control *Artist/Album/Song Navigation & Playback *Alphabetic scrolling *Playlist generation *Play Queue generation *Audio search *Shuffle and repeat *Usability testing *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 27 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Audio%20Player&product=MeeGo%20Handset%20Features
|-
|Photo
|*Basic photo viewer functionality *Photo album organization *Photo search *Photo mainipulation via popup menu *Photo Orientation *Thumbnail *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Photo%20viewer&product=MeeGo%20Handset%20Features
|-
|Video
|*Basic video playback *Video play control *Alphabetic scrolling *Video file mainipulation via popup menu *Video files filtering & search *Switch between different category *Stability testing *Negative testing *I18N 
|Handset
|P1
|The design input: UI design 28 Jul 2010 version and feature bugs: http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&query_format=advanced&component=Video%20player&product=MeeGo%20Handset%20Features
|}

==Features not to be tested==
Low level drivers will be not tested unless we need to perform the bug isolation.
middleware level like media framework(gstreamer) or tracker, qt-mobility, pulseaudio testing will not cover in this test plan.

==Dependency and Risks==
* Pulseaudio is buggy right now. Probably it will blocks the audio/video testing.
See Bug 150 - pulseaudio not work on CDK & Aava
* Video playback is not working also:
See Bug 5712 - Video playback is not working

==Test Strategy and Approach==
* UI testing
We will cover the media features via Music/Photo/Video application testing. Most of media usage models will be covered by this type of testing.
* User experience testing
Make sure user have comfortable usage experience like there no stuttering and no frame drop during video playback, no scratchy when playback audio and etc. There is no dedicate cases designed for it specially, it will be covered by normal usage testing.
* Negative testing
We will test boundary, negative input value and error input. Also some negative scenarios will be covered, such as playback non-media files or un-supported media file.
* Stability testing
Media is one type of resource intensive application (memory, cpu load) and user may put long time on one application like watch movie. So we test media apps' stability under long time run to make sure there is no crash etc.
* I18N testing
Test focus is for I18N mechanism in media application. Generally, we set another language environment (ex. Chinese Simplified) instead of default English environment and check the change of media application. Notes: only Chinese Simplified Translation will be covered by QA, for other languages, they will be covered by other teams.
* Performance testing
Performance testing will be covered by performance test separately.
* Others
For some low level testing such as pulseaudio, we may use pulseaudio compatible sample applications to cover the police config setting for typical usage models.

==Test Design==
====Audio====
* Audio playback
** Audio play/pause
** Audio mute/unmute
** Back to previous song
** Skip to next song
** Repeat playback
** Shuffle playback
* Artist/Album/Song Navigation & Playback
** Select given artist and browse the all albums within it
** Select given album and browse the all songs within it
** Playback all songs within one artist
** Playback all songs within one album
** Playback one song
* Alphabetic scrolling
** Scrolling to a given character head in My Music view
** Scrolling to a givne character head in playlist view
* Playlist manipulation
** Create one new playlist
** Delete one playlist
** Add all songs within one artist to a given playlist
** Add all songs wihtin one Album to a given playlist
** Add one song to a given playlist
** Playback a given playlist
* Audio search
** Search Artists/Albums/Songs by a given string
** All results/Artists/Albums/Songs tab view switch
** Playback the selected item from search result
** Add the selected item from search result into a given playlist
* Play Queue manipulation
** Switch to PlayQueue view
** Add a given song/album/artist to PlayQueue
** Check PlayQueue status display like show current song title in bottom bar
* Usability testing
** navigate the photos and listening music
** navigate the browsers and listening music
* Stability testing
** Playback the audio for a long time
** Browse great number of audio files
** Change the playback state randomly
* Negative testing
** Playback non-audio files
** Playback un-supported files
* I18N testing
** Switch language between Simplified Chinese and English

====Photo====
* Basic photo viewer functionality check
** Photo thumbnail
** View a given photo
** Photo slide show
** Photo navigation (Previous, Next)
* Photo format support (It should support common used free codec photo formats)
** JPEG
** BMP
** PNG
** GIF
* Photo album organization
** Organize photo album taken in similar time
** Organize photo album taken in similar place
* Photo search
** Search a photo by a given string
** Search an album by a given string
** Search result view switch between photo and album
* Photo mainipulation via popup menu
** Slide show
** Tag
** Copy
** Rotate CW
** Rotate CCW
** Delete
* Photo share
** email
** sms
** Facebook
** Picasa
** Bluetooth
** Flickr
* Photo Orientation
** Portrait
** Landscape
* Stability testing
** Slide show the photo files for a long time
** Browse great number of photo files
** view large photo file
* Negative testing
** View non-picture files
** View un-supported picture files
* I18N testing
** Switch language between Simplified Chinese and English

====Video====
In MeeGo 1.1 release, video player test points includes:
* Basic video playback check
* Video control bar functionality check
* Alphabetic scrolling
* Video file mainipulation via popup menu
* Video files filtering
* Video file search
* Video snapshot browsing
* Snapshot view switch between "My Movies", "Films" and "TV Shows"
* Stability testing
** Playback a long video file
** Browse great number of video files
** Change the playback state randomly
* Negative testing
** Playback non-video files
** Playback un-supported video files
* I18N testing
** Switch language between Simplified Chinese and English

==Test Environment==
===Hardware Platforms===
* Aava Koski EV2(Qual Sample) with Maxim PMIC (Firmware:2.46.4)
* Moorestown CDK with NEC PMIC (Firmware: IFWI 2.43, Kboot 0.10)

==QA Contact==
[mailto:shuang.wan@intel.com Wan Shuang]
 [mailto:jessica.ji@intel.com Jessica Ji]

==Developers==
* Video player
** [mailto:Halley.Zhao@intel.com Halley Zhao] and [mailto:huan.zheng@intel.com Huan Zheng]
* Audio
** Audio Player [mailto:forrest.zhao@intel.com Forrest Zhao]
* Photo viewer
** [mailto:long.bu@intel.com Bu Long], [mailto:ziv.chang@intel.com Ziv Chang]