Contents |
This is test plan for MeeGo 1.2 HandSet System Non-Functional Testing (NFT)
MeeGo 1.2 release Non-Functional Requirements (NFR) are listed in FeatureZilla We are not testing all of NFRs because many of them are not realistic anymore for current public HW (N900). Especially when we will be running release from memory card which decreases overall performance.
These features will be tested:
| Feature Name | Description | Comments |
|---|---|---|
| Feature 12089 - [FEA] Non-func - Application start-up time for pre-started applications | System shall start the most frequently used applications (which are pre-started) immediately (within 100ms) so that device feels responsive. | Pre-started applications not yet defined. As a basis we will use release 1.1 key applications (Dialer, SMS, Browser, Music Player, and Video Player) |
| Feature 12090 - [FEA] Non-func - Application start-up time for (non-pre-started) applications | System shall start all applications (excluding pre-started applications) within 0.5s so that device feels responsive | Pre-started applications not yet defined. As a basis we will use release 1.1 key applications => other are non-pre-started applications |
| Feature 12091 - [FEA] Non-func - Application start-up time for Browser | Browser application shall start and display meego.com page within 3s. This requirement assumes Internet connection is already present and active (WLAN 802.11g). | |
| Feature 12095 - [FEA] Non-func - Cold boot | System shall boot up from cold state as quickly as possible. Maximum boot time shall be 30s, from cold state to UI being completely usable. | |
| Feature 12096 - [FEA] Non-func - Complete shut down | The system shall power off quickly and without losing any data. The maximum time to complete shut down of the system shall be 10s. | |
| Feature 12097 - [FEA] Non-func - Screen orientation change | System shall render the UI in new orientation immediately after changing the device orientation. The UI shall be re-drawn in new orientation within 0.5s in all use scenarios. | |
| Feature 12101 - [FEA] Non-func - Video playback and incoming call | It shall be possible to hear the ringtone within 0.5s during an active video playback.
Note: the behaviour depends on the policy defined by device vendor whereas the purpose of this requirement is to define the required performance once above policy is applicable. MeeGo reference UXs policies are defined at the Policy requirements. | |
| Feature 12102 - [FEA] Non-func - Listen music and incoming call | It shall be possible to hear the ringtone within 0.5s while listening to music playback.
Note: the behaviour depends on the policy defined by device vendor whereas the purpose of this requirement is to define the required performance once above policy is applicable. MeeGo reference UXs policies are defined at the Policy requirements. | |
| Feature 12103 - [FEA] Non-func - Incoming call indication | System shall play ringtone for incoming call within 0.5s, when system is in idle state. |
Some of the NFRs are not realistic for N900 and that's why they are not measured. Here's a list of them:
| Feature Name | Description | Comments |
|---|---|---|
| Feature 12092 - [FEA] Non-func - Immediate feedback to user interactions | System shall respond immediately (within 100ms) to all user actions performed. | We're currently missing a tool for measuring this fast and accurate haptic feedback |
| Feature 12093 - [FEA] Non-func - UI framerate | System shall support rendering of animated effects such as transitions at the maximum frame rate supported by the HW. A common reference is minimum 60fps for fluent transitions. | Currently we are missing a proper tool for measurements but that will be hopefully solved |
| Feature 12094 - [FEA] Non-func - Reaction to power off & on | System shall give immediate feedback when pressing power button on the device
to turn it off. There shall be a visual and possibly other feedback such as haptic given within 100ms for power switch. | We're currently missing a tool for measuring this fast and accurate haptic feedback |
| Feature 12098 - [FEA] Non-func - 1080p video decoding performance | Once 1080p (Full HD) video decoding is supported the system shall provide minimum playback performance of 30FPS, with HW accelerated codecs supported by the system (eg. H.264 and AAC). Video bitrate upto 20Mbps, audio bitrate upto 448kbps. | Public MeeGo release supports only open source codecs. Not testable |
| Feature 12099 - [FEA] Non-func - Seeking in video playback | System shall be able to perform seek back- and forward functions within the currently playing video content. System shall resume the playback in new seek position without loosing the visual or audio quality eg. maintaining lip-sync at all times. | This FEA is tested in Video Player basic feature test set |
| Feature 12100 - [FEA] Non-func - 1080p video encoding performance | Once 1080p video encoding is supported, the system shall be able to capture video at a minimum of 30FPS using HW-accelerated video codecs present in the system (eg. H.264 and AAC). Video bitrate upto 20Mbps, audio bitrate upto 448kbps. | Public MeeGo release supports only open source codecs. Not testable |
Non-functional testing (NFT) can be divided in higher level to two different areas: performance and reliability testing. The official NFRs are mostly concentrating to response time but there are also couple ones related to framerate. Other performance areas and reliability area will be planned and tested based on 1.2 SW maturity status. Test areas that will be used:
Reporting of NFT will happen done to public wiki instead of QA Reports tool. Reporting will focus on trend metrics which will easily show how e.g. response times have been developing. Link to reports will be added here when reporting starts.