Meego Wiki
Views

Quality/TestSuite/MeeGo Graphics Test Plan

From MeeGo wiki
< Quality | TestSuite(Difference between revisions)
Jump to: navigation, search
(Created page with "= MeeGo Graphics Test Plan = == Component Summary == The graphics subsystem is composed of Xserver, Mesa, 2D driver (e.g. Xf86-video-intel) and Drm. It should provide the latest…")
(Undo revision 23800 by Shaofeng (Talk))
 
(19 intermediate revisions not shown)
Line 5: Line 5:
== Feature to be Tested ==
== Feature to be Tested ==
-
||<tablewidth="800px">'''Feature Name''' ||'''Description''' ||'''Platform''' ||'''Comments''' ||
+
What features to be tested are made of basic features + accepted features of MeeGO-1.1/MeeGo-1.2.
-
||''OpenGL ES 2.0'' || Should support OpenGL ES 2.0 || ''All'' || ''major feature'' ||
+
=== Table.1 Basic features ===
-
||''OpenVG'' || Should support OpenVG || ''ALL'' || ''major feature'' ||
+
-
||''Display'' || Should support Single/Multi display, Mode setting, Rotation setting  ||''All'' ||''major feature'' ||
+
-
||''Input'' || Should support multi-touch in X. || ''except for Non-touchscreen platform'' || ''major feature'' ||
+
-
||''Video'' || Should support XVideo and libVA || ''All'' || ''n/a'' ||
+
-
 
+
-
=== Table.1 MeeGo-1.1 ===
+
{| cellspacing="0" border="1"
{| cellspacing="0" border="1"
! ''FEA ID''
! ''FEA ID''
Line 20: Line 14:
! ''Priority''
! ''Priority''
|-
|-
-
|''2734''|| ''[MASTER] Bluetooth'' || ''Check package installation status. At least these packages need to be installed in image:<br>* bluez <br>* bluez-libs <br>* obexd (client) <br>* openobex'' || ''ALL'' || ''P1''  
+
|''n/a''|| ''X11'' || ''* Xserver should work well with Mesa, 2D driver and other components. <br>* Many X extensions should be supported, including Xrender, Xcomposite, Xdamage, Xrandr adn so on.'' || ''ALL'' || ''Major''
|-
|-
-
|''3002''|| ''FTP'' || ''FTP profile is to browse/send/receive/remote-create/remote-delete files with paired machine by OBEX. <br>* FTP browse obexd-server folder <br>* FTP send one file to obexd-server <br>* FTP receive(get) one file from obexd-server <br>* FTP create one file/folder on obexd-server's shared folder <br>* FTP delete one file/folder on obexd-server's shared folder '' || ''ALL'' || ''P1''  
+
|''n/a''|| ''OpenGL ES 2.0'' || ''Should support OpenGL ES 2.0'' || ''ALL'' || ''Major''  
|-
|-
-
|''3012''|| ''PBAP'' || ''PBAP profile is to get PhoneBook from mobile phone by OBEX'' || ''ALL'' || ''P1''  
+
|''n/a''|| ''OpenVG'' || ''Should support OpenVG'' || ''ALL'' || ''Major''  
|-
|-
-
|''3003''|| ''OPP'' || ''OPP profile is Object Push profile from device to mobile phone by OBEX. <br>Mutilple files (pitcures, text, music) will be sent to phone at one time.'' || ''ALL'' || ''P2''  
+
|''n/a''|| ''Display'' || ''* Should support Single/Multi display <br>* Mode setting  <br>* Rotation setting'' || ''ALL'' || ''Major''  
|-
|-
-
|''4716''|| ''SYNC'' || ''SYNC profile is to get/put contact with mobile phone by OBEX. <br>Test method is much like PBAP and OPP.'' || ''ALL'' || ''P2''  
+
|''n/a''||''Input''|| ''Should support proper event for input devices (i.e. multi-touch) in X'' || ''ALL'' || ''Major''  
|-
|-
-
|''3004''|| ''A2DP'' || ''A2DP profile is used enjoy music by Bluetooth Headset. <br>After connecting to BT Headset (pairing and redirect audio channel to headset), music could be heard by headset.'' || ''ALL'' || ''P1''  
+
|''n/a''|| ''Video'' || ''Should support XVideo and libVA'' || ''ALL'' || ''n/a''  
-
|-
+
|-  
-
|''4719''|| ''A2DP Direct streaming '' || ''Some streaming audio files can be played through A2DP. Need to setup A2DP connection firstly.<br>* AAC LC <br>* HE AAC <br>* WMA '' || ''ALL'' || ''P2''
+
|}
-
|-
+
 
-
|''3008''|| ''AVRCP'' || ''AVRCP profile is used for Bluetooth Headset control. Some keys on Headset will take effection:<br>* play/pause <br>* volume up/volume down <br>* next/pre --- special for user has a play list in media player'' || ''ALL'' || ''P1''
+
=== Table.2 MeeGo-1.1 ===
-
|-
+
{| cellspacing="0" border="1"
-
|''3006''|| ''PAN User'' || ''PANU profile is make Personal Area Network by BT to mobile phone which could reach internet.<br>* [Basic point] MeeGo device (PAN User) could get a person network connection to BNEP (PAN Server). <br>* [Enhanced] With same http_proxy, MeeGo device could log on internet web, such as www.google.com'' || ''non-Handset'' || ''P1''
+
! ''FEA ID''
-
|-
+
! ''Feature summary''
-
|''3005''|| ''HID'' || ''HID profile is to use Bluetooth human interface hardware. <br>* BT keyboard <br>* BT mouse <br>* [Enhanced] BT playjoy'' || ''ALL'' || ''P1''  
+
! ''Feature description and Test points''
-
|-
+
! ''Platform''
-
|''3011''|| ''HSP AG'' || ''HSP profile is Headset Profile, which is to make BT Headset to control a phone call. <br>* ring a call <br>* answer a call <br>* hang up <br>* adjust the volume'' || ''Handset'' || ''P2''  
+
! ''Priority''
-
|-
+
-
|''3007''|| ''HFP'' || ''HFP profile is to make driver Hands Free to take a call. <br>* Pair with mobile phone <br>* Redirect palseaudio to bluetooth HFP channel <br>* Hear and speak with the testing device to take a phone call'' || ''Automotive'' || ''P2''  
+
|-
|-
-
|''4717''|| ''SAP'' || ''SAP profile is commonly used in car, which is SIM Access Profile. No test point on non-automotive platforms'' || ''Automotive'' || ''P2''  
+
|''4877''|| ''[FEA] Provide libOpenVG and libOpenVGU packages'' || ''OpenVG is supported'' || ''ALL'' || ''Major''  
 +
|-
|}
|}
-
=== Table.2 MeeGo-1.2 ===
+
=== Table.3 MeeGo-1.2 ===
-
Those Bluetooth Low Energy features need BT LE support Hardware and BT LE driver support in kernel.
+
{| cellspacing="0" border="1"
{| cellspacing="0" border="1"
! ''FEA ID''
! ''FEA ID''
Line 56: Line 48:
! ''Priority''
! ''Priority''
|-
|-
-
|''7110''|| ''[MASTER] Bluetooth Low Energy (BT LE)'' || ''Qt-mobility should provide Bluetooth Low Energy APIs'' || ''ALL'' || ''P1''  
+
|''7614''|| ''[FEA] Multiple output (HDMI) support from Graphics and Window manager'' || ''* HDMI output is supported by graphics driver;  <br>* Multiple display is supported by Window Manager'' || ''HDMI platform'' || ''Major''  
|-
|-
-
|''7112''|| ''[BT LE] BT LE support in BlueZ '' || ''Bluetooth Low Energy APIs in bluez-libs '' || ''ALL'' || ''P1''
 
-
|-
 
-
|''7116''|| ''[BT LE] Secure data transfer'' || ''Bluetooth Low Energy should support 128-bits encryption. '' || ''ALL'' || ''P1''
 
-
|-
 
-
|''7114''|| ''[BT LE] Proximity tracking '' || ''Bluetooth Low Energy supports short-time device finding. '' || ''ALL'' || ''P1''
 
-
|-
 
-
|''7115''|| ''[BT LE] Non-secure data transfer '' || ''Bluetooth Low Energy no encryption mode '' || ''ALL'' || ''P2''
 
-
|-
 
-
|''7113''|| ''[BT LE] Bluetooth LE developer offering and APIs '' || ''In definition '' || ''ALL'' || ''P2''
 
-
|-
 
-
|''7111''|| ''[BT LE] HW verification in adaptation sw '' || ''In definition '' || ''ALL'' || ''P2''
 
|}
|}
-
== Feature not to be Tested ==
+
== Table.4 Feature not to be Tested ==
-
Bluez tests do not cover Drivers and Low Level BT Profiles
+
-
 
+
{| cellspacing="0" border="1"
{| cellspacing="0" border="1"
! ''Summary''
! ''Summary''
Line 80: Line 59:
! ''Reasons''
! ''Reasons''
|-
|-
-
|''BT driver''|| ''No BT driver for platform'' || ''NO'' || ''Use hciconfig to try to find info of the Blue device. <br>We do not support Hardware driver missing.''
+
|''OpenGL''|| ''OpenGL ES2.0 would be the default 3D backend'' || ''n/a'' || ''OpenGL not be supported by SGX540 h/w''
|-
|-
-
|''BT driver''|| ''HAL error'' || ''NO'' || ''Use hciconfig to try to find info of the Blue device. <br>We do not check other hardware error happening during bluetoothd starting process.''
 
-
|-
 
-
|''Low Level BT Profile''|| ''RFCOMM'' || ''NO'' || ''The RFCOMM profile is on L2CAP, works for serial connection. We do not test RFCOMM especially''
 
-
|-
 
-
|''Low Level BT Profile''|| ''L2CAP'' || ''NO'' || ''The L2CAP profile can be used by many Up-level profiles (such as FTP, OPP...). We do not test L2CAP especially''
 
|}
|}
== Test Strategy and Approach ==
== Test Strategy and Approach ==
-
Test is focused on Bluez user level profiles. Main goal is to ensure bluez stack can work well to comminucate with outside hardwares to implement different usages. All the commonly used BT profiles should be tested on different platforms either by GUI or by terminal commands. <br>
+
All the above features would be tested. You can get the latest cases from http://gitorious.org/qa-tools/mcts. <br>
-
For those low level profiles or Bluetooth drivers on platforms, there is no cover. <br>
+
-
<u>''Test Methodology:''</u>
+
== Table.5 Feature/Lib/Case mapping ==
-
 
+
{| cellspacing="0" border="1"
-
Bluez library provides many tools to do hardware BT connection. For example:<br>
+
! ''FEA name''
-
Nearly all the bluetooth test needs the first step: Bluetooth pairing. This can be done by '''simple-agent''' in bluez-test RPM. <br>
+
! ''Target Lib''
-
BT Keyboard and BT mouse connection uses HID profile, which could be done by HID daemon. '''hidd --connect <Keyboard MAC>''' and '''hidd --search''' can help to connect keyboard and mouse. <br>
+
! ''Test suite package name''
-
For the profiles through OBEX, we write python scripts to call OBEX server APIs to transfer data. <br>
+
|-
-
For PANU, we setup a unbutu10.04 machine as BNEP server. PAN user could link to the BNEP after pairing with the ubuntu machine. When PANU needs log on to internet, the BNEP server will do ipforwarding from the bnap0 to eth0 (eth0 is connecting to outside).<br> 
+
|''X11''||''libX11 libXdamage libXcomposite libXrandr libXrender libXv''||''blts-x11''
 +
|-
 +
|''OpenGL ES2.0''||''libEGL libGLESv2''||''blts-opengles2-perf''  
 +
|-
 +
|''OpenVG''||''libEGL libOpenVG''||''mcts-openvg-tests''
 +
|-
 +
|''Display''||''libXrandr''||''mcts-display-tests''  
 +
|-
 +
|''Input''||''libX11''||''blts-input-devices''
 +
|-
 +
|''Video''||''libXv''|| ''blts-xvideo''  
 +
|-
 +
|}
<u>''Test Types:''</u>
<u>''Test Types:''</u>
-
Although the test points are classified by each profiles, we also do negative, performance and even stress test to the test points.  
+
Most of cases are auto cases and provide function and performance testing.
-
* Positive test must pass
+
* Function test: All.
-
* If the system has GUI (such as gnome-bluetooth-meego), some tests are validated through GUI following user experience.
+
* Performance test: OpenGL ES2.0, OpenVG, Xrender.
-
* Negative and stress test will be especially run for FTP, for example:
+
-
** Transfer the file to wrong target, or target memory is full
+
-
** Transfer multiple files with small-size each
+
-
** Transfer one big files with hundreds of MB
+
-
* Performance Test will also be cared when doing FTP test, file transfer speed should be mor than 10K. Or we will treat BT connection not stable.  
+
<u>''Test level:  ''</u>
<u>''Test level:  ''</u>
-
Current test level is API level. All cases are based on middleware usage, no dependency on how GUI is like. <br>
+
All the cases are API level and inplemented with C language. Table.5 shows the target lib too. <br>
-
 
+
-
<u>''Flexibility:''</u>
+
-
 
+
-
Nokia shares some test cases calling bluez-libs API in C language. Although those cases are not in our test strategy, we still try to use bluez-libs to enlarge our profile test scope if possible. <br>
+
=== Test Automation ===
=== Test Automation ===
-
We will keep one PC as server machine for auto test, so automated cases are the profiles which can be used between two computers. Based on this strategy, we currently made FTP/OPP/PANU profiles test cases automated. <br>
+
All the cases tracked in mcts should be automatic. How to build/run cases, pls refer to README of every testing package.
-
There is no low level profiles automated by us. Nokia share some codes in MWTS and BLTS to test RFCOMM and L2CAP profiles. But unfortunately, all their cases are semi-automatic in fact. <br>
+
-
The test framework to run our auto cases is MCTS. Those cases are written by shell scripts, some utility tools (called by cases) are made by python. Before case running, a OBEXD-Server machine must be setup there and wait for obexd-client call. Detail information can be refered from the README file in MCTS:Bluez component. Git repository is http://gitorious.org/qa-tools/mcts
+
== Test Design ==
== Test Design ==
Line 130: Line 104:
== Test Environment ==
== Test Environment ==
-
Nearly all the bluez test cases need at least two devices. One is testing machine, the other may be some BT integrated device. <br>
+
It's OK to run the cases without UI home or Window manager, because our test target is graphics lib/stack only.
-
There are still some scripts needed for profile checking, since we do not depend on GUI APP.  
+
-
=== Hardware Platforms ===
+
=== Table.6 Hardware Platforms ===
{| cellspacing="0" border="1"
{| cellspacing="0" border="1"
! ''Test Platforms''
! ''Test Platforms''
-
! ''Networking''
+
! ''Graphics h/w''
-
! ''Other Devices''
+
! ''Priority''
! ''Priority''
|-
|-
-
|''Pinetrail Netbook: Acer NAV50''|| ''LAN, Wifi, BT'' || ''BT headset (Dell DH200), BT Keyboard (Dell)...'' || ''P1''
+
|''Pinetrail Netbook: EeePC 1005 HA''|| ''intel Pineview'' || ''P1''  
-
|-
+
-
|''Diamondville Netbook: EeePC 1005 HA''|| ''LAN, Wifi, BT'' || ''BT headset (Dell DH200), BT Keyboard (Dell)...'' || ''P2''  
+
|-
|-
-
|''Handset: Avaa''|| ''USB, BT-dongle (BT2.0+EDR)'' || ''BT headset (Dell DH200), BT Keyboard (Dell)...'' || ''P1''  
+
|''Handset: Avaa''|| ''PVR SGX535'' || ''P1''  
|}
|}
=== Tools ===
=== Tools ===
-
Some python scripts are used to check FTP/OPP/PBAP... functions in middleware (without GUI). They are from:<br>
+
N/A
-
* bluez source code: bluez/test/simple-agent --- For bluetooth pairing
+
-
* bluez source code: bluez/test/test-network --- For PANU connection setup with BNEP (PAN Server)
+
-
* obexd source code: obexd-0.32/test/ftp-client --- For FTP send/get/browse checking
+
== QA Contact ==
== QA Contact ==
-
jingke.zhang@intel.com<br>
+
kui.zheng@intel.com<br>
-
zhaojuan.guo@intel.com (intern)
+
xuetingx.jin@intel.com (intern)
-
== Developers ==
+
== Distrobution ==
-
majid.awad@intel.com <br>
+
peng.li@intel.com <br>
-
qiang.z.zhang@intel.com <br>
+
== Referrences ==
== Referrences ==
-
BT Profile introduction: http://en.wikipedia.org/wiki/Bluetooth_profile<br>
+
Xorg Fundation: http://www.x.org/wiki/<br>
-
Bluez source and tool downloading: http://www.bluez.org<br>
+
Intel Linux Graphics: http://intellinuxgraphics.org/index.html<br>
-
BIG offical webpage: http://www.bluetooth.org
+
MCTS QA-tools: http://gitorious.org/qa-tools/mcts

Latest revision as of 03:10, 9 December 2010

Contents

MeeGo Graphics Test Plan

Component Summary

The graphics subsystem is composed of Xserver, Mesa, 2D driver (e.g. Xf86-video-intel) and Drm. It should provide the latest graphics h/w features, as well as meet requirement of UI framework (e.g. Clutter, Qt), Window Manager even applications.

Feature to be Tested

What features to be tested are made of basic features + accepted features of MeeGO-1.1/MeeGo-1.2.

Table.1 Basic features

FEA ID Feature summary Feature description and Test points Platform Priority
n/a X11 * Xserver should work well with Mesa, 2D driver and other components.
* Many X extensions should be supported, including Xrender, Xcomposite, Xdamage, Xrandr adn so on. 		ALL Major
n/a OpenGL ES 2.0 Should support OpenGL ES 2.0 ALL Major
n/a OpenVG Should support OpenVG ALL Major
n/a Display * Should support Single/Multi display
* Mode setting
* Rotation setting 		ALL Major
n/aInput Should support proper event for input devices (i.e. multi-touch) in X ALL Major
n/a Video Should support XVideo and libVA ALL n/a

Table.2 MeeGo-1.1

FEA ID Feature summary Feature description and Test points Platform Priority
4877 [FEA] Provide libOpenVG and libOpenVGU packages OpenVG is supported ALL Major

Table.3 MeeGo-1.2

FEA ID Feature summary Test points Platform Priority
7614 [FEA] Multiple output (HDMI) support from Graphics and Window manager * HDMI output is supported by graphics driver;
* Multiple display is supported by Window Manager 		HDMI platform Major

Table.4 Feature not to be Tested

Summary Detail scenario Support Reasons
OpenGL OpenGL ES2.0 would be the default 3D backend n/a OpenGL not be supported by SGX540 h/w

Test Strategy and Approach

All the above features would be tested. You can get the latest cases from http://gitorious.org/qa-tools/mcts.

Table.5 Feature/Lib/Case mapping

FEA name Target Lib Test suite package name
X11libX11 libXdamage libXcomposite libXrandr libXrender libXvblts-x11
OpenGL ES2.0libEGL libGLESv2blts-opengles2-perf
OpenVGlibEGL libOpenVGmcts-openvg-tests
DisplaylibXrandrmcts-display-tests
InputlibX11blts-input-devices
VideolibXv blts-xvideo

Test Types:

Most of cases are auto cases and provide function and performance testing.

  • Function test: All.
  • Performance test: OpenGL ES2.0, OpenVG, Xrender.

Test level:

All the cases are API level and inplemented with C language. Table.5 shows the target lib too.

Test Automation

All the cases tracked in mcts should be automatic. How to build/run cases, pls refer to README of every testing package.

Test Design

Test points are written in Tested Feature Tables.
Test method is introduced in Test Strategy and Approach section.

Test Environment

It's OK to run the cases without UI home or Window manager, because our test target is graphics lib/stack only.

Table.6 Hardware Platforms

Test Platforms Graphics h/w Priority
Pinetrail Netbook: EeePC 1005 HA intel Pineview P1
Handset: Avaa PVR SGX535 P1

Tools

N/A

QA Contact

kui.zheng@intel.com
xuetingx.jin@intel.com (intern)

Distrobution

peng.li@intel.com

Referrences

Xorg Fundation: http://www.x.org/wiki/
Intel Linux Graphics: http://intellinuxgraphics.org/index.html
MCTS QA-tools: http://gitorious.org/qa-tools/mcts

Retrieved from "http://wiki.meego.com/Quality/TestSuite/MeeGo_Graphics_Test_Plan"
Personal tools