Contents 1 MeeGo SDK Test Plan (Content not ready) 1.1 Scope of this Document 1.2 Features to Be Tested 1.2.1 Tools to set up develop environment 1.2.2 Test Host & Target 1.2.3 Check Pointer 1.3 What will not Be Tested 1.4 Test Strategy and Approach 1.4.1 Test Coverage Definition 1.4.2 Component Test Plans and Coverage 1.4.3 Test Cycle 1.4.3.1 Weekly Testing 1.4.3.2 Milestone Testing 1.4.3.3 Regression Testing 1.4.4 Test Report 1.5 Environmental Needs 1.5.1 Hardware Platforms 1.5.2 Peripherals required 1.5.3 Network Environment 1.6 References

MeeGo SDK Test Plan (Content not ready)

Scope of this Document

This is overall test plan for MeeGo SDK(Software Development Kit) of MeeGo open source project, which defines test scope, test strategy, test configurations as well as test execution cycle for MeeGo SDK. It will give readers an overview of validation activities for MeeGo SDK of MeeGo open source releases. A series of component test plans will also be linked in this overall test plan to cover the detailed test approaches and test design for ingredients of MeeGo SDK. It will be the joint effort from Nokia, Intel and MeeGo community.

Features to Be Tested

Overall the MeeGo Core Testing will cover below host, target and testpoints:

• P1: Test focus with main functionality, make sure common case work
• P2: Nice to have for coming release
• TBD: not covered in coming release

Tools to set up develop environment

Tools	Priority
Emulator: Qemu	P1
Simulator: Xephyr + chroot	P2

Test Host & Target

Host\Target	Netbook	Handset	Tablet
Fedora13	P1	P1	P1
Fedora12	P2	P2	P2
Ubuntu10.04	P1	P1	P1
Ubuntu9.10	P2	P2	P2
OpenSuse 11.2	TBD	TBD	TBD
OpenSuse 11.3	TBD	TBD	TBD
WindowsXP	P1	P1	P1
Win 7	TBD	TBD	TBD
Meego Netbook	TBD	TBD	TBD
Mac OS	TBD	TBD	TBD

Check Pointer

Check Point		Priority
SDK installer works		P1
SDK images compliance with Meego images		P1
Boot SDK images with Qemu		P1
Boot SDK images with Xephyr		P2
Basic function check for SDK image	need more detail checklist here	P1
Qt Creator		P1
MADDE		P1
sysroots		P1
toolchains		P1
App development	CLI QT GUI QT MTF QT WRT	P1 P1 P1 TBD
Documents & Toturials	need detail checklist here	P1
Additional tools		TBD
Memory/performance/power check		TBD

What will not Be Tested

Following feature category won't be covered in MeeGo SDK testing.

• Only light test for SDK image, not check all of the functionalities or applications inside it, but only major deamon and service inside it.
• Not test for SDK package or image building, only check the built out package or images.
• Not cover Meego compliance test, but only check key package versions which should align with Meego release.
• ARM support are not tracked in this document yet, which should be owned by Nokia.
• MADDE, QT Creator will be fully tested by Nokia, no thorough test covered in this document.

Test Strategy and Approach

The overall objective of MeeGo Core testing is to ensure MeeGo middlewares provide stable hardware and usage model independent application services and APIs for building the vertical specific user experience. Each core component has different quality risk regarding to MeeGo integration. For example, some core component is mature in upstream and MeeGo do the integration without heavy new feature development; some component is contributed and open-sourced from proprietary component with heavy development. Considering most of MeeGo Core components will be adopted by multiple vertical usages and run on a number of MeeGo devices, we also need to take test execution efficiency into account when creating the test cases. Given this, we have below strategy considerations:

• Create one unified test suite (MeeGo Core Test Suite) for MeeGo Core validation. It's open-sourced and could start with the first batch of test cases contributed from Nokia and Intel, and will continually grow with the contribution from community. You can find the MeeGo Core Test Suite (MCTS) docs at MeeGo Quality page, and the code in Gitorious.
• Define appropriate test coverage (density) categories. The test coverage of each component is based on the quality risk regarding to MeeGo integration.
• Automate test cases as many as possible and be able to run under MeeGo auto testing framework.

Test Coverage Definition

In order to use resources efficiently and control risk level on component maturity. Testing is done in different levels as follows:

• Thorough
  • Full testing service for API
  • Target is to have 100% function/method coverage provided by API
  • Parameter coverage defined by using testing techniques as follows:
    • All Pairs (Pair wise)
    • Equivalence Partitioning
    • Boundary Value Analysis
    • Subjective test case selection (according to Expert opinion or similar)

• Average
  • Between Thorough and Light
  • Full API coverage (according to rules for Thorough) for new functionality.
  • Integration level testing for legacy features.

• Light
  • Integration level testing
  • Upstream Middleware components shall be tested with limited amount of test cases (sub set) ensuring that integration of component to MeeGo is successful and component is working as expected

• Not Covered
  • No Middleware testing, covered indirectly by UX Testing
  • There will be API’s not feasible to cover with test cases at middleware level but more efficiently tested indirectly by UX testing

Component Test Plans and Coverage

Category	Component	Test Coverage	Status
Comms Services	[[../oFono Test Plan | Telephony (oFono)]]	Thorough	test suite status
	[[../ConnMan Test Plan | Connection Management (ConnMan)]]	Thorough
	[[../Bluetooth Test Plan | Bluetooth (BlueZ)]]	Average
	[[../Telepathy Test Plan | VOIP, IM and Pres (Telepathy)]]	Light
Internet Services	Layout Engine (WebKit)	Not Covered
	[[../WRT Test Plan | Web RunTime (WebKit)]]	Open
	[[../Libsocialweb Test Plan | Web Services (libsocialweb)]]	Thorough
	[[../GeoClue Test Plan | Location (GeoClue)]]	Light
Visual Services	[[../3DGfx Test Plan | 3D Graphics (OpenGL/GLES)]]	Average
	[[../2DGfx Test Plan | 2D Graphics (QPainter)]]	Light
	[[../Clutter Test Plan | Clutter]]	Light
	[[../X Test Plan | X]]	Light
Media Services	[[../GStreamer Test Plan | Media Framework (GStreamer)]]	Light
	[[../Gstreamer_plugin Test Plan | Camera (Gstreamer plug-in)]]	Light
	Codecs (Gstreamer plug-in)	Light
	[[../PulseAudio Test Plan | Audio (PulseAudio)]]	Light
	[[../UPnP Test Plan | GUPnP]]	Light
Data Management	[[../Tracker Test Plan | Content Framework (Tracker)]]	Thorough
	[[../ContextKit Test Plan | Context Framework (ContextKit)]]	Thorough
	[[../Package Manager Test Plan | Package Manager (PackageKit)]]	Light
Device Services	[[../Device Health Test Plan | Device Health]]	Thorough
	[[../SensorFramework Test Plan | Sensor Framework]]	Average
	[[../Resource Manager Test Plan | Resource Manager]]	Average
	[[../BackupRestore Test Plan | Backup & Restore]]	Open
Personal Services	[[../PIM Test Plan | PIM services]]	Average
	[[../Buteo Test Plan | Device Sync (Buteo)]]	Thorough
	[[../Accts Test Plan | Accts & SSO]]	Open

Test Cycle

MeeGo Core will be tested from the following different test execution levels.

Weekly Testing

Weekly Testing is a test cycle against the weekly images released from distribution team. Usually the test frequency is twice a week, which aligns with distribution's weekly image release cadence.

It's an incremental testing and includes two parts in general: test the basic distribution health to report out regression, test new features when they are ready for testing. We will also do bug verification and regression test which is set of tests to verify that changes from the last build (code enhancements, bug fixes) don't introduce new issues to the previous working code. The purpose of weekly testing is to early test the new features, to track critical/major issues got fixed in timely fashion, and to report out the overall quality status of the whole image so that people could understand where we are.

Milestone Testing

In the formal test cycle for milestone test, after a new build passed the BAT test, test team will start the full validation testing for it, following test will be involved:

• Functional Test – which will follow the component test plan to run all tests.
• Non Functional Test – which are defined in the individual component test plan.

Regression Testing

This is a set of tests to verify that changes from the last build (code enhancements, bug fixes) don’t introduce new issues to the previous working code as well as new features work as expected. This cycle include the tests for previous major bug fixes and areas of the code that will be affected by new implementation.

The regression test will be taken in following test cycle:

• In the test cycle for milestone release, before the release criteria is satisfied, it is possible that there are several release candidates which includes bug fixes for previous release candidates. These bug-fixes need be tested
• In the intermediate regular release, only the modified features need to be heavily tested.
• Bug verification on weekly basis to make sure the bug fixes be verified within one week after bug fixing

Test Report

• Send Test report to mailing list
  • Weekly MeeGo Core Test Report (Bug verification, new feature exploration...)
  • Distribution Milestone Full Pass Test report
• Archive reports at MeeGo Quality wiki

Environmental Needs

Hardware Platforms

MeeGo v1.1

• Aava Koski
• MRST CDK
• Medfield CDK
• N900
• Netbook

Peripherals required

Category	Peripherals
Wireless	Wireless AP (abg)
Bluetooth Devices	Bluetooth dangles, Bluetooth keyboard, Bluetooth mouse,Bluetooth phone headset, Bluetooth A2DP headset
USB devices	Camera, keyboard, mouse, USB storage device , microphone, ...
3G	3G modem and Sim Card
SD Card	1G/2G/4G
MMC Card	512M/1G/2G
Speaker	5.1

Network Environment

The networking arrangements needed to conduct the testing:

• LAN
• WiFi network (abg)
• Internet
• 3G network

References

• MCTS (MeeGo Core Test Suite) code git tree: http://gitorious.org/qa-tools (to be moved meego.gitorious.org)
• MCTS docs http://wiki.meego.com/Quality/TestSuite/MCTS
• MeeGo Architecture: http://meego.com/developers/meego-architecture
• MeeGo Core test reports archive: http://wiki.meego.com/Quality/CoreTestReports
• MeeGo bugzilla: http://bugs.meego.com