Revision as of 08:14, 17 March 2011 (view source) Vilvo (Talk | contribs) (→Test automation images) ← Older edit		Revision as of 08:16, 17 March 2011 (view source) Vilvo (Talk | contribs) (→QA TODOs) Newer edit →
Line 1:		Line 1:
-	= QA TODOs =	+	= QA TODOs (in priority order) =
	== OTS setup and automated hourly testing ==		== OTS setup and automated hourly testing ==

---

Revision as of 08:16, 17 March 2011

Contents 1 QA TODOs (in priority order) 1.1 OTS setup and automated hourly testing 1.1.1 OTS setup 1.1.2 Test automation images 1.2 Applications for testing 1.3 Test asset 1.4 Test Execution Schedule 1.5 Crashdb support for ARM core dumps 1.6 6. QA Tasks For Developer Edition 1.6.1 Sanity Test Set 1.6.2 Feature Test Set

QA TODOs (in priority order)

OTS setup and automated hourly testing

OTS setup

• OTS server - DONE
• OTS worker(s) for core tests - DONE (2 workers up)
• OTS worker for UX tests
• Power consumption measurements
• Reporting to QA-reports
• Minimize automatic installation time

Test automation images

• Setup hourly image building for autotest image on own setup (maybe later with BOSS)
• We need to be able to control included test packages

If you need something else from QA please tell it to us :)

Applications for testing

• Package QML demo/example applications from http://doc.qt.nokia.com/latest/qdeclarativeexamples.html

Test asset

• Define what the asset includes
• Automating as many core/ux tests as possible
• Call/SMS cases will require parallel test execution support (If we want to automate those kind of tests)

Test Execution Schedule

• Core

• Handset UX weekly testing schedule

Day	Test set (status)	Release
Monday	Key feature (OK)	Preview
Monday	Acceptance (OK)	Testing trunk
Tuesday	Acceptance (Ok)	Testing trunk
Tuesday	Sanity (Ok)	Daily trunk
Tuesday	DE Dataflow (Ok)	Preview
Tuesday	DE use cases (Ok)	Preview
Wednesday	Acceptance (Ok)	Testing trunk
Wednesday	Key feature (Ok)	Weekly
Thursday	DE Dataflow (Ok)	Weekly
Thursday	DE use cases (Ok)	Weekly
Thursday	Acceptance (Ok)	Testing trunk
Thursday	Sanity Ok	Daily trunk
Thursday	DE Reliability (Ongoing)	Weekly
Thursday	DE Performance (Ongoing)	Weekly
Friday	Acceptance (Ok)	Testing trunk
Friday	Sanity (Ok)	Daily trunk

Crashdb support for ARM core dumps

Core dump processing and backtraces from crashing ARM processes.

6. QA Tasks For Developer Edition

There is a wiki article about the Developer Edition.

QA tasks for the Developer Edition differ from the usual N900 approach in that there are less features to be tested. There are currently 2 test sets for the Developer Edition, these are the Sanity Test Set and the Feature Test Set. They are described below.

Sanity Test Set

The sanity set should be run automatically on every image. As such it must meet the following requirements:

• 100% automated
• Testing only basic features

Feature Test Set

The feature set will be run periodically and will test the basic features as well as enablers for those features (e.g. PIM for phoning contacts). Performance will also be analysed, at the moment, this will include browser startup time and a CPU benchmark but this will be expanded later.

Suggestions are welcome.