Revision as of 05:01, 8 April 2011 (view source) Cindy (Talk | contribs) (→QA TODOs (in priority order)) ← Older edit		Revision as of 07:18, 11 April 2011 (view source) Rha (Talk | contribs) (→Crashdb support for ARM core dumps) Newer edit →
Line 39:		Line 39:
	# Sensor data cases (Qt Mobility, lower priority)		# Sensor data cases (Qt Mobility, lower priority)
-	== Crashdb support for ARM core dumps ==	+	== Crash analysis support for ARM core dumps ==
	Core dump processing and backtraces from crashing ARM processes.		Core dump processing and backtraces from crashing ARM processes.
Line 54:		Line 54:
	* Back-end server		* Back-end server
-	** Set up back-end server for core processing - ONGOING (rikhalon)	+	** Set up back-end server for core processing - ONGOING (rikhalon, sampos)
		+
		+	* Script to produce backtraces locally
		+	** Extend rich-core-extract to proceduce backtrace from rich-core file - NOT STARTED
		+	** Write howto guide in wiki - NOT STARTED
	== Boot time measurement ==		== Boot time measurement ==

---

Revision as of 07:18, 11 April 2011

Contents 1 QA TODOs (in priority order) 1.1 MeeGo.com IRC meetings 1.2 OTS setup and automated hourly testing 1.2.1 OTS setup 1.2.2 Test automation images 1.3 Applications for testing 1.4 Test asset 1.5 Crash analysis support for ARM core dumps 1.6 Boot time measurement 1.7 CPU load measurement during audio/video playback 1.8 Application Manager 1.9 Test Execution Schedule 1.10 QA Tasks For Developer Edition 1.10.1 Test Sets 1.10.1.1 Sanity Test Set 1.10.1.2 Feature Test Set 1.10.2 Core QA Team Backlog 1.11 QA Organization 1.12 Error Management 1.13 MeeGo.com N900 QA Tasks 1.13.1 Test execution schedule 1.14 Usefull links

QA TODOs (in priority order)

MeeGo.com IRC meetings

There are no special QA meeting, other DE team meetings can be found from here:

• MeeGo-Meeting IRC Schedule

OTS setup and automated hourly testing

OTS setup

• OTS server - Ville Ilvonen/Riku Halonen, DONE
• OTS worker(s) for core tests- Ville Ilvonen/Riku Halonen/Timo Harkonen - DONE
• Reporting of hourly tests to QA-reports - Ville Ilvonen/Esa-Pekka Miettinen/Timo Harkonen - DONE
• Minimize automatic installation time to 10mins - Timo Makimattila, ONGOING
• OTS worker for UX tests - NOTSTARTED
• Power consumption measurements - NOTSTARTED

Test automation images

• Setup hourly image building for autotest image on own setup - Riku Halonen/Timo Harkonen/Ville Ilvonen - DONE (in OTS subnet, see above)

• We need to be able to control included test packages - ONGOING (http://meego.gitorious.org/meego-quality-assurance/handset-hourly-automated-tests)

• Move to use images from release engineering (1. download image, 2. install automation enablers, core dumping enablers etc. using mic-chroot, 3. install image to device 4. test) - NOT STARTED

If you need something else from QA please tell it to us :)

Applications for testing

• qt-demos already available from the repos
• Small applications that use Qt mobility APIs to access things like sensors to help manual testing

Test asset

• List of open bugs for MCTS
  • see priorities below - discuss with Iekku about priorities of open bugs for MCTS tests

1. WLAN cases
2. Call/SMS cases
3. Audio policy framework cases (lower priority)
4. Camera cases (lower priority)
5. Sensor data cases (Qt Mobility, lower priority)

Crash analysis support for ARM core dumps

Core dump processing and backtraces from crashing ARM processes.

• Rich Core dumping
  • Make rich-core dumping to work in MeeGo N900 DE (basic functionality) - DONE (sampos, rikhalon)
    • Changes in MeeGo Gitorious (meego-n900de branch).
    • Dumps are generated in /home/meego/core-dumps
    • In file name, string "xxxx" is used instead of IMEI digits (privacy issue)
    • Get latest packages here
    • Add "-corewatcher" and "-corewatcher-applet" to .ks file to remove overlapping corewatcher.
  • Fix core-reducer (Something goes wrong, when reducer processes coredump.) - DONE (sampos)
  • Oopslog (and lifelog) functionality - NOT STARTED

• Back-end server
  • Set up back-end server for core processing - ONGOING (rikhalon, sampos)

• Script to produce backtraces locally
  • Extend rich-core-extract to proceduce backtrace from rich-core file - NOT STARTED
  • Write howto guide in wiki - NOT STARTED

Boot time measurement

• Measure and optimize N900 boot time (timakima, ONGOING)

CPU load measurement during audio/video playback

Application Manager

• Implement an application manager (similar to one in N900/Fremantle) to control install/uninstall/update applications and other packages. (kyranto, ONGOING).

Test Execution Schedule

• Core (Teivas)
• Handset UX weekly testing schedule (Rajala)

Day	Test set (status)	Release
Monday	Key feature (OK)	Preview
Monday	Acceptance (OK)	Testing trunk
Tuesday	Acceptance (Ok)	Testing trunk
Tuesday	Sanity (Ok)	Daily trunk
Tuesday	DE Dataflow (Ok)	Preview
Tuesday	DE use cases (Ok)	Preview
Wednesday	Acceptance (Ok)	Testing trunk
Wednesday	Key feature (Ok)	Weekly
Thursday	DE Dataflow (Ok)	Weekly
Thursday	DE use cases (Ok)	Weekly
Thursday	Acceptance (Ok)	Testing trunk
Thursday	Sanity Ok	Daily trunk
Thursday	DE Reliability (Ongoing)	Weekly
Thursday	DE Performance (Ongoing)	Weekly
Friday	Acceptance (Ok)	Testing trunk
Friday	Sanity (Ok)	Daily trunk

QA Tasks For Developer Edition

There is a wiki article about the Developer Edition.

QA tasks for the Developer Edition differ from the usual N900 approach in that there are less features to be tested. There are currently 2 test sets for the Developer Edition, these are the Sanity Test Set and the Feature Test Set. They are described below.

Test Sets

Sanity Test Set

The sanity set should be run automatically on every image. As such it must meet the following requirements:

• 100% automated
• Testing only basic features

Feature Test Set

The feature set will be run periodically and will test the basic features as well as enablers for those features (e.g. PIM for phoning contacts). Performance will also be analysed, at the moment, this will include browser startup time and a CPU benchmark but this will be expanded later.

Suggestions are welcome.

Core QA Team Backlog

The Core QA team intends to keep its backlog as public as possible. The limitation on this is the amount of work generated by doing this. We will attempt to keep an up-to-date lists of tasks and progress on these tasks. At the moment, the task list looks like this:

• Create weekly schedule for MRT (currently in draft form)
• Wiki clean/update
• Hourly Automation improvement
• Plan Feature testset
• Bug verification

QA Organization

Error Management

• Error Manager Iekku Huttunen

MeeGo.com N900 QA Tasks

Test execution schedule

• MeeGo.com N900 Core weekly test schedule (Tu Qingqing)

Day	Test set (status)	Release	Priority
Monday	Acceptance (OK)	Trunk:Testing	P1
Monday	Basic feature test	Preview	P2
Monday	Sanity (OK)	Trunk	P3
Tuesday	Acceptance (OK)	Trunk:Testing	P1
Tuesday	Sanity (OK)	Trunk	P2
Wednesday	Acceptance (OK)	Trunk:Testing	P1
Wednesday	Data Flow	Weekly	P2
Wednesday	Sanity (OK)	Trunk	P3
Thursday	Acceptance (OK)	Trunk:Testing	P1
Thursday	Sanity (OK)	Trunk	P2
Friday	Acceptance (OK)	Trunk:Testing	P1
Friday	Sanity (OK)	Trunk	P2

Usefull links

• Flashing instructions

Flashing tested with http://download.meego.com/testing-daily/builds/trunk/1.1.90.8.20110318.89/