Revision as of 15:14, 27 January 2011 (view source) Asinnela (Talk | contribs) m (→Roadmap: Fixing wiki links) ← Older edit		Revision as of 07:39, 28 January 2011 (view source) Slahtinen (Talk | contribs) (Updated roadmap item for parallel testing) Newer edit →
Line 155:		Line 155:
	| Test environment validation || TBD		| Test environment validation || TBD
	|-		|-
-	| Parallel testing || TBD	+	| [http://bugs.meego.com/showdependencytree.cgi?id=12454&hide_resolved=0 Parallel testing] || TBD
	|-		|-
	| [https://bugs.meego.com/show_bug.cgi?id=12810 Easy install] || TBD		| [https://bugs.meego.com/show_bug.cgi?id=12810 Easy install] || TBD

---

Revision as of 07:39, 28 January 2011

Contents 1 Quality Assurance (QA) Tools 1.1 Tools and Maintainers 1.1.1 Overview 1.2 Release Practices 1.2.1 Release checklist 1.3 Roadmap 1.4 Features and Bugs 1.5 Documentation 1.6 Design/ Planning 1.7 Meetings 1.8 Team Members and Collaboration Spaces

Quality Assurance (QA) Tools

Quality Assurance tools are developed to ensure MeeGo SW quality. QA tools team develops and maintains tools for Quality Assurance.

Open source tools  – available for all, available for development and contributions. Make people accountable for quality.

Anyone is welcome to contribute and non-member contributions will be treated with same process and review as member contributions. We follow MeeGo contribution guidelines. In addition, you may take personal clone from our git repositories and create merge request. Tool maintainers in our projects will review your contributions and decide on merge.

Targets:

• Improve MeeGo test reporting tools - target from MeeGo Quality Assurance
• Improve MeeGo test automation (execution and software installation) - target from MeeGo release engineering

"As Core OS release release manager I want to verify trunk:testing packages frequently so that I know the quality of nightly/weekly releases."

Tools and Maintainers

Tool maintainers are selected based on developer experience with particular tool/package or seniority. Tool maintainers have been agreed in the QA-tools meeting Tuesday September 7th 2010. Changes, if needed, are discussed also there.

In practice only tool maintainers will have commit and review right to particular repository - later several people may have rights to repository based on merit (as proposed by tool maintainer). Others must follow MeeGo contribution guidelines to submit patches or personal clone + merge request approach.

The maintainer of the tree shall update the changelog.

Tool (link to wiki page)	Gitorious	Maintainer	Substitute
test-definition	Gitorious	Sampo Saaristo	Timo Härkönen
testrunner-lite	Gitorious	Sampo Saaristo	Kyösti Ranto
Testrunner	Gitorious	Kyösti Ranto	Timo Härkönen
Testplanner	Gitorious	Kyösti Ranto	N/A
eat - enables automated testing	Gitorious	Timo Härkönen	Timo Mäkimattila
ots - open testing system	Gitorious	Teemu Vainio	Tom Galvin
MeeGo Automated installer	Gitorious	Timo Härkönen	N/A
MeeGo Core Test Suite	Gitorious	Matti Salmi	Jeff Zheng
MeeGo Netbook Test Suite	Gitorious	Jeff Zheng	N/A
Model-Based Testing adapter for qtuitest	Gitorious	Riku Halonen	N/A
MIN test framework	Gitorious	Sampo Saaristo	Timo Mäkimattila
Testability Driver	Gitorious	Petri Kiiskinen	Tatu Lahtela
Rich Core dumper	Gitorious	Carol Rus	Raimo Gratseff
Crash Reporter	Gitorious	Carol Rus	Raimo Gratseff
Crash Reporter settings	Gitorious	Carol Rus	Raimo Gratseff
Hardware Accessory for Testing (HAT)	Gitorious	Marko Junttila	Riku Halonen
QA Reports	Gitorious	Sami Hangaslammi	Jarno Keskikangas
Scripts and utils	Gitorious	N/A	N/A
handset_ux_tests	Gitorious	JessicaJi	N/A
MeeGo Fast Feedback Testing (MeeGo-FFT)	Gitorious	Alexey Kuznetsov	Timo Härkönen
Service OS based Flasher	Gitorious	Jing Wang	N/A
Qpid C wrapper library - libcqpid	Gitorious	Sami Lahtinen	N/A

You can install Testrunner, testrunner-lite, Testplanner, OTS, Meego-ai and eat from Tools:Testing repository. The instructions for setting up the repositories can be found here.

See the rest of our team members and our collaboration spaces below. If you are interested in the user experience work regarding these tools, you can find more information here.

Overview

The figure below tries to summarize the relations and tasks of the tools when used in test automation context.

xfig file:File:Qatools.fig

Release Practices

Here is the workflow for QA tools release practices.

Role	Description
Developer	Anyone who wants to participate in qa-tools development
VCS Maintainer	Component owner who has commit rights in version control system (VCS)
Package Maintainer	Integrator whose responsibility is the OBS packaging
Release Management	Third party who is responsible of trunk:testing releases(?)

Kivio file: File:Release.flw

1. Developer creates merge request(s) in gitorious.
2. VCS Maintainer tests and accepts merge requests.
3. VCS Maintainer checks/updates change logs.
4. VCS Maintainer tags a version.
5. VCS maintainer sends email to meego-qa mailing list based on the following template

Topic: Integration request: package-name version

PACKAGE: package-name
TAG: tag name
URL: link to sources
CHANGES: short description of changes containing bugs.meego.com bug numbers of fixed bugs

1. Package maintainer updates the OBS package.
2. Package maintainer tests the OBS package.
3. If the package belongs to tools:testing and passes testing, Package Maintainer may accept it. If the package belongs to trunk:testing, Package Maintainer creates a promotional request to Release Management. (If the package belongs to both repositories, we let the Release Management set bugs fixed by the package to RELEASED state).
4. Host side tools are updated to tools:testing after verifying functionality
5. Package maintainer replies to meego-qa list about the actions done with the updated package. e.g. 'Updated in devel:quality and sent promotion request to testing'
6. Release Management accepts the package. Or not. (Follow meego-packaging and meego-commits.)

YouTube videos

YouTube is a good way to communicate new features. You can find existing demo videos on meegoqatools channel on Youtube.

If you shoot a video to YouTube, promote it on meego-qa mailing list!

You can find some hints how to shoot, edit, and upload a video here: YouTube_Hints

Release checklist

To make sure fixes are released without delay, check that the following conditions are met

1. Change logs are updated and contain relevant references to MeeGo bugzilla
2. Created obs request include fixes bug numbers from MeeGo bugzilla
3. Bugzilla items listed in changes are set as resolved
4. Spec file matches MeeGo packaging guidelines
5. Rpmlint warnings are either fixed or explained by comments in the spec file. e.g. eat packages install files into root's home and the reasoning for it needs to be explained
6. Host side tool packages use the same source tar ball to produce debian and rpm packages

Roadmap

(Only partial, will be updated asap as we file rest of the bugs -timakima)

The release dates defined in MeeGo Release plans 1.1 and 1.2. These dates are the latest estimation. They will be updated as work progresses.

Feature	Release
Measurement support	1.2
MCTS coverage support	TBD
Test environment validation	TBD
Parallel testing	TBD
Easy install	TBD
Qt Creator integration	TBD
Events feature in automatic testing	TBD

Features and Bugs

Want to report an feature idea or bug to us? - Please do it here

• Assigned bugs and features - Working on it
• All open features and bugs in priority order

Bugzilla workflow: Bugzilla/how-report-bugs

We'd like to have the best guess of the moment about the delivery of features. Remember to set the target build of the bug you're working with according to: Release Engineering/Plans/1.1 and Release Engineering/Plans/1.2

When you add a new bug, add correct dependencies to the corresponding roadmap meta bug.

Documentation

This section will contain links to various guides and user documentation. See the wiki pages of the tools for tool-specific documentation.

• Autotest-Guide
• How to set up the repositories that are needed to install QA tools

Design/ Planning

• MCTS Test Automation (QA Tools support for MCTS)

Meetings

All meetings will be held in #meego-meeting on irc.freenode.net.

• Team meetings
  • QA tools team meetings will be held on need basis for specific topics. This was agreed in the last weekly meeting held on December 7th 2010
  • Meeting agendas and minutes

• Architecture meetings
  • Architecture meetings will be held on-demand basis.
  • Meeting agendas and minutes

• Workshops
  • Face-2-face meetings within the team.
  • Meeting agendas and minutes

Team Members and Collaboration Spaces

The current team members are (in no particular order):

Name	Role	Affiliation	IRC nickname
Ville Ilvonen	Team lead (act.)	Nokia	vilvo
Riku Halonen	Team member	Nokia	rikhalon
Kari Sievi	Team member	Digia	sievi
Timo Härkönen	Team member	Digia	timoph
Carol Rus	Team member	Digia	carrus
Sami Lahtinen	Team member	Digia	slahtinen
Raimo Gratseff	Team member	Digia	rrraimo
Kyösti Ranto	Team member	Digia	kyranto
Arto Sinnelä	Team member	Digia	asinnela
Joonas Kylänpää	Team member	Digia	Kaadlajk
Timo Mäkimattila	Team member	Digia	timakima
Elias Luttinen	Team member	Digia	eluttine
Ville Niutanen	Team member	Digia	Villen
Esa-Pekka Miettinen	Team member	Digia	E-P
Vesa Poikajärvi	Team member	Digia	vesse
Alexey Kuznetsov	Team member	Digia	alkuznet
Sergey Timofeev	Team member	Digia	setimofe
Daniil Chuiko	Team member	Digia	dachuiko
Jarmo Savinen	Team member	Digia	jasavi
Sampo Saaristo	Team member	Sofica	sampos
Ling Yu	Team member	Intel	-
Jing Wang	Team member	Intel	-
Teemu Vainio	Team member	Ixonos	tvainio
Tuomo Mäkinen	Team member	Ixonos	-
Jouni Leppäkases	Team member	Ixonos	jouni
Tom Galvin	Team member	Ixonos	-
Jarno Keskikangas	Team member	Leonidas	jakeskik
Janne Hietamäki	Team member	Leonidas	_janne
Sami Hangaslammi	Team member	Leonidas	sahangas

Team communication is in English. Our collaboration spaces are:

• meego-qa@lists.meego.com mailing list
• meego-dev@meego.com mailing list, please prefix with 'QA-tools' for team related topics.
  • Please also poke team members or Ville Ilvonen either by email or on IRC because of high traffic @ meego-dev
• #meego-qa-tools IRC channel on irc.freenode.net
  • #meego-qa-tools statistics
  • #meego-qa-tools irc logs
• Gitorious team, http://meego.gitorious.org/meego-quality-assurance/
• Youtube channel for demo videos
• QA-tools team blog
• MeeGo OBS - devel:quality
• This wiki area
• ServiceOS
• PXEInstallation