Meego Wiki
Views

Quality/QA tools development

From MeeGo wiki
(Difference between revisions)
Jump to: navigation, search
m
Line 2: Line 2:
This page provides information on the development activities and practices of QA tools. The focus is on presenting things that are relevant for people interested in developing the tools. Developer-oriented content from the [[Quality/QA-tools|main page of QA tools]] will be moved here and the main page will become more focused on end-user needs.
This page provides information on the development activities and practices of QA tools. The focus is on presenting things that are relevant for people interested in developing the tools. Developer-oriented content from the [[Quality/QA-tools|main page of QA tools]] will be moved here and the main page will become more focused on end-user needs.
 +
 +
== Tools and Maintainers ==
 +
 +
Tool maintainers are selected based on developer experience with particular tool/package or seniority. Tool maintainers have been agreed [[Quality/QA-tools/Meetings|in the QA-tools meeting]] Tuesday September 7th 2010. Changes, if needed, are discussed also there.
 +
 +
In practice only tool maintainers will have commit and review right to particular repository - later several people may have rights to repository based on merit (as proposed by tool maintainer). Others must follow [http://meego.com/about/contribution-guidelines MeeGo contribution guidelines] to submit patches or personal clone + merge request approach.
 +
 +
The maintainer of the tree shall update the changelog.
 +
 +
{| border="1" cellspacing="0"
 +
|+
 +
|'''Tool (link to wiki page)''' ||'''Gitorious'''||'''Maintainer''' ||'''Substitute'''
 +
|-
 +
| [[Quality/QA-tools/Test-definition|test-definition]]||[http://meego.gitorious.org/meego-quality-assurance/test-definition Gitorious] || Sampo Saaristo || Timo Härkönen
 +
|-
 +
| [[Quality/QA-tools/Testrunner-lite|testrunner-lite]] ||[http://meego.gitorious.org/meego-quality-assurance/testrunner-lite Gitorious] || Sampo Saaristo || Kyösti Ranto
 +
|-
 +
| [[Quality/QA-tools/Testrunner|Testrunner]] || [http://meego.gitorious.org/meego-quality-assurance/testrunner Gitorious] || Kyösti Ranto || Timo Härkönen
 +
|-
 +
| [[Quality/QA-tools/Testplanner|Testplanner]] ||[http://meego.gitorious.org/meego-quality-assurance/testplanner Gitorious] ||  Kyösti Ranto || N/A
 +
|-
 +
| [[Quality/QA-tools/Eat|eat - enables automated testing]] ||[http://meego.gitorious.org/meego-quality-assurance/enables-automated-testing Gitorious] ||  Timo Härkönen || Timo Mäkimattila
 +
|-
 +
| [[Quality/QA-tools/OTS|ots - open testing system]] ||[http://meego.gitorious.org/meego-quality-assurance/ots Gitorious] ||  Teemu Vainio || Tom Galvin
 +
|-
 +
| [[Quality/QA-tools/Autotest-guide#Automatic_image_installations|MeeGo Automated installer]] ||[http://gitorious.org/qa-tools/meego-ai Gitorious] ||  Timo Härkönen || N/A
 +
|-
 +
| [[Quality/TestSuite/MCTS|MeeGo Core Test Suite]] || [http://meego.gitorious.org/meego-quality-assurance/mcts/ Gitorious] ||Matti Salmi || Jeff Zheng
 +
|-
 +
| [[Quality/Netbook Test Suite and Utilities|MeeGo Netbook Test Suite]] || [http://gitorious.org/qa-tools/mnts/ Gitorious] ||Jeff Zheng || N/A
 +
|-
 +
| [[Quality/QA-tools/qtuitest-mbt-adapter|Model-Based Testing adapter for qtuitest]]  || [http://gitorious.org/qa-tools/qtuitest-mbt-adapter Gitorious] || Riku Halonen || N/A
 +
|-
 +
| [[Quality/QA-tools/Min|MIN test framework]] || [http://meego.gitorious.org/meego-quality-assurance/min Gitorious] ||  Sampo Saaristo || Timo Mäkimattila
 +
|-
 +
| [[Quality/QA-tools/TDriver|Testability Driver]] || [http://gitorious.org/TDriver Gitorious] ||  Petri Kiiskinen || Tatu Lahtela
 +
|-
 +
| Rich Core dumper || [http://meego.gitorious.org/meego-quality-assurance/rich-core Gitorious] || Carol Rus || Raimo Gratseff
 +
|-
 +
| Crash Reporter || [http://meego.gitorious.org/meego-quality-assurance/crash-reporter Gitorious] || Carol Rus || Raimo Gratseff
 +
|-
 +
| Crash Reporter settings || [http://meego.gitorious.org/meego-quality-assurance/crash-reporter-settings-public Gitorious] || Carol Rus || Raimo Gratseff
 +
|-
 +
| [[Quality/QA-tools/hat-control|Hardware Accessory for Testing (HAT)]] || [http://meego.gitorious.org/meego-quality-assurance/hat-control Gitorious] || Marko Junttila || Riku Halonen
 +
|-
 +
| [[Quality/QA-tools/QAReports|QA Reports]] || [http://meego.gitorious.org/meego-quality-assurance/qa-reports Gitorious] || Sami Hangaslammi || Jarno Keskikangas
 +
|-
 +
| [[Quality/QA-tools/Scripts_and_utils|Scripts and utils]] || [http://meego.gitorious.com/meego-quality-assurance/scripts-and-utils Gitorious] || N/A || N/A
 +
|-
 +
| [[Quality/TestSuite/handset-test-suite/handset-ux-test_Releases | handset_ux_tests]] || [http://meego.gitorious.com/meego-quality-assurance/handset-ux-tests Gitorious] || JessicaJi || N/A
 +
|-
 +
| [[Quality/QA-tools/MeeGo_Fast_Feedback_Testing|MeeGo Fast Feedback Testing (MeeGo-FFT)]] || [http://meego.gitorious.org/meego-quality-assurance/meego-testing-hudson-plugin Gitorious] || Alexey Kuznetsov || Timo Härkönen
 +
|-
 +
| Service OS based Flasher || [http://meego.gitorious.org/meego-quality-assurance/meego-ai-serviceos Gitorious] || Jing Wang || N/A
 +
|-
 +
| Qpid C wrapper library - libcqpid || [http://meego.gitorious.org/meego-quality-assurance/libcqpid Gitorious] || Sami Lahtinen || N/A
 +
|}
 +
 +
If you are interested in the user experience work regarding these tools, you can find more information [[Quality/QA-tools/User experience|here]].
 +
 +
=== Overview ===
 +
 +
The figure below tries to summarize the relations and tasks of the tools when used in test automation context.
 +
 +
[[File:testautomationtools.png]]
 +
 +
xfig file:[[File:qatools.fig]]
 +
 +
== Release Practices ==
 +
 +
Here is the workflow for QA tools release practices.
 +
 +
{| border="1" cellspacing="0"
 +
|+
 +
|'''Role'''
 +
|'''Description'''
 +
|-
 +
| Developer || Anyone who wants to participate in qa-tools development
 +
|-
 +
| VCS Maintainer || Component owner who has commit rights in version control system (VCS)
 +
|-
 +
| Package Maintainer || Integrator whose responsibility is the OBS packaging
 +
|-
 +
| Release Management || Third party who is responsible of trunk:testing releases(?)
 +
|}
 +
 +
[[File:Release.png]]
 +
 +
Kivio file: [[File:Release.flw]]
 +
 +
# Developer creates merge request(s) in gitorious.
 +
# VCS Maintainer tests and accepts merge requests.
 +
# VCS Maintainer checks/updates change logs.
 +
# VCS Maintainer tags a version.
 +
# VCS maintainer sends email to [http://lists.meego.com/pipermail/meego-qa/ meego-qa mailing list] based on the following template
 +
Topic: Integration request: package-name version
 +
PACKAGE: package-name
 +
TAG: tag name
 +
URL: link to sources
 +
CHANGES: short description of changes containing bugs.meego.com bug numbers of fixed bugs
 +
# Package maintainer updates the OBS package.
 +
# Package maintainer tests the OBS package.
 +
# If the package belongs to tools:testing and passes testing, Package Maintainer may accept it. If the package belongs to trunk:testing, Package Maintainer creates a promotional request to Release Management. (If the package belongs to both repositories, we let the Release Management set bugs fixed by the package to RELEASED state).
 +
# Host side tools are updated to tools:testing after verifying functionality
 +
# Package maintainer replies to meego-qa list about the actions done with the updated package. e.g. 'Updated in devel:quality and sent promotion request to testing'
 +
# Release Management accepts the package. Or not. (Follow meego-packaging and meego-commits.)
 +
 +
''' YouTube videos '''
 +
 +
YouTube is a good way to communicate new features. You can find existing demo videos on [http://www.youtube.com/user/meegoqatools meegoqatools channel on Youtube].
 +
 +
If you shoot a video to YouTube, promote it on meego-qa mailing list!
 +
 +
You can find some hints how to shoot, edit, and upload a video here: [[YouTube_Hints]]
 +
 +
=== Release checklist ===
 +
To make sure fixes are released without delay, check that the following conditions are met
 +
 +
# Change logs are updated and contain relevant references to MeeGo bugzilla
 +
# Created obs request include fixes bug numbers from MeeGo bugzilla
 +
# Bugzilla items listed in changes are set as resolved
 +
# Spec file matches [[Packaging/Guidelines|MeeGo packaging guidelines]]
 +
# Rpmlint warnings are either fixed or explained by comments in the spec file. e.g. eat packages install files into root's home and the reasoning for it needs to be explained
 +
# Host side tool packages use the same source tar ball to produce debian and rpm packages
 +
 +
== Design/ Planning ==
 +
 +
* [[Quality/QA-tools/MCTS test automation design|MCTS Test Automation]] (QA Tools support for MCTS)

Revision as of 14:07, 15 February 2011

Contents

QA tools development

This page provides information on the development activities and practices of QA tools. The focus is on presenting things that are relevant for people interested in developing the tools. Developer-oriented content from the main page of QA tools will be moved here and the main page will become more focused on end-user needs.

Tools and Maintainers

Tool maintainers are selected based on developer experience with particular tool/package or seniority. Tool maintainers have been agreed in the QA-tools meeting Tuesday September 7th 2010. Changes, if needed, are discussed also there.

In practice only tool maintainers will have commit and review right to particular repository - later several people may have rights to repository based on merit (as proposed by tool maintainer). Others must follow MeeGo contribution guidelines to submit patches or personal clone + merge request approach.

The maintainer of the tree shall update the changelog.

Tool (link to wiki page) GitoriousMaintainer Substitute
test-definitionGitorious Sampo Saaristo Timo Härkönen
testrunner-lite Gitorious Sampo Saaristo Kyösti Ranto
Testrunner Gitorious Kyösti Ranto Timo Härkönen
Testplanner Gitorious Kyösti Ranto N/A
eat - enables automated testing Gitorious Timo Härkönen Timo Mäkimattila
ots - open testing system Gitorious Teemu Vainio Tom Galvin
MeeGo Automated installer Gitorious Timo Härkönen N/A
MeeGo Core Test Suite Gitorious Matti Salmi Jeff Zheng
MeeGo Netbook Test Suite Gitorious Jeff Zheng N/A
Model-Based Testing adapter for qtuitest Gitorious Riku Halonen N/A
MIN test framework Gitorious Sampo Saaristo Timo Mäkimattila
Testability Driver Gitorious Petri Kiiskinen Tatu Lahtela
Rich Core dumper Gitorious Carol Rus Raimo Gratseff
Crash Reporter Gitorious Carol Rus Raimo Gratseff
Crash Reporter settings Gitorious Carol Rus Raimo Gratseff
Hardware Accessory for Testing (HAT) Gitorious Marko Junttila Riku Halonen
QA Reports Gitorious Sami Hangaslammi Jarno Keskikangas
Scripts and utils Gitorious N/A N/A
handset_ux_tests Gitorious JessicaJi N/A
MeeGo Fast Feedback Testing (MeeGo-FFT) Gitorious Alexey Kuznetsov Timo Härkönen
Service OS based Flasher Gitorious Jing Wang N/A
Qpid C wrapper library - libcqpid Gitorious Sami Lahtinen N/A

If you are interested in the user experience work regarding these tools, you can find more information here.

Overview

The figure below tries to summarize the relations and tasks of the tools when used in test automation context.

Testautomationtools.png

xfig file:File:Qatools.fig

Release Practices

Here is the workflow for QA tools release practices.

Role Description
Developer Anyone who wants to participate in qa-tools development
VCS Maintainer Component owner who has commit rights in version control system (VCS)
Package Maintainer Integrator whose responsibility is the OBS packaging
Release Management Third party who is responsible of trunk:testing releases(?)

Release.png

Kivio file: File:Release.flw

  1. Developer creates merge request(s) in gitorious.
  2. VCS Maintainer tests and accepts merge requests.
  3. VCS Maintainer checks/updates change logs.
  4. VCS Maintainer tags a version.
  5. VCS maintainer sends email to meego-qa mailing list based on the following template

Topic: Integration request: package-name version 

PACKAGE: package-name
TAG: tag name
URL: link to sources
CHANGES: short description of changes containing bugs.meego.com bug numbers of fixed bugs
  1. Package maintainer updates the OBS package.
  2. Package maintainer tests the OBS package.
  3. If the package belongs to tools:testing and passes testing, Package Maintainer may accept it. If the package belongs to trunk:testing, Package Maintainer creates a promotional request to Release Management. (If the package belongs to both repositories, we let the Release Management set bugs fixed by the package to RELEASED state).
  4. Host side tools are updated to tools:testing after verifying functionality
  5. Package maintainer replies to meego-qa list about the actions done with the updated package. e.g. 'Updated in devel:quality and sent promotion request to testing'
  6. Release Management accepts the package. Or not. (Follow meego-packaging and meego-commits.)

YouTube videos

YouTube is a good way to communicate new features. You can find existing demo videos on meegoqatools channel on Youtube.

If you shoot a video to YouTube, promote it on meego-qa mailing list!

You can find some hints how to shoot, edit, and upload a video here: YouTube_Hints

Release checklist

To make sure fixes are released without delay, check that the following conditions are met

  1. Change logs are updated and contain relevant references to MeeGo bugzilla
  2. Created obs request include fixes bug numbers from MeeGo bugzilla
  3. Bugzilla items listed in changes are set as resolved
  4. Spec file matches MeeGo packaging guidelines
  5. Rpmlint warnings are either fixed or explained by comments in the spec file. e.g. eat packages install files into root's home and the reasoning for it needs to be explained
  6. Host side tool packages use the same source tar ball to produce debian and rpm packages

Design/ Planning

Retrieved from "http://wiki.meego.com/Quality/QA_tools_development"
Personal tools