(→Documentation) |
m (→Tools and Maintainers) |
||
| (33 intermediate revisions not shown) | |||
| Line 24: | Line 24: | ||
|'''Tool (link to wiki page)''' ||'''Gitorious'''||'''Maintainer''' ||'''Substitute''' | |'''Tool (link to wiki page)''' ||'''Gitorious'''||'''Maintainer''' ||'''Substitute''' | ||
|- | |- | ||
| - | | [[Quality/QA-tools/Test-definition|test-definition]]||[http://meego.gitorious.org/meego-quality-assurance/test-definition Gitorious] || | + | | [[Quality/QA-tools/Test-definition|test-definition]]||[http://meego.gitorious.org/meego-quality-assurance/test-definition Gitorious] || Sami Lahtinen || Timo Härkönen |
|- | |- | ||
| - | | [[Quality/QA-tools/Testrunner-lite|testrunner-lite]] ||[http://meego.gitorious.org/meego-quality-assurance/testrunner-lite Gitorious] || | + | | [[Quality/QA-tools/Testrunner-lite|testrunner-lite]] ||[http://meego.gitorious.org/meego-quality-assurance/testrunner-lite Gitorious] || Sami Lahtinen || Kyösti Ranto |
|- | |- | ||
| [[Quality/QA-tools/Testrunner|Testrunner]] || [http://meego.gitorious.org/meego-quality-assurance/testrunner Gitorious] || Kyösti Ranto || Timo Härkönen | | [[Quality/QA-tools/Testrunner|Testrunner]] || [http://meego.gitorious.org/meego-quality-assurance/testrunner Gitorious] || Kyösti Ranto || Timo Härkönen | ||
| Line 34: | Line 34: | ||
| [[Quality/QA-tools/Eat|eat - enables automated testing]] ||[http://meego.gitorious.org/meego-quality-assurance/enables-automated-testing Gitorious] || Timo Härkönen || Timo Mäkimattila | | [[Quality/QA-tools/Eat|eat - enables automated testing]] ||[http://meego.gitorious.org/meego-quality-assurance/enables-automated-testing Gitorious] || Timo Härkönen || Timo Mäkimattila | ||
|- | |- | ||
| - | | [[Quality/QA-tools/OTS| | + | | [[Quality/QA-tools/OTS|OTS - Open Test System]] ||[http://meego.gitorious.org/meego-quality-assurance/ots Gitorious] || Elias Luttinen || Ville Niutanen |
|- | |- | ||
| [[Quality/QA-tools/Autotest-guide#Automatic_image_installations|MeeGo Automated installer]] ||[http://gitorious.org/qa-tools/meego-ai Gitorious] || Timo Härkönen || N/A | | [[Quality/QA-tools/Autotest-guide#Automatic_image_installations|MeeGo Automated installer]] ||[http://gitorious.org/qa-tools/meego-ai Gitorious] || Timo Härkönen || N/A | ||
| Line 44: | Line 44: | ||
| [[Quality/QA-tools/qtuitest-mbt-adapter|Model-Based Testing adapter for qtuitest]] || [http://gitorious.org/qa-tools/qtuitest-mbt-adapter Gitorious] || Riku Halonen || N/A | | [[Quality/QA-tools/qtuitest-mbt-adapter|Model-Based Testing adapter for qtuitest]] || [http://gitorious.org/qa-tools/qtuitest-mbt-adapter Gitorious] || Riku Halonen || N/A | ||
|- | |- | ||
| - | | [ | + | | [http://min.sourceforge.net/ MIN test framework] || [http://meego.gitorious.org/meego-quality-assurance/min Gitorious] || Joonas Kylänpää || Timo Mäkimattila |
|- | |- | ||
| [[Quality/QA-tools/TDriver|Testability Driver]] || [http://gitorious.org/TDriver Gitorious] || Petri Kiiskinen || Tatu Lahtela | | [[Quality/QA-tools/TDriver|Testability Driver]] || [http://gitorious.org/TDriver Gitorious] || Petri Kiiskinen || Tatu Lahtela | ||
| Line 67: | Line 67: | ||
|- | |- | ||
| Qpid C wrapper library - libcqpid || [http://meego.gitorious.org/meego-quality-assurance/libcqpid Gitorious] || Sami Lahtinen || N/A | | Qpid C wrapper library - libcqpid || [http://meego.gitorious.org/meego-quality-assurance/libcqpid Gitorious] || Sami Lahtinen || N/A | ||
| + | |- | ||
| + | | [[Quality/QA-tools/QmlUITestlib|MeeGo QML GUI Testlib]] || [http://meego.gitorious.org/meego-quality-assurance/auto-ux-testlib Gitorious] || Shao-Feng Tang|| Helia Correia | ||
| + | |- | ||
| + | | [[Testkit_Howto|testkit-lite]] || [http://meego.gitorious.org/meego-quality-assurance/testkit-lite Gitorious] || Jing Wang|| N/A | ||
| + | |- | ||
| + | | [[Testkit_Howto|testkit-manager]] || [http://meego.gitorious.org/meego-quality-assurance/testkit-manager Gitorious] || Shao-Feng Tang|| N/A | ||
|} | |} | ||
| Line 137: | Line 143: | ||
== Roadmap == | == Roadmap == | ||
| - | |||
| - | + | These are the items that we're currently working on. | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
The features in the roadmap are followed with META FEA bugs. The features are split to small tool specific FEA:s that block the feature META bug. The META bug is then the last bug to be closed when the feature is finished. You can also follow the tool bug progression from the dependency trees of the META bugs. | The features in the roadmap are followed with META FEA bugs. The features are split to small tool specific FEA:s that block the feature META bug. The META bug is then the last bug to be closed when the feature is finished. You can also follow the tool bug progression from the dependency trees of the META bugs. | ||
| Line 155: | Line 156: | ||
{| border="1" cellspacing="0" | {| border="1" cellspacing="0" | ||
|+ | |+ | ||
| - | |'''Feature | + | |'''Feature''' |
|- | |- | ||
| - | | [https://bugs.meego.com/showdependencytree.cgi?id=12980&hide_resolved=0 Measurement support] | + | | [https://bugs.meego.com/showdependencytree.cgi?id=12980&hide_resolved=0 Measurement support] |
|- | |- | ||
| - | | [ | + | | [[Quality/QA-tools/OTS/Roadmap|OTS releases]] |
| + | |} | ||
| + | |||
| + | == Future development == | ||
| + | |||
| + | The focus is currently on [[ARM/N900/QA|N900 Developer Edition QA]]. Hence the N900 related list below doesn't have a comprehensive information about target builds or priorities. The items listed here are not currently worked on but are planned to be implemented in the long run. | ||
| + | |||
| + | A list of defined features: | ||
| + | |||
| + | {| border="1" cellspacing="0" | ||
| + | |+ | ||
| + | |'''Feature''' | ||
|- | |- | ||
| - | | [ | + | | [https://bugs.meego.com/show_bug.cgi?id=12810 Easy install] |
|- | |- | ||
| - | | [ | + | | [http://bugs.meego.com/showdependencytree.cgi?id=12454&hide_resolved=0 Parallel testing] |
|- | |- | ||
| - | | [https://bugs.meego.com/showdependencytree.cgi?id= | + | | [https://bugs.meego.com/showdependencytree.cgi?id=12981&hide_resolved=0 Events feature in automatic testing] |
|- | |- | ||
| - | | [ | + | | [https://bugs.meego.com/showdependencytree.cgi?id=12937&hide_resolved=0 MCTS coverage support] |
|- | |- | ||
| - | | [http://bugs.meego.com/showdependencytree.cgi?id= | + | | [http://bugs.meego.com/showdependencytree.cgi?id=12833&hide_resolved=0 Test environment validation] |
|- | |- | ||
| - | | [ | + | | [http://bugs.meego.com/showdependencytree.cgi?id=13092&hide_resolved=0 Test equipment control] |
|- | |- | ||
| - | | [https://bugs.meego.com/ | + | | [https://bugs.meego.com/showdependencytree.cgi?id=12971&hide_resolved=0 Qt Creator integration] |
|- | |- | ||
| - | | [ | + | | [https://bugs.meego.com/show_bug.cgi?id=13190 A tool that combines Testrunner and Testplanner] |
|} | |} | ||
| Line 184: | Line 196: | ||
== Documentation == | == Documentation == | ||
| - | Please make sure that all the relevant documentation regarding the tools is kept up-to-date whenever you update any of the tools. Every tool should have a wiki page that contains at least 1) Overview of the tool and its benefits 2) Installation instructions 3) Basic use tutorial 4) Contact information. | + | Please make sure that all the relevant documentation regarding the tools is kept up-to-date whenever you update any of the tools. Please consider the end users of the tool as the primary readers of the documentation and make the documentation task-oriented for end user needs. Separate developer documentation can exist as well. |
| + | |||
| + | '''Wiki guidelines''' | ||
| + | |||
| + | Every tool should have a wiki page that contains at least 1) Overview of the tool and its benefits 2) Installation instructions 3) Basic use tutorial 4) Contact information. | ||
| + | |||
| + | Make sure that you are familiar with [[Wiki contribution guidelines|MeeGo wiki contribution guidelines]] before creating new content. Currently, there are quite a lot problems relating to e.g. page names, wiki links, deep hierarchies and obsolete pages in our wiki section. | ||
| + | |||
| + | '''Documentation checklist''' | ||
| - | The following table lists the tool-specific documentation that should exist (marked with 'x'). These | + | The following table lists the tool-specific documentation that should exist (marked with 'x'). These should be checked frequently and more in depth in the future. Bugs will be filed for the missing or inaccurate documentation (marked as 'NOK' i.e. "Not OK"). |
{| border="1" cellspacing="0" | {| border="1" cellspacing="0" | ||
| Line 194: | Line 214: | ||
| test-definition || || || || x<br />Checked: asinnela 2011-03-03 | | test-definition || || || || x<br />Checked: asinnela 2011-03-03 | ||
|- | |- | ||
| - | | testrunner-lite || x || x<br />Checked: asinnela 2011-03-07 || x<br />Checked: asinnela 2011-03-07, NOK || x<br />Checked: asinnela 2011- | + | | testrunner-lite || x || x<br />Checked: asinnela 2011-03-07 || x<br />Checked: asinnela 2011-03-07, NOK<br />[http://bugs.meego.com/show_bug.cgi?id=14074 Bug 14074] || x<br />Checked: asinnela 2011-06-07 |
|- | |- | ||
| Testrunner || x || x<br />Checked: asinnela 2011-03-07 || || x<br />Checked: asinnela 2011-03-07 | | Testrunner || x || x<br />Checked: asinnela 2011-03-07 || || x<br />Checked: asinnela 2011-03-07 | ||
|- | |- | ||
| - | | Testplanner || x || x<br />Checked: asinnela 2011-03- | + | | Testplanner || x || x<br />Checked: asinnela 2011-03-10 || || x<br />Checked: asinnela 2011-03-07 |
|- | |- | ||
| - | | eat - enables automated testing || || || || x | + | | eat - enables automated testing || || || || x<br />Checked: asinnela 2011-03-08 |
|- | |- | ||
| - | | | + | | OTS - Open Test System || x || x || x || x<br />Checked: asinnela 2011-03-25 |
|- | |- | ||
| - | | MIN test framework || x || x || x || | + | | MIN test framework || x || x || x || |
|- | |- | ||
| - | | Scripts and utils || || || || x | + | | Scripts and utils || || || || x<br />Checked: asinnela 2011-03-11 |
|- | |- | ||
| - | | MeeGo Fast Feedback Testing || x || x || || x | + | | MeeGo Fast Feedback Testing || x || x || || x<br />Checked: asinnela 2011-03-11 |
|- | |- | ||
| - | | Qpid C wrapper library - libcqpid || x || || || x | + | | Qpid C wrapper library - libcqpid || x || || || x<br />Page created, content to be added. |
|} | |} | ||
| - | Other documentation | + | '''Other documentation''' |
| + | |||
* [[Quality/QA-tools/Autotest-guide|Autotest-Guide]] | * [[Quality/QA-tools/Autotest-guide|Autotest-Guide]] | ||
* [[Quality/QA-tools/How_to_set_up_repositories|How to set up the repositories that are needed to install QA tools]] | * [[Quality/QA-tools/How_to_set_up_repositories|How to set up the repositories that are needed to install QA tools]] | ||
| Line 222: | Line 243: | ||
* Team meetings | * Team meetings | ||
| - | ** QA tools | + | ** QA-tools weekly meetings are held on every Tuesday 08:00 UTC in #meego-meeting2 on irc.freenode.net (please note the "2") from May 3rd 2011 onwards. |
** [[Quality/QA-tools/Meetings|Meeting agendas and minutes]] | ** [[Quality/QA-tools/Meetings|Meeting agendas and minutes]] | ||
| Line 239: | Line 260: | ||
* [http://lists.meego.com/listinfo/meego-dev meego-dev@meego.com mailing list], please prefix with 'QA-tools' for team related topics. | * [http://lists.meego.com/listinfo/meego-dev meego-dev@meego.com mailing list], please prefix with 'QA-tools' for team related topics. | ||
** Please also poke team members or Ville Ilvonen either by email or on IRC because of high traffic @ meego-dev | ** Please also poke team members or Ville Ilvonen either by email or on IRC because of high traffic @ meego-dev | ||
| - | * [http://webchat.freenode.net/?channels=meego-qa | + | * [http://webchat.freenode.net/?channels=meego-qa #meego-qa IRC channel on irc.freenode.net] |
| - | ** [http://timoph.fi/qa-tools-stats/ #meego-qa-tools statistics] | + | ** [http://timoph.fi/qa-tools-stats/ Old #meego-qa-tools statistics] |
| - | ** [http://timoph.fi/qa-tools-logs/ #meego-qa-tools | + | ** [http://timoph.fi/qa-tools-logs-archive/ Archived #meego-qa-tools logs] |
* Gitorious team, http://meego.gitorious.org/meego-quality-assurance/ | * Gitorious team, http://meego.gitorious.org/meego-quality-assurance/ | ||
* [http://www.youtube.com/user/meegoqatools Youtube channel for demo videos] | * [http://www.youtube.com/user/meegoqatools Youtube channel for demo videos] | ||
| Line 261: | Line 282: | ||
| '''IRC nickname''' | | '''IRC nickname''' | ||
|- | |- | ||
| - | |||
| - | |||
| Riku Halonen || Team member || Nokia || rikhalon | | Riku Halonen || Team member || Nokia || rikhalon | ||
|- | |- | ||
| Line 276: | Line 295: | ||
|- | |- | ||
| Kyösti Ranto || Team member || Digia || kyranto | | Kyösti Ranto || Team member || Digia || kyranto | ||
| - | |||
| - | |||
|- | |- | ||
| Joonas Kylänpää || Team member || Digia || Kaadlajk | | Joonas Kylänpää || Team member || Digia || Kaadlajk | ||
| Line 286: | Line 303: | ||
|- | |- | ||
| Ville Niutanen || Team member || Digia || Villen | | Ville Niutanen || Team member || Digia || Villen | ||
| - | |||
| - | |||
|- | |- | ||
| Vesa Poikajärvi || Team member || Digia || vesse | | Vesa Poikajärvi || Team member || Digia || vesse | ||
|- | |- | ||
| Alexey Kuznetsov || Team member || Digia || alkuznet | | Alexey Kuznetsov || Team member || Digia || alkuznet | ||
| - | |||
| - | |||
|- | |- | ||
| Daniil Chuiko || Team member || Digia || dachuiko | | Daniil Chuiko || Team member || Digia || dachuiko | ||
| Line 304: | Line 317: | ||
|- | |- | ||
| Jing Wang || Team member || Intel || - | | Jing Wang || Team member || Intel || - | ||
| - | |||
| - | |||
|- | |- | ||
| Tuomo Mäkinen || Team member || Ixonos || - | | Tuomo Mäkinen || Team member || Ixonos || - | ||
|- | |- | ||
| Jouni Leppäkases || Team member || Ixonos || jouni | | Jouni Leppäkases || Team member || Ixonos || jouni | ||
| - | |||
| - | |||
|- | |- | ||
| Jarno Keskikangas || Team member || Leonidas || jakeskik | | Jarno Keskikangas || Team member || Leonidas || jakeskik | ||
Contents |
This page provides information on the development activities and practices of QA tools. The focus is on presenting things that are relevant for people interested in developing the tools. The main page for the end users is here.
Anyone is welcome to contribute and non-member contributions will be treated with same process and review as member contributions. We follow MeeGo contribution guidelines. In addition, you may take personal clone from our git repositories and create merge request. Tool maintainers in our projects will review your contributions and decide on merge.
Targets:
"As Core OS release release manager I want to verify trunk:testing packages frequently so that I know the quality of nightly/weekly releases."
Tool maintainers are selected based on developer experience with particular tool/package or seniority. Tool maintainers have been agreed in the QA-tools meeting Tuesday September 7th 2010. Changes, if needed, are discussed also there.
In practice only tool maintainers will have commit and review right to particular repository - later several people may have rights to repository based on merit (as proposed by tool maintainer). Others must follow MeeGo contribution guidelines to submit patches or personal clone + merge request approach.
The maintainer of the tree shall update the changelog.
| Tool (link to wiki page) | Gitorious | Maintainer | Substitute |
| test-definition | Gitorious | Sami Lahtinen | Timo Härkönen |
| testrunner-lite | Gitorious | Sami Lahtinen | Kyösti Ranto |
| Testrunner | Gitorious | Kyösti Ranto | Timo Härkönen |
| Testplanner | Gitorious | Kyösti Ranto | N/A |
| eat - enables automated testing | Gitorious | Timo Härkönen | Timo Mäkimattila |
| OTS - Open Test System | Gitorious | Elias Luttinen | Ville Niutanen |
| MeeGo Automated installer | Gitorious | Timo Härkönen | N/A |
| MeeGo Core Test Suite | Gitorious | Matti Salmi | Jeff Zheng |
| MeeGo Netbook Test Suite | Gitorious | Jeff Zheng | N/A |
| Model-Based Testing adapter for qtuitest | Gitorious | Riku Halonen | N/A |
| MIN test framework | Gitorious | Joonas Kylänpää | Timo Mäkimattila |
| Testability Driver | Gitorious | Petri Kiiskinen | Tatu Lahtela |
| Rich Core dumper | Gitorious | Carol Rus | Raimo Gratseff |
| Crash Reporter | Gitorious | Carol Rus | Raimo Gratseff |
| Crash Reporter settings | Gitorious | Carol Rus | Raimo Gratseff |
| Hardware Accessory for Testing (HAT) | Gitorious | Marko Junttila | Riku Halonen |
| QA Reports | Gitorious | Sami Hangaslammi | Jarno Keskikangas |
| Scripts and utils | Gitorious | N/A | N/A |
| handset_ux_tests | Gitorious | JessicaJi | N/A |
| MeeGo Fast Feedback Testing (MeeGo-FFT) | Gitorious | Alexey Kuznetsov | Timo Härkönen |
| Service OS based Flasher | Gitorious | Jing Wang | N/A |
| Qpid C wrapper library - libcqpid | Gitorious | Sami Lahtinen | N/A |
| MeeGo QML GUI Testlib | Gitorious | Shao-Feng Tang | Helia Correia |
| testkit-lite | Gitorious | Jing Wang | N/A |
| testkit-manager | Gitorious | Shao-Feng Tang | N/A |
If you are interested in the user experience work regarding these tools, you can find more information here.
The figure below tries to summarize the relations and tasks of the tools when used in test automation context.
xfig file:File:Qatools.fig
Here is the workflow for QA tools release practices.
| Role | Description |
| Developer | Anyone who wants to participate in qa-tools development |
| VCS Maintainer | Component owner who has commit rights in version control system (VCS) |
| Package Maintainer | Integrator whose responsibility is the OBS packaging |
| Release Management | Third party who is responsible of trunk:testing releases(?) |
Kivio file: File:Release.flw
Topic: Integration request: package-name version
PACKAGE: package-name TAG: tag name URL: link to sources CHANGES: short description of changes containing bugs.meego.com bug numbers of fixed bugs
YouTube videos
YouTube is a good way to communicate new features. You can find existing demo videos on meegoqatools channel on Youtube.
If you shoot a video to YouTube, promote it on meego-qa mailing list!
You can find some hints how to shoot, edit, and upload a video here: YouTube_Hints
To make sure fixes are released without delay, check that the following conditions are met
These are the items that we're currently working on.
The features in the roadmap are followed with META FEA bugs. The features are split to small tool specific FEA:s that block the feature META bug. The META bug is then the last bug to be closed when the feature is finished. You can also follow the tool bug progression from the dependency trees of the META bugs.
Adding a new feature to the roadmap:
| Feature |
| Measurement support |
| OTS releases |
The focus is currently on N900 Developer Edition QA. Hence the N900 related list below doesn't have a comprehensive information about target builds or priorities. The items listed here are not currently worked on but are planned to be implemented in the long run.
A list of defined features:
Please make sure that all the relevant documentation regarding the tools is kept up-to-date whenever you update any of the tools. Please consider the end users of the tool as the primary readers of the documentation and make the documentation task-oriented for end user needs. Separate developer documentation can exist as well.
Wiki guidelines
Every tool should have a wiki page that contains at least 1) Overview of the tool and its benefits 2) Installation instructions 3) Basic use tutorial 4) Contact information.
Make sure that you are familiar with MeeGo wiki contribution guidelines before creating new content. Currently, there are quite a lot problems relating to e.g. page names, wiki links, deep hierarchies and obsolete pages in our wiki section.
Documentation checklist
The following table lists the tool-specific documentation that should exist (marked with 'x'). These should be checked frequently and more in depth in the future. Bugs will be filed for the missing or inaccurate documentation (marked as 'NOK' i.e. "Not OK").
| Tool | Doxygen | Help | Man pages | Wiki |
| test-definition | x Checked: asinnela 2011-03-03 | |||
| testrunner-lite | x | x Checked: asinnela 2011-03-07 | x Checked: asinnela 2011-03-07, NOK Bug 14074 | x Checked: asinnela 2011-06-07 |
| Testrunner | x | x Checked: asinnela 2011-03-07 | x Checked: asinnela 2011-03-07 | |
| Testplanner | x | x Checked: asinnela 2011-03-10 | x Checked: asinnela 2011-03-07 | |
| eat - enables automated testing | x Checked: asinnela 2011-03-08 | |||
| OTS - Open Test System | x | x | x | x Checked: asinnela 2011-03-25 |
| MIN test framework | x | x | x | |
| Scripts and utils | x Checked: asinnela 2011-03-11 | |||
| MeeGo Fast Feedback Testing | x | x | x Checked: asinnela 2011-03-11 | |
| Qpid C wrapper library - libcqpid | x | x Page created, content to be added. |
Other documentation
All meetings will be held in #meego-meeting on irc.freenode.net.
Team communication is in English. Our collaboration spaces are:
The current team members are (in no particular order):
| Name | Role | Affiliation | IRC nickname |
| Riku Halonen | Team member | Nokia | rikhalon |
| Kari Sievi | Team member | Digia | sievi |
| Timo Härkönen | Team member | Digia | timoph |
| Carol Rus | Team member | Digia | carrus |
| Sami Lahtinen | Team member | Digia | slahtinen |
| Raimo Gratseff | Team member | Digia | rrraimo |
| Kyösti Ranto | Team member | Digia | kyranto |
| Joonas Kylänpää | Team member | Digia | Kaadlajk |
| Timo Mäkimattila | Team member | Digia | timakima |
| Elias Luttinen | Team member | Digia | eluttine |
| Ville Niutanen | Team member | Digia | Villen |
| Vesa Poikajärvi | Team member | Digia | vesse |
| Alexey Kuznetsov | Team member | Digia | alkuznet |
| Daniil Chuiko | Team member | Digia | dachuiko |
| Jarmo Savinen | Team member | Digia | jasavi |
| Sampo Saaristo | Team member | Sofica | sampos |
| Ling Yu | Team member | Intel | - |
| Jing Wang | Team member | Intel | - |
| Tuomo Mäkinen | Team member | Ixonos | - |
| Jouni Leppäkases | Team member | Ixonos | jouni |
| Jarno Keskikangas | Team member | Leonidas | jakeskik |
| Janne Hietamäki | Team member | Leonidas | _janne |
| Sami Hangaslammi | Team member | Leonidas | sahangas |