Quality Assurance Team

Revision as of 14:28, 25 October 2010

The MeeGo Quality assurance is one of the pieces in MeeGo Development Structure (see http://meego.com/sites/all/files/MeeGoDevStructureTSG_May5.pdf)

Today there is no procedure defined but we can improvise with some common sense.

Mission

Learning from the former labor at Maemo the QA team will actively conduct testing and simulations. The QA team will stimulate community participation in testing and quality efforts to make sure MeeGo is as usable and polished as competitor's smartphone OSs, or better.

Structure

Areas

The main areas within the scope of the working group are (in no particular order):

• Specifications  : Identify new needed features or features that arise from user stories or bug reports and make a spec out of them.
• Documentation  : Develop release notes documentation and workaround HOWTOs until a specific issue is solved.
• Testing Control Groups: Organize control groups with members of the community for integrational testing (e.g. above unit level) of new major features or drastic UI or human-machine interface changes.
• Ongoing Testing : Through well defined test plans, do integrational/functional testing with each new feature, bug fix or a release (minor and major)
• Reduce load on bug-squads: Bug trackers and bug squads should be the last resort. Proper *active* testing will make their work easier and lighter in load.
• Head to head spec extraction: Doing detailed comparisons with other OSs to identify soft spots and spec out how to improve/fix them.

[Rudely added by V-PV - Feel free to remove/edit/include to original]

How about areas like (not in particular order either):

• Testing definition: Containing testing levels and testing types
• Test specifications: Defining how to document and store
• Performance (and other non-functional parameters) targets and quality of the "Requirements"
• Intented quality of the MeeGo releases
• Testing approach for MeeGo distributions
• Test planning, what is planned to be done and what is ongoing
• Test run (test results) repository
• Quality guidelines for developers: What should be done in Projects
• Compliancy testing of eg. applications

Rational

I have drafted what I envision as the desired procedures, building on the experience I gained through Ubuntu.

While bug squads are important and vital part of the development process, I think that centrally lead, milestone oriented QA team is the only way to get to the polish other smartphone OSs on the market display.

I propose to lead the efforts, do the wiki gardening and user story conversion to bug reports and specifications, and to recruit prospective community members that want in the game as a non-development, no prerequisite contribution entry point.

Process

Meetings

Please expand.

Tasks

Started tasks

• Have a subdomain qa/quality.meego.com -- the name will be qa.meego.com and request has been posted to IT.
  • Reasoning for having qa/quality.meego.com subdomain: Open source software doesn't test itself. MeeGo relies on the testing efforts of community members to help find bugs and verify products. Not everyone may be cut out to hack on MeeGo's codebase, but anyone who can use a web browser and has a supported hardware can help us test it. In order to do that one needs to have collaboration space having test cases for contribution, test runs for participation and test results for viewing. In addition one could also have possibility to dip her or his toes into development with QA tools and test automation with QA core team.
• Define Testability Checklist for ensuring requirement quality - Please, contribute by commenting.
• Define Test Design Process and Guideline - Please, contribute by commenting.

Suggested tasks

• Have a mailing list (or channel in discussion forum or decision for having certain tag (e.g. [QA]) in the beginning of the title.)
• Have a irc-channel
• I propose we start by getting a thorough specification of what functionality each currently available component (=package?) provides, and then create unit test plans to test cover it with every iteration of the release (minor/alpha/beta/RC/final).
• I then propose that we move on to integration test plan authoring, meaning to cover test cases of component X that is supposed to integrate with with component Y and Z and write test plans that verify the integration is working as expected. That of course would require us to first survey and specify what higher level functionality each integration combination yields.
• Define Testing approach for Release Creation (moment of involvements) and related quality targets
• Define Quality Guidelines for Projects / Application developers
• QA Process (Test Design, Test Asset management, Test Execution and reporting)
• How community members can efficiently contribute to bug triaging while following our processes?

Completed tasks

• Modify Quality Assurance wiki from Working Group format to Team area format (see http://forum.meego.com/showpost.php?p=3088&postcount=3)
• Have a role description for Test Collection Maintainer (a.k.a test suite maintainer) (see http://wiki.meego.com/Quality_Assurance_Glossary_QA_Glossary)
• Agree the "acronym" used for Quality assurance (used for domain, mailing list, irc, etc.)
  • qa and quality were the names suggested
  • according "votes" given in email tread qa is the choice.
• Define test plan template (see http://wiki.meego.com/TestPlanTemplate)
• Define test report template (see http://wiki.meego.com/TestReportTemplateCollection)

How to contribute

See the Process section above for current approved work and proposed tasks.

Contributors

Coordinators of this working group:

• Sivan Greenberg - Veteran Ubuntu developer, QA and integration specialist. Interested in triage, reporting and automation.

Meego members interested in taking an active role in this working group. Please detail your interests and what you can contribute to the group :

• Lew Krohon - I like things properly done and I am willing to expend time on helping MeeGo. The QA Working Group seems the best place to use the few skills I might have.
• Randall Arnold/Texrat - Maemo Community Council member, former Nokia QA engineer for Maemo devices and best practices blogger. Facilitator for MeeGo User Experience Framework proposal and eager to implement and support QA best practices.
• Timo Paatola - Principal Engineer, QA. Full disclosure: I am a Nokia employee.
• Jari Tahvanainen - Testing Specialist being Testing and Quality Obsessed from both theory and practical point of views. Working for Nokia.
• Veli-Pekka Vatula - Head of Maemo SW Testing. Industry background of SW Development. Past years in Testing of different SW platforms and Products. Working for Nokia.
• Ville Ilvonen - Quality assurance tools development for MeeGo and MeeGo Devices(Nokia). Nokia employee.
• Shanthappa Chaitra - Application Testing team for MeeGo. Working for Nokia.

Glossary

Some terminology for aligning ourselves can be found here Quality_Assurance_Glossary QA Glossary. See and comment (maybe by using discussion area of the page).

History

Page was initially done as a proposal for working group. Format changed based on instructions given in http://forum.meego.com/showpost.php?p=3088&postcount=3