= Quality assurance (QA) tools =

''' We moved our discussion to #meego-qa IRC channel as of March 24th 2011. Please join us there. '''

[[File:Tool_meegon.png|400px]]

Quality Assurance tools are developed to ensure MeeGo SW quality. QA tools team develops and maintains tools for quality assurance.

* "Open source tools – available for all, available for development and contributions. Make people accountable for quality."

'''Please note''' that developer-oriented content has been moved to [[Quality/QA tools development|QA tools development]] page. We apologize for any inconvenience. The target is to make this page clearer and more informative for the actual users of QA tools.

== Overview ==

QA tools support test planning, test execution and test reporting. The most important tool offering relating to this is illustrated in the figure below. In addition, there are also a couple of concept videos available at our [http://www.youtube.com/user/meegoqatools YouTube channel]:
* [http://www.youtube.com/watch?v=WdXBqGdLz8I Tool offering for manual testing]
* [http://youtu.be/Ry4MM-CX2b8 Test automation]

[[File:Meego-qa-tools.png|600px]]

== Tools and other utilities ==

If you want to contribute to the development of the following tools, test suites and utilities, please visit [[Quality/QA_tools_development|QA tools development page]].

'''Test tools'''

Each wiki page will contain more detailed description, installation instructions and tutorial on the basic usage. For now, please [[Quality/QA-tools#Collaboration spaces|contact us]] if you need help. Currently, you can install Testrunner, testrunner-lite, test-definition, Testplanner, OTS, Meego-ai, libcqpid, eat and MIN from Tools:Testing repository. The instructions for setting up the repositories can be found [[Quality/QA-tools/How_to_set_up_repositories|here]].

{| border="1" cellspacing="0"
|+
|'''Tool (link to wiki page)''' ||'''Short description'''
|-
| Crash Reporter || On-device tool for sending core dumps to core dump data base
|-
| [[Quality/QA-tools/OTS|OTS - open testing system]] || Test automation system
|-
| [[Quality/QA-tools/Testrunner-lite|testrunner-lite]] ||Command line tool for test execution
|-
| [[Quality/QA-tools/TDriver|Testability Driver]] || Comprehensive tool for UI testing
|-
| [[Quality/QA-tools/QAReports|QA Reports]] || Reporting tool for publishing test reports
|-
| [[Quality/QA-tools/CrashReports|Crash Reports Web UI]] || Web UI for Crash Reports
|-
| [[Quality/QA-tools/QADashboard|QA Dashboard]] || Quality at a glance
|-
| Application Monitor || Monitor runtime indicators - CPU, memory, power for specified apps
|-
| Video performance || Collect FPS, CPU information when do video playback
|-
| [[Quality/QA-tools/QmlUITestlib|MeeGo QML GUI Testlib]] || Test cases and Cucumber steps for MeeGo QML UI applications
|}

'''Other utilities'''

{| border="1" cellspacing="0"
|+
|'''Name (link to wiki page)''' ||'''Short description'''
|-
|[[Quality/QA-tools/Test-definition|test-definition]]|| XML schema for test plan and result files
|-
| [[Quality/QA-tools/Eat|eat - enables automated testing]] || Test automation configuration packages
|-
| [[Quality/QA-tools/Autotest-guide#Automatic_image_installations|MeeGo Automated installer]] ||
|-
| [[Quality/QA-tools/rich-core|Rich core dumper (sp-rich-core)]] || A tool to collect core dump and context data from process crashes
|-
| Crash Reporter settings || Settings package for Crash Reporter
|-
| [[Quality/QA-tools/hat-control|Hardware Accessory for Testing (HAT)]] ||
|-
| [[Quality/QA-tools/Scripts_and_utils|Scripts and utils]] ||
|-
| Service OS based Flasher || Flasher to support Intel netbook/tablet
|-
| Environment collector || Collect detail system environment information
|-
| [[Quality/QA-tools/Corelysis|Corelysis]] || A daemon to process rich core dumps in the back-end server
|}

'''Tools and utils not in active development'''

Tools listed here are in maintenance and no new features are planned to be implemented in the near future to them.

{| border="1" cellspacing="0"
|+
|'''Tool (link to wiki page)''' ||'''Short description'''
|-
| [[Quality/QA-tools/Testrunner|Testrunner]] || Test execution tool with graphical UI
|-
| [[Quality/QA-tools/Testplanner|Testplanner]] || Simple tool for creating and editing test plan files
|-
| [[Quality/QA-tools/MeeGo_Fast_Feedback_Testing|MeeGo Fast Feedback Testing (MeeGo-FFT)]] ||
|-
| [http://min.sf.net/ MIN Test Framework] || Advanced test framework
|-
| [[Quality/QA-tools/qtuitest-mbt-adapter|Model-Based Testing adapter for qtuitest]] ||
|-
| [[Quality/QA-tools/Qpid C wrapper|Qpid C wrapper library - libcqpid]] ||
|}

== Feature requests and bugs ==

Want to report an feature idea or bug to us? - [http://bugs.meego.com/enter_bug.cgi?product=MeeGo%20Quality%20Assurance Please do it here]

* [http://bugs.meego.com/buglist.cgi?query_format=advanced&order=Importance&bug_status=ASSIGNED&component=eat&component=Fast%20Feedback%20Testing&component=hat&component=libcqpid&component=meego-qa-reports&component=min&component=ots&component=TDriver&component=testdefinition&component=Testplanner&component=Testrunner&component=testrunner-lite&component=Others&classification=MeeGo%20Projects&product=MeeGo%20Quality%20Assurance Assigned bugs and features - Working on it]
* [http://bugs.meego.com/buglist.cgi?query_format=advanced&order=Importance&bug_status=NEW&bug_status=NEEDINFO&bug_status=ASSIGNED&bug_status=WAITING%20FOR%20UPSTREAM&bug_status=REOPENED&bug_status=RESOLVED&component=eat&component=Fast%20Feedback%20Testing&component=hat&component=libcqpid&component=meego-qa-reports&component=min&component=ots&component=TDriver&component=testdefinition&component=Testplanner&component=Testrunner&component=testrunner-lite&component=Others&classification=MeeGo%20Projects&product=MeeGo%20Quality%20Assurance All open features and bugs in priority order]

Bugzilla instructions:
* [[Quality/How To Report Bugs|How to report bugs]]
* [[Quality/Bug Life Cycle and Handling|Bugzilla workflow]]

== Collaboration spaces ==

Team communication is in English. Our collaboration spaces are:
* [http://lists.meego.com/listinfo/meego-qa meego-qa@lists.meego.com mailing list]
* [http://webchat.freenode.net/?channels=meego-qa #meego-qa IRC channel on irc.freenode.net]
* [http://www.youtube.com/user/meegoqatools YouTube channel for demo videos]
* [http://meegoqatools.wordpress.com/ QA-tools team blog]

== Nice to have ==

None of the existing tools answer your needs? Here is where you should enter any suggestion.

* Relieve localization testing effort by automating all or part of it ([[Quality/QA-tools/Automation of L10n testing|Detail]]).

 
[[Category:QA]]

2011-06-03T10:37:24Z

Rha: /* Installation */

Quality/QA-tools/Corelysis

2011-06-03T10:36:16Z

Rha: /* Corelysis */

2011-03-18T08:24:17Z

Rha: /* Test automation images */

= QA TODOs (in priority order) =

== OTS setup and automated hourly testing ==

=== OTS setup ===
* OTS server - Ville Ilvonen/Riku Halonen, DONE
* OTS worker(s) for core tests - Ville Ilvonen/Riku Halonen/Timo Harkonen - ONGOING (2 workers up)
* OTS worker for UX tests
* Power consumption measurements
* Reporting to QA-reports
* Minimize automatic installation time to 10mins - Timo Makimattila, ONGOING

=== Test automation images ===
* Setup hourly image building for autotest image on own setup (maybe later with BOSS) Riku Halonen ONGOING (setting up PC for creating images)
* We need to be able to control included test packages

If you need something else from QA please tell it to us :)

== Applications for testing ==
* Package QML demo/example applications from http://doc.qt.nokia.com/latest/qdeclarativeexamples.html

== Test asset ==
* Define what the asset includes
* Automating as many core/ux tests as possible
* Call/SMS cases will require parallel test execution support (If we want to automate those kind of tests)

== Crashdb support for ARM core dumps ==

* Core dump processing and backtraces from crashing ARM processes.

== Test Execution Schedule ==
* Core

* Handset UX weekly testing schedule
{| cellpadding="5" cellspacing="0" border="1"
! Day !! Test set (status) !! Release
|-
| Monday
| Key feature (OK)
| Preview
|-
| Monday
| Acceptance (OK)
| Testing trunk
|-
|Tuesday
|Acceptance (Ok)
|Testing trunk
|-
|Tuesday
|Sanity (Ok)
|Daily trunk
|-
|Tuesday
|DE Dataflow (Ok)
|Preview
|-
|Tuesday
|DE use cases (Ok)
|Preview
|-
|Wednesday
|Acceptance (Ok)
|Testing trunk
|-
|Wednesday
|Key feature (Ok)
|Weekly
|-
|Thursday
|DE Dataflow (Ok)
|Weekly
|-
|Thursday
|DE use cases (Ok)
|Weekly
|-
|Thursday
|Acceptance (Ok)
|Testing trunk
|-
|Thursday
|Sanity Ok
|Daily trunk
|-
|Thursday
|DE Reliability (Ongoing)
|Weekly
|-
|Thursday
|DE Performance (Ongoing)
|Weekly
|-
|Friday
|Acceptance (Ok)
|Testing trunk
|-
|Friday
|Sanity (Ok)
|Daily trunk
|}

== QA Tasks For Developer Edition ==
There is a wiki article about the [[ARM/N900/DeveloperEdition|Developer Edition]].

QA tasks for the Developer Edition differ from the usual N900 approach in that there are less features to be tested. There are currently 2 test sets for the Developer Edition, these are the Sanity Test Set and the Feature Test Set. They are described below.

=== Sanity Test Set ===
The sanity set should be run automatically on every image. As such it must meet the following requirements:
* 100% automated
* Testing only basic features

=== Feature Test Set ===
The feature set will be run periodically and will test the basic features as well as enablers for those features (e.g. PIM for phoning contacts). Performance will also be analysed, at the moment, this will include browser startup time and a CPU benchmark but this will be expanded later.

Suggestions are welcome.

== Error Management ==
* Error Manager Iekku Huttunen

[[Category:N900]]