Latest revision as of 13:24, 6 July 2011

MeeGo QML UI Testlib

Overview

MeeGo QML UI Testlib is an open tool for testing against MeeGo QML GUI application automatically. The test framework is based on "TDriver", and provide a set of pre-defined cucumber steps for designer to prepare test cases with natural language. Also BAT and L10N test cases of MeeGo QML UI applications are involved in Testlib.

Architecture

BAT and L10N Test cases and a pre-defined library, in which many Cucumber steps are involved for covering all actions on MeeGo QML UI applications, are included in this Testlib.

Installation

This tool is based on cucumber and TDriver. Make sure if TDriver and cucumber already work well on your machine.

Verifying if TDriver works well:

$> tdriver_visualizer &

Check if visualizer could connect to device via sut_qt.

Verifying if Cucumber works well

$> cucumber --help

For TDriver installation, please refer to Quality/QA-tools/TDriver#Installation. For cucumber, please refer to Cucumber Home site

For downloading this tools,

$>git clone http://git.gitorious.org/meego-quality-assurance/auto-ux-testlib.git

Enabling Localization

TDriver support execute test cases with multiple languages. For enabling this feature in our testlib, please follow below steps.

1\ copy below configurations into tdriver_parameters.xml

<parameter name="localisation_server_ip" value="localhost" />
<parameter name="localisation_server_username" value="user" />
<parameter name="localisation_server_password" value="password" />
<parameter name="localisation_server_database_name" value="database_name" />  

and fill the corresponding user/password/database_name.

2\ copy below parameters into the configuration of sut.

<parameter name="language" value="en-US" />
<parameter name="localisation_server_database_tablename" value="tr_qt_default" />

The language of sut and the table name of the translation info are set at here.

3\ Connect to the sut device.

4\ Execute below command to load the translation information which are deployed on the sut devices into MySQL.

$> export RUBYOPT=rubygems
$> ruby create_localization_db_tables.rb

Usage

Run existing test cases

Some BAT test cases are also provided for MeeGo applications. For running them, follow below steps:

$> cd <Testlib_HOME>
$> cucumber

If you just want to execute some features.

$> cd <Testlib_HOME>
$> cucumber features/meego_qml_calculator.feature

If you just want to execute some special scenario, regular expression is your friend

$> cd <Testlib_HOME>
$> cucumber features/meego_qml_calculator.feature --name "^Cal"

Prepare your own scenarios (test cases)

Besides running the existed test cases, you can also define your own scenario (test cases) using the predefined cucumber steps.

For instance, you can define a new test cases for verifying if MeeGo browser can access a new web site (for instance, Amazon).

 Scenario: Check if browser works for Amazon
   Given I launch meego browser
   And I sleep "5" second
   And I input "www.amazon.com" into "urlTextInput"
   And I sleep "5" second
   Then In meego browser, I should see "Amazon"
   And In meego browser, I should see "Sign in"

Test Result

Beside the result of cucumber scripts, snapshot will be captured automatically to help QA and developer to investigate what happen when the verification failed. And the snapshot is named as

xxxx_xxxxx.png

First part is the name of the testing application, such as email, im, browser Second part is a short description of the failed verification.

For instance, when the device isn't connected with internet, the test case "Input URL into location bar" will fail. Then below snapshot will be captured automatically to help investigating this failure.

Contribution Guide

You are welcome to contribute some test cases or cucumber steps into this project. For designing the cucumber steps on MeeGo QML applications, please keep it in mind that the step should be designed to be easily re-used in other test cases. For this purpose, something you need to notice.

1\ A easily re-used steps should just take 1 action of user.

such as "tap a button", "input a string", or "launch an application"

2\ Just verify 1 thing in 1 step.

Bad example: "I should received a mail with title "XXXXX", and I can open the mail" Above step should be defined as 2 steps a\"I should see a received mail with title "XXXX"", b\ "I can open the mail "XXXXX"".

Obviously, 2 short and simple steps are move easily to be re-used.

3\ Localization method should be referred in the definition of step for the label which is shown on QML widget.

Below is a example to invoke the localization method to extract the translation info.

 itemtext = MeegoLocalizationHelpers::getTranslation(@__sut, @app, "Find...", "meego-app-browser")

4\ The identifier of QML widget should be defined as parameter in the definition of steps

For instance, below step could be re-defined as a re-usable step.

 Given /^I press the search button$/ do

As below,

 Given /^I press the button "([^\"]*)" in meego application$/ do |button_id|

5\ Using optional parameter to avoid duplicated steps

For instance, below 2 steps could be merged

 S1\ Given /^I press the button\(label\) "([^\"]*)"$/ do |button_text|
 S2\ Given /^I press the button\(label\) "([^\"]*)" within "([^"]*)"$/ do |button_text, container_id|

As below:

 Given /^I press the button\(label\) "([^\"]*)"(?: within "([^"]*)")?$/ do |button_text, container_id|

6\ It isn't recommend to define Ruby method in step-definition file.

We recommend you define some small steps, and then the small one could be invoked in your big step, instead of Ruby method. If you need to get the return from a Ruby method, maybe the "World" of cucumber is helpful which is defined under /feature/support/

7\ Global variable shouldn't be used in step-definition. All variables which are used in step definition should only work in the scope of one step. Otherwise, the variable which is available in more than 1 steps will make the steps un-reusable.

To be completed

Dependency

Because the cucumber steps of testlib are designed based on the attribute "QML_ID" of QML widget, the patch "[Add public method to allow users to retrieve id string from object in a given context]" should be applied in the package "libqtdeclarative4" and "libqt-devel". We recommend you merge this patch and re-build the qt-related package for yourself. Only if there are some problems to apply this patch, a temporary repository is available as a reference. Follow below steps to update the qt-related packages

$> cd /etc/zypp/repo.d/

$> wget http://download.meego.com/live/home:/shaofeng/MeeGo_1.2/home:shaofeng.repo

$> sudo zypper up

$> sudo zypper in libqt-devel

How to check if the patch is installed in your device.

$> vi /usr/include/qt4/QtDeclarative/qdeclarativecontext.h

In 91 lines, below method should be announced.

     QString getStringId(const QObject *) const;

For enabling the "QML_ID" patch on TDriver side, the package agent_qt should also be re-compiled.

$> git clone git://gitorious.org/tdriver/agent_qt.git

$> sudo zypper in libqtwebkit-devel libXtst-devel libX11-devel libXext-devel libXi-devel

$> qmake -r CONFIG+=no_mobility CONFIG+=qml_id

$> make

$> sudo make install

Below is the updated package list:

libqt-devel-4.7.2-8.18.i586.rpm

libqtcore4-4.7.2-8.18.i586.rpm

libqtdbus4-4.7.2-8.18.i586.rpm

libqtdeclarative4-4.7.2-8.18.i586.rpm

libqtdeclarative4-folderlistmodel-4.7.2-8.18.i586.rpm

libqtdeclarative4-gestures-4.7.2-8.18.i586.rpm

libqtdeclarative4-particles-4.7.2-8.18.i586.rpm

libqtdesigner4-4.7.2-8.18.i586.rpm

libqtgui4-4.7.2-8.18.i586.rpm

libqthelp4-4.7.2-8.18.i586.rpm

libqtnetwork4-4.7.2-8.18.i586.rpm

libqtopengl-devel-4.7.2-8.18.i586.rpm

libqtopengl4-4.7.2-8.18.i586.rpm

libqtscript4-4.7.2-8.18.i586.rpm

libqtscripttools4-4.7.2-8.18.i586.rpm

libqtsql4-4.7.2-8.18.i586.rpm

libqtsql4-sqlite-4.7.2-8.18.i586.rpm

libqtsvg4-4.7.2-8.18.i586.rpm

libqttest4-4.7.2-8.18.i586.rpm

libqtxml4-4.7.2-8.18.i586.rpm

libqtxmlpatterns4-4.7.2-8.18.i586.rpm

qt-debuginfo-4.7.2-8.18.i586.rpm

qt-demos-4.7.2-8.18.i586.rpm

qt-designer-4.7.2-8.18.i586.rpm

qt-devel-tools-4.7.2-8.18.i586.rpm

qt-qmake-4.7.2-8.18.i586.rpm

qt-qmlviewer-4.7.2-8.18.i586.rpm