Meego Wiki
Views

Quality/QA-tools/TDriver

From MeeGo wiki
< Quality | QA-tools(Difference between revisions)
Jump to: navigation, search
(TDriver kickstart)
(→Installation on MeeGo Netbook)
 
(31 intermediate revisions not shown)
Line 1: Line 1:
-
== Testability Driver (TDriver) ==
+
=Testability Driver (TDriver)=
-
TDriver is a test automation driver enabling acceptance testingthrough the UI or using business  logic directly. It is written in Ruby language and enables using any test framework written for Ruby.
+
New to TDriver? [https://projects.forum.nokia.com/Testabilitydriver/wiki/introduction_to_tdriver Look for an introduction to TDriver] from Forum Nokia web site.
-
== Installation on MeeGo ==
+
TDriver is a test automation driver enabling acceptance testing through the UI or using business  logic directly. It is written in Ruby language and enables using any test framework written for Ruby.
-
Note! Repo locations will change as soon the official location is decided.
+
For a sample test run, see [http://www.youtube.com/watch?v=UL0XmcPKAp8 TDriver in action]. The cucumber tests in the example can be found from [http://gitorious.org/tdriver/tests/trees/master/examples/MeeGo/N900/cucumber/features here].  
-
=== Installing agent to the target SUT ===
+
==Overview==
-
sudo zypper -p http://download.meego.com/live/devel:/quality/testing install qttas-server
+
TDriver consists of the following modules:
-
This will install the agent. Once you reboot the first time the agent should start automatically. (TODO: This will be changed to lazy load later)
+
* [[#Agent|Agent]]
 +
* [[#Driver|Driver]]
 +
* [[#Visualizer|Visualizer]]
-
You can start the agent manually by typing:
+
===Agent===
-
qttasserver &
+
TDriver has one agent for Qt based systems where Qt applications on any supported OS can be tested. The Qt agent is called qttasserver.
 +
Agent (qttas-server) is the only component that needs to be installed to the SUT (system under test). It will handle communication with the testing framework and applications. When Qt application starts it will load a testability plugin into the application process. The plugin will register the application into qttasserver. Once the application is registered to qttasserver, it can be controlled and queried from the testing framework.
-
=== Installing driver ===
+
MeeGo touch applications will automatically load the testability plugin. If your application is a standard Qt application, use the "-testability" argument:
-
sudo zypper -p http://download.meego.com/live/devel:/quality/testing install rubygem-testability-driver-qt-sut-plugin
+
<pre>
 +
myapp -testability
 +
</pre>
-
This will install the ruby gem for handling qt SUTs and all the dependencies like for example ruby interpreter it self.
+
That will cause the testability plugin to be loaded automatically.
 +
===Driver===
-
=== Testing the installation ===
+
Driver is the component that will provide the glue between test harness and SUT. You can install driver inside SUT or you can also use another host for driver that will connect to sut over tcp/ip.
-
You can test the installation using irb easily:
+
TDriver is a ruby class libary which is packaged as a Ruby gem. Package 'rubygem-testability-driver' provides the core functionality and 'rubygem-testability-driver-qt-sut-plugin' provides the qt specific support.
 +
 
 +
===Visualizer===
 +
 
 +
Visualizer is a helper tool that provides access to UI components of the application. It will help testers to know what objects are contained and what is possible to do with them. You will see a SUT screenshot and an object tree containing all visible object. For Qt applications the objects are fetched from the object tree of the application registered to qttasserver. Visualizer also contains documentation and a small editor to help in scripting and debugging.
 +
 
 +
==Installation==
 +
 
 +
On a typical test automation setup, tests are executed on the host. See the [[Quality/QA-tools/TDriver/Getting Started|Getting Started Guide]] for host based installation. If you are going to install TDriver on MeeGo netbook, see the [[#Installation on MeeGo Netbook|Installation on MeeGo Netbook]] section.
 +
 
 +
===Adding Agent On The Target Device===
 +
 
 +
On host based testing, the only required package for the image is the qttas-server. Add this package to your ks. file if you want it into your image.
 +
 
 +
On a preexisting image, you can add it by
 +
<pre>
 +
sudo zypper install qttas-server
 +
</pre>
 +
 
 +
This will install the agent. Once you reboot the agent should start automatically.
 +
 
 +
If you do not reboot the device, you can start the agent manually by:
 +
<pre>
 +
qttasserver &
 +
</pre>
 +
 
 +
''' Note - qttasserver should always be run with the same user as the applications, e.g. meego '''
 +
 
 +
===Adding Everything On The Image===
 +
 
 +
You can also run the entire testability driver on the device. Currently packages are only in devel:quality. Add the devel:quality repository into the ks. file:
 +
<pre>
 +
repo --name=devel-quality-testing  --baseurl=http://download.meego.com/live/devel:/quality/testing/
 +
</pre>
 +
And add the meta package:
 +
<pre>
 +
testability-driver
 +
</pre>
 +
 
 +
Package rubygem-cucumber is also available if you have [http://cukes.info Cucumber] tests.
 +
 
 +
===Installation on MeeGo Netbook===
 +
 
 +
You need to enable devel:quality repo as not all components in Trunk:Testing
 +
<pre>
 +
sudo zypper ar http://download.meego.com/live/devel:/quality/testing/ devel:quality.repo
 +
sudo zypper update
 +
</pre>
 +
 
 +
To install all required packages install the testability-driver and visualizer packages. But lets install the tests package so we can easily test the installation:
 +
<pre>
 +
sudo zypper install testability-driver-tests testability-driver-visualizer
 +
</pre>
 +
 
 +
You can test the installation using irb:
<pre> [matti@matti-desktop tmp]$ irb
<pre> [matti@matti-desktop tmp]$ irb
Line 34: Line 94:
irb(main):002:0> </pre>
irb(main):002:0> </pre>
-
If you get 'true' as the result then everything is installed ok and you are set. In case it complains about tdriver not found make sure you have env variable RUBYOPT defined.
 
 +
To test the visualizer, make sure qttasserver is running and start testapp:
 +
 +
<pre>
 +
qttasserver &
 +
testapp &
 +
</pre>
 +
 +
Now make sure you have RUBYOPT set and start visualizer:
 +
<pre>
  export RUBYOPT=rubygems
  export RUBYOPT=rubygems
 +
tdriver_visualizer &
 +
</pre>
 +
 +
Now you should see the visualizer application. Select sut_qt from device menu and select File -> Refresh. You should now see screenshot of testapp and object tree which contains all visible objects from the application.
 +
 +
You can also try to run our unit tests to see that automated tests work:
 +
<pre>
 +
ruby /usr/share/testability-driver-tests/tc_testapp.rb
 +
</pre>
 +
 +
And the script will use the app 'testapp' to test basic things like launching and closing app, tapping, dragging, flick etc.
 +
 +
== Enabling Logs On The Agent ==
 +
 +
In order to debug you can enable logs on the device. Just create a folder and give appropriate write access so that qttasserver and applications can write in there:
 +
 +
mkdir -p /logs/testability
 +
sudo chmod a+w /logs/testability
-
This will tell ruby to use gem packages.
+
Now if you restart qttasserver you will see a log file qttasserver.log being created. For each application loading testability plugin there will be also a file <application_name>.log

Latest revision as of 07:16, 8 December 2010

Contents

Testability Driver (TDriver)

New to TDriver? Look for an introduction to TDriver from Forum Nokia web site.

TDriver is a test automation driver enabling acceptance testing through the UI or using business logic directly. It is written in Ruby language and enables using any test framework written for Ruby.

For a sample test run, see TDriver in action. The cucumber tests in the example can be found from here.

Overview

TDriver consists of the following modules:

Agent

TDriver has one agent for Qt based systems where Qt applications on any supported OS can be tested. The Qt agent is called qttasserver.

Agent (qttas-server) is the only component that needs to be installed to the SUT (system under test). It will handle communication with the testing framework and applications. When Qt application starts it will load a testability plugin into the application process. The plugin will register the application into qttasserver. Once the application is registered to qttasserver, it can be controlled and queried from the testing framework.

MeeGo touch applications will automatically load the testability plugin. If your application is a standard Qt application, use the "-testability" argument: 

myapp -testability

That will cause the testability plugin to be loaded automatically.

Driver

Driver is the component that will provide the glue between test harness and SUT. You can install driver inside SUT or you can also use another host for driver that will connect to sut over tcp/ip.

TDriver is a ruby class libary which is packaged as a Ruby gem. Package 'rubygem-testability-driver' provides the core functionality and 'rubygem-testability-driver-qt-sut-plugin' provides the qt specific support.

Visualizer

Visualizer is a helper tool that provides access to UI components of the application. It will help testers to know what objects are contained and what is possible to do with them. You will see a SUT screenshot and an object tree containing all visible object. For Qt applications the objects are fetched from the object tree of the application registered to qttasserver. Visualizer also contains documentation and a small editor to help in scripting and debugging.

Installation

On a typical test automation setup, tests are executed on the host. See the Getting Started Guide for host based installation. If you are going to install TDriver on MeeGo netbook, see the Installation on MeeGo Netbook section.

Adding Agent On The Target Device

On host based testing, the only required package for the image is the qttas-server. Add this package to your ks. file if you want it into your image.

On a preexisting image, you can add it by 

sudo zypper install qttas-server

This will install the agent. Once you reboot the agent should start automatically.

If you do not reboot the device, you can start the agent manually by: 

qttasserver &

Note - qttasserver should always be run with the same user as the applications, e.g. meego

Adding Everything On The Image

You can also run the entire testability driver on the device. Currently packages are only in devel:quality. Add the devel:quality repository into the ks. file: 

repo --name=devel-quality-testing  --baseurl=http://download.meego.com/live/devel:/quality/testing/

And add the meta package: 

testability-driver

Package rubygem-cucumber is also available if you have Cucumber tests.

Installation on MeeGo Netbook

You need to enable devel:quality repo as not all components in Trunk:Testing 

sudo zypper ar http://download.meego.com/live/devel:/quality/testing/ devel:quality.repo
sudo zypper update

To install all required packages install the testability-driver and visualizer packages. But lets install the tests package so we can easily test the installation: 

sudo zypper install testability-driver-tests testability-driver-visualizer

You can test the installation using irb: 

 [matti@matti-desktop tmp]$ irb
irb(main):001:0> require 'tdriver'
=> true
irb(main):002:0>


To test the visualizer, make sure qttasserver is running and start testapp: 

 qttasserver &
 testapp &

Now make sure you have RUBYOPT set and start visualizer: 

 export RUBYOPT=rubygems
 tdriver_visualizer &

Now you should see the visualizer application. Select sut_qt from device menu and select File -> Refresh. You should now see screenshot of testapp and object tree which contains all visible objects from the application.

You can also try to run our unit tests to see that automated tests work: 

ruby /usr/share/testability-driver-tests/tc_testapp.rb

And the script will use the app 'testapp' to test basic things like launching and closing app, tapping, dragging, flick etc.

Enabling Logs On The Agent

In order to debug you can enable logs on the device. Just create a folder and give appropriate write access so that qttasserver and applications can write in there: 

mkdir -p /logs/testability
sudo chmod a+w /logs/testability

Now if you restart qttasserver you will see a log file qttasserver.log being created. For each application loading testability plugin there will be also a file <application_name>.log

Retrieved from "http://wiki.meego.com/Quality/QA-tools/TDriver"
Personal tools