Meego Wiki
Views

User talk:Zhdou

From MeeGo wiki
Jump to: navigation, search

Contents

Test Process

Test process.JPG

Test Plan

Test plan is a XML file that contains your test suites, cases and steps. Tool support for validating test plan and automating test plan execution is provided with testrunner-lite.

Example:

Test plan example.JPG

Test Plan definition consists of : 

suite, set, case : Hierarchical structure of the tests.
feature, subfeature, requirement : Information about why the tested software has been implemented in the first place.
type : Information about the viewpoint of the tests (which quality aspect of the software they are testing).
level : Information about which test level the tests belong to.
domain : Information about which architectural domains the tests are focused on.
description : Description
step :  Execution instructions (for automated tests), which determine the actual commands to execute to run each test.

Test Package

Test packaging is set of simple rules to wrap tests with test plan inside rpm package. A source package with test cases must:

  • build binary rpm package with name which ends with "-tests" (this is a test package)

A test package must:

  • contain all tests, scripts and configuration files required to run tests
  • define dependencies - the ones it tests and the test tools it depends on (if any) contain test plan located at /usr/share/<packagename>-tests/tests.xml

more detail can be checked from [1]

Create Test Images

To perform automated testing you need to create images that contain test automation enablers and the tests. There are two kinds of test images "host based test" and "on device test".

  • For reference kickstart files for test images, see: http://gitorious.org/qa-tools/eat/commits/ks
  • For host based testing the image needs to have the eat-device package installed and optionally eat-syslog-device in it. In addition to these configuration packages you need in to install test packages to the image since OTS fetches the tests from the image
  • For on device testing install eat-selftest package along with your test packages to the image. This package finds and executes all tests found in the image automatically after boot and writes test results to $HOME/testresults.

Running Test Images

Testrunner-lite reads planned tests from test plan xml file which is validated against test-definition schema. The tool can be run on the device under test or from a test control pc.

  • Tests can be run automatically on a MeeGo image by adding package eat-selftest to the image.
  • If testing is performed by OTS the test packages are fetched from the device and you don't need to manually start the test execution. To manually start test execution from the test control pc (host machine) you need to have the test plan xml files available on the host. To start executing tests in the device under test you need to tell testrunner-lite the device's IP address. For example testrunner-lite -f tests.xml -v -o results.xml -t root@192.168.2.15

Test Result File

Test result example.jpg

more details can be get from [2]

Interface of testrunner-lite

The program is executed from command line 

/usr/bin/testrunner-lite [options]
Options

-h, --help
Display short help and exit

-V, --version
Display version information and exit

-f FILE, --file FILE
 Input file with test definitions in XML (required)

-o FILE, --output FILE
 Output file format; FORMAT can be xml or text (Default xml)

-v, -vv, --verbose[={INFO|DEBUG}]
 Enable verbosity mode; -v and --verbose=INFO are equivalent outputting INFO, ERROR and WARNING messages. Similarly -vv and --verbose=DEBUG are equivalent, outputting also debug messages. Default behaviour is silent mode.

-L URL, --logger=URL
 Remote HTTP logger. Log messages are sent to given URL in a HTTP POST message. URL format is [http://]host[:port][/path/], where host may be a hostname or an IPv4 address.

-a, --automatic
 Enable only automatic tests to be executed

-S, --syslog
 Enable logging to syslog

-P,  --print-step-output
 Output standard streams from test steps.

-m, --manual
 Enable only manual tests to be executed

-c, --ci
 Disable validation of test definition against schema

-s, --semantic
 Enable validation of test definition against stricter (semantics) schema

-A, --validate-only
 Do only input xml validation, do not execute tests

-H, --no-hwinfo
 Do not try to obtain hardware information

-t [USER@]ADDRESS, --target=[USER@]ADDRESS
 Enable host-based testing. If given, commands are executed from test control PC (host) side. ADDRESS is the ipv4 adress of the system under test.

OTS

OTS's power is to distribute tests to many kind of devices and the ability to process lots of testruns in parallel.

  • OTS server has an xmlrpc interface for starting the testruns.
  • Result files are sent to the user as an email. Result file processing can be extended with a custom plug-in.

OTS Arch.jpg

OTS XMLRPC Interface

Usage: ots_trigger.py [options] 

-s SERVER, --server=SERVER    OTS server URL
-b ID, --build_id=ID    Build request ID 
-i IMAGE, --image=IMAGE    URL to image file
-d DEVICE, --device=DEVICE    Device properties 
-e EMAIL, --email=EMAIL   Email addresses as a comma separated list
-t TESTPACKAGES, --testpackages=TESTPACKAGES    List of test packages separated with space
-o HOSTTEST, --hostpackages=HOSTTEST    List of host test packages separated with space
-f FILTER, --filter=FILTER    Test filter string
-g ENGINES, --engine=ENGINES    Test engines as a CSV
-n PLUGIN, --input_plugin=PLUGIN    input_pluging to use
-c DISTMODEL, --distribution=DISTMODEL    task distribution model(for example 'perpackage')

Test to BOSS

  • Make the test plan for test packages
  • Intall BOSS and test packages
  • Run the testrunner-lite with the test plan
  • Analysis the test result file
  • Mail the test result file or create a defect in bugzilla
  • Not use host based test and on device test

Resource

Retrieved from "http://wiki.meego.com/User_talk:Zhdou"
Personal tools