Views

Quality/TestSuite/MCTS/Run MCTS

From MeeGo wiki

1 How to run MCTS

How to run MCTS

Generic

A quick collection for anybody running MCTS. Please read generic section first before got to specific component.

There are 3 test suite categories for MCTS. Test suite name in each category starts with mcts-, mwts- and blts- seperately. These 3 test categories have some differences to run. To avoid confusion, we call test suites that starts with mcts- as mcts- test suites, we call other two categories as mwts and blts test suites respectively.

This section use mcts-connman-test, mwts-network and blt-wlan-core as example.

Install MCTS from RPM

# export http_proxy=<proxy> if you are in Intranet
zypper addrepo http://download.meego.com/live/devel:/quality:/tests/Trunk/  mcts
zypper in mwts-network-generic-all # In general, you can install MWTS by install mwts-<component>-generic-all
zypper in mcts-connman-tests
zypper in blts-wlan-core-tests
# Or you might prefer to install all MCTS by
zypper in mcts* mwts* blts*

Install MCTS from Source

# export http_proxy=<proxy> if you are in Intranet
git clone http://git.gitorious.org/meego-quality-assurance/mcts.git
cd mcts/mcts-connman-tests
./pack.sh
# Using ./pack.sh -r for MWTS and BLTS
# You need make sure that you have already installed necessary build required packages
rpm -ivh mcts-connman-tests.<version>.rpm
# You need make sure that you have already installed required packages

Execute

Install testrunner-lite

zypper in testrunner-lite

/usr/share/<test suite name>/tests.xml defines all test cases for the test suite. You can run all automatic test cases in the test suite by

# Run test suite by testrunner-lite. You need to define where the test result is.
# MWTS and BLTS also generate additional test log at /var/log/tests
# It's best to run on a xterm of test machine. Some test suites need X environment. 
# By default, MCTS test suite needs to be run with root user
testrunner-lite -a -f /usr/share/mcts-connman-tests/tests.xml -o results.xml

Test configure file

MWTS and BLTS have configure file for testing, so you need to modify the configure file before you run these test suite. Please refer to section for specific component for detail.
MWTS configure file is at /usr/lib/tests
BLTS configure file is at /etc/blts

README

There is a README for each test suite. Please follow the README for execution

MCTS- test suite README is at /opt/mcts-<component>-tests/
MWTS test suite README (README and MWTS.README) is at /usr/share/doc/mwts-<component>/
BLTS test suite README is at /usr/share/doc/blts-<component>-tests/

MCTS- category

mcts-bluez-tests

mcts-connman-tests

mcts-display-tests

mcts-geoclue-tests

mcts-gupnp-tests

mcts-lsw-tests

mcts-ofono-tests

mcts-openvg-tests

mcts-packagekit-tests

mcts-pulseaudio-tests

mcts-qtgfx-tests

mcts-system-tests

mcts-tracker-tests

MWTS category

MWTS test assets uses Min test framework, it provides script interface that MWTS can script its  test cases. 
These MWTS test cases are defined in min script files (*.cfg at /usr/lib/min/). To run test case user can 
run "min -c -t 'test-case-title'" where 'test-case-title' are defined as title in the *.cfg file
Many MWTS test cases need configurations, including:
* test asset configure file (*.conf at /usr/lib/tests). Like GstreamerTest.conf for mwts-gstreamer
* min test case script file (*.cfg at /usr/lib/min). Like mwts-gstreamer-audio-streaming.cfg for mwts-gstreamer
* test data that defined in both test asset configure file and min test case script file
MWTS wraps these min test cases into tests.xml to be compliant with MCTS.

MWTS-Bluetooth

mwts-location