MeeGo wiki - User contributions [en]

Quality/QA-tools/OTS/UserDocumentation/Installation rpm

2011-11-23T08:30:19Z

Villen: /* Preparation */

= OTS 0.8.x installation RPM =

These instructions have been tested with Fedora 13. Execute commands as root.

In MeeGo netbook, at least following packages needs to be installed manually:
python-amqp, python-django, minixsv, rabbitmq-server

== Network Setup ==

Using static IP addresses on all OTS machines is strongly recommended. It is possible to setup the system also with dynamic IP addresses but it might cause problems every now and then because IP's and hostnames change.

'''Note''' All OTS machines need to be able to access other OTS machines based on their hostname.

How to check that DNS names are correct:

1. Check OTS server's hostname
hostname
2. Ping the OTS server's hostname from the OTS worker
ping <hostname>

If ping fails, add manually OTS server's hostname to OTS workers '''/etc/hosts''' file
<IP> <hostname>

== Preparation ==

There are two ways of getting OTS binary packages: from '''the source code'''
or from the '''MeeGo Tools:Testing repository'''. From the repository you can get
the stable version and from the source code the development version.

* Add Tools:Testing source to /etc/yum.repos.d/
cd /etc/yum.repos.d/
yum install wget nano
wget http://download.meego.com/live/Tools:/Testing/Fedora13/Tools:Testing.repo

* Update the repositories
yum check-update

'''Build packages from source (optional)'''
* Install git core
yum install git-core

* Get sources
git clone git://gitorious.org/meego-quality-assurance/ots.git

* List tags
git tag -l

* Checkout the latest tag
git checkout <tag_name>

* Get building tools
yum install rpm-build python-setuptools

* Build packages
mv ots python-ots-0.8.x
tar czf python-ots-0.8.x.tar.gz python-ots-0.8.x
rpmbuild -tb python-ots-0.8.x.tar.gz

== OTS server installation ==

* Install dependencies
yum install Django python-amqplib python-configobj rabbitmq-server test-definition
easy_install minixsv
yum install python-setuptools

* Install OTS server components
yum install python-ots-common python-ots-results python-ots-server
or if you are using own built packages
rpm -i ...

* Add rabbitmq to start on boot
chkconfig --add rabbitmq-server
chkconfig rabbitmq-server on

* Start rabbitmq server
/etc/init.d/rabbitmq-server start

* Configure OTS server settings
sudo nano /etc/ots/server.conf

Recommended to change following:
'example_sw_product', example to 'meego_product'
'devicegroup', example to 'meego_n900'

* Install OTS server plug-ins

See plug-ins from [[Quality/QA-tools/OTS/Plugins|OTS Plug-ins]]

* Start OTS server
ots_server

== OTS worker installation ==

* Install dependencies
yum install python-amqplib testrunner-lite

* Install OTS worker packages
yum install python-ots-common python-ots-worker

* Configure OTS worker settings
sudo nano /etc/ots/worker.conf

Change 'devicegroup', example meego_n900

Change 'host' to server's address

* Start OTS worker (as root)
ots_worker

NOTE: if worker failed to start up with error like "socket.error: [Errno 113] No route to host", check that ots-worker is able to connect to ots-server

== OTS tools ==

Tools are used for triggering test runs from the command line.

* Install OTS tools
yum install python-ots-tools

* Trigger a test run
ots_trigger -s localhost:8080 -b test_build -i http://url_to_image -p example_sw_product -e test@man.com

* Kickstart file for automation can be found from
http://meego.gitorious.org/meego-quality-assurance/enables-automated-testing/blobs/ks/meego-n900-autotest.ks

* You need to build the image with mic-image-creator with parameters:
--save-kernel --package=tar.gz

== How to install N900 automation tools to OTS worker ==

* Install dependencies
yum install meego-ai-flaster eat-host

* Get N900 flasher from the Nokia's tablets-dev portal
http://tablets-dev.nokia.com/maemo-dev-env-downloads.php?f=maemo_flasher-3.5_2.5.2.2.tar.gz
Compile the flasher

* Configure conductor to use rpm and fetch /var/log/messages
sudo nano /etc/ots/conductor.conf

* Test manually that everything is OK, this is executed in the worker
sudo conductor -u <image url>
* Start OTS worker
sudo ots_worker

Quality/QA-tools/OTS/Plugins/Commands

2011-11-18T08:33:27Z

Villen: Created page with "= Commands plug-in = Released in OTS 0.8.11. == Installation == Installation is made to OTS worker. 1. Install plug-in package sudo apt-get install python-ots-plugin-conduct..."

= Commands plug-in =

Released in OTS 0.8.11.

== Installation ==

Installation is made to OTS worker.

1. Install plug-in package
sudo apt-get install python-ots-plugin-conductor-commands
On Fedora
sudo yum install python-ots-plugin-conductor-commands

2. Configure commands plug-in settings
sudo nano /etc/ots/plugins/conductor_commands.conf

5. Restart OTS worker

Quality/QA-tools/OTS/DeveloperDocs/Plugins

2011-11-18T08:24:13Z

Villen: /* Conductor Plugins */

= Plugins =

== Overview ==

OTS currently consists of two distributed '''Applications''':

* Worker

* Server

These Applications are made up of '''Components'''. The '''Components''' perform a specific task in the system (and are distributable in their own right).

Some of the '''Components''' are essential for the '''Applications''' to operate. The '''Plugins''' are the non-essential (pluggable!) '''Components'''.

== Technology ==

OTS allows for a configurable distribution by using a plug-in mechanism using [http://peak.telecommunity.com/DevCenter/PkgResources Package Resources]

== Supported Plugins ==

Some of the plug-ins are requiring [[Quality/QA-tools/OTS/Plugins/Django|Django environment]]. This is mentioned in the plug-ins instructions.

=== Distribution Model Plugins ===

{|border="1" cellpadding="5"
|-
|'''Name'''||'''Entry point'''||'''Purpose'''
|-
|[[Quality/QA-tools/OTS/Plugins/History|History]]|| ots.plugin.history.distribution_model:get_model || Test package distribution model based on last execution time
|}

=== Publisher Plugins ===

{|border="1" cellpadding="5"
|-
|'''Name'''||'''Entry point'''||'''Purpose'''
|-
|[[Quality/QA-tools/OTS/Plugins/HTTP_logger|HTTP Logger]]|| ots.plugin.logger.logger_plugin:LoggerPlugin || Plug-in for enabling HTTP based logging and monitoring
|-
|[[Quality/QA-tools/OTS/Plugins/Email|Email]]|| ots.plugin.email.email_plugin:EmailPlugin || Plug-in for enabling email sending
|-
|[[Quality/QA-tools/OTS/Plugins/QAReports|QA-Reports]]|| ots.plugin.qareports.qareports_plugin:QAReportsPlugin || Plug-in for sending results to QA Reports
|-
|[[Quality/QA-tools/OTS/Plugins/History|History]]|| ots.plugin.history.history_plugin:HistoryPlugin || Plug-in for storing execution times to database
|-
|[[Quality/QA-tools/OTS/Plugins/Monitor|Monitor]]|| ots.plugin.monitor.monitor_plugin:MonitorPlugin || Plug-in for collecting statistical information
|}

=== Conductor Plugins ===

{|border="1" cellpadding="5"
|-
|'''Name'''||'''Entry point'''||'''Purpose'''
|-
|[[Quality/QA-tools/OTS/Plugins/Rich-core|Rich-core]]|| ots.plugin.conductor.richcore.richcore_plugin:RichCorePlugin || Plug-in for uploading rich-core dumps to post-processing. Rich-core plug-in is part of [[ARM/N900/QA#Crash_analysis_support_for_ARM_core_dumps|crash reporting functionality]].
|-
|[[Quality/QA-tools/OTS/Plugins/Commands|Commands]]|| ots.plugin.conductor.commands.commands_plugin:CommandsPlugin || Plug-in for running commands before, in between and after test packages.
|}

Quality/QA-tools/OTS

2011-11-09T07:28:50Z

Villen: /* Releases */

= OTS Index =

Welcome to the Open Test System (OTS) Wiki Index.

== Introduction ==

OTS is the '''Open Test System''' for testing software systems on evolving hardware architectures.

It was born out of the test framework incrementally developed for use on [http://maemo.org/ Maemo] project and is in use for 24/7 testing service by [http://www.nokia.com/ Nokia].

More detailed info can be found from [[Quality/QA-tools/OTS/About|about]] page.

== Releases ==

Current release [http://meego.gitorious.org/meego-quality-assurance/ots tag 0.8.10]. See [[Quality/QA-tools/OTS/UserDocumentation/Upgrading|upgrading]] instructions!

Next release [http://bugs.meego.com/show_bug.cgi?id=23933 0.8.11]

See OTS [[Quality/QA-tools/OTS/Roadmap| roadmap.]]

== Documentation ==

* [[Quality/QA-tools/OTS/UserDocumentation | User Documentation]]
** [[Quality/QA-tools/OTS/UserDocumentation/Installation | Installation Ubuntu]]
** [[Quality/QA-tools/OTS/UserDocumentation/Installation_rpm | Installation RPM]]
** [[Quality/QA-tools/OTS/UserDocumentation/Upgrading | Upgrading]]
* [[Quality/QA-tools/OTS/DeveloperDocs | Developer Documentation]]
* [[Quality/QA-tools/OTS/AdminDocs | System Administration]]
* [[Quality/QA-tools/OTS/Glossary | Glossary]]
* [[Quality/QA-tools/OTS/ExampleSetup | Example OTS Service Setup]]

Quality/QA-tools/OTS/UserDocumentation/Upgrading

2011-11-09T07:26:49Z

Villen: /* Upgrading OTS */

= Upgrading OTS =
== 0.8.9 - 0.8.10 ==
New conductor configuration parameter 'remote_logger'. Add it to existing conductor configuration files (/etc/ots/conductor.conf and /et/ots/conductor_*.conf) or replace old file(s) with new one. 'remote_logger' allows one to use amqp as logging channel instead of http. Currently http is the default logging method.

== 0.8.8 - 0.8.9 ==
Upgrade to meego-ai-flasher 0.1.8, older versions doesn't work with OTS 0.8.9.

== 0.8.7 - 0.8.8 ==
New conductor configuration parameter 'target_username'. Add it to existing conductor configuration files (/etc/ots/conductor.conf and /etc/ots/conductor_*.conf) or replace old file(s) with new one.

== 0.8.6 - 0.8.7 ==
Locations and names of configuration files have been changed. All configuration files are now located in /etc/ots and the plugin configuration files are in /etc/ots/plugins.

If you wish to keep your old settings you have to replace the new files.
On the server side:
sudo mv /etc/ots_server.ini /etc/ots/server.conf
sudo mv /etc/ots_plugin_qareports.conf /etc/ots/plugins/qareports.conf
And email plugin has it's own configuration file so you should move the [ots.email_plugin] section from the old /etc/ots_server.ini file to /etc/ots/plugins/email.conf.

On the worker side:
sudo mv /etc/ots.ini /etc/ots/worker.conf
sudo mv /etc/conductor.conf /etc/ots/conductor.conf
sudo mv /etc/ots_plugin_conductor_richcore.conf /etc/ots/plugins/conductor_richcore.conf

== 0.8.5 - 0.8.6 ==
Backup all configuration files and database before upgrading.

If you have logger plugin installed update the templates by giving following commands after upgrade:
sudo cp -r /usr/share/ots/django/* /var/www/
sudo cp -r /usr/share/ots/plugin/* /var/www/

Upgrade also the latest meego-ai-flasher.

=== Updating the database ===

If you want to keep your old log database, add new (user_defined_id) column to the log_messages table. Run database command line interface for OTS database. By default OTS uses sqlite database at /opt/ots/ots.sqlite.
sudo sqlite3 /opt/ots/ots.sqlite
Add new column to the log_messages table by executing following command:
ALTER TABLE log_messages ADD COLUMN user_defined_id varchar(255) NOT NULL DEFAULT <nowiki>''</nowiki>;

If there's no need to keep the old database you can remove the database file (/opt/ots/ots.sqlite) and generate a new one using [[Quality/QA-tools/OTS/Plugins/Django#Create.2Fupdate_databases|the instructions]].

== 0.8.4 - 0.8.5 ==
* Rich-core conductor plug-in available
** Changes to /etc/conductor.conf
** New configuration file for plug-in: /etc/ots_plugin_conductor_richcore.conf

== 0.8.3 - 0.8.4 ==
* Backup all configuration files before updating
* Multiple device support added, even if you use '''only one device''' check following:
** Changes to /etc/conductor.conf
** Update meego-ai-flasher (>= 0.1.4) and [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] packages!
** Even if you use only one device, remove usb0 configurations from /etc/network/interfaces
** [[Quality/QA-tools/OTS/UserDocumentation/Multiple_device| See this instructions!]]
* Changes to QAReports configuration file /etc/ots_plugins_qareports.conf
** Added proxy and attachment upload settings

== 0.8.2 - 0.8.3 ==
* Changes in ots_plugin_qareports.conf
** Add new parameters
* Only new features
** Test plan based execution
** Chroot support
** See [[Quality/QA-tools/OTS/UserDocumentation/Usage|how to]] use these features

== 0.8.1 - 0.8.2 ==
* New django package added, effects to all plug-ins
** See [[Quality/QA-tools/OTS/Plugins/Django|Django instructions]].
** See [[Quality/QA-tools/OTS/DeveloperDocs/Plugins|plug-ins instructions]].

== 0.8.0 - 0.8.1 ==
* Plug-in namespace changed, effects to Logger and QA-Reports plug-ins
** For logger update web page content, see step 8 from [[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger documentation]].
** For QA-Reports, configuration file name has changed. Move /etc/ots_qareports_plugin.conf to /etc/ots_plugin_qareports.conf

== 0.1.x - 0.8.0 ==

See [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration document]].

Quality/QA-tools/OTS/UserDocumentation/Usage

2011-07-12T11:06:35Z

Villen: /* Using OTS trigger */

= OTS Usage =

== Test execution ==

=== Using conductor ===

Conductor is the executer in the ots-worker. You can use conductor to test device flashing and for semi-automatic test executions.

To test flashing and basic test execution:
conductor --imageurl=http://192.168.1.1/image.tar.gz

The conductor flasher the image to the device and executes all available test packages.

To list all supported options:
conductor --help

=== Using OTS trigger ===

'''Note:''' Before starting to execute tests with ots_trigger, verify that your ots-worker is configured correctly. That can be done with conductor, see the topic above.

OTS trigger is tool for triggering a testrun to OTS server.

[[Quality/QA-tools/OTS/UserDocumentation/Installation#OTS_tools| How to install OTS trigger]].

Before you can trigger a testrun, you need to have all following:
* OTS server up-and-running [http://wiki.meego.com/Quality/QA-tools/OTS/UserDocumentation#Installation]
* OTS worker with hardware configured [http://wiki.meego.com/Quality/QA-tools/OTS/UserDocumentation#Installation]
* Image with eat-device package (enabled test automation environment to hardware) [http://wiki.meego.com/Quality/QA-tools/Autotest-guide]

And one of the following options:
* Test package installed to the device
* Test package installed to the ots-worker or
* Test plan [TODO LINK]

Okey, now we are ready to get dirty.

If we have test package(s) installed into the device and we want to execute all of them:
ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--email=my.name@company.com

Let me explain the parameters.
* '''server''': Defines the address of the OTS server. If you are not using logger plug-in, the address is then 192.168.1.1:8080.
* '''build_id''': Free string to define the image build id.
* '''image''': Url path where the image is located. Yes, it must be on the http server.
* '''sw_product''': The default SW test execution options, you have defined the sw_product in /etc/ots/server.conf.
* '''email''': Who has requested the testrun and to whom to send notifications.

That was simple. Next you want to execute only one package, no problem, just define test package name to OTS trigger:
ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--email=my.name@company.com --testpackages=my-test-package1-tests

In to the '''testpackages''' option, you can define multiple packages like this:
--testpackages=my-test-package1-tests,my-test-package2-tests

If you want to add more people to receive email after the execution is done, just add more email addresses to --email option:
ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--email=my.name@company.com,second.name@company.com

If you have test package installed to your ots-worker machine, that can be executed with --hostpackages:
ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--email=my.name@company.com --hostpackages=my-test-package1-tests

For optimizing the execution time, see distribution models section below.

To list all supported options:
ots_trigger --help

== Test plan based executions ==

Test plan based execution means that the user can create a test plan and give it directly to ots_trigger without making any test package. This brings benefits when executing different test against the same test image or release.

The user can create the test plan with [http://wiki.meego.com/Quality/QA-tools/Testplanner Testplanner].

And then give the test plan to ots_trigger with --deviceplan or --hostplan. Difference between these options are that 'deviceplan' is executed in the device and 'hostplan' in the ots-worker.
ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--email=my.name@company.com --deviceplan=/path/to/my/test_plan.xml

== Distribution models ==

These are models how the OTS is distributing test executions to ots-workers. There are 2 build-in models and other ones are distribution model plug-ins.

Distribution model can be specified to ots_trigger with --distribution option.

=== Default ===

This is the default model, if nothing else has been specified.

Default model executes all test packages in a single task, so only one ots-worker will handle the test execution. Only exception to this model is test plan based executions, they are always executed as an own task and therefor distributed to multiple workers.

=== Perpackage ===

'''---distribution=perpackage'''

Perpackage is the second build-in distribution model.

This models creates always an own task for each test package and test plan, so multiple ots-worker can handle the testrun at the same time.

=== Optimized ===

'''---distribution=optimized'''

Optimized is a distribution plug-in and it is part of history plug-in.

History plug-in collects test package execution times to a database. When the user triggers a testrun, he/she can specify how long the execution should take and how many device can be used. The optimized model then tries to optimize the execution time to fit the user's requirements.

See how to use the optimized plug-in from the [[Quality/QA-tools/OTS/Plugins/History|plug-in subpage]].

== Running tests in chroot environment ==

Chroot functionality makes possible to use chroot environment with specific software to run the tests. This way a test case which needs specific software can be started at any OTS worker and a worker will stay clean of additional software. Chroot based testing is primarily targeted at executing tests with the [[Quality/QA-tools/TDriver|Testability Driver framework]].

=== How to create rootstrap? ===

Basically you can use any i386 chroot environment you like, but MeeGo Netbook edition is good starting point. First [[Image_Creation|create an image]] with all packages you need for testing and then unpack and compress it to a tar.gz package:

mic-chroot --unpack-only –save-to=<chroot_directory> <image>
cd <chroot_directory>
tar -Pczf ../rootstrap.tar.gz .

Now you have a rootstrap ready in rootstrap.tar.gz file.

=== Triggering a chroot test ===

To trigger a chroot test you need to give two arguments for OTS trigger:
* --chrootpackages=<test_packages>
** Test packages to be executed on chroot environment
* --rootstrap=<rootstrap_file>
** The rootstrap URL

Example like this:

ots_trigger --server=192.168.1.1/xmlrpc --build_id=nightly_testing --image=http://url.to.image/image.tar.gz --sw_product=meego_n900 \
--chrootpackages=mwts-filesystem --rootstrap=http://url.to.rootstrap/rootstrap.tar.gz --email=my.name@company.com

Quality/QA-tools/OTS/UserDocumentation/Multiple device

2011-07-12T11:00:39Z

Villen: /* conductor configuration */

= Multiple device support =

== Introduction ==

This feature has been added to 0.8.4 version.

Multiple device support means that one OTS worker can handle multiple devices simultaneously and execute tests parallel. One worker can include multiple ots-worker instances and they work individually. Only limitation is amount of the switchbox ports (if using USB devices), this limitation in HAT switchbox is 2.

'''Note!''' For MeeGo N900, you need the latest [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] and [https://meego.gitorious.org/meego-quality-assurance/meego-ai-flasher meego-ai-flasher]!

There is a flasher plug-in example in the OTS source code under the example folder.

See example video from youtube: http://www.youtube.com/watch?v=exmJ6T-fjlM

== Setup and configuration ==

This guideline assumes that you have HAT switchbox and two N900 devices.

Install OTS worker normally, [[Quality/QA-tools/OTS/UserDocumentation|check installation guidelines]].

Remove the '''network-manager''' and remove all usb* settings from the '''/etc/network/interfaces'''.

=== Wiring ===

[[File:20110414_002.jpg]]

You need one HAT, two N900 devices and lots of cables:
* 3 x USB A-B cables from the PC to HAT
* 1 x Power cable to HAT
* 2 x Power cable from HAT to JIGs
* 2 x USB A-microB cables from HAT to JIGs

=== ots-worker configuration ===

Basically you need to start two ots-worker programs in one PC.

If both devices are using the same OTS server and queue, then you can use the default /etc/ots/worker.conf file. Start workers as '''root''' in different shells or screen windows:
ots_worker -n 1
ots_worker -n 2

If you want to dedicated the other device to some other server or queue, create a copy from the default worker.conf file and make your changes to that. Start workers as '''root''' in different shells or screen windows:
ots_worker -n 1
ots_worker -n 2 -c /your/new/worker.conf

Both workers should be able to connect to OTS server normally, check that they have different worker id:
ots.worker.worker - INFO - Starting the worker 1...

=== conductor configuration ===

You need always own conductor.conf per device. Copy the default conductor.conf to first device:
sudo cp /etc/ots/conductor.conf /etc/ots/conductor_1.conf

And for second device:
sudo cp /etc/ots/conductor.conf /etc/ots/conductor_2.conf

The most important thing in those configuration files is to define the default flasher and IP addresses. '''You cannot use 192.168.2.15 address for testing! That is used for setting up the devices!'''

Recommended IPs for devices:
* conductor_1.conf, host: 192.168.3.14, device: 192.168.3.15
* conductor_2.conf, host: 192.168.4.14, device: 192.168.4.15

Test that conductors are configured correctly and flashing is working:
sudo su
export OTS_WORKER_NUMBER=1
conductor -u http://url/to/image.bin -v

You should see something like:
...
INFO using config file /etc/ots/conductor_1.conf
...
INFO Loaded flasher 'meego-ai-flasher-n900'
...
INFO Host IP 192.168.3.14;Device IP 192.168.3.15
...

Quality/QA-tools/OTS/UserDocumentation/Multiple device

2011-07-12T10:59:10Z

Villen: /* ots-worker configuration */

= Multiple device support =

== Introduction ==

This feature has been added to 0.8.4 version.

Multiple device support means that one OTS worker can handle multiple devices simultaneously and execute tests parallel. One worker can include multiple ots-worker instances and they work individually. Only limitation is amount of the switchbox ports (if using USB devices), this limitation in HAT switchbox is 2.

'''Note!''' For MeeGo N900, you need the latest [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] and [https://meego.gitorious.org/meego-quality-assurance/meego-ai-flasher meego-ai-flasher]!

There is a flasher plug-in example in the OTS source code under the example folder.

See example video from youtube: http://www.youtube.com/watch?v=exmJ6T-fjlM

== Setup and configuration ==

This guideline assumes that you have HAT switchbox and two N900 devices.

Install OTS worker normally, [[Quality/QA-tools/OTS/UserDocumentation|check installation guidelines]].

Remove the '''network-manager''' and remove all usb* settings from the '''/etc/network/interfaces'''.

=== Wiring ===

[[File:20110414_002.jpg]]

You need one HAT, two N900 devices and lots of cables:
* 3 x USB A-B cables from the PC to HAT
* 1 x Power cable to HAT
* 2 x Power cable from HAT to JIGs
* 2 x USB A-microB cables from HAT to JIGs

=== ots-worker configuration ===

Basically you need to start two ots-worker programs in one PC.

If both devices are using the same OTS server and queue, then you can use the default /etc/ots/worker.conf file. Start workers as '''root''' in different shells or screen windows:
ots_worker -n 1
ots_worker -n 2

If you want to dedicated the other device to some other server or queue, create a copy from the default worker.conf file and make your changes to that. Start workers as '''root''' in different shells or screen windows:
ots_worker -n 1
ots_worker -n 2 -c /your/new/worker.conf

Both workers should be able to connect to OTS server normally, check that they have different worker id:
ots.worker.worker - INFO - Starting the worker 1...

=== conductor configuration ===

You need always own conductor.conf per device. Copy the default conductor.conf to first device:
sudo cp /etc/conductor.conf /etc/conductor_1.conf

And for second device:
sudo cp /etc/conductor.conf /etc/conductor_2.conf

The most important thing in those configuration files is to define the default flasher and IP addresses. '''You cannot use 192.168.2.15 address for testing! That is used for setting up the devices!'''

Recommended IPs for devices:
* conductor_1.conf, host: 192.168.3.14, device: 192.168.3.15
* conductor_2.conf, host: 192.168.4.14, device: 192.168.4.15

Test that conductors are configured correctly and flashing is working:
sudo su
export OTS_WORKER_NUMBER=1
conductor -u http://url/to/image.bin -v

You should see something like:
...
INFO using config file /etc/conductor_1.conf
...
INFO Loaded flasher 'meego-ai-flasher-n900'
...
INFO Host IP 192.168.3.14;Device IP 192.168.3.15
...

Quality/QA-tools/OTS

2011-07-12T09:47:55Z

Villen: /* Releases */

= OTS Index =

Welcome to the Open Test System (OTS) Wiki Index.

== Introduction ==

OTS is the '''Open Test System''' for testing software systems on evolving hardware architectures.

It was born out of the test framework incrementally developed for use on [http://maemo.org/ Maemo] project and is in use for 24/7 testing service by [http://www.nokia.com/ Nokia].

More detailed info can be found from [[Quality/QA-tools/OTS/About|about]] page.

== Releases ==

Current release [http://meego.gitorious.org/meego-quality-assurance/ots tag 0.8.7]. See [[Quality/QA-tools/OTS/UserDocumentation/Upgrading|upgrading]] instructions!

Next release [http://bugs.meego.com/show_bug.cgi?id=20753 0.8.8]

See OTS [[Quality/QA-tools/OTS/Roadmap| roadmap.]]

== Documentation ==

* [[Quality/QA-tools/OTS/UserDocumentation | User Documentation]]
** [[Quality/QA-tools/OTS/UserDocumentation/Installation | Installation Ubuntu]]
** [[Quality/QA-tools/OTS/UserDocumentation/Installation_rpm | Installation RPM]]
** [[Quality/QA-tools/OTS/UserDocumentation/Upgrading | Upgrading]]
* [[Quality/QA-tools/OTS/DeveloperDocs | Developer Documentation]]
* [[Quality/QA-tools/OTS/AdminDocs | System Administration]]
* [[Quality/QA-tools/OTS/Glossary | Glossary]]
* [[Quality/QA-tools/OTS/ExampleSetup | Example OTS Service Setup]]

Quality/QA-tools/OTS/Configuration

2011-07-11T09:56:34Z

Villen: /* Location */

= Configuration =

Each distributable '''Application''' has a configuration file.

== Location ==

The default location for the files is /etc/ots/
and they use the .conf postfix

== Components ==

The configuration of the '''Component''' is contained in sub-headings of the configuration file by name.

e.g. An ots.wiki would have the configuration parameters as follows

<code>
[ots.wiki]
foo = 1
bar = 2
baz = 3
</code>

== SW Products Default Options ==

The default [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options.py Options] can be configured to vary depending on the Software Product under test.

These defaults override the base defaults in Options.

'''WIP this is currently as a [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options_defaults.yaml YAML] file but is likely to change.

<code>
example_sw_product:
device:
devicegroup : examplegroup
</code>

== Format ==

=== ots.server.hub ===

==== The Timeout ====

The [[Quality/QA-tools/OTS/DeveloperDocs/OTS_Timeouts|Global Timeout from the Server]] is configured here.

<code>
timeout = 30
</code>

==== The Storage Host ====

<code>
storage_host = Hostname
storage_port = 1982
</code>

=== ots.server.xmlrpc ===

Configure the OTS server XMLRPC interface

<code>
host = "your_hostname_here"
port = 8080
</code>

Quality/QA-tools/OTS/Routing

2011-07-11T09:53:12Z

Villen: /* Worker */

== Testrun Routing ==

=== OTS 0.1.2 and Newer ===

Starting from version 0.1.2 OTS now supports additional device properties.

==== Worker ====

'''''The config file format has changed and is not compatible with older OTS versions!'''''

/etc/ots/worker.conf now includes new ''[Device]'' section. Devicegroup is still the only mandatory parameter but in addition to that it's possible to define devicename and deviceid.

<pre>
###########################################
# REPLACE `fix_me` WITH YOUR DEVICEGROUP
###########################################
[Device]
devicegroup = fix_me

#############################
# Optional device properties
#############################

#devicename = name of the device type
#deviceid = Specific HW id of the device
</pre>

Workers are now able to listen to multiple queues. Queues for the worker are generated from these 3 parameters:

* Only devicegroup defined:
** ''devicegroup''
* Devicegroup and devicename defined:
** ''devicegroup''
** ''devicegroup.devicename''
* Devicegroup, devicename and deviceid defined:
** ''devicegroup''
** ''devicegroup.devicename''
** ''devicegroup.devicename.deviceid''

Deviceid is a subproperty of devicename so it is not possible to define deviceid only. An exception will be raised in that situation.

The more specific queues have higher priority than the more generic ones.

==== Server ====

On server side the routing key for a particular testrun is defined from the properties as follows:

* Only devicegroup defined:
** ''devicegroup''
* Devicegroup and devicename defined:
** ''devicegroup.devicename''
* Devicegroup, devicename and deviceid:
** ''devicegroup.devicename.deviceid''

Defining a deviceid without devicename will raise an exception and cause the testrun to fail.

The format of the device option is:

devicegroup:<GROUP_1> devicename:<NAME> deviceid:<ID>;devicegroup:<GROUP_1> devicename:<NAME> deviceid:<ID>

This triggers two separate testruns.

For example with ots_trigger the -d option can be used as follows:

* Route the testrun to group handset, use n900 device with deviceid 1

-d "devicegroup:handset devicename:n900 deviceid:1"

* Route the testrun to default devicegroup, use n900 device, don't care about deviceid (Default devicegroup is defined in ots server config)

-d "devicename:n900"

* Route the testrun to "meego_coffee" devicegroup, don't care about devicename or deviceid.

-d "devicegroup:meego_coffee"

=== OTS 0.1.1 and Older ===

Testruns are routed to workers based on devicegroup. Devicegroup is directly used as the AMQP routing key and queue name. AMQP values are directly defined in the ots.ini config file.

Quality/QA-tools/OTS/Plugins/Rich-core

2011-07-11T09:44:16Z

Villen: /* Installation */

= Rich-core plug-in =

Released in OTS 0.8.5.

== Installation ==

Installation is made to OTS worker.

1. Install plug-in package
sudo apt-get install python-ots-plugin-conductor-richcore
On Fedora
sudo yum install python-ots-plugin-conductor-richcore

2. Make sure that testrunner-lite 1.6.1 or newer is installed

3. Set rich_core_dumps_folder parameter for conductor
sudo nano /etc/ots/conductor.conf

4. Configure rich-core plug-in settings
sudo nano /etc/ots/plugins/conductor_richcore.conf

5. Restart OTS worker

Quality/QA-tools/OTS/Plugins/QAReports

2011-07-11T09:43:02Z

Villen: /* Installation */

= QA-Reports plugin =

Released in OTS 0.8.1.

== Installation ==

Installation is made to OTS server.

1. Install plug-in
sudo apt-get install python-ots-plugin-qareports
Fedora:
sudo yum install python-ots-plugin-qareports
2. Instructions how to get token ring
[[Quality/QA-tools/QAReports/API]]
3. Configure QA-Reports settings
sudo nano /etc/ots/plugins/qareports.conf
4a. Restart apache
sudo service apache2 restart
Fedora:
sudo /etc/init.d/httpd restart
4b. Restart OTS server

== Usage ==

Default settings are taken from the configuration file, but if you want to define a testrun specific parameters, you can use "--options" command line parameter in ots_trigger:
<pre>
ots_trigger --options="qa_hwproduct:<HWPRODUCT> qa_testtype:<TESTTYPE> qa_target:<TARGET> qa_release_version:<RELEASE_VERSION>"
</pre>

Example:
ots_trigger --options="qa_hwproduct:N900 qa_testtype:Nightly qa_target:Core qa_release_version:1.2"

Quality/QA-tools/OTS/Plugins/Email

2011-07-11T09:41:19Z

Villen: /* Installation */

= Email plug-in =

Released in OTS 0.8.0.

== Installation ==

Installation is made to OTS server.

1. Install plug-in package
sudo apt-get install python-ots-plugin-email
Fedora
sudo yum install python-ots-plugin-email
2. Configure email settings
sudo nano /etc/ots/plugins/email.conf
3a. Restart apache
sudo service apache2 restart
Fedora:
sudo /etc/init.d/httpd restart
3b. Restart OTS server

Quality/QA-tools/OTS/DeveloperDocs

2011-07-11T09:38:51Z

Villen: /* XmlFileFormats */

= OTS - Open Testing Service =

== Related Documentation ==

* [[Quality/QA-tools/OTS/DevelopmentGuidelines | Development Guidelines]]
* [[Quality/QA-tools/OTS/Packages | Packages]]
* [[Quality/QA-tools/OTS/ErrorCodes | Error Codes]]
* [[Quality/QA-tools/OTS/Glossary]]
* [[Quality/QA-tools/OTS/DeveloperDocs/Transition0.1-0.8|Making transition from 0.1 to 0.8]]

== Synopsis ==

OTS is a Distributed Test System that takes a Data Driven Approach using XML Files to allow parallel testing of Applications on evolving Hardware platforms.

Design has been driven by the need for a language neutral test platform that allows testing under different device architectures in parallel.

== High Level Overview ==

OTS allows a Test Sequence to be fanned out from a hub known as a '''Distributor''' to a remote machine known as a '''Worker'''.
Each '''Worker''' has a '''Device Group''' associated with it, this is defined in the configuration. The Tests are routed to the '''Worker'''
on the basis of the '''Device Group'''.

An XML '''Test Definition File''' describes the Tests that are run. The results are returned to the '''Distributor''' in an XML '''Results File'''.

The '''Distributor''' takes a '''Device Group''' and '''Timeout''' as inputs, as well as the '''Command''' for running '''Tasks'''. Tests are run within a '''Task''' which
is currently run as a process. The '''Command''' will typically take a path to the '''Test Definition File''' as an input.
If the process exceeds the '''Timeout''' the '''Task''' is stopped (the process is killed).

A number of '''Tasks''' can be added to a '''Testrun'''. The '''Testrun''' will normally wait for all '''Tasks''' to finish before completion.

Data (status, error and results) are communicated back from the '''Task''' to the '''Worker'''. The '''Results Object''' is a container for the files outputted: '''Test Definition File''', '''Results File'''
and any other files produced by the test as well as metadata relating to the run.

== Technology ==

OTS is written in [http://www.python.org/ Python 2.6].

[http://www.amqp.org/confluence/display/AMQP/Advanced+Message+Queuing+Protocol AMQP] protocol is used for communication.

=== XmlFileFormats ===

The XML file formats provide the datum for the system.

Download [http://meego.gitorious.org/meego-quality-assurance/test-definition/trees/master them] and add the path to the '''results_xsd''' parameter in your [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/master/ots.server/ots/server/server.conf server.conf] file

=== Third Party Dependencies ===

==== RabbitMQ ====

You need to install [http://www.rabbitmq.com/server.html RabbitMQ], this is AMQP message server.

===== Managing the Queues =====

If you need to delete or empty the queues there are [http://meego.gitorious.org/meego-quality-assurance/ots/trees/master/ots.server/ots/server/distributor/dev_utils utilities] to help with this.

==== Python Packages ====

The following python packages are needed

* [http://pypi.python.org/pypi/setuptools setuptools]

* [http://www.voidspace.org.uk/python/configobj.html configobj]

* [http://pypi.python.org/pypi/multiprocessing multiprocessing]

* [http://code.google.com/p/py-amqplib/ pyamqplib] (version 0.6.1 or newer)

* [http://www.familieleuthe.de/DownloadMiniXsv.html minixsv]

* [http://www.djangoproject.com/download/1.1.2/tarball/ django]

* [http://code.google.com/p/python-nose/ nose]

== Diving In ==

The following steps show how to get a Development Environment up and running

=== 1. Install Dependencies ===

* Install the Third Party Dependencies

* Get the XML File Formats and set the OTS_TESTDEFINITION environment variable

=== 2. Build the Eggs ===

From the root directory

<code>
./setup.sh
source ./paths.sh
</code>

=== 3. Unittests ===

Run the unittests from the root directory

<code>
./nose.sh
</code>

=== 4. Component Tests ===

Component Tests run high level tests on an OTS Component using Mocks of the dependencies.

You can run these with the component_tests shell script in the root directory.

<code>
./component_tests.sh
</code>

=== 5. Hello OTS World ===

The source contains a simple demonstration of the way the key elements work.

==== 1. Open two terminal windows. ====

==== 2. In the first window run the server ====

<code>
cd $SERVER
cd hub/demos
python demo_hub.py
</code>

==== 3. You should see this: ====

<code>
<snip>
20XX-XX-XX XX:XX:XX,XXX - ots.server.distributor.taskrunner - DEBUG - Sending command '['echo', 'hello world']' with key 'foo'
</code>

==== 4. In the second window run the worker ====

<code>
cd $WORKER
python worker.py -c config.ini
</code>

==== 5. Now in the Server terminal the command should run to completion. You should see the following line in the logs: ====

<code>
20XX-XX-XX XX:XX:XX,XXX - ots.worker.command - DEBUG - process.communicate() returned echo hello world
</code>

It will raise a PackageException. This is normal as no Tasks that are run return Packages to the Server

==== 6. And on the logs on the Worker side should contain: ====

<code>
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.command - DEBUG - process.communicate() returned echo hello world
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.task_broker - DEBUG - Task in state: 'finish'
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.task_broker - INFO - Recommence consume on queue: foo
</code>

== Installation ==

It is intended that the '''Distributor''' and each '''Worker''' should be installed to separate machines.

== Configuration ==

=== Distributor ===

The format of the configuration file is as follows.

The `host` is the name of the RabbitMQ server

<code>

[Client]
host = localhost
vhost = /
port = 5672
username = guest
password = guest
queue = FIXME
consumer_tag = worker
services_exchange = services

</code>

=== Worker ===

Edit your config.ini so that the

* `queue`
* `routing_key`
* `services_exchange`

Are set for your '''Device Group'''.

And the `host` is the name of your RabbitMQ server.

<code>
[Worker]
host = localhost
vhost = /
port = 5672
username = guest
password = guest
queue = foo
routing_key = foo
services_exchange = foo
</code>

==== Starting the Worker ====

To start your Worker:

<code>
python worker.py -c config.ini
</code>

== Automated System Tests ==

Automated system tests are available in ots.system_tests. They trigger various testruns with xmlrpc and check return value and log messages.

=== Requirements ===

* Fully functional OTS system with one worker.
* A SW Product "ots-system-tests" configured so that it defaults to the worker.
* A meego image with test-definition-tests and testrunner-lite-regression-tests.
* Django based advanced OTS setup.
* BeautifulSoup python library.

=== Configuration ===

* make the system test image available for the workers. (for example copy it to your apache static content directory /var/www/static/)
* Modify system_tests.conf in ots.system_tests directory to match your system.
* Make sure all the urls in system_tests.conf work.

=== Notes ===

* System tests execute multiple full testruns. Running all tests can easily take more than 1.5 hours with N900 worker.
* Make sure there's no other activities ongoing in the system while running the tests.
* Some error handling tests expect certain devicegroups and sw products _not_ to be available. If these test cases fail you can remove those queues with ots tools.

== License ==

OTS is distributed under an LGPL license.

== Contributions ==

OTS welcomes contributions.

The following areas would benefit greatly from your help:

* Adding to the list of freely available Publisher Plugins
* Examples of OTS in use with different hardware architectures
* Improving the ease of deployment

Please check out the [[Quality/QA-tools/OTS/DevelopmentGuidelines | Guidelines]] and [[Quality/QA-tools/OTS/DevelopmentGuidelines/Review | Review process]].

== Wiki ==

[[Quality/QA-tools/OTS | OTS Wiki Home]]

== Source Code Documentation ==

OTS docstrings are in [http://epydoc.sourceforge.net/manual-epytext.html epytext] format which enables the use of [http://epydoc.sourceforge.net/index.html epydoc] in API documentation generation.

== Mailing List ==

[http://lists.meego.com/listinfo/meego-qa MeeGo QA] mailing list

== IRC Channel ==

The #meego-qa irc channel in freenode

== Bugs ==

The [http://bugs.meego.com/ Meego bugs page]

== Platform Dependencies ==

OTS is tested to run on Linux only.

Tested on Ubuntu (8.04, 10.04, 10.10), Fedora 13, MeeGo and Redhat.

Binary support for Ubuntu, Fedora and MeeGo.

== Architecture ==

* [[Quality/QA-tools/OTS/DeveloperDocs/Components | Components]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Server | Server]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Plugins | Plugins]]

* [[Quality/QA-tools/OTS/DeveloperDocs/DataFlow | DataFlow]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Behaviour | Behaviour]]

== APIs ==

==== Server ====

The top level [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/api.py api] for external clients utilises parameters in a dictionary.

The parameters are explicitly defined in [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options.py options.py]. The conversion from the loosely defined dictionary to a clear API is undertaken by the [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options_factory.py options_factory.py]

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/distributor/api.py Distributor] ====

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.results/ots/results/api.py Results] ====

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.worker/ots/worker/api.py Worker] ====

== Extending OTS ==

You can extend OTS with your own [[Quality/QA-tools/OTS/CreatingAPublisher|Publisher]]
and with [[Quality/QA-tools/OTS/DeveloperDocs/CreatingACustomDistributionModel|custom package distribution models.]]

== Experimental ==

Documentation for experimental branches, spikes, WIPs etc. can be found in [[Quality/QA-tools/OTS/DeveloperDocs/Experimental]].

== Analysis ==

[[Quality/QA-tools/OTS/DeveloperDocs/ErrorSituations| OTS 0.1 Error Situations]]

[[Quality/QA-tools/OTS/DeveloperDocs/TypicalUsage| Typical OTS users and usage scenarios]]

== Project Planning ==

* [[Quality/QA-tools/OTS/Roadmap | Roadmap]]
* [[Quality/QA-tools/OTS/Meetings | Meetings]]
* [[Quality/QA-tools/OTS/ReleaseChecklist | ReleaseChecklist]]

==== Release Practices ====

OTS releases are done when a new release makes sense from OTS point of view. There are no scheduled releases.

The [[Quality/QA tools development#Release_Practices | QA Tools Release Practices]] gives more details.

Quality/QA-tools/OTS/UserDocumentation/Installation rpm

2011-07-11T09:33:28Z

Villen: /* OTS 0.8.x installation RPM */

= OTS 0.8.x installation RPM =

These instructions have been tested with Fedora 13. Execute commands as root.

In MeeGo netbook, at least following packages needs to be installed manually:
python-amqp, python-django, minixsv, rabbitmq-server

== Network Setup ==

Using static IP addresses on all OTS machines is strongly recommended. It is possible to setup the system also with dynamic IP addresses but it might cause problems every now and then because IP's and hostnames change.

'''Note''' All OTS machines need to be able to access other OTS machines based on their hostname.

How to check that DNS names are correct:

1. Check OTS server's hostname
hostname
2. Ping the OTS server's hostname from the OTS worker
ping <hostname>

If ping fails, add manually OTS server's hostname to OTS workers '''/etc/hosts''' file
<IP> <hostname>

== Preparation ==

There are two ways of getting OTS binary packages: from '''the source code'''
or from the '''MeeGo Tools:Testing repository'''. From the repository you can get
the stable version and from the source code the development version.

* Add Tools:Testing source to /etc/yum.repos.d/
cd /etc/yum.repos.d/
yum install wget nano
wget http://download.meego.com/live/Tools:/Testing/Fedora13/Tools:Testing.repo

* Update the repositories
yum check-update

'''Build packages from source (optional)'''
* Install git core
yum install git-core

* Get sources
git clone git://gitorious.org/meego-quality-assurance/ots.git

* List tags
git tag -l

* Checkout the latest tag
git checkout <tag_name>

* Get building tools
yum rpm-build python-setuptools

* Build packages
mv ots python-ots-0.8.x
tar czf python-ots-0.8.x.tar.gz python-ots-0.8.x
rpmbuild -tb python-ots-0.8.x.tar.gz

== OTS server installation ==

* Install dependencies
yum install Django python-amqplib python-configobj rabbitmq-server test-definition
easy_install minixsv
yum install python-setuptools

* Install OTS server components
yum install python-ots-common python-ots-results python-ots-server
or if you are using own built packages
rpm -i ...

* Add rabbitmq to start on boot
chkconfig --add rabbitmq-server
chkconfig rabbitmq-server on

* Start rabbitmq server
/etc/init.d/rabbitmq-server start

* Configure OTS server settings
sudo nano /etc/ots/server.conf

Recommended to change following:
'example_sw_product', example to 'meego_product'
'devicegroup', example to 'meego_n900'

* Install OTS server plug-ins

See plug-ins from [[Quality/QA-tools/OTS/Plugins|OTS Plug-ins]]

* Start OTS server
ots_server

== OTS worker installation ==

* Install dependencies
yum install python-amqplib testrunner-lite

* Install OTS worker packages
yum install python-ots-common python-ots-worker

* Configure OTS worker settings
sudo nano /etc/ots/worker.conf

Change 'devicegroup', example meego_n900

Change 'host' to server's address

* Start OTS worker (as root)
ots_worker

NOTE: if worker failed to start up with error like "socket.error: [Errno 113] No route to host", check that ots-worker is able to connect to ots-server

== OTS tools ==

Tools are used for triggering test runs from the command line.

* Install OTS tools
yum install python-ots-tools

* Trigger a test run
ots_trigger -s localhost:8080 -b test_build -i http://url_to_image -p example_sw_product -e test@man.com

* Kickstart file for automation can be found from
http://meego.gitorious.org/meego-quality-assurance/enables-automated-testing/blobs/ks/meego-n900-autotest.ks

* You need to build the image with mic-image-creator with parameters:
--save-kernel --package=tar.gz

== How to install N900 automation tools to OTS worker ==

* Install dependencies
yum install meego-ai-flaster eat-host

* Get N900 flasher from the Nokia's tablets-dev portal
http://tablets-dev.nokia.com/maemo-dev-env-downloads.php?f=maemo_flasher-3.5_2.5.2.2.tar.gz
Compile the flasher

* Configure conductor to use rpm and fetch /var/log/messages
sudo nano /etc/ots/conductor.conf

* Test manually that everything is OK, this is executed in the worker
sudo conductor -u <image url>
* Start OTS worker
sudo ots_worker

Quality/QA-tools/OTS/UserDocumentation/Installation

2011-07-11T09:30:34Z

Villen: /* OTS 0.8.x installation Ubuntu */

= OTS 0.8.x installation Ubuntu =

If using 0.1.x version, see [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration from 0.1.x to 0.8.x]] document.

== Used hardware ==

:1 x Server (Ubuntu 10.10)
:1 x Client (Ubuntu 10.10)
::1 x HAT controller
:: 1 x Power cable
:: 1 x Power supply
:: 2 x USB A-B cable
:: 1 x USB A-microB cable
:: 1 x N900

== Network Setup ==

Using static IP addresses on all OTS machines is strongly recommended. It is possible to setup the system also with dynamic IP addresses but it might cause problems every now and then because IP's and hostnames change.

'''Note''' All OTS machines need to be able to access other OTS machines based on their hostname.

How to check that DNS names are correct:

1. Check OTS server's hostname
hostname
2. Ping the OTS server's hostname from the OTS worker
ping <hostname>

If ping fails, add manually OTS server's hostname to OTS workers '''/etc/hosts''' file
<IP> <hostname>

== Preparation ==

There are two ways of getting OTS binary packages: from '''the source code'''
or from the '''MeeGo Tools:Testing repository'''. From the repository you can get
the stable version and from the source code the development version.

=== Set up Tools:Testing repository ===

By setting up the repository it allows you to install OTS and other MeeGo QA tools via apt-get command. Follow the [[Quality/QA-tools/How_to_set_up_repositories|instructions]].

=== Build packages from source (optional) ===

If you which to build your own packages and install OTS from them here are the instructions. But it is recommended to install OTS using the repository.

* Install git core
sudo apt-get install git-core

* Get sources
git clone git://gitorious.org/meego-quality-assurance/ots.git

* Get building tools
sudo apt-get install dpkg-dev debhelper python-setuptools

* Build packages
cd ots
dpkg-buildpackage -rfakeroot

== OTS server installation ==

* Install dependencies
sudo apt-get install python-django python-amqplib python-configobj rabbitmq-server test-definition

* Install OTS server components
sudo apt-get install python-ots-common python-ots-results python-ots-server
Or if you are using own built packages
sudo dpkg -i ...

* Configure OTS server settings
sudo nano /etc/ots/server.conf

Recommended to change following:
'example_sw_product', example to 'meego_product'
'devicegroup', example to 'meego_n900'

* Install OTS server plug-ins
[[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger plug-in]] (recommended)
[[Quality/QA-tools/OTS/Plugins/Email|Email plug-in]] (recommended)

See more plug-ins from [[Quality/QA-tools/OTS/Plugins|OTS Plug-ins]]

* If logger plug-in is installed
Change permissions to logging directory
sudo chown www-data:www-data /var/log/ots

Now the OTS server can be used via apache (django_xmlrpc).

* If logger plug-in is not installed
Change user permissions to logging directory
sudo chown <user>:<group> /var/log/ots

* Now you can start OTS server
ots_server

== OTS worker installation ==

* Install dependencies
sudo apt-get install python-amqplib testrunner-lite

* Install OTS worker packages
sudo apt-get install python-ots-common python-ots-worker

* Configure OTS worker settings
sudo nano /etc/ots/worker.conf

Change 'devicegroup', example meego_n900

Change 'host' to server's address

* Start OTS worker
sudo ots_worker

== OTS tools ==

Tools are used for triggering test runs from the command line.

* Install OTS tools
sudo apt-get install python-ots-tools

* Trigger a test run
ots_trigger -s localhost/xmlrpc -b test_build -i http://url_to_image -p example_sw_product -e test@man.com

== How to install N900 automation tools to OTS worker ==

* Remove network-manager, because it breaks the USB connection. See more information from autotest-guide.

* Install dependencies
sudo apt-get install meego-ai-flasher eat-host-ubuntu

* Get N900 flasher from the Nokia's tablets-dev portal
http://tablets-dev.nokia.com/maemo-dev-env-downloads.php?f=maemo_flasher-3.5_2.5.2.2_i386.deb
sudo dpkg -i maemo_flasher-3.5_2.5.2.2_i386.deb

* Start HAT driver
sudo hat_drv &

* Test manually that everything is OK, this is executed in the worker
sudo conductor -u <image url>

* Start OTS worker
sudo ots_worker

Quality/QA-tools/OTS/UserDocumentation/Installation

2011-07-11T09:29:32Z

Villen: /* OTS server installation */

= OTS 0.8.x installation Ubuntu =

If using 0.1.x version, see [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration from 0.1.x to 0.8.x]] document.

== Used hardware ==

:1 x Server (Ubuntu 10.10)
:1 x Client (Ubuntu 10.10)
::1 x HAT controller
:: 1 x Power cable
:: 1 x Power supply
:: 2 x USB A-B cable
:: 1 x USB A-microB cable
:: 1 x N900

== Network Setup ==

Using static IP addresses on all OTS machines is strongly recommended. It is possible to setup the system also with dynamic IP addresses but it might cause problems every now and then because IP's and hostnames change.

'''Note''' All OTS machines need to be able to access other OTS machines based on their hostname.

How to check that DNS names are correct:

1. Check OTS server's hostname
hostname
2. Ping the OTS server's hostname from the OTS worker
ping <hostname>

If ping fails, add manually OTS server's hostname to OTS workers '''/etc/hosts''' file
<IP> <hostname>

== Preparation ==

There are two ways of getting OTS binary packages: from '''the source code'''
or from the '''MeeGo Tools:Testing repository'''. From the repository you can get
the stable version and from the source code the development version.

=== Set up Tools:Testing repository ===

By setting up the repository it allows you to install OTS and other MeeGo QA tools via apt-get command. Follow the [[Quality/QA-tools/How_to_set_up_repositories|instructions]].

=== Build packages from source (optional) ===

If you which to build your own packages and install OTS from them here are the instructions. But it is recommended to install OTS using the repository.

* Install git core
sudo apt-get install git-core

* Get sources
git clone git://gitorious.org/meego-quality-assurance/ots.git

* Get building tools
sudo apt-get install dpkg-dev debhelper python-setuptools

* Build packages
cd ots
dpkg-buildpackage -rfakeroot

== OTS server installation ==

* Install dependencies
sudo apt-get install python-django python-amqplib python-configobj rabbitmq-server test-definition

* Install OTS server components
sudo apt-get install python-ots-common python-ots-results python-ots-server
Or if you are using own built packages
sudo dpkg -i ...

* Configure OTS server settings
sudo nano /etc/ots/server.conf

Recommended to change following:
'example_sw_product', example to 'meego_product'
'devicegroup', example to 'meego_n900'

* Install OTS server plug-ins
[[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger plug-in]] (recommended)
[[Quality/QA-tools/OTS/Plugins/Email|Email plug-in]] (recommended)

See more plug-ins from [[Quality/QA-tools/OTS/Plugins|OTS Plug-ins]]

* If logger plug-in is installed
Change permissions to logging directory
sudo chown www-data:www-data /var/log/ots

Now the OTS server can be used via apache (django_xmlrpc).

* If logger plug-in is not installed
Change user permissions to logging directory
sudo chown <user>:<group> /var/log/ots

* Now you can start OTS server
ots_server

== OTS worker installation ==

* Install dependencies
sudo apt-get install python-amqplib testrunner-lite

* Install OTS worker packages
sudo apt-get install python-ots-common python-ots-worker

* Configure OTS worker settings
sudo nano /etc/ots.ini

Change 'devicegroup', example meego_n900

Change 'host' to server's address

* Start OTS worker
sudo ots_worker

== OTS tools ==

Tools are used for triggering test runs from the command line.

* Install OTS tools
sudo apt-get install python-ots-tools

* Trigger a test run
ots_trigger -s localhost/xmlrpc -b test_build -i http://url_to_image -p example_sw_product -e test@man.com

== How to install N900 automation tools to OTS worker ==

* Remove network-manager, because it breaks the USB connection. See more information from autotest-guide.

* Install dependencies
sudo apt-get install meego-ai-flasher eat-host-ubuntu

* Get N900 flasher from the Nokia's tablets-dev portal
http://tablets-dev.nokia.com/maemo-dev-env-downloads.php?f=maemo_flasher-3.5_2.5.2.2_i386.deb
sudo dpkg -i maemo_flasher-3.5_2.5.2.2_i386.deb

* Start HAT driver
sudo hat_drv &

* Test manually that everything is OK, this is executed in the worker
sudo conductor -u <image url>

* Start OTS worker
sudo ots_worker

Quality/QA-tools/OTS/UserDocumentation/Upgrading

2011-07-11T09:27:23Z

Villen: /* Upgrading OTS */

= Upgrading OTS =

== 0.8.6 - 0.8.7 ==
Locations and names of configuration files have been changed
All configuration files are now located in /etc/ots
And the plugin configuration files are in /etc/ots/plugins

If you wish to keep your old settings you have to replace the new files.
On the server side:
sudo mv /etc/ots_server.ini /etc/ots/server.conf
sudo mv /etc/ots_plugin_qareports.conf /etc/ots/plugins/qareports.conf
And email plugin has it's own configuration file so you should move the [ots.email_plugin] section from the old /etc/ots_server.ini file to /etc/ots/plugins/email.conf

On the worker side:
sudo mv /etc/ots.ini /etc/ots/worker.conf
sudo mv /etc/conductor.conf /etc/ots/conductor.conf
sudo mv /etc/ots_plugin_conductor_richcore.conf /etc/ots/plugins/conductor_richcore.conf

== 0.8.5 - 0.8.6 ==
Backup all configuration files and database before upgrading.

If you have logger plugin installed update the templates by giving following commands after upgrade:
sudo cp -r /usr/share/ots/django/* /var/www/
sudo cp -r /usr/share/ots/plugin/* /var/www/

Upgrade also the latest meego-ai-flasher.

=== Updating the database ===

If you want to keep your old log database, add new (user_defined_id) column to the log_messages table. Run database command line interface for OTS database. By default OTS uses sqlite database at /opt/ots/ots.sqlite.
sudo sqlite3 /opt/ots/ots.sqlite
Add new column to the log_messages table by executing following command:
ALTER TABLE log_messages ADD COLUMN user_defined_id varchar(255) NOT NULL DEFAULT <nowiki>''</nowiki>;

If there's no need to keep the old database you can remove the database file (/opt/ots/ots.sqlite) and generate a new one using [[Quality/QA-tools/OTS/Plugins/Django#Create.2Fupdate_databases|the instructions]].

== 0.8.4 - 0.8.5 ==
* Rich-core conductor plug-in available
** Changes to /etc/conductor.conf
** New configuration file for plug-in: /etc/ots_plugin_conductor_richcore.conf

== 0.8.3 - 0.8.4 ==
* Backup all configuration files before updating
* Multiple device support added, even if you use '''only one device''' check following:
** Changes to /etc/conductor.conf
** Update meego-ai-flasher (>= 0.1.4) and [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] packages!
** Even if you use only one device, remove usb0 configurations from /etc/network/interfaces
** [[Quality/QA-tools/OTS/UserDocumentation/Multiple_device| See this instructions!]]
* Changes to QAReports configuration file /etc/ots_plugins_qareports.conf
** Added proxy and attachment upload settings

== 0.8.2 - 0.8.3 ==
* Changes in ots_plugin_qareports.conf
** Add new parameters
* Only new features
** Test plan based execution
** Chroot support
** See [[Quality/QA-tools/OTS/UserDocumentation/Usage|how to]] use these features

== 0.8.1 - 0.8.2 ==
* New django package added, effects to all plug-ins
** See [[Quality/QA-tools/OTS/Plugins/Django|Django instructions]].
** See [[Quality/QA-tools/OTS/DeveloperDocs/Plugins|plug-ins instructions]].

== 0.8.0 - 0.8.1 ==
* Plug-in namespace changed, effects to Logger and QA-Reports plug-ins
** For logger update web page content, see step 8 from [[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger documentation]].
** For QA-Reports, configuration file name has changed. Move /etc/ots_qareports_plugin.conf to /etc/ots_plugin_qareports.conf

== 0.1.x - 0.8.0 ==

See [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration document]].

Quality/QA-tools/OTS/UserDocumentation/Upgrading

2011-06-13T05:28:55Z

Villen: /* Upgrading OTS */

= Upgrading OTS =

== 0.8.5 - 0.8.6 ==
* If you want to keep your old log database, add new (user_defined_id) column to the log_messages table
Run database command line interface for ots database. By default ots uses sqlite database at /opt/ots/ots.sqlite
sudo sqlite3 /opt/ots/ots.sqlite
Add new column to the log_messages table by executing following command:
ALTER TABLE log_messages ADD COLUMN user_defined_id varchar(255) NOT NULL DEFAULT <nowiki>''</nowiki>;

== 0.8.4 - 0.8.5 ==
* Rich-core conductor plug-in available
** Changes to /etc/conductor.conf
** New configuration file for plug-in: /etc/ots_plugin_conductor_richcore.conf

== 0.8.3 - 0.8.4 ==
* Backup all configuration files before updating
* Multiple device support added, even if you use '''only one device''' check following:
** Changes to /etc/conductor.conf
** Update meego-ai-flasher (>= 0.1.4) and [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] packages!
** Even if you use only one device, remove usb0 configurations from /etc/network/interfaces
** [[Quality/QA-tools/OTS/UserDocumentation/Multiple_device| See this instructions!]]
* Changes to QAReports configuration file /etc/ots_plugins_qareports.conf
** Added proxy and attachment upload settings

== 0.8.2 - 0.8.3 ==
* Changes in ots_plugin_qareports.conf
** Add new parameters
* Only new features
** Test plan based execution
** Chroot support
** See [[Quality/QA-tools/OTS/UserDocumentation/Usage|how to]] use these features

== 0.8.1 - 0.8.2 ==
* New django package added, effects to all plug-ins
** See [[Quality/QA-tools/OTS/Plugins/Django|Django instructions]].
** See [[Quality/QA-tools/OTS/DeveloperDocs/Plugins|plug-ins instructions]].

== 0.8.0 - 0.8.1 ==
* Plug-in namespace changed, effects to Logger and QA-Reports plug-ins
** For logger update web page content, see step 8 from [[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger documentation]].
** For QA-Reports, configuration file name has changed. Move /etc/ots_qareports_plugin.conf to /etc/ots_plugin_qareports.conf

== 0.1.x - 0.8.0 ==

See [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration document]].

Quality/QA-tools/OTS/UserDocumentation/Upgrading

2011-06-10T12:24:42Z

Villen: /* 0.8.5 - 0.8.6 */

= Upgrading OTS =

== 0.8.5 - 0.8.6 ==
* If you want to keep your old log database, add new (user_defined_id) column to the log_messages table
ALTER TABLE log_messages ADD COLUMN user_defined_id varchar(255) NOT NULL DEFAULT <nowiki>''</nowiki>;

== 0.8.4 - 0.8.5 ==
* Rich-core conductor plug-in available
** Changes to /etc/conductor.conf
** New configuration file for plug-in: /etc/ots_plugin_conductor_richcore.conf

== 0.8.3 - 0.8.4 ==
* Backup all configuration files before updating
* Multiple device support added, even if you use '''only one device''' check following:
** Changes to /etc/conductor.conf
** Update meego-ai-flasher (>= 0.1.4) and [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] packages!
** Even if you use only one device, remove usb0 configurations from /etc/network/interfaces
** [[Quality/QA-tools/OTS/UserDocumentation/Multiple_device| See this instructions!]]
* Changes to QAReports configuration file /etc/ots_plugins_qareports.conf
** Added proxy and attachment upload settings

== 0.8.2 - 0.8.3 ==
* Changes in ots_plugin_qareports.conf
** Add new parameters
* Only new features
** Test plan based execution
** Chroot support
** See [[Quality/QA-tools/OTS/UserDocumentation/Usage|how to]] use these features

== 0.8.1 - 0.8.2 ==
* New django package added, effects to all plug-ins
** See [[Quality/QA-tools/OTS/Plugins/Django|Django instructions]].
** See [[Quality/QA-tools/OTS/DeveloperDocs/Plugins|plug-ins instructions]].

== 0.8.0 - 0.8.1 ==
* Plug-in namespace changed, effects to Logger and QA-Reports plug-ins
** For logger update web page content, see step 8 from [[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger documentation]].
** For QA-Reports, configuration file name has changed. Move /etc/ots_qareports_plugin.conf to /etc/ots_plugin_qareports.conf

== 0.1.x - 0.8.0 ==

See [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration document]].

Quality/QA-tools/OTS/UserDocumentation/Upgrading

2011-06-10T11:41:17Z

Villen: /* Upgrading OTS */

= Upgrading OTS =

== 0.8.5 - 0.8.6 ==
* If you want to keep your old log database, add new (user_defined_id) column to the log_messages table
ALTER TABLE log_messages ADD COLUMN user_defined_id varchar(255) NOT NULL DEFAULT ''

== 0.8.4 - 0.8.5 ==
* Rich-core conductor plug-in available
** Changes to /etc/conductor.conf
** New configuration file for plug-in: /etc/ots_plugin_conductor_richcore.conf

== 0.8.3 - 0.8.4 ==
* Backup all configuration files before updating
* Multiple device support added, even if you use '''only one device''' check following:
** Changes to /etc/conductor.conf
** Update meego-ai-flasher (>= 0.1.4) and [http://meego-qa-tools.digia.com/meego-ai/meego-ai-optimized.tar.gz meego-ai] packages!
** Even if you use only one device, remove usb0 configurations from /etc/network/interfaces
** [[Quality/QA-tools/OTS/UserDocumentation/Multiple_device| See this instructions!]]
* Changes to QAReports configuration file /etc/ots_plugins_qareports.conf
** Added proxy and attachment upload settings

== 0.8.2 - 0.8.3 ==
* Changes in ots_plugin_qareports.conf
** Add new parameters
* Only new features
** Test plan based execution
** Chroot support
** See [[Quality/QA-tools/OTS/UserDocumentation/Usage|how to]] use these features

== 0.8.1 - 0.8.2 ==
* New django package added, effects to all plug-ins
** See [[Quality/QA-tools/OTS/Plugins/Django|Django instructions]].
** See [[Quality/QA-tools/OTS/DeveloperDocs/Plugins|plug-ins instructions]].

== 0.8.0 - 0.8.1 ==
* Plug-in namespace changed, effects to Logger and QA-Reports plug-ins
** For logger update web page content, see step 8 from [[Quality/QA-tools/OTS/Plugins/HTTP_logger|Logger documentation]].
** For QA-Reports, configuration file name has changed. Move /etc/ots_qareports_plugin.conf to /etc/ots_plugin_qareports.conf

== 0.1.x - 0.8.0 ==

See [[Quality/QA-tools/OTS/UserDocumentation/Migration_0.1_0.8| migration document]].

Quality/QA-tools/OTS/DeveloperDocs

2011-04-07T09:31:21Z

Villen: /* Source Code Documentation */

= OTS - Open Testing Service =

== Related Documentation ==

* [[Quality/QA-tools/OTS/DevelopmentGuidelines | Development Guidelines]]
* [[Quality/QA-tools/OTS/Packages | Packages]]
* [[Quality/QA-tools/OTS/ErrorCodes | Error Codes]]
* [[Quality/QA-tools/OTS/Glossary]]
* [[Quality/QA-tools/OTS/DeveloperDocs/Transition0.1-0.8|Making transition from 0.1 to 0.8]]

== Synopsis ==

OTS is a Distributed Test System that takes a Data Driven Approach using XML Files to allow parallel testing of Applications on evolving Hardware platforms.

Design has been driven by the need for a language neutral test platform that allows testing under different device architectures in parallel.

== High Level Overview ==

OTS allows a Test Sequence to be fanned out from a hub known as a '''Distributor''' to a remote machine known as a '''Worker'''.
Each '''Worker''' has a '''Device Group''' associated with it, this is defined in the configuration. The Tests are routed to the '''Worker'''
on the basis of the '''Device Group'''.

An XML '''Test Definition File''' describes the Tests that are run. The results are returned to the '''Distributor''' in an XML '''Results File'''.

The '''Distributor''' takes a '''Device Group''' and '''Timeout''' as inputs, as well as the '''Command''' for running '''Tasks'''. Tests are run within a '''Task''' which
is currently run as a process. The '''Command''' will typically take a path to the '''Test Definition File''' as an input.
If the process exceeds the '''Timeout''' the '''Task''' is stopped (the process is killed).

A number of '''Tasks''' can be added to a '''Testrun'''. The '''Testrun''' will normally wait for all '''Tasks''' to finish before completion.

Data (status, error and results) are communicated back from the '''Task''' to the '''Worker'''. The '''Results Object''' is a container for the files outputted: '''Test Definition File''', '''Results File'''
and any other files produced by the test as well as metadata relating to the run.

== Technology ==

OTS is written in [http://www.python.org/ Python 2.6].

[http://www.amqp.org/confluence/display/AMQP/Advanced+Message+Queuing+Protocol AMQP] protocol is used for communication.

=== XmlFileFormats ===

The XML file formats provide the datum for the system.

Download [http://meego.gitorious.org/meego-quality-assurance/test-definition/trees/master them] and add the path to the '''results_xsd''' parameter in your [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/ots_server.ini ots_server.ini] file

=== Third Party Dependencies ===

==== RabbitMQ ====

You need to install [http://www.rabbitmq.com/server.html RabbitMQ], this is AMQP message server.

===== Managing the Queues =====

If you need to delete or empty the queues there are [http://meego.gitorious.org/meego-quality-assurance/ots/trees/master/ots.server/ots/server/distributor/dev_utils utilities] to help with this.

==== Python Packages ====

The following python packages are needed

* [http://pypi.python.org/pypi/setuptools setuptools]

* [http://www.voidspace.org.uk/python/configobj.html configobj]

* [http://pypi.python.org/pypi/multiprocessing multiprocessing]

* [http://code.google.com/p/py-amqplib/ pyamqplib] (version 0.6.1 or newer)

* [http://www.familieleuthe.de/DownloadMiniXsv.html minixsv]

* [http://www.djangoproject.com/download/1.1.2/tarball/ django]

* [http://code.google.com/p/python-nose/ nose]

== Diving In ==

The following steps show how to get a Development Environment up and running

=== 1. Install Dependencies ===

* Install the Third Party Dependencies

* Get the XML File Formats and set the OTS_TESTDEFINITION environment variable

=== 2. Build the Eggs ===

From the root directory

<code>
./setup.sh
source ./paths.sh
</code>

=== 3. Unittests ===

Run the unittests from the root directory

<code>
./nose.sh
</code>

=== 4. Component Tests ===

Component Tests run high level tests on an OTS Component using Mocks of the dependencies.

You can run these with the component_tests shell script in the root directory.

<code>
./component_tests.sh
</code>

=== 5. Hello OTS World ===

The source contains a simple demonstration of the way the key elements work.

==== 1. Open two terminal windows. ====

==== 2. In the first window run the server ====

<code>
cd $SERVER
cd hub/demos
python demo_hub.py
</code>

==== 3. You should see this: ====

<code>
<snip>
20XX-XX-XX XX:XX:XX,XXX - ots.server.distributor.taskrunner - DEBUG - Sending command '['echo', 'hello world']' with key 'foo'
</code>

==== 4. In the second window run the worker ====

<code>
cd $WORKER
python worker.py -c config.ini
</code>

==== 5. Now in the Server terminal the command should run to completion. You should see the following line in the logs: ====

<code>
20XX-XX-XX XX:XX:XX,XXX - ots.worker.command - DEBUG - process.communicate() returned echo hello world
</code>

It will raise a PackageException. This is normal as no Tasks that are run return Packages to the Server

==== 6. And on the logs on the Worker side should contain: ====

<code>
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.command - DEBUG - process.communicate() returned echo hello world
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.task_broker - DEBUG - Task in state: 'finish'
XXXX-XX-XX XX:XX:XX,XXX - ots.worker.task_broker - INFO - Recommence consume on queue: foo
</code>

== Installation ==

It is intended that the '''Distributor''' and each '''Worker''' should be installed to separate machines.

== Configuration ==

=== Distributor ===

The format of the configuration file is as follows.

The `host` is the name of the RabbitMQ server

<code>

[Client]
host = localhost
vhost = /
port = 5672
username = guest
password = guest
queue = FIXME
consumer_tag = worker
services_exchange = services

</code>

=== Worker ===

Edit your config.ini so that the

* `queue`
* `routing_key`
* `services_exchange`

Are set for your '''Device Group'''.

And the `host` is the name of your RabbitMQ server.

<code>
[Worker]
host = localhost
vhost = /
port = 5672
username = guest
password = guest
queue = foo
routing_key = foo
services_exchange = foo
</code>

==== Starting the Worker ====

To start your Worker:

<code>
python worker.py -c config.ini
</code>

== Automated System Tests ==

Automated system tests are available in ots.system_tests. They trigger various testruns with xmlrpc and check return value and log messages.

=== Requirements ===

* Fully functional OTS system with one worker.
* A SW Product "ots-system-tests" configured so that it defaults to the worker.
* A meego image with test-definition-tests and testrunner-lite-regression-tests.
* Django based advanced OTS setup.
* BeautifulSoup python library.

=== Configuration ===

* make the system test image available for the workers. (for example copy it to your apache static content directory /var/www/static/)
* Modify system_tests.conf in ots.system_tests directory to match your system.
* Make sure all the urls in system_tests.conf work.

=== Notes ===

* System tests execute multiple full testruns. Running all tests can easily take more than 1.5 hours with N900 worker.
* Make sure there's no other activities ongoing in the system while running the tests.
* Some error handling tests expect certain devicegroups and sw products _not_ to be available. If these test cases fail you can remove those queues with ots tools.

== License ==

OTS is distributed under an LGPL license.

== Contributions ==

OTS welcomes contributions.

The following areas would benefit greatly from your help:

* Adding to the list of freely available Publisher Plugins
* Examples of OTS in use with different hardware architectures
* Improving the ease of deployment

Please check out the [[Quality/QA-tools/OTS/DevelopmentGuidelines | Guidelines]] and [[Quality/QA-tools/OTS/DevelopmentGuidelines/Review | Review process]].

== Wiki ==

[[Quality/QA-tools/OTS | OTS Wiki Home]]

== Source Code Documentation ==

OTS docstrings are in [http://epydoc.sourceforge.net/manual-epytext.html epytext] format which enables the use of [http://epydoc.sourceforge.net/index.html epydoc] in API documentation generation.

== Mailing List ==

[http://lists.meego.com/listinfo/meego-qa MeeGo QA] mailing list

== IRC Channel ==

The #meego-qa-tools irc channel in freenode

== Bugs ==

The [http://bugs.meego.com/ Meego bugs page]

== Platform Dependencies ==

OTS is tested to run on Linux only.

Tested on Ubuntu (8.04, 10.04, 10.10), Fedora 13, MeeGo and Redhat.

Binary support for Ubuntu, Fedora and MeeGo.

== Architecture ==

* [[Quality/QA-tools/OTS/DeveloperDocs/Components | Components]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Server | Server]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Plugins | Plugins]]

* [[Quality/QA-tools/OTS/DeveloperDocs/DataFlow | DataFlow]]

* [[Quality/QA-tools/OTS/DeveloperDocs/Behaviour | Behaviour]]

== APIs ==

==== Server ====

The top level [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/api.py api] for external clients utilises parameters in a dictionary.

The parameters are explicitly defined in [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options.py options.py]. The conversion from the loosely defined dictionary to a clear API is undertaken by the [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/hub/options_factory.py options_factory.py]

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.server/ots/server/distributor/api.py Distributor] ====

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.results/ots/results/api.py Results] ====

==== [http://meego.gitorious.org/meego-quality-assurance/ots/blobs/dev_branch_0.8/ots.worker/ots/worker/api.py Worker] ====

== Extending OTS ==

You can extend OTS with your own [[Quality/QA-tools/OTS/CreatingAPublisher|Publisher]]
and with [[Quality/QA-tools/OTS/DeveloperDocs/CreatingACustomDistributionModel|custom package distribution models.]]

== Experimental ==

Documentation for experimental branches, spikes, WIPs etc. can be found in [[Quality/QA-tools/OTS/DeveloperDocs/Experimental]].

== Analysis ==

[[Quality/QA-tools/OTS/DeveloperDocs/ErrorSituations| OTS 0.1 Error Situations]]

[[Quality/QA-tools/OTS/DeveloperDocs/TypicalUsage| Typical OTS users and usage scenarios]]

== Project Planning ==

* [[Quality/QA-tools/OTS/Roadmap | Roadmap]]
* [[Quality/QA-tools/OTS/Meetings | Meetings]]
* [[Quality/QA-tools/OTS/ReleaseChecklist | ReleaseChecklist]]

==== Release Practices ====

OTS releases are done when a new release makes sense from OTS point of view. There are no scheduled releases.

The [[Quality/QA tools development#Release_Practices | QA Tools Release Practices]] gives more details.

Quality/QA-tools/OTS/DeveloperDocs/Experimental

2011-03-01T13:17:50Z

Villen:

== Experimental Area ==

Documentation for experimental branches, spikes, WIPs etc. You can also add documentation related to your merge requests or patches here.

The docs should be moved to proper place in OTS wiki after the feature gets integrated to master branch.

'''This area is relevant for OTS developers only! Information under this area does not apply to master or released OTS versions.'''

=== 0.1 ===

* <s>[[Quality/QA-tools/OTS/DeveloperDocs/Experimental/Django|Advanced OTS server using django]] (Relates to branch dev_branch_0.1_django_server) </s>
**Released in 0.1.1 version. Documentation moved to [[Quality/QA-tools/OTS/UserDocumentation/Installation#Django_Based_Server]]

=== 0.8.2 ===

=== How to monitor ots? ===

We have monitor DTO:s. Probably some kind of hierarchy of dto monitor objects.

* Testrun Events for all testrun states (flashing, bootup, testing, test result processing...):
** Test pkg execution event (start/stop) (minimum requirement for [FEA] 9036)
** Flashing Event (start/stop) (n:th try)
** Bootup event (n:th try)
** Testsuite/set/case/step event? (What level of detail we want to monitor?) Remember that logging still exists...
** Testrun overall result

* Worker events:
** Worker state (Start/stop/busy(testrun id)/free) (information about queues)
** Heartbeat?

* Failure event dto? (Flashing failed, device failure...)?

What about statistics monitoring? What do we need?

* Number of testruns?
* Number of workers?
* Active testruns?
* PASS/FAIL/ERROR ratios?
* Error statistics?
** Most common errors
** Some kind of grouping of errors and statistics for them (user fault vs catastrophic ots failure vs device/image error vs...)

Notes:

* One host can have multiple worker instances running. Dto:s need to have information about the instance, not just hostname.
*

==== Who receives monitor dto:s? ====

Current implementation uses the same testrun response queue and process for monitor dto:s. This is simple to implement for testrun related events but very limited for generic monitoring of the whole system. For example most of the worker events cannot be sent to testrun queue because they are not related to a specific testrun and we don't know if there even is any testrun response queues around.

Having a separate permanent queue and a separate listener process for monitoring would allow monitoring also outside a testrun.

Pros:

* Can be deployed as a completely separate component
** even into a different machine
** Maintenance, update etc. does not affect test execution in any way

Cons:

* One more process/daemon to maintain
** More complicated install
* Might cause errors.
** Not running => Lots of messages pile up in the queue
* Dependencies
** What if for example the distribution model relies on monitor component to provide up to date data?
* How to provide testrun specific monitor info to Publishers?

==== WebUI performance ====
Tested with sqlite and MySQL
It looks like that MySQL does some caching because when a page is loaded for the first time it takes much longer to render the data than on following loads.
This means that on fixed time frame the pages can be loaded quite quickly, but when the time frame changes then the page loading takes more time.
<br>
200000 testruns and 1160443 events within fixed time frame average page loading time:
{|
! page
! sqlite
! MySQL
|-
|main
|4.5322
|0.0378
|-
|group details
|61.3120
|28.2552
|-
|requestor details
|0.5142
|0.0653
|-
|testrun details
|0.0033
|0.0128
|-
|testrun list
|1.0347
|0.0730
|}
<br>
200000 testruns and 1160443 events within dynamic time frame:
{|
! page
! MySQL
|-
|main
|6.4605
|-
|group details
|43.5432
|-
|requestor details
|0.2901
|-
|testrun details
|0.0957
|-
|testrun list
|0.5161
|}
<br>
1000 testruns and 5802 events average page loading time:
{|
! page
! sqlite
|-
|main
|0.0619
|-
|group details
|0.3595
|-
|requestor details
|0.0111
|-
|testrun details
|0.0030
|-
|testrun list
|0.0173
|}

==== TODO ====

* Current server side implementation does not send monitor dtos to Publishers in real time. They are sent only after the testrun is done.

=== Bug 9036 - [FEA] Test package distribution based on history (last execution time of a package) ===

Proposal:

* Data provided by monitor dto:s
* Data stored to a DB
** A separate DB for this plugin only or dependency to a more common monitor DB?
* A custom distribution model that reads data from DB and creates tasks based on that

Notes:

* We cannot limit to testpackage level. Implementation for set level distribution is already ongoing.

Quality/QA-tools/OTS/Plugins/Ots.logger plugin

2010-11-26T10:44:32Z

Villen: /* Installation */

= ots.logger_plugin =

'''Version:''' 0.8

'''Author:''' Ville Niutanen

'''Plugin type:''' Publisher plugin

== Installation ==
Install ots.logger_plugin:
cd ots/ots.logger_plugin/
sudo python setup.py install

== Usage ==