Views

Quality/QA-tools/OTS/UserDocumentation/Installation

From MeeGo wiki

(Difference between revisions)

Jump to: navigation, search

Revision as of 13:53, 29 December 2010

1 Installation Instructions for OTS 0.1

Installation Instructions for OTS 0.1

These instructions have been tested with Ubuntu 10.04.

Network Setup

Using static IP addresses on all OTS machines is strongly recommended. It is possible to setup the system also with dynamic IP addresses but it might cause problems every now and then because IP's and hostnames change.

All OTS machines need to be able to access other OTS machines based on their hostname.

Server

prerequisites

An smtp server is needed for OTS to be able to send result emails.
- You can use the smtp server provided by your ISP or organization
- Or you can setup your own. for example postfix

Basic Setup

Download and install RabbitMQ server from http://www.rabbitmq.com/server.html

Install dependencies from ubuntu repository:

sudo apt-get update
sudo apt-get install python-setuptools python-amqplib python-django

Install rest of the dependencies with easy_install:

sudo easy_install minixsv

Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (latest tag)

Install ots.common:

cd ots/ots.common/
sudo python setup.py install

Install ots.server:

cd ots/ots.server/
sudo python setup.py install

Install ots.tools: (Required only for triggering testruns easily from command line over xmlrpc)

cd ots/ots.tools/
sudo python setup.py install

Download and install test definition schema files from http://meego.gitorious.org/meego-quality-assurance/test-definition

Create a directory for testrun logs and make sure the user running ots.server has write permissions to it:

sudo mkdir /var/log/ots

At this point it should be possible to trigger testruns from python shell if you have a worker already set up and configured to use this server. Here's an example ipython session:

In [1]: from ots.server.xmlrpc.public import request_sync

In [4]: program = "example_sw_product"

In [5]: request = "5"

In [6]: notify_list = ['tvainio@localhost']

In [7]: options = dict()

In [8]: options['image'] = "http://somewhere/some_image.bin"

In [9]: options['email'] = "off"

In [13]: options["device"] = "devicegroup:default"            # Make this match the worker 

In [14]: request_sync(program, request, notify_list, options)
Out[14]: 'FAIL'

The result will be PASS/FAIL/ERROR. ERROR means that something went wrong in server side and testrun was not even started. FAIL means failed testrun and PASS means that everything went fine and all tests passed. Testrun logs with more information about the possible errors are available in /var/log/ots/

Configuring the Server

The previous example was not very useful. To get OTS server running as a real server you have two options. A simple xmlrpc server or a more advanced django based server. First thing you need to do is setup a custom config file.

Custom Config File

Ots server tries to import a configuration module ots_extensions.ots_config. If it does not succeed it uses the default from ots.server.testrun_host.default_ots_config. You can set up your own ots_config by following these instructions:

Create a directory for your extensions:

mkdir my_ots_extensions

Add it to your python path

export PYTHONPATH=/full/path/to/my_ots_extensions/:$PYTHONPATH

Create custom ots_extensions python package:

mkdir my_ots_extensions/ots_extensions
touch my_ots_extensions/ots_extensions/__init__.py

Copy the default config file from ots/ots.server/ots/server/testrun_host/default_ots_config.py to my_ots_extensions/ots_extensions/ots_config.py

Modify the file to suit your needs. Check at least the following settings:
- Email settings (The smtp server OTS uses to send testrun results)
- xmlrpc server config (if you are setting the simple xmlrpc server setup. More info below.)
- The default options for your sw product:

You can define default values for all options here. The example_sw_product contains and example how to define options. You cannot use an sw_product if it does not contain the default options in ots_config. Here's a small example. I'm developing a Meego based coffeemaker and creating custom software images for it. To automate the testing I create an sw_product called "my_meego_variant": TODO: Document all options

#
# Defaults for my_meego_variant                             
#                                                                                                                                   

# Lets start with the global_defaults
default_options["my_meego_variant"] = global_defaults.copy()

# The images work only on meego coffeemakers so the default devicegroup will be "meego_coffeemaker"
default_options["my_meego_variant"]['device'] = {'devicegroup':'meego_coffeemaker'}

# Coffeemaker needs to be fast so we define a 3 minutes global timeout for the test execution.
default_options["my_meego_variant"]['timeout'] = 3

Extension Points

OTS uses plugins to do various things during the testrun. You can write your own plugins to add more functionality by implementing a module ots_extensions.extension_points. An example file can be found from ots/ots.server/ots/server/testrun_host/example_extension_points.py

Simple XMLRPC Server

After you have implemented your custom config file and defined suitable xmlrpc_host and xmlrpc_port values you can start the simple xmlrpc server:

ots_server

After that you can start testruns on your ots_server by calling the request_sync() method over xmlrpc just like we did locally in the example above. There's also a command line tool called ots_trigger in ots.tools package.

A testrun can be triggered from command line with ots_trigger:

ots_trigger -s your_xmlrpc_host:your_xmlrpc_port -b 333 -i url_to_dummyimage -p my_meego_variant -e your_email_address

The result should be FAIL and if your email server settings are correct you should also receive an email.

If we look into the new testrun log file in /var/log/ots/ we should find something like this:

2010-09-24 15:48:21,188  conductorengine ERROR    Device group 'meego_coffeemaker' does not exist
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/ots.server-0.1dev-py2.6.egg/ots/server/conductorengine/conductorengine.py", line 150, in execute
    self._taskrunner.run()
  File "/usr/local/lib/python2.6/dist-packages/ots.server-0.1dev-py2.6.egg/ots/server/distributor/taskrunner.py", line 417, in run
    (self._routing_key))
OtsQueueDoesNotExistError: No queue for meego_coffeemaker

Next step is to set up an OTS worker into the devicegroup meego_coffeemaker. Instructions can be found below.

Django Based Server

With Django and Apache it's possible to set up a more advanced ots server.

There's for example a global logger application available in ots.server.logger. By setting up the logger it is possible to get log messages for a specific testrun from all parts of the system (ots server, ots conductor and testrunner-lite). The logs are stored into database and can be accessed with a web UI. It's also easy to implement a result db and web views for the test results.

If you are not familiar with Django you can study the basics from the tutorial: http://docs.djangoproject.com/en/1.2/intro/tutorial01/

These instructions assume that you have already done the setup described in: [[1]]

There is an example django project in ots.server.django_ots. It is possible to use that directly but in most cases you should create your own django project to be able to modify the settings.

Installation Using the Default Django Project in ots.server.django_ots

Install required packages:

sudo apt-get update
sudo apt-get install python-django python-setuptools rabbitmq-server python-amqplib

Install django-xmlrpc from https://launchpad.net/django-xmlrpc to somewhere in your pythonpath

After that you should be able to execute tests for the ots django applications:

python ots/server/django_ots/manage.py test logger testrun_db

Create the database. The default ots django project uses sqlite. The DB file is located in /opt/ots.sqlite.

python ots/server/django_ots/manage.py syncdb

The script asks about creating a superuser for the Django's auth system. OTS does not use the auth system at the moment but it might be useful to create the account if you start writing custom django applications or want to use django admin interface at some point.

If you get errors like "unable to open database file" make sure your user has write permissions to /opt/

Install apache and mod-python: (OTS requires the prefork version of apache)

sudo apt-get install apache2-mpm-prefork libapache2-mod-python

Setup your custom ots_config and extension points
- Instructions are available in [[2]]
- Use ots/server/django_ots/extension_points.py as the basis of your extension points. It takes the testrun_db into use.
- Changes http_loggin_enabled to True in your ots_config.py if you want to have global testrun logs:

http_logging_enabled = True

Configure your django_ots to /etc/apache2/apache2.conf by adding the following:

<Location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE ots.server.django_ots.settings
PythonDebug On
PythonPath "['ADD HERE THE PATH WHERE YOUR ots_extensions directory with CUSTOM OTS ENTRYPOINTS AND CONFIGFILE CAN BE FOUND'] + sys.path"
</Location>

Open http://localhost/logger/view/ in your web browser. You should see an empty logging message page.
- If you get "Unable to open database file" make sure that the user apache is running on has read and write permissions to the db file (/opt/ots.sqlite).
Try triggering a testrun with ots_trigger (note the /xmlrpc in server url):

ots_trigger -s localhost/xmlrpc -b 333 -i url_to_dummyimage -p my_meego_variant -e your_email_address

If everything went ok you should see a succesful testrun and logs from OTS server, OTS Worker.conductor and testrunner-lite appearing in the http://localhost/logger/view/ and http://localhost/logger/view/ots/<your_testrun_id>/

If you only see log messages from your server most probably yuor hostnames are not set up correctly. The worker machine needs to be able to access the server by it's host name. Or you can try to play around with the "results_storage_config["host"]" setting in ots_config.py. It is the hostname ots_server sends to worker to use for global logging.

Installation Using a Custom Django Project

If you want to use different django setup you can create a custom django project. You can use ots/server/django_ots/ as an example. Django documentation contains lots of information about the features and settings: http://www.djangoproject.com/

The most important files are:

settings.py - DB settings, installed django applications, email etc.
urls.py - Url mappings. For example the "xmlrpc/" url is defined here.

You also need to modify your apache configuration to point to your own django project instead of the default one. Also note that if you change your code, mod_python does not automatically reload the changed code so you need to restart apache to make sure you are using the right version of your code. If you do a full apache restart all ongoing testruns will fail. If you do a graceful restart ("apache2ctl graceful")the testruns won't fail but updated code is reloaded only after the testruns have fininshed.

Uploading Results to Meego QA Reports Tool

Starting from version 0.1.6 OTS can automatically upload results to Meego QA Reports tool.

Make sure ots.qareports_plugin is installed.
Modify values in /etc/ots_qareports_plugin.conf to match your setup:

# More information about the options is available in
# http://wiki.meego.com/Quality/QA-tools/QAReports/API

[ots.qareports_plugin]
host = "localhost"
url = "/api/import"
auth_token = ""
release_version = "1.0"
target = "Core"
testtype = "Acceptance"
hwproduct = "N900"

Modify "get_resultbackends()" in your custom extension_points.py to take QaReportsBackend into use:

from ots.qareports_plugin.qareports_backend import QAReportsBackend


def get_resultbackends(testrun):
    """Returns custom ResultBackends as a list"""
    return [QAReportsBackend()]

Dynamic Report Control

Starting from OTS 0.1.7 it's possible to control reporting options dynamically when triggering a testrun with "--options" command line parameter in ots_trigger:

ots_trigger --options="qa_hwproduct:<HWPRODUCT> qa_testtype:<TESTTYPE> qa_target:<TARGET> qa_release_version:<RELEASE_VERSION>"

Worker

Install dependencies:

sudo apt-get update
sudo apt-get install python-setuptools python-amqplib

Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (latest tag)

Install ots.common:

cd ots/ots.common/
sudo python setup.py install

Install ots.worker:

cd ots/ots.worker/
sudo python setup.py install

Configure the worker by editing devicegroup and server address in /etc/ots.ini:

###########################################
# REPLACE `fix_me` WITH YOUR DEVICEGROUP
###########################################
[Device]
devicegroup = fix_me                     # Change this to the devicegroup this worker belongs to

#############################
# Optional device properties
#############################

#devicename = name of the device type    # Uncomment these if you want more detailed testrun routing
#deviceid = Specific HW id of the device # See http://wiki.meego.com/Quality/QA-tools/OTS/Routing for details

#############################
# ADVANCED CONFIGURATION
#############################
[Worker]
host = your_server_address               # Change this to your ots server address
vhost = /
port = 5672
username = guest
password = guest
log_file = /var/log/ots.log

More info about testrun routing is available here: Quality/QA-tools/OTS/Routing

Start the worker:

sudo ots_worker

If everything went ok, you should see something like this:

2010-11-08 12:55:10,790 - ots.worker.worker - DEBUG - Initialising the server
2010-11-08 12:55:10,790 - ots.worker.connection - DEBUG - Connecting to RabbitMQ
2010-11-08 12:55:10,894 - amqplib - DEBUG - Start from server, version: 8.0, properties: {u'platform': 'Erlang/OTP', u'product': 'RabbitMQ', u'version': '2.1.0', u'copyright': 'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
2010-11-08 12:55:10,935 - amqplib - DEBUG - Open OK! known_hosts []
2010-11-08 12:55:10,935 - amqplib - DEBUG - using channel_id: 1
2010-11-08 12:55:10,937 - amqplib - DEBUG - Channel open
2010-11-08 12:55:10,938 - ots.worker.worker - DEBUG - Starting the worker. server: 192.168.1.102:5672, device_properties: {'devicegroup': 'meego_coffeemaker'}
2010-11-08 12:55:10,957 - ots.worker.task_broker - INFO - consume on queue: meego_coffeemaker
2010-11-08 12:55:10,958 - ots.worker.task_broker - DEBUG - Starting main loop...

There's also an init script available in ots.worker/debian/python-ots-worker.ots-worker.init to make running the worker easier

After that the worker should be ready for receiving tasks. To make sure the actual test execution works please see the instructions for ots.worker.conductor: Conductor

Retrieved from "http://wiki.meego.com/Quality/QA-tools/OTS/UserDocumentation/Installation"

@@ Line 28: / Line 28: @@
   sudo easy_install minixsv
-* Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (tag 0.1.6)
+* Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (latest tag)
 * Install ots.common:
@@ Line 250: / Line 250: @@
      """Returns custom ResultBackends as a list"""
      return [QAReportsBackend()]
+</pre>
+===== Dynamic Report Control =====
+Starting from OTS 0.1.7 it's possible to control reporting options dynamically when triggering a testrun with "--options" command line parameter in ots_trigger:
+<pre>
+ots_trigger --options="qa_hwproduct:<HWPRODUCT> qa_testtype:<TESTTYPE> qa_target:<TARGET> qa_release_version:<RELEASE_VERSION>"
 </pre>
@@ Line 259: / Line 264: @@
   sudo apt-get install python-setuptools python-amqplib
-* Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (tag 0.1.6)
+* Get source code from http://meego.gitorious.org/meego-quality-assurance/ots/ (latest tag)
 * Install ots.common: