Contents |
Automated test execution in MeeGo is can be done using generic test execution tool called testrunner-lite. Testrunner-lite reads planned tests from test plan xml file which is validated agains test-definition schema. The tool can be run on the device under test or from a test control pc.
To enable automatically setting up the test environment we have the created `eat` (Enable(s) Automated Testing) package. Eat package splits into different sub-packages listed below.
eat-device/host package configures ssh-key based passwordless root logins to the device.
Tests can be run automatically on a MeeGo image by adding package eat-selftest to the image.
eat-selftest package pulls testrunner-lite and test-definition as dependencies to the image. The package itself installs an init script that finds all installed test packages and calls testrunner-lite to execute them. Test packages need to installed in /usr/share/<packagename>-tests/tests.xml otherwise the script doesn't find them.
Results from the test and logs from the test runs are written to /var/local/testresults/ (The results location needs to be checked)
Note: This is currently only tested with N900 MeeGo images. The other images most probably will not work.
RPM packages produced by obs are only tested to work with MeeGo and Fedora 13.
Install the package eat-host to you host machine. This package will install ssh-keys needed for passwordless authentication and testrunner-lite and test-definition as dependencies. If you need also to receive syslogs from the device under test, also install the eat-syslog-host/device packages.
After installing you need to configure the host machine to use usb networking with the device under test. With Fedora you can do this by right-clicking the network managet icon and selecting edit connections. Edit the "Auto usb0" -connections IPv4 Settings and set Method to Manual and define the following Address to it:
Address: 192.168.2.14 Netmask: 255.255.255.0 Gateway: 0.0.0.0
or if you don't use network manager to manage your networking do the following
# ifconfig usb0 192.168.2.14 up
After this your host machine should be ready to act as a test control pc. Have fun testing with it.
If you're using Ubuntu 10.04 as your host you can compile the host configuration package from test tools git repository
Compile the package by doing the following
git clone git@gitorious.org:qa-tools/eat.git cd eat dpkg-buildpackage -rfakeroot -b
this creates package named `eat-host-ubuntu`. When installed it set ssh-key used to access the device under test to the host machine's root user. The package also configures also host machines rsyslog to receive syslog messages from the device under test.
Use the package only if you know what you're doing. There is currently no plan to upload this package to any repository.
To enable host based test execution with N900 MeeGo image you need to install eat-device package to the image. The package will install ssh keys to enable passwordless logins from the host machine. Package eat-syslog-devuce can also be used to configure the images syslog settings so that syslogs get sent to the host machine.
MeeGo images for the N900 have usb networking enabled by default. The devices IP address is 192.168.2.15
Current assumption is that we will use ethernet connection to netbook. The netbook should have ip 192.168.2.15 and eat-device package installed. Configuring DHCP for netbook is explained in http://wiki.meego.com/PXE_boot_server.
If the device under test uses IP 192.168.2.15 and the test control PC uses IP 192.168.2.14 then you can make the set up by using the eat-device and eat-host packages. The default IP addresses need to be those to support OTS. If you need to use some other IP addresses you have to do some manual work to get host based test execution working.
To enable host based test execution do the following:
Add the test control PC's public ssh-key to the devices authorized_keys by
cat ~/.ssh/id_rsa.pub | ssh root@[devices's IP address] "mkdir -p ~/.ssh ; cat >> ~/.ssh/authorized_keys"
you may also need to increase the device's sshd startups by
echo "MaxStartUps 1024" >> /etc/ssh/sshd_config
after this you should be able to run testrunner-lite from the host machine. If you need to get the device's syslog sent to the test control PC you have to edit the device's /etc/rsyslog.conf or /etc/syslog.conf depending if the image is using rsyslog or sysklogd. Newer MeeGo images are using sysklogd. For sysklogd do the following
cp /etc/syslog.conf /etc/syslog.conf.back echo "*.*;auth,authpriv.none @[control PC's IP]"\ >> /etc/syslog.conf cp /etc/sysconfig/sysklogd /etc/sysconfig/sysklogd.back sed -e 's/SYSLOGD_OPTIONS.*/SYSLOGD_OPTIONS=\"-m 0 -r\"/' \ /etc/sysconfig/sysklogd.back \ > /etc/sysconfig/sysklogd
After that setup the test control PC to receive the syslogs
cp /etc/sysconfig/sysklogd /etc/sysconfig/sysklogd.back cp /etc/syslog.conf /etc/syslog.conf.back echo "SYSLOGD_OPTIONS=\"-m 0 -r\"" >> /etc/sysconfig/sysklogd echo ":fromhost-ip, isequal, "[device's IP]" /var/log/eat.log"\ >> /etc/syslog.conf
To start executing tests in the device under test you need to tell testrunner-lite the device's IP address. For example
testrunner-lite -f tests.xml -v -o results.xml -t root@192.168.2.15
Some users have a use case where they want to start a server process in a test step to be used by later test steps. If you fork a new process it is possible that the ssh session running freezes during logout until given timeout or when the process terminates. This blocks further test steps. Reason for this is that each test step is currently in its own ssh session and the forked process holds on to its standard pipes and working directory. This is discussed in more technical detail in bug 5718.
One solution to this issue is daemonizing the new process. Then it redirects it's input, output and error pipes to "/dev/null" and changes the working directory to root. Here's a short code snippet how to daemonize a C program.
#include <stdlib.h>
int main( int argc, char *argv[] ) {
daemon(0, 0);
/* actual code */
return 0;
}
The server process must be terminated by a later test step so that it won't stay running in the device after test case.
MeeGo images can be installed to the N900 automatically by using autoinstaller-n900.sh script. The script is included in File:Autoinstaller-n900.tar.gz. The package also contains a custom initrd image and kernel that are used in the installation process. The package however does not include the Nokia flasher utility required by the process. The utility is available for N900 owners from Nokia
Autoinstaller for N900 images is available here: File:Autoinstaller-n900.tar.gz
Check that you have the following conditions met before starting the installation
After all the pre-requirements are met do the following
cd /path/to/directory/containing/script/ sudo ./autoinstaller-n900.sh /path/to/raw-image /path/to/kernel
Note the the kernel given to the script is the kernel that comes with the raw image.
sudo kill -USR1 `pidof dd`
ifconfig
bring the interface up by
ifconfig usb0 192.168.2.14 up
The installation tries to bring the interface up automatically if it can't get a response from the device by doing the above command
Note that USB hubs, etc. can slow down the installation significantly.
The automated install of netbooks is still work in progress. Current idea is that the test control PC acts as a PXE boot server for the SUT (=netbook). We boot into initrd image over ethernet and from there on do pretty much the same as with the N900. The following has been tested with Samsung NC 10 laptop.
Autoinstaller for netbooks: File:Autoinstaller-netbook.tar.gz
Starting netcat for initial connection check
cd /path/to/directory/containing/script/ sudo ./autoinstaller-netbook.sh /path/to/raw-image
No solution for other targets currently available
Tools maintained by MeeGo QA-tools
Testrunner-lite is a generic test execution tool that enabled tests to be execution from a host machine through ssh or locally on the device under tests. More information about the tool from testrunner-lite wiki page
testrunner-ui is a graphical user interface for testrunner-lite
test-definition includes XML schemas for test plan validation
Testplanner is a graphical editor for test plan xml files.
eat contains different configuration package for test automation
min is an advanced unit/integration test framework for Linux based devices. MIN can be integrated with standard toolchain in Linux (make). It makes testing easy, efficient and what is most important: rapid.
MeeGo-ai contains setups and configurations for automated MeeGo installations. Sources for it can be found from http://gitorious.org/qa-tools/meego-ai