This presentation is to describe the details about how to use this test framework to test BOSS and participants. BATS system support openSUSE 11.2 (verified).
All the codes of BATS have been packaged and uploaded to relevant repositories. To deployment the test environemt, please follow the steps:
1. Add repositories url(openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo zypper ar http://maemo-repo.europe.nokia.com/Tools:/MeeGoInfra/openSUSE_11.2/Tools:MeeGoInfra.repo zypper ar http://download.opensuse.org/repositories/devel:/languages:/ruby:/extensions/openSUSE_11.2/devel:languages:ruby:extensions.repo zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.2/devel:languages:python.repo zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo zypper ar http://download.meego.com/live/Tools:/Building/openSUSE_11.2/Tools:Building.repo zypper ar http://download.opensuse.org/repositories/home:/victorliu:/bats/openSUSE_11.2/home:victorliu:bats.repo zypper ref
2. Install BATS package
zypper in bats
3. Configure and start BATS QA Reports system
bats-qa-reports setup bats-qa-reports start
4. Download sample test suites, then have a try
git clone http://git.gitorious.org/bats/bats-test-suites.git cd bats-test-suites bats-testrunner -c bats-test-ruote.conf # check results in your BATS QA reports system
Then, the whole test environment is ready include BOSS, participants (not all participants are depoyed now, such as imager), BATS and all existed test cases has been integrated into BATS.
In this step, you need to launch the BOSS and the participants you want to test.
1. start boss
2. config and start need participants (repeat)
configure participants in /etc/boss service <participant_name> start
Before running the test script, you have a chance to choose executing testing with default setting or customize the testing. There are 2 config files inside BATS to specify the test sets, the QA report system account, etc... Follows are the sample of global config file (default located in /etc/bats.conf):
# user name in the BATS QA report system USER_NAME=boss # user email , user can login BATS QA report system by this email to operate on test result. # user also gets test result notification mail through this address USER_EMAILemail@example.com # user password is used by user to login BATS QA report system USER_PASSWORD=88888888
The other one is to be specified the test sets which will be executed:
# root directory used to run these test cases TEST_PLAN_DIR=/home/boss/Downloads/meego-infrastructure-tools-boss/ruote/ruote/test # test plan xml file TEST_PLAN_XML=/home/boss/work/bats-test-suites/ruote-2.1-func-test.xml # 1st test category, test target can be "ruote", "participant" etc... TEST_TARGET=ruote # 2st test category, test version must only contains letter, number and "_" TEST_VERSION=ruote_2_1 # 3st test category, test type can be "functional", "unit" and "performance" etc... TEST_TYPE=functional # repeat above to specify another test set # ...
or as a XML file:
<?xml version="1.0" encoding="UTF-8"?> <test_plans> <test_plan> <test_plan_dir>/usr/lib/ruby/gems/1.8/gems/ruote-2.1.11/test</test_plan_dir> <test_plan_xml>/home/boss/work/bats-test-suites/ruote-2.1-func-test.xml</test_plan_xml> <test_target>engine</test_target> <test_type>functional</test_type> <test_version>ruote_2_1</test_version> </test_plan> <test_plan> ... </test_plan> ... </test_plans>
Defaultly each BATS environment includes one BATS QA Reports system, you can run test and check results on same host. If you want a remote or centralized BATS QA Reports system to hold all test reports from multiple BATS environment on different hosts(clients), you need to do following steps on remote host:
1. Deploy BATS QA Reports system and mysql
zypper ar http://download.opensuse.org/repositories/home:/victorliu:/bats/openSUSE_11.2/home:victorliu:bats.repo zypper in bats-qa-reports mysql bats-qa-reports setup
2. Configure the BATS QA Reports system
change /srv/www/bats-qa-reports/config/database.yml as following example: development: adapter: mysql host: localhost database: db_test username: tester password: 12345 encoding: utf8 timeout: 5000
3. Create and migrate database with following commands:
cd /srv/www/bats-qa-reports bin/rake db:create bin/rake db:migrate
4. Configure database to grant rights for client accessing(take "db_test" as example database and "tester" as user)
mysql -h localhost -u root -p GRANT ALL ON db_test.* TO 'tester'@'%' IDENTIFIED BY '12345'; FLUSH PRIVILEGES;
5. Start rails server
After this, you can access your QA reports system in browser from anywhere by "http://<remote IP>:3000"
On client side, connect a remote QA report system to upload the test results to this server.
1. Deploy BATS following introduction in "Step 1"
2. Configure the BATS QA Reports system
change /srv/www/bats-qa-reports/config/database.yml as following example: delopment: adapter: mysql host: <server name/server IP points to remote host> database: db_test username: tester password: <password> encoding: utf8 timeout: 5000
3. Execute test and upload test result, refer to Step4.
All the test cases are organized as XML file. A XML file means a test set(plan). You can config this testing with several test sets. To add a new test set, just need append an additional section in test.cfg file.
About the details of the test plan xml file, please refer to http://wiki.meego.com/Quality/QA-tools/Test_plan
BTAS invokes different test tools/framworks to do the real testing. Follow table is the list of test tools used by BATS and relevant guide. You can refer the sample test cases also.
|Test Framework||Test Level||Guide|
|participanttester.py||integration, functionality (participant)||http://wiki.meego.com/Release_Infrastructure/BOSS_Testing/Participants_Testing|
You also can refer to the existed test cases we have implemented (mentioned in Step1).
If the new test cases are ready, you need change the exist test plan (http://wiki.meego.com/Quality/QA-tools/Test_plan ) or create a new one ("Add a Test Set")
Two code coverage tools are installed in BATS environment - "rcov" for Ruby and "coverage" for Python. To use them getting code coverage for ruby code in BOSS engine and Python code in various participants, you need to start up boss and participants with these tools. Then you can get code coverage reports after executing your test cases.
bast-testrunner is used to run testsuite and import test result to bats-qa-reports.
Usage ----- bats-testrunner [Options] For help use: bats-testrunner -h Options ------- -h, --help Displays help message -v, --version Display the version, then exit -V, --verbose Verbose output -c <bats_cnf>, --config <bats_cnf> set <bats_cnf> as BATS test config
During the execution, bast-testrunner will generate the test result as a xml file for per test plan. Then upload the results to QA report system. After all the activities are done, you will get an email with the link of test result. Such as:
1. startup the bats-qa-reports under bats-qa-reports root directory:
2. check http://localhost:3000/boss to access local QA report