BATS "BOSS Automatic Testing System"
- BATS is a system which is used to ensure the quality of BOSS. This system is developed to make the testing easely. It will manage all the precess of auto-testing: environemt deployment, test exectuion and result management. This system covers different levels of tests for BOSS:
- Unit Test
- Functional Test
- Integration Test
- Performance Test
- Test cases shall be easy to extend and customize with simple interface.
- Anything is automatic include real test environment deployment. Only one command is needed to triger the testing.
- The target of BATS is to cover all components of BOSS and the dependencies, which can be found at: https://build.opensuse.org/project/packages?project=Maemo:MeeGo-Infra
- At the beginning, we're focusing more on packages BOSS and various participants.
- More test cases are expecting coming from BOSS developers/integrators or from upstream directly.
- BOSS developers
- BOSS integrators
- BOSS QA
- For BOSS developer:
- Modify participant A
- Create or modify test cases upon his updating to A
- Packaging work
- Deploy BATS then deploy updated A, or update A in existing BATS
- Trigger unit/functional testing
- Trigger regression testing
- If no error, publish A
- For BOSS integrator:
- New BOSS package coming or new BOSS usage scenario coming
- Change BATS configuration to include new package or update test case upon new scenario
- Trigger integration testing and regression testing
- If no error, add new package or scenario
- For QA:
- Run various test cases upon BOSS updating
- BATS is based on OTS. See our previous work: http://wiki.meego.com/Release_Infrastructure/BOSS_Testing/BOSS_Tesing_OTS
- The existed test cases from upstream are integrated into this system with the used test tools.
- BATS deploys the environment automatically by some scripts and configuration files
- Test cases are easy to add/modify based on OTS xml format
- A new test tool is implemented to covers the functionality test of participant and integration test of BOSS.
- Test result is base on OTS xml format. And it is easy to integrate existed report management system.
- More design detail is under discussion, update later...
- Source code can be found at gitorious: (TBD)
Test Techniques in BATS
- For Unit/Functional test, following table lists existing packags/projects status:
| Test Framework || Targets || Package/Project
| RSepc || performance testing, functionality testing || BOSS, ruote-amqp, AIR ...
| Test::Unit || unit test || ruote
| py.test || unit test, functionality testing || MeeGo-IMG, AIR, ruote-amqp-pyclient, boss-participant-*, boss-launcher-*, ...
| unittest || unit test || python-amqlib
| participanttester.py || integration testing, functionality testing || boss-participant-*, BOSS
Current Test Status
About how to get the engine scripts and start to triger the test framework, please refer to http://wiki.meego.com/Release_Infrastructure/BOSS_Testing/BTF/Usage