Quality/TestSuite/MCTS/MCTS Development Guideline

Revision as of 09:03, 9 June 2010

MeeGo Core Test Suite(CTS) Development Guideline

Test Case Naming

        - Show the purpose clearly, avoid simply use number.
        - Name length should less than 30
        - Use '_' to connect words
  

Test Definition

        - XML format :http://gitorious.org/qa-tools/test-definition
        - One test package corresponds one test case definition file
        

File header

        - Copyright
        - License
        - Author
        - Changelog
     

Check list - README - src/ : contain source files - utils/: contain utility and tools if any - Makefile - tests.xml - data/: contain data files - pack.sh: the script to make package - test.spec: spec files to create package

Makefile (??? If need automake) - make : build test package. - make install: install test binary or script into /usr/bin, tests.xml into /usr/share/<testpackagename>/tests.xml - make clean: remove all object file and tempory files - make uninstall: remove test binary or scripts, tests.xml under target installation directory. - make package: create rpm for current test package

Coding Style

        - C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
        - Python: http://www.python.org/dev/peps/pep-0008/
        - Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design Common Rule:

        - Automate test under condition of stability
        - Main function should keep simple and structure keep neat.
        - Define common function file shared between test cases.
        - Have no dependency on other test case.
        - Have no dependency on UX or vertical specific apps.
        - Have no dependency on test framework.
        - Platform irrelevant
        - Clean environment after test complete.
        - Avoid hard coding - String, Digital
        - If parameters number greater than 3, use getopts
        - Suitable comments
        

Test Level Specific Rules:

        Thorough
        - All APIs should be covered.
        - A test case targets an usage (API combination) instead of an API itself
        - Test need cover param pairs with equivalence partitioning and boundary condition  (All-pairs optional)

        Average
        - All cared APIs should be covered (e.g. new functionality relevant APIs) with thorough test rules
        - Legacy or low risk APIs are covered only with positive tests
        
        Lightweight
        - Test most ciritical functionalities 
        - With most typical parm-pairs