Quality/TestSuite/MCTS/MCTS Development Guideline

MeeGo Core Test Suite(CTS) Development Guideline

Test Case Naming

        * Show the purpose clearly, avoid simply use number.
        * Name length should less than 30
        * Use '_' to connect words 

Test Definition

        * XML format :http://gitorious.org/qa-tools/test-definition
        * One test package corresponds one test case definition file        

File header

        * Copyright
        * License
        * Author
        * Changelog     

Check list

• README
• src/ : contain source files
• utils/: contain utility and tools if any
• Makefile
• tests.xml
• data/: contain small data files (For big data such as media content, need separate package)
• pack.sh: the script to make package
• test.spec: spec files to create package

Makefile (??? If need automake)

• make : build test package.
• make install: install test binary or script into /usr/bin, tests.xml into /usr/share/<testpackagename>/tests.xml
• make clean: remove all object file and tempory files
• make uninstall: remove test binary or scripts, tests.xml under target installation directory.
• make package: create rpm for current test package

Coding Style

        * C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
        * Python: http://www.python.org/dev/peps/pep-0008/
        * Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design Common Rule:

        - Automate test under condition of stability
        - Main function should keep simple and structure keep neat.
        - Define common function file shared between test cases.
        - Have no dependency on other test case.
        - Have no dependency on UX or vertical specific apps.
        - Have no dependency on test framework.
        - Platform irrelevant
        - Clean environment after test complete.
        - Avoid hard coding - String, Digital
        - If parameters number greater than 3, use getopts
        - Suitable comments
        

Test Level Specific Rules:

        Thorough
        - All APIs should be covered.
        - A test case targets an usage (API combination) instead of an API itself
        - Test need cover param pairs with equivalence partitioning and boundary condition  (All-pairs optional)

        Average
        - All cared APIs should be covered (e.g. new functionality relevant APIs) with thorough test rules
        - Legacy or low risk APIs are covered only with positive tests
        
        Lightweight
        - Test most ciritical functionalities 
        - With most typical parm-pairs