Quality/TestSuite/MCTS/MCTS Development Guideline

Contents 1 MeeGo Core Test Suite(CTS) Development Guideline 1.1 Test Definition 1.2 Test Case Naming 1.3 File header 1.4 Check list 1.5 Makefile (??? If need automake) 1.6 Coding Style 1.7 Test Case Design

MeeGo Core Test Suite(CTS) Development Guideline

Test Definition

• XML format :http://gitorious.org/qa-tools/test-definition
• One test package corresponds one test case definition file

Test Case Naming

• Show the purpose clearly, avoid simply use number.
• Name length should less than 30
• Use '_' to connect words

File header

• Copyright
• License
• Author
• Changelog

Check list

• README
• src/ : contain source files
• utils/: contain utility and tools if any
• Makefile
• tests.xml
• data/: contain small data files (For big data such as media content, need separate package)
• pack.sh: the script to make package
• test.spec: spec files to create package

Makefile (??? If need automake)

• make : build test suite.
• make install: install test binary or script into /usr/bin, and tests.xml into /usr/share/<testpackagename>/tests.xml
• make clean: remove all object file and tempory files
• make uninstall: remove test binary, scripts, and tests.xml out of target installation directory.
• make package: create rpm package

Coding Style

• C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
• Python: http://www.python.org/dev/peps/pep-0008/
• Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design

• Common Rule:
  • Automate test under condition of stability
  • Main function should keep simple and structure keep neat.
  • Define common function file shared between test cases.
  • Has no dependency on other test case.
  • Has no dependency on UX or vertical specific apps.
  • Has no dependency on test framework.
  • Platform irrelevant
  • Clean environment after test complete.
  • If parameters number greater than 3, use getopts
  • Enough comments

• Test Coverage Category Rules:
  • Thorough
    • Cover all APIs.
    • Test case targets for usage (API combination) instead of single API.
    • With Parm-pairs of equivalence partitioning and boundary condition (All-pairs optional)
  • Average
    • High risk APIs(e.g. new functionality relevant APIs) should be covered with thorough test rules
    • Legacy or low risk APIs are covered only with positive tests
  • Lightweight
    • Test most critical functionalities
    • With most typical parm-pairs