Quality/TestSuite/MCTS/MCTS Development Guideline

Contents 1 MeeGo Core Test Suite(CTS) Development Guideline 1.1 Test Case Naming 1.2 Test Definition 1.3 File header 1.4 Check list 1.5 Makefile (??? If need automake) 1.6 Coding Style 1.7 Test Case Design

MeeGo Core Test Suite(CTS) Development Guideline

Test Case Naming

• Show the purpose clearly, avoid simply use number.
• Name length should less than 30
• Use '_' to connect words

Test Definition

• XML format :http://gitorious.org/qa-tools/test-definition
• One test package corresponds one test case definition file

File header

• Copyright
• License
• Author
• Changelog

Check list

• README
• src/ : contain source files
• utils/: contain utility and tools if any
• Makefile
• tests.xml
• data/: contain small data files (For big data such as media content, need separate package)
• pack.sh: the script to make package
• test.spec: spec files to create package

Makefile (??? If need automake)

• make : build test package.
• make install: install test binary or script into /usr/bin, tests.xml into /usr/share/<testpackagename>/tests.xml
• make clean: remove all object file and tempory files
• make uninstall: remove test binary or scripts, tests.xml under target installation directory.
• make package: create rpm for current test package

Coding Style

• C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
• Python: http://www.python.org/dev/peps/pep-0008/
• Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design

• Common Rule:
  • Automate test under condition of stability
  • Main function should keep simple and structure keep neat.
  • Define common function file shared between test cases.
  • Have no dependency on other test case.
  • Have no dependency on UX or vertical specific apps.
  • Have no dependency on test framework.
  • Platform irrelevant
  • Clean environment after test complete.
  • Avoid hard coding - String, Digital
  • If parameters number greater than 3, use getopts
  • Suitable comments

• Test Level Specific Rules:
  • Thorough
    • All APIs should be covered.
    • A test case targets an usage (API combination) instead of an API itself
    • Test need cover param pairs with equivalence partitioning and boundary condition (All-pairs optional)
  • Average
    • Legacy or low risk APIs are covered only with positive tests
  • Lightweight
    • Test most critical functionalities
    • With most typical parm-pairs