Quality/TestSuite/MCTS/MCTS Development Guideline

Revision as of 09:14, 9 June 2010

MeeGo Core Test Suite(CTS) Development Guideline

Test Case Naming

• Show the purpose clearly, avoid simply use number.
• Name length should less than 30
• Use '_' to connect words

Test Definition

• XML format :http://gitorious.org/qa-tools/test-definition
• One test package corresponds one test case definition file

File header

• Copyright
• License
• Author
• Changelog

Check list

• README
• src/ : contain source files
• utils/: contain utility and tools if any
• Makefile
• tests.xml
• data/: contain small data files (For big data such as media content, need separate package)
• pack.sh: the script to make package
• test.spec: spec files to create package

Makefile (??? If need automake)

• make : build test package.
• make install: install test binary or script into /usr/bin, tests.xml into /usr/share/<testpackagename>/tests.xml
• make clean: remove all object file and tempory files
• make uninstall: remove test binary or scripts, tests.xml under target installation directory.
• make package: create rpm for current test package

Coding Style

• C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
• Python: http://www.python.org/dev/peps/pep-0008/
• Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design

• Common Rule:
  • Automate test under condition of stability
  • Main function should keep simple and structure keep neat.
  • Define common function file shared between test cases.
  • Have no dependency on other test case.
  • Have no dependency on UX or vertical specific apps.
  • Have no dependency on test framework.
  • Platform irrelevant
  • Clean environment after test complete.
  • Avoid hard coding - String, Digital
  • If parameters number greater than 3, use getopts
  • Suitable comments

• Test Level Specific Rules:
  • Thorough
    • All APIs should be covered.
    • A test case targets an usage (API combination) instead of an API itself
    • Test need cover param pairs with equivalence partitioning and boundary condition (All-pairs optional)
  • Average
    • Legacy or low risk APIs are covered only with positive tests
  • Lightweight
    • Test most critical functionalities
    • With most typical parm-pairs