Quality/TestSuite/MCTS/MCTS Development Guideline

Revision as of 00:46, 10 June 2010

MeeGo Core Test Suites(CTS) Development Guideline

Test Definition

• XML format :http://gitorious.org/qa-tools/test-definition
• One test package corresponds one test case definition file

Test Case Naming

• Show the purpose clearly, avoid simply use number.
• Name length should be less than 30
• Use '_' to connect words

File header

• Copyright
• License
• Author
• Changelog

Check list

• README
• src/ : contain source files
• utils/: contain utility and tools if any
• Makefile
• tests.xml
• data/: contain small data files (For big data such as media content, need separate package)
• pack.sh: the script to make package
• test.spec: spec files to create package

Makefile (??? If need automake)

• make : build test suite.
• make install: install test binary or script into /usr/bin, and tests.xml into /usr/share/<testpackagename>/tests.xml
• make clean: remove all object file and tempory files
• make uninstall: remove test binary, scripts, and tests.xml out of target installation directory.
• make package: create rpm package

Coding Style

• C/C++ : http://www.chris-lott.org/resources/cstyle/indhill-cstyle.html
• Python: http://www.python.org/dev/peps/pep-0008/
• Shell : http://hub.opensolaris.org/bin/view/Community+Group+on/shellstyle

Test Case Design

• Common Rule:
  • Automate test under condition of stability
  • Main function should keep simple and structure keep neat.
  • For across-test common function, create separated lib file.
  • Has no dependency on other test case.
  • Has no dependency on UX or vertical specific apps.
  • Has no dependency on test framework.
  • Platform irrelevant
  • Clean environment after test complete.
  • If parameters number greater than 3, use getopts
  • Enough comments

• Test Coverage Category Rules:
  • Thorough
    • Cover all APIs.
    • Test case targets for usage (API combination) instead of single API.
    • With Parm-pairs of equivalence partitioning and boundary condition (All-pairs optional)
  • Average
    • High risk APIs(e.g. new functionality relevant APIs) should be covered with thorough test rules
    • Legacy or low risk APIs are covered only with positive tests
  • Lightweight
    • Test most critical functionalities
    • With most typical parm-pairs

    Test package list see http://wiki.meego.com/MeeGo_Core_Test_Suites_Packaging_draft