Quality/QA-tools/MCTS test automation design/meeting-2010-11-30

Architecture

• Agreed architecture appeared to be something which had something from both option 1 and option 2 presented.
  • Option 1: Media:Components_ext_verification_option1.png
  • Option 2: Media:Components_ext_verification_option2.png

Messaging System

• agreed not to use [STAF]
• system needs to be able to run in Linux and Windows
• independent system
  • test case/testrunner/single component can join/communicate with the system
• needs to provide a two-way communication
  • events/ signals
  • variable exchange in messages
  • synchronous/ asynchronous messages
• test case might need real-time messaging
• AMQP seems to be the best option to handle messaging
  • provides messaging logging
• AMQP implementations
  • RabbitMQ
    • http://www.rabbitmq.com/
    • OTS uses this. We don't have to use the same as the system doesn't have to communicate with OTS
    • poor C/C++ support. C/C++ bindings not available for the AMQP version RabbitMQ uses. An option would be to contribute the support.
  • OpenAMQ
    • http://www.openamq.org/
    • has C-language binding provided by OpenAMQ
  • Apache QPid
    • http://qpid.apache.org/
  • ZeroMQ
    • http://www.zeromq.org/
    • Doesn't provide message broker. Message delivery is therefore not guaranteed.

Test Definition Changes

• support to define, where the command is executed (host, DUT, external device) (http://wiki.meego.com/User:Esmietti#2._Doesn.27t_support_host_and_device_commands_in_the_same_tests.xml)
• support to define measurement values (http://wiki.meego.com/User:Esmietti#4._Measurement_values)

Testrunner-lite Changes

• executing command in different devices
• Pass/ Fail resolution based on measurement values
• Plugin to communicate with messaging system

Device Interfaces

• Similar external devices should have the same interface
  • two different network simulators should have same interface
  • create voice call, answer to call, measure throughput

Test Execution

• Both manual and automated execution shall be possible
• Manual mode to be provided with testrunner-lite/ testrunner-ui. Which would consume messages instead of external devices and requests manual verification for user.

Other Things Discussed

• Device capabilities
  • information provided in tests.xml
  • devices to register to "resource manager" and telling capabilities
• MIN master-slave interface could be made more general and utilized to handle communication from device to messaging
• To support test case development some sort of stubs are needed to simulate external devices
• Desing to be moved in wiki