Contents 1 MWTS-Asset : USB 2 Introduction 2.1 Overview 2.2 Getting the Latest Version 2.3 Support & Contacts 2.4 Documentation 2.5 Licensing 2.6 Supported Devices and SW 2.7 Test Cases 2.8 Coverage 3 Running the Tests 3.1 Hardware setup and test environment 3.2 Building packages 3.3 Running test cases 3.4 Test result analysis 3.5 Troubleshooting 3.6 Configuration file examples 4 Module Design and Architecture 4.1 MIN Scripter Interface 5 Known issues 6 Future improvements 7 References 7.1 Version History

MWTS-Asset : USB

Introduction

Overview

MWTS-USB contains tests for USB network functionalities: upload/download throughput and reliability.

Throughput is measured with iperf measurement tool and also by transferring real files. File sizes are 512kB, 5MB and 640MB. Scp is used in file transfer.

USB mass storage is out of scope. Mass storage test cases are located in MWTS-filesystem.

Getting the Latest Version

• The source code is available at:

http://gitorious.org/qa-tools/mcts/trees/master/mcts-mwts/generic/mwts-usb

Support & Contacts

• In case of a problem read carefully through this document. Then search bugzilla to see if someone else has had similar problems.
• If your questions is still valid, please prefer the public #meego-qa-mcts @ irc.freenode.net IRC-channel or meego-qa-tools mailing list. This way you get the best possible attention to your questions.
• If you think that you need to directly contact package maintainer then contact
  • Rauno Vartiainen, rauno.vartiainen@digia.com

Documentation

• See document MWTS.README for detailed general information about mwts test modules.
• Run command

doxygen

to generate html documentation

Licensing

• Please see the file called COPYING

Supported Devices and SW

Supported devices

• N900
• nCDK

Test Cases

Test cases can be found in MIN scripter files

File name	Description
mwts-usb.cfg	Basic functional test cases for USB
mwts-usb-nft.cfg	Test cases for verifying USB network performance and reliability
mwts-usb-slave.cfg	Slave test cases for master-slave

Basic functional test cases for USB

Test case name	Description
FUTE-USB-Send_throughput
FUTE-USB-Receive_throughput
FUTE-USB-Send_512kb_file
FUTE-USB-Receive_512kb_file

Test cases for verifying USB network performance and reliability

Test case name	Description
NFT-USB-Send_throughput-THRO
NFT-USB-Receive_throughput-THRO
NFT-USB-Send_512kb_file-THRO
NFT-USB-Receive_512kb_file-THRO
NFT-USB-Send_512kb_file-ITER
NFT-USB-Receive_512kb_file-ITER
NFT-USB-Send_5MB_file-THRO
NFT-USB-Receive_5MB_file-THRO
NFT-USB-Send_5MB_file-ITER
NFT-USB-Receive_5MB_file-ITER
NFT-USB-Send_640MB_file-THRO
NFT-USB-Receive_640MB_file-THRO
NFT-USB-Send_throughput_2h-LOLA
NFT-USB-Receive_throughput_2h-LOLA

Slave test cases for master-slave

Test case name	Description
FUTE-USB-Start_throughput_server-SLAVE
FUTE-USB-Send_throughput-SLAVE
NFT-USB-Send_throughput-ITER-SLAVE
NFT-USB-Send_throughput_2h-LOLA-SLAVE

Coverage

Coverage target

• The aim of this test module is to verify and measure USB networking throughput and reliability on the target device.

Running the Tests

Hardware setup and test environment

Required HW

• One test device with MeeGo (all MeeGo versions should work)
• One desktop PC
• USB cable between desktop PC and the target device

Required SW

• Qt version 4.7.0
• mwts-common (use the latest version)
• iperf version 2.0.2 or later
  NOTE: iperf must be installed into location available in PATH variable.

Building packages

To build package from source code:

• using build script

sh ./buildrpm

• manually

qmake
make
make install (as root)

• Packages will be provided

Running test cases

In order to execute all tests on MWTS-USB, you need the following:

• Passwordless SSH login to target device from desktop PC
  • Create your own ssh keys in the desktop PC by typing
    ssh-keygen
    in the command line and leave passphrase empty
  • Copy public SSH pair to the target device by typing
    ssh-copy-id root@192.168.2.15
  • Now you should be able to create SSH connection without password
• Default path for test files is /usr/share/mwts/testdata/
  • Create files using dd:
    sudo dd if=/dev/zero of=/usr/share/mwts/testdata/0512k.bin bs=1024 count=5
sudo dd if=/dev/zero of=/usr/share/mwts/testdata/5MB.bin bs=1024 count=5000
sudo dd if=/dev/zero of=/usr/share/mwts/testdata/640MB.bin bs=1024 count=640000

After you have successfully installed required packages to the target device and to your desktop PC you can execute test cases.

Connect USB cable to device and configure IP addresses. For desktop PC use 192.168.2.14 and for target device use 192.168.2.15. Check that you can use SSH without password.

Give command

mind

in your target device to start the MIN daemon, which will run the slave tests automatically.

Give command

min -s 192.168.2.15

in your desktop PC to start MIN in master-slave mode. This brings up MIN user interface where you can select and start test cases.

Alternatively you can specify the name of the test case as a command line parameter

min -s 192.168.2.15 -c -t 'FUTE-USB-Send_throughput'

Note that you have to execute send test case before receive test case! Without that the receive test case will fail because there will be no file to transfer in the target device.

Check the MWTS.README for general instructions.

Test result analysis

Test result are available in the MIN UI. In addition, test result log files can be found from device in directory

/var/log/tests/

Result files are named according to test cases, <test_case_name>.result file contains overall information about test run and measured values.

Log file are created if the environment variable MWTS_DEBUG has been set to 1.

export MWTS_DEBUG=1 

To print debug info in console set environment variable MWTS_LOG_PRINT to 1.

export MWTS_LOG_PRINT=1

Troubleshooting

Configuration file examples

No configuration file is used with MWTS-USB test asset.

Module Design and Architecture

MIN Scripter Interface

Table below describes the MIN scripter interface functions. Please refer to MIN documentation (1) for further technical details.

MIN	Parameters	Description
SendFile	ip <target_device_ip> srcfile <file_to_send> destfile <destination_path>	Measure file send throughput. Parameter ip defines the IP address of the target device. Parameter srcfile is the source file that is sent. Parameter destfile defines the destination path on the target device for the file being sent.
ReceiveFile	ip <target_device_ip> srcfile <file_to_receive> destfile <destination_path>	Measure file receive throughput. Parameter ip defines the IP address of the target device. Parameter srcfile is the source file that is received. Parameter destfile defines the destination path for the file being received.

Examples:
SendFile ip 192.168.2.15 srcfile /usr/share/mwts/testdata/0512k.bin destfile /home/meego/
ReceiveFile ip 192.168.2.15 srcfile /usr/share/mwts/testdata/0512k.bin destfile /tmp/

MWTS-USB test asset also uses the following MIN functions from MWTS-Common:

MIN	Parameters	Description
StartClientThroughput	<server_ip> <time>	Starts iperf in client mode. Parameter server_ip defines the IP address where iperf connects. Parameter time defines how long iperf is run. Time parameter is optional.
StartServerThroughput		Starts iperf in server mode. No parameters.
StopThroughput		Stops the iperf tool, if it is still running. No parameters.

Known issues

When transferring a big file (640MB) eMMC must be mounted in the device. Otherwise the device will run out of disk space.

Future improvements

References

• (1) https://garage.maemo.org/docman/?group_id=853

Version History

Version	Revision	Date	Handled By	Comments
0.0.1	r1	05 Jan-2011	Rauno Vartiainen	Initial version