Quality/TestSuite/Bluetooth Driver Test Specification

Latest revision as of 12:14, 17 December 2010

Introduction

Bluez is the current official Bluetooth protocol stack on Linux. It consists of kernel modules and a userspace library, with a set of tools for low-level configuration. It is used (roughly) analogously to the network subsystem, using sockets for communication.

The blts-bluetooth test package (0.2.9) includes functional tests for the Bluez stack, currently consisting of device scan and simple L2CAP and RFCOMM connectivity tests, a test for link-level (only) connection using HCI and simple HCI transfer/receive ACL test data package test. A test case for probing the system for installed Bluetooth drivers and Bluez userspace components is included. Pairing (without services from system Bluetooth daemon) is also tested.

Test Coverage

Software Coverage

The test package uses Bluez sockets with the libbluetooth functions hci_get_route(), hci_open_dev(), hci_inquiry() and hci_read_remote_name(). The L2CAP tests also check some connection parameters with direct hci_send_req() calls. HCI link-level connection is tested with hci_create_connection() and hci_disconnect(). HCI transfer/receive ACL data tests utilize hci_filter structs and socket() function with parameters AF_BLUETOOTH, SOCK_RAW and BTPROTO_HCI while receiving ACL test data. Incoming connections are audited by tracing the sequence with hcidump and analysing the trace.

The HCI name change test cases utilize hci library functions hci_read_local_name(), hci_write_local_name() to change and reset name of the device and hci_read_remote_name() function to verify changed name. The class change test cases utilize hci library functions hci_read_class_of_dev() and hci_write_class_of_dev() to change and reset the class of the device and hci_inquiry() function to verify changed device class.

The HCI controller information test cases utilize hci library functions hci_read_{local/remote}_version(), hci_read_{local/remote}_features(), hci_read_{local/remote}_ext_features() to collect data and the connected link information test cases utilize hci library functions hci_read_rssi(), hci_read_link_quality(), hci_read_transmit_power_level(), hci_read_afh_map(), hci_read_clock(), hci_read_clock_offset() to collect data.

The pairing test performs authentication and pairing for a connected link, according to the Bluetooth specification. Due to possible interference from the Bluetooth daemon in the system, it is shut down for the duration of the test, and the test needs to perform simplified device initialisation.

Current coverage of the Bluetooth HCI API:

• [BT-HCI-coverage.ods]

Hardware Coverage

Hardware coverage includes the data path through the default Bluetooth adapter to the antenna.

Limitations

While the device scan does list all found devices, the actual validation of the result is left to the tester.

Future Improvements

Future test cases may include further tests for HCI link control, and exploring different connection parameter combinations in existing tests. Testing protocols and profiles further up in the Bluetooth stack is currently beyond scope for the test suite.

Running the tests

Hardware setup and test environment

The tests require two test devices with Bluetooth adapters, both running the test program. Several test cases are meant to run in pairs; see below.

hciconfig can be used to find out the required mac addresses:

$ hciconfig

This will print out all active bluetooth devices and their addresses. The output is similar to following:

hci0:   Type: USB
        BD Address: 00:15:83:18:C0:30 ACL MTU: 310:10 SCO MTU: 64:8
        UP RUNNING PSCAN 
        RX bytes:1007 acl:0 sco:0 events:30 errors:0
        TX bytes:367 acl:0 sco:0 commands:28 errors:0

The test environment must have following packages installed:

• libbltscommon1
• bluez
• bluez-hcidump

Building packages

The build environment must have the following packages installed:

• libbltscommon-devel
• bluez-libs-devel

The source package includes the pack.sh script, which is used for building RPM and Debian packages and source tar packages. To build an RPM package with the script, execute this command in the top level source directory of asset:

./pack.sh -r

To get help about the packaging script, execute

./pack.sh -h

The asset can also be built manually by executing:

./autogen.sh
./configure
make

Configuration package

The bluetooth test package is not dependent on any configuration file package like some other blts test packages. The configuration file /etc/blts/blts-bluetooth-tests.cnf is copied in the test device during RPM installation.

Test asset specific instructions

The test package can be installed in the device with a command:

rpm -ivh blts-bluetooth-tests_{version}.rpm 

Test case execution instructions for high-level cases:

The Scan and Software check tests run on a single device, and don't require extra hardware. Having visible Bluetooth devices present during the Scan test may help evaluating the test results.

The L2CAP and RFCOMM test cases are executed with one device running the test "Core-Bluetooth receive L2CAP connection" or "Core-Bluetooth receive RFCOMM connection", and the other the corresponding client test cases.

Test case execution instructions for HCI test cases:

When executing HCI tests, the first step is to verify that hci device is available by typing hciconfig and then try to open and initialize it with command hciconfig hciX up.

Some of the HCI test cases are counter parts to other HCI test cases and these must be executed at the same time with two devices so that the changing/receiving side must be executed first.

Change name with HCI <-> Verify name with remote HCI
The verify test case connects to the other bluetooth device and verifies that the device name is expected one while the changing side is waiting until other bluetooth device is verified name change.

Change class with HCI <-> Verify class with remote HCI
The verify test case connects to the other bluetooth device and verifies that the class of the device is expected one while the changing side is waiting until other bluetooth device is verified class change.

HCI transfer ACL data package <-> HCI receive ACL data package
The transfer test case sends ACL test data packet containing predefined test data while the receive test case waits and receives the ACL test data packet and verifies the content.

Read HCI controller information local <-> Read HCI controller information remote
The remote test case collects information about other bluetooth device, connect to that device and sends collected information package while the local test case waits and receives the information package and verifies that locally collected information matches with received data.

Read connected link information local <-> Read connected link information remote
The remote test case connects to other bluetooth device using multiple L2CAP channels, collects information about connected link and sends collected information package while the local test case waits and receives the information package and verifies that locally collected information matches with received data.

Authentication with pairing as master / slave
As initiating device (master), the test case connects to another Bluetooth device and requests authentication, triggering pairing. The pairing sequence is completed with a predetermined PIN. The slave device waits for the connection and attempts pairing as the request arrives. Either end of the test can be substituted with any Bluetooth device supporting the functionality. The device(s) running this test case must not have the Bluetooth daemon (bluetoothd) running.

Test case arguments

Arguments:

Running test cases

Test cases execution must be done with root privileges.

Test cases marked with bold require that remote device MAC is defined in configuration file or -m command line parameter is given. NOTE: hciconfig tool can be used to check BD Address in the remote device.

Test case execution notices:

Test case descriptions and execution commands:

• Core-Bluetooth scan Parameters ( no parameters )

* Perform inquiry with first available Bluetooth adapter, resolving names for any found remote devices.

* $ blts-bluetooth-tests -en "Core-Bluetooth scan"

• Core-Bluetooth drivers and userspace check Parameters ( no parameters )

* Verify that Bluez-related components are available and correctly installed on the system.

* $ blts-bluetooth-tests -en "Core-Bluetooth drivers and userspace check"

• Core-Bluetooth receive L2CAP connection Parameters ( no parameters )

* Start test server for use with L2CAP tests. The server stops after one client connection has been served.

* This test requires "Core-Bluetooth connect with L2CAP" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth receive L2CAP connection"

• Core-Bluetooth connect with L2CAP Parameters <server-mac>

* Connect with L2CAP to test server running on device <server-mac>, then disconnect.

* This test requires "Core-Bluetooth receive L2CAP connection" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth connect with L2CAP" -m 00:00:00:00:00:00

• Core-Bluetooth ping-pong transfer with L2CAP Parameters <server-mac>

* Connect with L2CAP to test server running on device <server-mac>, transmit a short message and verify the returned reply, then disconnect.

* This test requires "Core-Bluetooth receive L2CAP connection" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth ping-pong transfer with L2CAP" -m 00:00:00:00:00:00

• Core-Bluetooth receive RFCOMM connection Parameters ( no parameters )

* Start test server for use with RFCOMM tests. The server stops after one client connection has been served.

* This test requires "Core-Bluetooth connect with RFCOMM" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth receive RFCOMM connection"

• Core-Bluetooth connect with RFCOMM Parameters <server-mac>

* Connect with RFCOMM to test server running on device <server-mac>, then disconnect.

* This test requires "Core-Bluetooth receive RFCOMM connection" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth connect with RFCOMM" -m 00:00:00:00:00:00

• Core-Bluetooth ping-pong transfer with RFCOMM Parameters <server-mac>

* Connect with RFCOMM to test server running on device <server-mac>, transmit a short message and verify the returned reply, then disconnect.

* This test requires "Core-Bluetooth receive RFCOMM connection" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth ping-pong transfer with RFCOMM" -m 00:00:00:00:00:00

• Core-Bluetooth connect with HCI Parameters <remote-mac>

* Connect just the link to device <remote-mac>, wait a while, the disconnect.

* $ blts-bluetooth-tests -en "Core-Bluetooth connect with HCI" -m 00:00:00:00:00:00

• Core-Bluetooth receive ACL data with HCI Parameters ( no parameters )

* Start receiving ACL data packets from hci device until ACL test data packet arrives.

* This test requires "Core-Bluetooth transfer ACL data with HCI" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth receive ACL data with HCI"

• Core-Bluetooth transfer ACL data with HCI Parameters <remote-mac>

* Connect to device <remote-mac>, and send ACL data packet.

* This test requires "Core-Bluetooth receive ACL data with HCI" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth transfer ACL data with HCI" -m 00:00:00:00:00:00

• Core-Bluetooth change name with HCI Parameters ( no parameters )

* Change local name and wait until other Bluetooth device is verified name change.

* This test requires "Core-Bluetooth verify name with remote HCI" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth change name with HCI"

• Core-Bluetooth verify name with remote HCI Parameters <remote-mac>

* Connect to device <remote-mac>, and verify name of the device.

* This test requires "Core-Bluetooth change name with HCI" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth verify name with remote HCI" -m 00:00:00:00:00:00

• Core-Bluetooth change class with HCI Parameters ( no parameters )

* Change class of device and wait until other Bluetooth device is verified class change.

* This test requires "Core-Bluetooth verify class with remote HCI" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth change class with HCI"

• Core-Bluetooth verify class with remote HCI Parameters <remote-mac>

* Connect to device <remote-mac>, and verify class of the device.

* This test requires "Core-Bluetooth change class with HCI" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth verify class with remote HCI" -m 00:00:00:00:00:00

• Core-Bluetooth reset connection with HCI Parameters <remote-mac>

* Connect, reset connection and connect again to <remote-mac>, wait a while and disconnect.

* $ blts-bluetooth-tests -en "Core-Bluetooth reset connection with HCI" -m 00:00:00:00:00:00

• Core-Bluetooth audit incoming HCI connection Parameters ( no parameters )

* Wait for incoming connection and trace HCI events and commands. Sequence is audited for test pass/fail. Test requires tester to establish some level of Bluetooth connection to the device; for example, starting pairing with a mobile handset and cancelling out is enough. Running any connect test from the counterpart also works. Start this test case first in the test device and after that, when test case is waiting incoming connection, do the pairing step or execute some other HCI connection test case e.g. "Core-Bluetooth connect with HCI".

* $ blts-bluetooth-tests -en "Core-Bluetooth audit incoming HCI connection"

• Core-Bluetooth Read HCI controller information local Parameters ( no parameters )

* Read information about local device, create L2CAP socket and wait for information packet from remote device. Verify that locally collected information matches with received data.

* This test requires "Core-Bluetooth Read HCI controller information remote" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth Read HCI controller information local"

• Core-Bluetooth Read HCI controller information remote Parameters <remote-mac>

* Collect information about remote device (other DUT), connect to device <remote-mac>, and send collected information package with L2CAP connection to remote side.

* This test requires "Core-Bluetooth Read HCI controller information local" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth Read HCI controller information remote" -m 00:00:00:00:00:00

• Core-Bluetooth Read connected link information local Parameters ( no parameters )

* Create L2CAP sockets with multiple channels, read information about connected links, and wait for link information packet from remote device. Verify that locally collected link information matches with received data.

* This test requires "Core-Bluetooth Read connected link information remote" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth Read connected link information local"

• Core-Bluetooth Read connected link information remote Parameters <remote-mac>

* Connect to device <remote-mac> using multiple channels (other DUT), collect information about connected links, and send collected link information package with L2CAP connection to remote side.

* This test requires "Core-Bluetooth Read connected link information local" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth Read connected link information remote" -m 00:00:00:00:00:00

• Core-Bluetooth authentication with pairing as master Parameters <remote-mac>

* Perform pairing with another bluetooth device. The device under test initiates the pairing.

* Running this test requires stopping bluetoothd on the test device (as root, try $ invoke-rc.d bluetooth stop or $ initctl stop bluetoothd depending on distribution).

* Set the <remote-mac> to the Bluetooth MAC of the other device.

* Only one device needs to run the test software; the other can be any Bluetooth device (that can use the given PIN). In this case, the tester needs to enter the PIN manually within the timeout period on the other device. The PIN used is "1234". The timeout is 10 seconds for each test step. If both ends run the test (one as master, one as slave), this is taken care of automatically.

* This test can be run with "Core-Bluetooth authentication with pairing as slave" as counterpart and must be executed when remote side is waiting.

* $ blts-bluetooth-tests -en "Core-Bluetooth authentication with pairing as master" -m 00:00:00:00:00:00

• Core-Bluetooth authentication with pairing as slave Parameters (none)

* Perform pairing with another bluetooth device. The device under test waits for incoming pairing request.

* Running this test requires stopping bluetoothd on the test device (as root, try $ invoke-rc.d bluetooth stop or $ initctl stop bluetoothd depending on distribution).

* Only one device needs to run the test software; the other can be any Bluetooth device (that can use the given PIN). In this case, the tester needs to enter the PIN manually within the timeout period on the other device. The PIN used is "1234". The timeout is 10 seconds for each test step. If both ends run the test (one as master, one as slave), this is taken care of automatically.

* This test can be run with "Core-Bluetooth authentication with pairing as master" as counterpart and must be executed first.

* $ blts-bluetooth-tests -en "Core-Bluetooth authentication with pairing as slave"

Configuration file examples

To run test cases with variable data you need to specify where configuration file is located or if no configuration file is given, test driver tries to use default /etc/blts/blts-bluetooth-tests.cnf

The configuration file must be edited to match your test environment (optionally command line parameter -m can be used to override mac_address parameter)

To run "Core-Bluetooth connect with HCI" case for example, execute the command:

blts-bluetooth-tests -l /var/log/tests/Core-Bluetooth_connect_with_HCI.log -en "Core-Bluetooth connect with HCI" -m 00:00:00:00:00:00 -C /etc/blts/blts-bluetooth-tests.cnf

Example of BT configuration file:

# Parameter configuration for blts-bluetooth

# MAC address - used in test cases where another BT device is needed
# NOTE: command line parameter -m (mac_address) overrides this parameter

[globals]
	mac_address "00:00:00:00:00:00"
[end_globals]

Module Design and Architecture

The test module can be compiled to self-sufficient command line program. Both Debian and RPM packaging is also provided.

Architecture diagram

References

Change History