Quality/TestSuite/Camera Driver Test Specification

Revision as of 07:46, 8 December 2010

Introduction

V4L2 API is a kernel interface for analog radio and video capture and output drivers.

Blts-v4l2 package (0.3.8) is meant to test V4L2 functionality. The blts-v4l2 package tests camera driver functionality by checking e.g. capabilities and controls. There is also tests that stream frames (using XVideo) and take snapshots (saved as JPG pictures) using different resolutions. Some of these test cases also measure FPS rates. The audio part of the V4L2 API is not in the scope.

Test Coverage

The test cases are:

• Core-Camera-Open device: Try to open a V4L2 device
• Core-Camera-Read device: Open V4L2 device to requested width and height
• Core-Camera-Check device controls: Enumerate through V4L2 device controls
• Core-Camera-Check device capabilites: Check and print V4L2 device capabilities
• Core-Camera-Check device formats: Enumerate and print V4L2 device formats
• Core-Camera-Measure device stepping down resolution: Step down resolutions and calculate FPS using V4L2 device
• Core-Camera-Stream from device to screen: Stream frames from V4L2 device to screen using XVideo
• Core-Camera-Measure device FPS with defined resolution: Use a given resolution and calculate FPS using V4L2 device
• Core-Camera-Take picture as JPEG with device: Take one frame from V4L2 device and create a JPG file
• Core-Camera-Device enumeration: Enumerate V4L2 input and output devices and print supported standards
• Core-Camera-Stream from device and crop corners: Stream frames from V4L2 device to screen using XVideo and use cropping
• Core-Camera-Read from device using poll: As "Measure device FPS", but use poll() to detect device readiness.
• Core-Camera-Test standard control settings: Enumerate all standard controls, and test each available control with suitable values
• Core-Camera-Test extended control settings: Enumerate all extended controls, and test each available control with suitable values
• Core-Camera-Check device FD priority: Init a device and check if it supports priorities
• Core-Camera-Stream from device and vary frame rate: Stream frames from V4L2 device to screen and vary frame rates and calculate FPS
• Core-Camera-Get stream quality setting: Stream frames from V4L2 and get the quality setting
• Core-Camera-Set stream quality setting to high: Set stream quality setting to high, stream from the device and check the setting is used
• Core-Camera-Check debug capability: Get state from driver to kernel log, API function doesn't need to be supported
• Core-Camera-Select output and stream: Selects and tests possible outputs of device
• Core-Camera-Select input and stream: Selects and tests possible inputs of device.
• Core-Camera-Test input device standards: Loops through V4L2 input devices and tests video standards

Software Coverage

Current IOCTL coverage for v4l2 tests:

• [V4l2-coverage.ods]

Hardware Coverage

At the moment test cases are configured to test main and front cameras. The audio functionality of the test device is excluded.

Limitations

• Test suite functionality depends on implementation of used device driver in v4l2.

Future Improvements

• More coverage with tests for v4l2.

Running the tests

Hardware setup and test environment

The test environment must have following packages installed:

• libbltscommon1
• libbltspixelformat1
• libjpeg
• libX11
• libXv

Building packages

The v4l2 test package comes with a packaging script that can be used to build tarball out of the sources, RPM package or Debian package. To build RPM package, in the root directory execute command:

./pack.sh -r

To get packaging script help, execute command:

./pack.sh -h

The building environment requires the following RPM packages installed:

• libbltscommon-devel
• libbltspixelformat-devel
• libjpeg-devel
• libX11-devel
• libXv-devel

Configuration package

The blts-v4l2-tests package is dependent on blts-v4l2-tests-config configuration file package. The packaging script pack.sh creates RPM package blts-v4l2-tests-config-example_{version}.rpm and it must be installed first. The configuration file /etc/blts/blts-v4l2-tests.cnf is copied in the test device during RPM installation.

The configuration file package can be installed in the device with a command:

rpm -ivh blts-v4l2-tests-config-example_{version}.rpm

Test asset specific instructions

The test package can be installed in the device with a command:

rpm -ivh blts-v4l2-tests_{version}.rpm 

The control setting test cases support external image verification tool.

Image verification

The standard and extended control tests support verification of the camera snapshot image using an external tool. Image verification is enabled by specifying values for the image_save_path and image_verify_tool global configuration parameters, and by giving the -i command line flag when running the test. Once a camera snapshot has been saved for a certain control setting, the tool given in image_verify_tool is run with a command line like this: verify_tool <snapshot image name> <control name> <V4L2 control id> <control value>

The verification tool should return zero when the image is verified successfully, and a non-zero value when the verification fails. The output of the tool is appended to the log file of the test.

Control value limits and test values

The minimum and maximum values reported by the V4L2 driver can be compared with values specified in the configuration file. The test succeeds if the values match exactly, and fails otherwise. The minimum and maximum values are specified as global configuration parameters, with a special naming format:

dev<number of device>_<escaped control name>_min = <minimum value> dev<number of device>_<escaped control name>_max = <maximum value>

As the global configuration value identifiers cannot contain spaces or any of the /()[] characters, which may be present in the name of a V4L2 control, the control name must be escaped by replacing spaces with underscores, and by removing any prohibited characters from the name. For example, values for the control named Exposure Time (us) on the device /dev/video0 could be provided like this:

dev0_Exposure_Time_us_min = 1000 dev0_Exposure_Time_us_max = 100000

By default, the control tests use the minimum and maximum values reported by the V4L2 driver to test a control. Additional test values can be provided as global configuration parameters in the configuration file, using a format similar to the min/max format:

dev<number of device>_<escaped control name>_testvalues = "<space-delimited list of values>"

For example:

dev0_Exposure_Time_us_testvalues = "25000 50000 75000"

Test case arguments

Arguments:

Test cases must be run with root privileges.

Test cases

Test case execution notices:

Test case descriptions and execution commands:

• Core-Camera-Open device

* Try to open a V4L2 device

* $ blts-v4l2-tests -en "Core-Camera-Open device"

• Core-Camera-Read device

* Open V4L2 device to requested width and height

* $ blts-v4l2-tests -en "Core-Camera-Read device"

• Core-Camera-Check device controls

* Enumerate through V4L2 device controls

* $ blts-v4l2-tests -en "Core-Camera-Check device controls"

• Core-Camera-Check device capabilites

* Check and print V4L2 device capabilities

* $ blts-v4l2-tests -en "Core-Camera-Check device capabilites"

• Core-Camera-Check device formats

* Enumerate and print V4L2 device formats

* $ blts-v4l2-tests -en "Core-Camera-Check device formats"

• Core-Camera-Measure device stepping down resolution

* Step down resolutions and calculate FPS using V4L2 device

* $ blts-v4l2-tests -en "Core-Camera-Measure device stepping down resolution"

• Core-Camera-Stream from device to screen

* Stream frames from V4L2 device to screen using XVideo

* $ blts-v4l2-tests -en "Core-Camera-Stream from device to screen"

• Core-Camera-Measure device FPS with defined resolution

* Use a given resolution and calculate FPS using V4L2 device

* $ blts-v4l2-tests -en "Core-Camera-Measure device FPS with defined resolution"

• Core-Camera-Take picture as JPEG with device

* Take one frame from V4L2 device and create a JPG file

* $ blts-v4l2-tests -en "Core-Camera-Take picture as JPEG with device"

• Core-Camera-Device enumeration

* Enumerate V4L2 input and output devices and print supported standards

* $ blts-v4l2-tests -en "Core-Camera-Device enumeration"

• Core-Camera-Stream from device and crop corners

* Stream frames from V4L2 device to screen using XVideo and use cropping

* $ blts-v4l2-tests -en "Core-Camera-Stream from device and crop corners"

• Core-Camera-Read from device using poll

* Read/stream frames from device as with measure device FPS case, but use

* poll() instead of select() to find out when the device is ready.

* $ blts-v4l2-tests -en "Core-Camera-Read from device using poll"

• Core-Camera-Test standard control settings

* Enumerate all standard controls, and test each available control with

* min/max values and any configured values.

* $ blts-v4l2-tests -en "Core-Camera-Test standard control settings"

• Core-Camera-Test extended control settings

* Enumerate all extended controls, and test each available control with

* min/max values and any configured values.

* $ blts-v4l2-tests -en "Core-Camera-Test extended control settings"

• Core-Camera-Check device FD priority

* Init a device and check if it supports priorities - VIDIOC_G_PRIORITY

* $ blts-v4l2-tests -en "Core-Camera-Check device FD priority"

• Core-Camera-Stream from device and vary frame rate

* Stream frames from V4L2 device to screen using default frame rate and

* calculate FPS, change frame rate and repeat streaming to compare FPS values.

* $ blts-v4l2-tests -en "Core-Camera-Stream from device and vary frame rate"

• Core-Camera-Get stream quality setting

* Stream frames from V4L2 device and get a default quality setting

* $ blts-v4l2-tests -en "Core-Camera-Get stream quality setting"

• Core-Camera-Set stream quality setting to high

* Stream frames from V4L2 device, set stream quality setting to high, stream from the device and check that high setting is used

* $ blts-v4l2-tests -en "Core-Camera-Set stream quality setting to high"

• Core-Camera-Check debug capability

* Requests the driver to dump current state to kernel log. This API

* call is specified so that driver doesn't have to support this.

* $ blts-v4l2-tests -en "Core-Camera-Check debug capability"

• Core-Camera-Select input and stream

* Tries different inputs and streams from those to the screen.

* If no input available with device this test will force test - VIDIOC_S_INPUT

* $ blts-v4l2-tests -en "Core-Camera-Select input and stream"

• Core-Camera-Select output and stream

* Tries different outputs of device.

* If no output available with device this test will force test - VIDIOC_S_OUTPUT

* $ blts-v4l2-tests -en "Core-Camera-Select output and stream"

• Core-Camera-Test input device standards

* Loops through V4L2 input devices and tries to get current video standard,

* then enumerates and sets supported standards one by one.

* $ blts-v4l2-tests -en "Core-Camera-Test input device standards"

Configuration file examples

To run test case with variable data you need to specify where configuration file is located or if no configuration file is given, test driver tries to use default /etc/blts/blts-v4l2-tests.cnf

Edit the configuration file yourself if necessary.

To run "Core-Camera-Stream from device to screen" case you can use: blts-v4l2 -l /var/log/tests/Core-Camera-Stream_from_device_to_screen.log -en "Core-Camera-Stream from device to screen" -C example-blts-v4l2.cnf

Example of Stream case variation (example-blts-v4l2.cnf):

# Valid video devices for HW.
[parameter]
   name video_device
   const "/dev/video0" "/dev/video1"
[end_parameter]

[parameter]
   name io_method_stream
   const "MMAP" "userptr"
[end_parameter]

# Pixel formats (fourcc, as kernel v4l2 entries)
[parameter]
   name camera_pixel_format
   const "YUYV"
[end_parameter]

[parameter]
   name camera_resolution_stream_all
   const "4016x3016" "2400x1812" "848x480" "640x480" "320x240"
[end_parameter]

[test]
   name "Core-Camera-Stream from device to screen"
   params video_device camera_pixel_format io_method_stream camera_resolution_stream_all
[end_test]

Module Design and Architecture

Architecture diagram

References

Change History