Meego Wiki
Views

Quality/Compliance tools

From MeeGo wiki
< Quality(Difference between revisions)
Jump to: navigation, search
(dist-compliance-tool)
(Result Format)
 
(12 intermediate revisions not shown)
Line 1: Line 1:
= Introduction =
= Introduction =
-
Compliance tools are used to check MeeGo compliance for MeeGo Distributions or MeeGo Applications. The compliance reference input is the MeeGo core package/file list and a minimal MeeGo reference image created by MIC2, which will contain all the required core packages, as well as src rpms.
+
The compliance tools are used to check distributions using the MeeGo software stack, and applications targeting such distributions, for compliance with the MeeGo requirements. Checking is done against reference information which tells the tools what is mandatory.
-
For MeeGo distribution compliance checking, the input needs to include the OS image and the src.rpms for all the packages. And for MeeGo Application compliance checking, the input is the application rpm file.
+
For MeeGo distribution compliance checking, the candidate distribution needs to supply an image of the distribution in suitable format, which will include source rpm packages. The other input to the tool is an official MeeGo reference image (which also includes source packages), and a list of the mandatory packages.
 +
 
 +
For application compliance checking, the candidate rpm package is supplied as input.  Other inputs are an official MeeGo reference image (this one does not include sources), which will be used to test that installation actually works, as well as a list of packages and a list of packages that make up the MeeGo API.  The latter two are used to determine valid dependencies and to perform a check of whether the application is a MeeGo API application or one that uses different/additional libaries.
The tools will generate report files (in HTML currently) to show the result.
The tools will generate report files (in HTML currently) to show the result.
= Installation =
= Installation =
-
Compliance tools can be installed using released rpm and deb packages, or installed by source code.
+
The compliance tools can be installed using released rpm and deb packages (available for a variety of distrubutions), or installed from source code if using an unsupported distribution.
== Packages and Repos ==
== Packages and Repos ==
-
The package in the MeeGo OBS server for this project is [http://build.meego.com/package/show?package=meego-compliance-tools&project=devel:tools:building devel:tools:building/meego-compliance-tools]. And all the rpm and deb packages for release can be generated by the MeeGo OBS server. The following Linux distributions are supported:
+
 
 +
The code lives in the MeeGo git repository, and will be built by the OBS for all the supported distributions automatically.  The home for this is  
 +
[http://build.meego.com/package/show?package=meego-compliance-tools&project=devel:tools:building devel:tools:building/meego-compliance-tools].  
 +
 
 +
The following Linux distributions are supported at the moment by OBS (this is a changing list, check the repositories which may be more up to date than this page):
* Debian 5.0
* Debian 5.0
* xUbuntu 9.10, 10.04, 10.10
* xUbuntu 9.10, 10.04, 10.10
-
* Fedora 11, 12, 13
+
* Fedora 11, 12, 13, 14
-
* openSUSE 11.1, 11.2, 11.3
+
* openSUSE 11.1, 11.2, 11.3, 11.4
-
* MeeGo 1.0, Trunk (1.2)
+
* MeeGo 1.0, 1.1, Trunk (1.2)
-
All the corresponding package repos can be used to install the tools conveniently,and will be published in http://repo.meego.com.
+
The packages appear in the tools repository at [http://repo.meego.com/MeeGo/tools/repos/ tools/repos]. rpm-based distributions have usable .repo files in each distribution directory which can be used by zypper or yum, but the deb-based distributions do not; the sources.list.d entry needs to be constructed from the path.  For example, for Ubuntu 10.10 the following in /etc/apt/sources.list.d/meego-tools.list:
 +
  deb http://repo/meego.com/MeeGo/tools/repos/ubuntu/10.10/ ./
-
For rpm packaging ditros, there are three packages available:
+
For distros which use rpms natively, there are three packages available:
* meego-compliance-tools (common data, will be installed automatically as the dependency of the two packages below)
* meego-compliance-tools (common data, will be installed automatically as the dependency of the two packages below)
* meego-dist-compliance-tools (for MeeGo distro compliance checking)
* meego-dist-compliance-tools (for MeeGo distro compliance checking)
* meego-app-compliance-tools. (for MeeGo application compliance checking)
* meego-app-compliance-tools. (for MeeGo application compliance checking)
 +
 +
For distros which use debs natively, only the meego-compliance-tools package is needed.
== Source Code ==
== Source Code ==
-
The tools can also be installed by source code. After fetching the code, under the code directory, run:
+
The tools can also be installed from source (which is all Python). After fetching the code, under the code directory, run:
   $ [sudo] make install
   $ [sudo] make install
Line 33: Line 42:
   [http://meego.gitorious.org/meego-developer-tools/meego-compliance-tools meego-compliance-tools@gitorious.org]
   [http://meego.gitorious.org/meego-developer-tools/meego-compliance-tools meego-compliance-tools@gitorious.org]
-
== MIC2 ==
+
A typical fetch would be:
-
To prepare the images used in compliance tools, you need [[Image_Creation|MIC2]] with new features (version 0.22.0 or later).
+
  git clone git://gitorious.org/meego-developer-tools/meego-compliance-tools.git
 +
 
 +
== Compliance Image Preparation ==
 +
 
 +
To prepare the images used in compliance tools, you need [[Image_Creation|MIC2]] with new features (version 0.22.0 or later).  The mic2 package from the same tools repository described above will provide a current version.
=  Features =
=  Features =
== dist-compliance-tool ==
== dist-compliance-tool ==
-
It executes compliance checking for MeeGo distributions. The following features are supported:
+
Executes compliance checking for distributions build from the MeeGo code base. The following features are supported:
-
* Binary pkg comparison:
+
* Binary package comparison:
** Exist in target image?
** Exist in target image?
** Version matches the requirement?
** Version matches the requirement?
-
*** flexible version checking: support rules with comparison operators like ">=" in core pkg list (kernel >= 2.6.35)
+
*** flexible version checking: support rules with comparison operators like ">=" in core package list (kernel >= 2.6.35)
* Content in source rpm comparison:
* Content in source rpm comparison:
** This step will be skipped if binary rpm checking fails
** This step will be skipped if binary rpm checking fails
Line 50: Line 63:
** Patches and the content
** Patches and the content
-
* File checking: For example check the following file if exists or not. (/usr/lib/libGLES2.so.1)
+
* File checking:  
 +
** Exist in target image
 +
*** For example check /usr/lib/libGLES2.so.1 if exists in target image or not.
== app-compliance-tool ==
== app-compliance-tool ==
Line 63: Line 78:
Before using the tools, the reference image needs to be created by using MIC2 (version 0.22.0 or later):
Before using the tools, the reference image needs to be created by using MIC2 (version 0.22.0 or later):
   $ [sudo] mic-image-creator --run-mode=0 --cache=mycachedir --format=fs  --config=<ks file> --package=tar.bz2 --include-source
   $ [sudo] mic-image-creator --run-mode=0 --cache=mycachedir --format=fs  --config=<ks file> --package=tar.bz2 --include-source
-
The sample ks file can be downloaded at:
+
The sample ks file is at /usr/share/compliance-tools/share dir after installation or can be downloaded at:
-
   [[media:Meego-minimal-compliance-1.0.99.1.ks]]
+
   Meego-1.1:[[media:Meego-minimal-compliance-1.0.99.1.ks|MeeGo-minimal-compliance-1.0.99.1.ks]]
-
The sample core list file can be downloaded at:
+
  Meego-1.2:[[media:MeeGo-minimal-compliance-1.1.99.7.ks|MeeGo-minimal-compliance-1.1.99.7.ks]]
-
   [[media:Core-list-1.0.99.1‎]]
+
The sample core list file is at /usr/share/compliance-tools/share dir after installation or can be downloaded at:
 +
   Meego-1.1:[[media:Core-list-1.0.99.1‎|Core-list-1.0.99.1]]
 +
  Meego-1.2:[[media:Meego-full-compliance-pkg-1.2.txt|Core-list-1.1.99.7]]
And the ks file and core list should be synced with and conform to the latest [[Quality/Compliance|MeeGo compliance spec]].
And the ks file and core list should be synced with and conform to the latest [[Quality/Compliance|MeeGo compliance spec]].
 +
 +
For application compliance checking, the source packages are not needed, so the --include-source option can be omitted.
 +
 +
In future, official pre-made images will be available, when this occurs the appropriate image should be downloaded instead of constructed.
To use the dist compliance check tool (dist_compliance), the needed target image can also be generated by mic2 using the above options. But the target images
To use the dist compliance check tool (dist_compliance), the needed target image can also be generated by mic2 using the above options. But the target images
-
will not be limited to this specific format(fs in tar.bz2). If the exist target image is in other formats, please unpack it to a directory manually using format
+
will not be limited to this specific format (fs in tar.bz2). If the exist target image is in other formats, please unpack it to a directory manually using format
specific instructions and use this directory as the command line input.
specific instructions and use this directory as the command line input.
Line 81: Line 102:
       -h, --help            show this help message and exit
       -h, --help            show this help message and exit
       -c CORELIST, --corelist=CORELIST
       -c CORELIST, --corelist=CORELIST
-
                             File to store core list of MeeGo
+
                             File containing MeeGo Core package list
       -r REF, --reference=REF
       -r REF, --reference=REF
-
                             Path of reference image or unpacked directory
+
                             Path to reference image or unpacked directory
-
       -t TGT, --target=TGT  Path of target image or unpacked directory
+
       -t TGT, --target=TGT  Path to target image or unpacked directory
       -o OUTDIR, --outdir=OUTDIR
       -o OUTDIR, --outdir=OUTDIR
                             Output directory of report
                             Output directory of report
-
       -d, --debug          Save intermedia log data for debugging
+
       -d, --debug          Save intermediate log data for debugging
== Apps Compliance Check ==
== Apps Compliance Check ==
Line 97: Line 118:
       -h, --help            show this help message and exit
       -h, --help            show this help message and exit
       -r REFIMAGE, --refimage=REFIMAGE
       -r REFIMAGE, --refimage=REFIMAGE
-
                             Filepath of reference image
+
                             Path to reference image
       -c CORELIST, --corelist=CORELIST
       -c CORELIST, --corelist=CORELIST
-
                             File to store core list of MeeGo
+
                             File containing MeeGo Core package list
       -m MEEGO_API_PKGLIST, --meego_api_pkglist=MEEGO_API_PKGLIST
       -m MEEGO_API_PKGLIST, --meego_api_pkglist=MEEGO_API_PKGLIST
-
                             File to store MeeGo API package list of MeeGo
+
                             File containing MeeGo API package list
       -o OUTDIR, --outdir=OUTDIR
       -o OUTDIR, --outdir=OUTDIR
                             Output directory of report
                             Output directory of report
-
       -d, --debug          Save intermedia log data for debugging
+
       -d, --debug          Save intermediate log data for debugging
The sample MeeGo API package list file can be downloaded at:  
The sample MeeGo API package list file can be downloaded at:  
-
     [[media:MeeGo-API-pkglist.txt]]
+
     [[media:MeeGo-API-pkglist.txt|MeeGo-API-pkglist.txt]]
= Result Format =
= Result Format =
-
Currently, the output of checking tools will be store in HTML files.
+
Currently, the output of checking tools will be stored in HTML files.
-
Screen capture of sample distribution report:
+
 
 +
A distribution check report will look like this:
[[File:DistSummaryReport.png|none|600px|Sample Report]]
[[File:DistSummaryReport.png|none|600px|Sample Report]]
 +
 +
An application check report will look like this:
 +
 +
[[File:Appcheck2.png|none|600px|Sample Report]]
= Bug Report =
= Bug Report =
-
If problems were found in compliance tools, please file bugs in bugs.meego.com.
+
If problems were found in the compliance tools, please file bugs at bugs.meego.com. There is a top-level category for compliance - use "MeeGo Spec and Compliance", product "Compliance Tools".
-
The corresponding section is: "Development Tools/Tools".
+
= TODO =
= TODO =
Line 124: Line 149:
* use new image format with separated src.rpm pack
* use new image format with separated src.rpm pack
* Web UI for commenting the result
* Web UI for commenting the result
 +
* Web service for app checking?
 +
* Eliminate need for a reference compliance image for dist checking

Latest revision as of 22:46, 6 June 2011

Contents

Introduction

The compliance tools are used to check distributions using the MeeGo software stack, and applications targeting such distributions, for compliance with the MeeGo requirements. Checking is done against reference information which tells the tools what is mandatory.

For MeeGo distribution compliance checking, the candidate distribution needs to supply an image of the distribution in suitable format, which will include source rpm packages. The other input to the tool is an official MeeGo reference image (which also includes source packages), and a list of the mandatory packages.

For application compliance checking, the candidate rpm package is supplied as input. Other inputs are an official MeeGo reference image (this one does not include sources), which will be used to test that installation actually works, as well as a list of packages and a list of packages that make up the MeeGo API. The latter two are used to determine valid dependencies and to perform a check of whether the application is a MeeGo API application or one that uses different/additional libaries.

The tools will generate report files (in HTML currently) to show the result.

Installation

The compliance tools can be installed using released rpm and deb packages (available for a variety of distrubutions), or installed from source code if using an unsupported distribution.

Packages and Repos

The code lives in the MeeGo git repository, and will be built by the OBS for all the supported distributions automatically. The home for this is devel:tools:building/meego-compliance-tools.

The following Linux distributions are supported at the moment by OBS (this is a changing list, check the repositories which may be more up to date than this page):

  • Debian 5.0
  • xUbuntu 9.10, 10.04, 10.10
  • Fedora 11, 12, 13, 14
  • openSUSE 11.1, 11.2, 11.3, 11.4
  • MeeGo 1.0, 1.1, Trunk (1.2)

The packages appear in the tools repository at tools/repos. rpm-based distributions have usable .repo files in each distribution directory which can be used by zypper or yum, but the deb-based distributions do not; the sources.list.d entry needs to be constructed from the path. For example, for Ubuntu 10.10 the following in /etc/apt/sources.list.d/meego-tools.list:

 deb http://repo/meego.com/MeeGo/tools/repos/ubuntu/10.10/ ./

For distros which use rpms natively, there are three packages available:

  • meego-compliance-tools (common data, will be installed automatically as the dependency of the two packages below)
  • meego-dist-compliance-tools (for MeeGo distro compliance checking)
  • meego-app-compliance-tools. (for MeeGo application compliance checking)

For distros which use debs natively, only the meego-compliance-tools package is needed.

Source Code

The tools can also be installed from source (which is all Python). After fetching the code, under the code directory, run:

 $ [sudo] make install

The source code is hosted at:

 meego-compliance-tools@gitorious.org

A typical fetch would be:

 git clone git://gitorious.org/meego-developer-tools/meego-compliance-tools.git

Compliance Image Preparation

To prepare the images used in compliance tools, you need MIC2 with new features (version 0.22.0 or later). The mic2 package from the same tools repository described above will provide a current version.

Features

dist-compliance-tool

Executes compliance checking for distributions build from the MeeGo code base. The following features are supported:

  • Binary package comparison:
    • Exist in target image?
    • Version matches the requirement?
      • flexible version checking: support rules with comparison operators like ">=" in core package list (kernel >= 2.6.35)
  • Content in source rpm comparison:
    • This step will be skipped if binary rpm checking fails
    • Spec filename
    • Spec content
    • Sources and md5sum
    • Patches and the content
  • File checking:
    • Exist in target image
      • For example check /usr/lib/libGLES2.so.1 if exists in target image or not.

app-compliance-tool

For compliance checking for MeeGo applications, the following features are supported:

  • check if all dependent packages exist in core package list
  • check if the target package can be installed in the minimal image
  • verify the installation location of the application
  • verify the minimal requirement of the desktop file and icon for the application

Usage

Image Preparation

Before using the tools, the reference image needs to be created by using MIC2 (version 0.22.0 or later):

 $ [sudo] mic-image-creator --run-mode=0 --cache=mycachedir --format=fs  --config=<ks file> --package=tar.bz2 --include-source

The sample ks file is at /usr/share/compliance-tools/share dir after installation or can be downloaded at:

 Meego-1.1:MeeGo-minimal-compliance-1.0.99.1.ks
 Meego-1.2:MeeGo-minimal-compliance-1.1.99.7.ks

The sample core list file is at /usr/share/compliance-tools/share dir after installation or can be downloaded at:

 Meego-1.1:Core-list-1.0.99.1
 Meego-1.2:Core-list-1.1.99.7

And the ks file and core list should be synced with and conform to the latest MeeGo compliance spec.

For application compliance checking, the source packages are not needed, so the --include-source option can be omitted.

In future, official pre-made images will be available, when this occurs the appropriate image should be downloaded instead of constructed.

To use the dist compliance check tool (dist_compliance), the needed target image can also be generated by mic2 using the above options. But the target images will not be limited to this specific format (fs in tar.bz2). If the exist target image is in other formats, please unpack it to a directory manually using format specific instructions and use this directory as the command line input.

Devices Compliance Check

The command "dist_compliance" is for checking compliance of MeeGo devices, or distributions.

   Usage: [sudo] dist_compliance [options]
   Options:
     --version             show program's version number and exit
     -h, --help            show this help message and exit
     -c CORELIST, --corelist=CORELIST
                           File containing MeeGo Core package list
     -r REF, --reference=REF
                           Path to reference image or unpacked directory
     -t TGT, --target=TGT  Path to target image or unpacked directory
     -o OUTDIR, --outdir=OUTDIR
                           Output directory of report
     -d, --debug           Save intermediate log data for debugging

Apps Compliance Check

The command "app_compliance" is for applications compliance checking.

   Usage: [sudo] app_compliance [options] <app-rpm>
   Options:
     --version             show program's version number and exit
     -h, --help            show this help message and exit
     -r REFIMAGE, --refimage=REFIMAGE
                           Path to reference image
     -c CORELIST, --corelist=CORELIST
                           File containing MeeGo Core package list
     -m MEEGO_API_PKGLIST, --meego_api_pkglist=MEEGO_API_PKGLIST
                           File containing MeeGo API package list
     -o OUTDIR, --outdir=OUTDIR
                           Output directory of report
     -d, --debug           Save intermediate log data for debugging

The sample MeeGo API package list file can be downloaded at:

   MeeGo-API-pkglist.txt

Result Format

Currently, the output of checking tools will be stored in HTML files.

A distribution check report will look like this:

Sample Report

An application check report will look like this:

Sample Report

Bug Report

If problems were found in the compliance tools, please file bugs at bugs.meego.com. There is a top-level category for compliance - use "MeeGo Spec and Compliance", product "Compliance Tools".

TODO

  • add checking for contents of binary packages
  • use new image format with separated src.rpm pack
  • Web UI for commenting the result
  • Web service for app checking?
  • Eliminate need for a reference compliance image for dist checking
Personal tools