(→Feature to be Tested) |
Xiaoqiang0 (Talk | contribs) (→How to report a bug) |
||
| (38 intermediate revisions not shown) | |||
| Line 7: | Line 7: | ||
==Feature to be Tested== | ==Feature to be Tested== | ||
| - | The features listed here are all from [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Package%20management&product=MeeGo%20Core%20OS%20Features&query_format=advanced&order=bug_id%20DESC&query_based_on= Package management feature zilla], OS update(and software management) test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as package management GUI, CLI, i18/l10n etc. | + | The features listed here are all from [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Package%20management&product=MeeGo%20Core%20OS%20Features&query_format=advanced&order=bug_id%20DESC&query_based_on= Package management feature zilla], Package managment and OS update(and software management) test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as package management GUI, CLI, i18/l10n etc. |
{|cellspacing="0" border="1" | {|cellspacing="0" border="1" | ||
!|Feature ID | !|Feature ID | ||
| Line 16: | Line 16: | ||
|11231 | |11231 | ||
|[FEA] download package should be supported in Core OS. | |[FEA] download package should be supported in Core OS. | ||
| - | |rowspan= | + | |rowspan=21 | Package managment (GUI/CLI) |
| - | | | + | |Test Case: pkcon download [directory] [packgename] |
|- | |- | ||
|11130 | |11130 | ||
|[FEA] search file support in Package Management Subsystem | |[FEA] search file support in Package Management Subsystem | ||
| - | | | + | |Test Case: pkcon file name [full-path-name] |
| + | eg: pkcon search file /usr/bin/gedit, if gedit is installed "Installed" status would be show, other wise "Avaliable" status would be show | ||
|- | |- | ||
|9698 | |9698 | ||
| Line 29: | Line 30: | ||
|9378 | |9378 | ||
|[FEA] Total download size | |[FEA] Total download size | ||
| - | | | + | |Test Case: |
| + | *pkcon get-detail [packagename] | ||
| + | *pkcon get-update-details [patch name] | ||
| + | |||
|- | |- | ||
|9377 | |9377 | ||
| Line 41: | Line 45: | ||
|9374 | |9374 | ||
|[FEA] SW update bearers | |[FEA] SW update bearers | ||
| + | |. | ||
| + | |- | ||
| + | |9371 | ||
| + | |[FEA] Full stack SW update | ||
| + | |. | ||
| + | |- | ||
| + | |9370 | ||
| + | |[FEA] Install SW from vendor source | ||
| + | |. | ||
| + | |- | ||
| + | |9369 | ||
| + | |[FEA] Support for stand alone installation from .rpm files | ||
| + | | Test Case: pkcon install-local XXX.rpm | ||
| + | |- | ||
| + | |9367 | ||
| + | |[FEA] Dependency management fetch new SW | ||
| + | |already verified please referenced the comments in featurezilla. | ||
| + | |- | ||
| + | |9366 | ||
| + | |[FEA] Dependency management | ||
| + | |. | ||
| + | |- | ||
| + | |9365 | ||
| + | |[FEA] Installation progress | ||
| + | | Already integrated an verified, please refrenced the comments in feature zilla. | ||
| + | |- | ||
| + | |9361 | ||
| + | |[FEA] Independent SW delivery | ||
| + | |. | ||
| + | |- | ||
| + | |9359 | ||
| + | |[FEA] Update notification | ||
| + | |. | ||
| + | |- | ||
| + | |9358 | ||
| + | |[FEA] Uninstalling user installed SW | ||
| + | |. | ||
| + | |- | ||
| + | |9356 | ||
| + | |[FEA] Download state handling | ||
| + | |. | ||
| + | |- | ||
| + | |9355 | ||
| + | |[FEA] Download error handling | ||
| + | |Steps test this feature: | ||
| + | |||
| + | * Set proxy if needed. | ||
| + | * Zypper install qt-web-runtime-examples, | ||
| + | * check the status of downloading: | ||
| + | ls -l /var/cache/zypp/packages/core/i586/ | ||
| + | * killall -9 zypper | ||
| + | * zypper install qt-web-runtime-examples and check rpm in | ||
| + | /var/cache/zypp/packages/core/i586/ to see if it’s resume downloading. | ||
| + | |- | ||
| + | |9354 | ||
| + | |[FEA] Download progress indication | ||
| + | | Already integrated an verified, please refrenced the comments in feature zilla. | ||
| + | |- | ||
| + | |8003 | ||
| + | |[FEA] OSV independent package manager API | ||
| + | |. | ||
| + | |- | ||
| + | |7864 | ||
| + | |[FEA] Protected Data Update | ||
|. | |. | ||
|- | |- | ||
|11184 | |11184 | ||
|[FEA] Configure applications to store their settings in ~/.config/ | |[FEA] Configure applications to store their settings in ~/.config/ | ||
| - | |rowspan= | + | |rowspan=18 | Application managment |
|. | |. | ||
|- | |- | ||
| Line 58: | Line 126: | ||
|9373 | |9373 | ||
|[FEA] Unistalling vendor pre-installed 3rd party applications | |[FEA] Unistalling vendor pre-installed 3rd party applications | ||
| + | |. | ||
| + | |- | ||
| + | |7863 | ||
| + | |[FEA] Multiple Application Classes | ||
| + | |. | ||
| + | |- | ||
| + | |4856 | ||
| + | |[FEA] Installed Application List - App Usage statistics | ||
| + | |. | ||
| + | |- | ||
| + | |4855 | ||
| + | |[FEA] Installed Application List - App Size | ||
| + | | This feature can be down with following steps: | ||
| + | 1) got all the application installed on system: | ||
| + | pkcon search group Application filter=installed | ||
| + | 2) get the size of eatch application: | ||
| + | pkcon get-detail [packagename] | ||
| + | |||
| + | There are related PackageKit Dbus API used in these two steps. | ||
| + | |||
| + | |- | ||
| + | |4854 | ||
| + | |[FEA] Installed Application List - App Location | ||
| + | |. | ||
| + | |- | ||
| + | |4852 | ||
| + | |[FEA] Application removal | ||
| + | |. | ||
| + | |- | ||
| + | |4851 | ||
| + | |[FEA] Application Update Installation | ||
| + | |This feature has already been integrated in PackageKit now. | ||
| + | |||
| + | Test case steps: | ||
| + | * Prepare update repo | ||
| + | * pkcon get-updates | ||
| + | * pkcon update <updatesname> | ||
| + | |- | ||
| + | |4850 | ||
| + | |[FEA] Application Update-Check Scheduling | ||
| + | | This feature should be implemented in Package Management UI | ||
| + | |- | ||
| + | |4849 | ||
| + | |[FEA] Application Update Notification | ||
| + | |This feature should be implemented in Package Management UI | ||
| + | |||
| + | |- | ||
| + | |4848 | ||
| + | |[FEA] Installed Application List | ||
| + | |use "pkcon search group Applications --filter=installed" to test this | ||
| + | feature. | ||
| + | |- | ||
| + | |4847 | ||
| + | |[FEA] Application Installation | ||
| + | |Test Case: | ||
| + | pkcon install-local XXX.rpm | ||
| + | |- | ||
| + | |4846 | ||
| + | |[FEA] Application Download | ||
| + | | pleaes use the following steps to verify this feature: | ||
| + | |||
| + | * set proxy if needed | ||
| + | * refresh repo cache if needed (pkcon refresh) | ||
| + | * Using "pkcon download . PackageKit-docs" to download applications | ||
| + | |- | ||
| + | |4845 | ||
| + | |[FEA] Available Application Announcement | ||
| + | | Test Case: | ||
| + | pkcon search group Applications --filter=~installed | ||
| + | |- | ||
| + | |4844 | ||
| + | |[FEA] Application Discovery | ||
| + | | Test Case: pkcon search group Applications --filter=~installed | ||
| + | |- | ||
| + | |2731 | ||
| + | |[FEA] [MASTER] Application lifecycle | ||
|. | |. | ||
|- | |- | ||
|10910 | |10910 | ||
|[FEA] MeeGo Package Changelogs are broken | |[FEA] MeeGo Package Changelogs are broken | ||
| - | |Package validation | + | |rowspan=4 | Package validation |
| + | |. | ||
| + | |- | ||
| + | |9372 | ||
| + | |[FEA] Update error handling | ||
| + | |. | ||
| + | |- | ||
| + | |9368 | ||
| + | |[FEA] Installation error handling | ||
| + | |. | ||
| + | |- | ||
| + | |9357 | ||
| + | |[FEA] Support for Security FW in package manager | ||
|. | |. | ||
|- | |- | ||
|9379 | |9379 | ||
|[FEA] Selected repository refresh | |[FEA] Selected repository refresh | ||
| - | |Package repos | + | |rowspan=2 | Package repos |
| + | |. | ||
| + | |- | ||
| + | |9360 | ||
| + | |[FEA] Enabling SW sources | ||
| + | |. | ||
| + | |- | ||
| + | |9363 | ||
| + | |[FEA] Prioritized OS update | ||
| + | |rowspan=6 | System/OS update (GUI/CLI) | ||
| + | |. | ||
| + | |- | ||
| + | |9362 | ||
| + | |[FEA] OS update and enough disk space for updates | ||
| + | |. | ||
| + | |- | ||
| + | |8007 | ||
| + | |[FEA] lsb_release support | ||
| + | |. | ||
| + | |- | ||
| + | |7842 | ||
| + | |[FEA] OS Update | ||
| + | |. | ||
| + | |- | ||
| + | |7841 | ||
| + | |[FEA] Services Upgrade | ||
| + | |. | ||
| + | |- | ||
| + | |7840 | ||
| + | |[FEA] Firmware Upgrade | ||
|. | |. | ||
|- | |- | ||
| Line 73: | Line 258: | ||
==Feature not to be Tested== | ==Feature not to be Tested== | ||
| - | * Independent applcations functionalities testing: Packages/Applictaions installation/update/remove are in package management test scope, but | + | * Independent applcations functionalities testing: Packages/Applictaions installation/update/remove are in package management test scope, but applications themselves functionalities are not in this component's test coverage; |
* System update will only update from default distribution repos, unconfirmed new OS update will not in this scope; | * System update will only update from default distribution repos, unconfirmed new OS update will not in this scope; | ||
| - | |||
| - | |||
==Test Strategy and Approach== | ==Test Strategy and Approach== | ||
''' Isolate logical function testing from GUI testing: ''' | ''' Isolate logical function testing from GUI testing: ''' | ||
| - | + | Package managment and system update are to be validated from two perspectives, logic function testing and GUI testing. The former focuses on features’ functionalities, while the later dedicates to UI visual appearance and user interaction. As the characteristic of event driven application, there are multiple event paths can reach the same function. Although separated from GUI layer, logic function testing is still achieved by go through one of equivalent event paths in GUI navigation, leaving most of event paths being covered in GUI testing. | |
''' Focus on features new implemented or customized for MeeGo system: ''' | ''' Focus on features new implemented or customized for MeeGo system: ''' | ||
| Line 101: | Line 284: | ||
* OS update need tester manually check update status, and need also check whether it can workable after update process, so it's hard to automation the test like other GUI or midware components; | * OS update need tester manually check update status, and need also check whether it can workable after update process, so it's hard to automation the test like other GUI or midware components; | ||
* Pakcage management automation need more investigation. [TBD] | * Pakcage management automation need more investigation. [TBD] | ||
| - | |||
| - | |||
| - | |||
==Test Design== | ==Test Design== | ||
| Line 119: | Line 299: | ||
** Default repo files stays in right folder | ** Default repo files stays in right folder | ||
** Default repo can work while using them to install packages | ** Default repo can work while using them to install packages | ||
| + | ** Default repo have not any dependency issue (Using 'zypper ve' to verify this) | ||
* Third-party or Customized repository | * Third-party or Customized repository | ||
** Add a customized repo, check whether package manager could detect it | ** Add a customized repo, check whether package manager could detect it | ||
** Install pakcages from customized repo and check whether it can work | ** Install pakcages from customized repo and check whether it can work | ||
|- | |- | ||
| - | |Package Manager GUI and functionality | + | |Package/Application Manager GUI and functionality |
| | | | ||
* GUI layout and navigation | * GUI layout and navigation | ||
| Line 141: | Line 322: | ||
** Check these application desk files from home screen or other place | ** Check these application desk files from home screen or other place | ||
|- | |- | ||
| - | |Package Manager command line | + | |Package/Application Manager command line |
| | | | ||
* Package searching and reviewing | * Package searching and reviewing | ||
| - | ** pkcon search | + | ** pkcon search [name|details|group|file] [data] |
** zypper search | ** zypper search | ||
* Package installation and removing | * Package installation and removing | ||
| Line 154: | Line 335: | ||
** pkcon refresh | ** pkcon refresh | ||
** zypper refresh | ** zypper refresh | ||
| + | * Package operation progress(install/download) | ||
| + | ** pkcon [directory] [packages] | ||
| + | ** pkcon install | ||
| + | * local rpm installation | ||
| + | ** pkcon install-local XXX.rpm | ||
| + | * get the size of package and update(patch) | ||
| + | ** pkcon get-update-detail [patch name] | ||
| + | ** pkcon get-details [package] | ||
| + | |||
|- | |- | ||
|Package validation | |Package validation | ||
| Line 211: | Line 401: | ||
|- | |- | ||
|1 | |1 | ||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
|Wired network | |Wired network | ||
| | | | ||
| Line 221: | Line 406: | ||
* Using proxy | * Using proxy | ||
* Without proxy | * Without proxy | ||
| - | | | + | | |
| + | Method to set proxy | ||
| + | * set proxy in repo file: add "proxy=hostname:port" to special repo file, if you want to diable proxy for special repo, add "proxy=_none_"; | ||
| + | * set proxy in terminal: export http_proxy=hostname:port, which is used for command line | ||
|- | |- | ||
| - | | | + | |2 |
|Wireless network | |Wireless network | ||
| | | | ||
| Line 239: | Line 427: | ||
===Hardware Platforms=== | ===Hardware Platforms=== | ||
| + | * Netbook | ||
| + | * Handset | ||
* Wired network | * Wired network | ||
* Wireless network | * Wireless network | ||
| - | |||
===Tools=== | ===Tools=== | ||
| Line 251: | Line 440: | ||
==Developers== | ==Developers== | ||
| + | * Zhang Qiang <qiang.z.zhang@intel.com> | ||
* | * | ||
| - | * | + | * |
| - | * | + | == How to report a bug == |
| + | === PackageKit or Package Manager UI bug === | ||
| + | For Package Management component issue, please catch more logs and attach them when you reporting a bug. | ||
| + | For PackageKit or Package Manager UI issue | ||
| + | Steps to get logs: | ||
| + | * Setup system Environment | ||
| + | * Install PackageKit-debuginfo, libzypp-debuginfo, which should match the version of PackageKit/libzypp | ||
| + | * start packagekitd manually. In one terminal execute: | ||
| + | #killall -9 packagekitd | ||
| + | #/usr/libexec/packagekitd -v >packagekitd.log | ||
| + | * In another terminal execute your script/command to reproduce bug. | ||
| + | * Then you'll get packagekitd.log and /var/log/pk_backend_zypp, and these two log files are very important to fix bug in the first time. Please attach them. | ||
| + | === zypper command bug === | ||
| + | For Zypper issue while you using zypper command, using the following steps to get logs: | ||
| + | * Setup system Environment | ||
| + | #rm /var/log/zypper.log -f | ||
| + | * set log level and reproduce bug using zypper | ||
| + | #export ZYPP_LIBSAT_FULLLOG=1 | ||
| + | #zypper xxx yyy | ||
| + | or using the following log level | ||
| + | #export ZYPP_FULLLOG=1 | ||
| + | #zypper xxx yyy | ||
| + | * check /var/log/zypper.log and attach them. The log generated using ZYPP_LIBSAT_FULLLOG would be very large(40M), but includde more information about packages. while the log generated using ZYPP_FULLLOG is smaller (1M). | ||
== References == | == References == | ||
| - | * | + | * Software management and system update features on feature zilla [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Package%20management&product=MeeGo%20Core%20OS%20Features&query_format=advanced&order=bug_id%20DESC&query_based_on= Package management and update features] |
| - | [http://bugs.meego.com/buglist.cgi?classification=MeeGo%20Features&component=Package%20management&product=MeeGo%20Core%20OS%20Features&query_format=advanced&order=bug_id%20DESC&query_based_on= Package management and update features] | + | * [[Quality/Plans/MeeGo Core Test Plan|MeeGo Core Test Plan]] |
| - | * MeeGo Core Test Plan | + | |
| - | + | [[Category:Packaging]] | |
Contents |
Package Management and Update is an important way for system function expansion, stability enhancement and issue fix. The Mechanism of software installation and update decide the MeeGo system's security and the complexity of user operation. Software Installation and Update application currently contains 2 components:
The features listed here are all from Package management feature zilla, Package managment and OS update(and software management) test will cove all these features with required priority, at the same time, the test will also cover other features which are not listed in this table, such as package management GUI, CLI, i18/l10n etc.
| Feature ID | Feature | Feature Set | Comments |
|---|---|---|---|
| 11231 | [FEA] download package should be supported in Core OS. | Package managment (GUI/CLI) | Test Case: pkcon download [directory] [packgename] |
| 11130 | [FEA] search file support in Package Management Subsystem | Test Case: pkcon file name [full-path-name]
eg: pkcon search file /usr/bin/gedit, if gedit is installed "Installed" status would be show, other wise "Avaliable" status would be show | |
| 9698 | [FEA] [MASTER] Package Management and App Lifecycle | . | |
| 9378 | [FEA] Total download size | Test Case:
| |
| 9377 | [FEA] Parallel SW downloading | . | |
| 9375 | [FEA] Multiple SW installation | . | |
| 9374 | [FEA] SW update bearers | . | |
| 9371 | [FEA] Full stack SW update | . | |
| 9370 | [FEA] Install SW from vendor source | . | |
| 9369 | [FEA] Support for stand alone installation from .rpm files | Test Case: pkcon install-local XXX.rpm | |
| 9367 | [FEA] Dependency management fetch new SW | already verified please referenced the comments in featurezilla. | |
| 9366 | [FEA] Dependency management | . | |
| 9365 | [FEA] Installation progress | Already integrated an verified, please refrenced the comments in feature zilla. | |
| 9361 | [FEA] Independent SW delivery | . | |
| 9359 | [FEA] Update notification | . | |
| 9358 | [FEA] Uninstalling user installed SW | . | |
| 9356 | [FEA] Download state handling | . | |
| 9355 | [FEA] Download error handling | Steps test this feature:
ls -l /var/cache/zypp/packages/core/i586/
/var/cache/zypp/packages/core/i586/ to see if it’s resume downloading. | |
| 9354 | [FEA] Download progress indication | Already integrated an verified, please refrenced the comments in feature zilla. | |
| 8003 | [FEA] OSV independent package manager API | . | |
| 7864 | [FEA] Protected Data Update | . | |
| 11184 | [FEA] Configure applications to store their settings in ~/.config/ | Application managment | . |
| 9380 | [FEA] Application revocation | . | |
| 9376 | [FEA] Support for .install file | . | |
| 9373 | [FEA] Unistalling vendor pre-installed 3rd party applications | . | |
| 7863 | [FEA] Multiple Application Classes | . | |
| 4856 | [FEA] Installed Application List - App Usage statistics | . | |
| 4855 | [FEA] Installed Application List - App Size | This feature can be down with following steps:
1) got all the application installed on system: pkcon search group Application filter=installed 2) get the size of eatch application: pkcon get-detail [packagename] There are related PackageKit Dbus API used in these two steps. | |
| 4854 | [FEA] Installed Application List - App Location | . | |
| 4852 | [FEA] Application removal | . | |
| 4851 | [FEA] Application Update Installation | This feature has already been integrated in PackageKit now.
Test case steps: * Prepare update repo * pkcon get-updates * pkcon update <updatesname> | |
| 4850 | [FEA] Application Update-Check Scheduling | This feature should be implemented in Package Management UI | |
| 4849 | [FEA] Application Update Notification | This feature should be implemented in Package Management UI | |
| 4848 | [FEA] Installed Application List | use "pkcon search group Applications --filter=installed" to test this
feature. | |
| 4847 | [FEA] Application Installation | Test Case:
pkcon install-local XXX.rpm | |
| 4846 | [FEA] Application Download | pleaes use the following steps to verify this feature:
| |
| 4845 | [FEA] Available Application Announcement | Test Case:
pkcon search group Applications --filter=~installed | |
| 4844 | [FEA] Application Discovery | Test Case: pkcon search group Applications --filter=~installed | |
| 2731 | [FEA] [MASTER] Application lifecycle | . | |
| 10910 | [FEA] MeeGo Package Changelogs are broken | Package validation | . |
| 9372 | [FEA] Update error handling | . | |
| 9368 | [FEA] Installation error handling | . | |
| 9357 | [FEA] Support for Security FW in package manager | . | |
| 9379 | [FEA] Selected repository refresh | Package repos | . |
| 9360 | [FEA] Enabling SW sources | . | |
| 9363 | [FEA] Prioritized OS update | System/OS update (GUI/CLI) | . |
| 9362 | [FEA] OS update and enough disk space for updates | . | |
| 8007 | [FEA] lsb_release support | . | |
| 7842 | [FEA] OS Update | . | |
| 7841 | [FEA] Services Upgrade | . | |
| 7840 | [FEA] Firmware Upgrade | . |
Isolate logical function testing from GUI testing: Package managment and system update are to be validated from two perspectives, logic function testing and GUI testing. The former focuses on features’ functionalities, while the later dedicates to UI visual appearance and user interaction. As the characteristic of event driven application, there are multiple event paths can reach the same function. Although separated from GUI layer, logic function testing is still achieved by go through one of equivalent event paths in GUI navigation, leaving most of event paths being covered in GUI testing.
Focus on features new implemented or customized for MeeGo system: The basic idea is to validate the MeeGo Software manager and system update functions from the end users’ perspective. Test efforts will mostly be taken on the features new implemented or customized for MeeGo system.
Software manager Test Strategy: Besides the MeeGo distribution repositories, QA package repository server will also be setup to simulate scenarios of package installation and updating. Below scenarios will be covered.
System update test strategy: Test system update from the end users' perspective, check the user private setting and data remains after the updating of system, this test simulates tow user updating scene: update system day by day and once update from final release image to the latest status.
Package security installation test strategy: Packages security installation depends on MeeGo's signature validation mechanism, both of released repository and released packages have MeeGo's official signature. Set up various signature package and repository to simulate a real situation.
Test automation:
Package management and OS update test will from end user side, follow the test design guideline and application test design method, and refer to other similar dirtibution OS like fedora, then get the following test point:
| Feature Set | Test Point |
|---|---|
| Package repos |
|
| Package/Application Manager GUI and functionality |
|
| Package/Application Manager command line |
|
| Package validation |
|
| System/OS update GUI |
|
| System/OS update from CLI |
|
| System/OS update effect |
|
| I18N & L10N |
|
Mainly for the conditions that the application will meet, maily introduced by platform characteristic. For example, devices may in different wireless environment, users may use different input methods to write, etc.
| Item ID | Name | Details | Comments |
|---|---|---|---|
| 1 | Wired network |
Connects to wired network:
|
Method to set proxy
|
| 2 | Wireless network |
Connects to wireless network:
| . |
For Package Management component issue, please catch more logs and attach them when you reporting a bug. For PackageKit or Package Manager UI issue Steps to get logs:
#killall -9 packagekitd #/usr/libexec/packagekitd -v >packagekitd.log
For Zypper issue while you using zypper command, using the following steps to get logs:
#rm /var/log/zypper.log -f
#export ZYPP_LIBSAT_FULLLOG=1 #zypper xxx yyy
or using the following log level
#export ZYPP_FULLLOG=1 #zypper xxx yyy