Meego Wiki
Views

Quality/Plans/MeeGo Security Test Plan

From MeeGo wiki
Jump to: navigation, search

Contents

MeeGo Security Test Plan

Component Summary

MeeGo Architecture Domain View defines Security domain is responsible of security deployment across the system. It provides enablers for platform security, user security and user identity.
Picture1. Security in whole MeeGo Architecture
Security component.png

From bottom to up, the functions of those sub-components are:

  • Access Control Framework - Access control enforcement and access control policy for the device. (See Test Section1)
  • Software Distribution Security - Security aspects of software distribution including new application installations and updates. (See Test Section2)
  • Certificate Manager - Services for storing and validation of security certificates for various purposes (such as email, Wi-Fi, and browsing). Test is defined in Test Section3.
  • Integrity Protection Framework - Integrity protection of executables, configuration, and data files. (See Test Section4)
  • Accounts - Provides a storage solution for user accounts. Applications which need to store and access user settings for the service they provide over a user account will use the Accounts API. Instant messaging, e-mail, calendar, and sharing are examples of such applications. (Lower priority test, see Test Section5)
  • Single Sign-On - Responsible for providing secure storage for credentials and framework for authentication plug-ins to different services. (Lower priority test, see Test Section5)
  • Security Adaptation - Platform specific abstraction of security and crypto services. (This is not tested in the plan)

MeeGo Security Component is a Mobile Simplified Security Framework, which provides:

  • Protection for the User
    • Disallow loss of owner’s personal data
    • Mis-use of the device
  • Protection for the Device
    • Must meet regulatory requirements and specifications
    • Disallow changing RF or WiFi values
  • Protection for the Business
    • Disallow breaking SIM/Subsidy Lock
    • Limit what can be installed on the device
  • Enable New Services
    • Allow services such as Music Store or App Store and support copy protection
    • Mobile payments and Billing

Picture2. Security Framework
Security architecture.png

Feature to be Tested

This test plan follows MeeGo v1.1 and v1.2 Security Features, covering functions MeeGo implements and potential defects checking, which are filled with basic test points and also marked with different test priorities. Table instructions:
FEA ID
N/A – there is no any bug reported in bugs.meego.com
No. – FEATURE bug number in bugs.meego.com
FEA Summary
This contains general feature information.
FEA Description and Test points
Detail feature requirement will be described here. And basic test points will cover the sub-features if there are some.
Priority
P1 – User impact big, potential to have more bugs, or risk is HIGH
P2 – User impact, potential bug, or risk is MIDDLE
P3 – User impact is small bug, or risk is LOW, or an enhancement

Section1. Access Control Framework

FEA ID Feature summary Feature description and Test points Priority
N/A Authority to install/uninstall APPs Some APPs must be checked certification authority before being installed. Once installed, some sensitive APPs cannot be uninstalled.

Test should cover:

  • When installing applications, system should check CA to make sure the APP is trusted.
  • From GUI, when user uninstall some sensitive package (such as system key APP), there should be dialog prompted.
  • In command line, when doing RPM remove, system should have ability to reject the command if the APP removing is dangerous.
P1
N/A Security Unlock mechanism support in developer tool Provide a security unlock mechanism in order to allow developers to work without signing their pplications. It also works on the IMEI (International Mobile Equipment Identity) basis

Test should cover:

  • Ring 0 code should be able to be executed by ADMIN
  • Ring 0 code should also be able to be executed by developers as long as they are not doing digital rights management.
  • Developers can setup self-repository without sign.
  • The Unlock mechanism should also be related to IMEI. (Hardware checking, P3 priority).
P2
N/A Privileged mechanism for APIs There are PRIVILEGED APIs for runtime usages.

Test should cover:

  • Voice call APIs
  • Message sending APIs
  • data network connection APIs (CSD, GPRS, EDGE, 3G, HSDPA, WiFi)
  • User private data access, including:
    • Identity, MSISDN
    • Location
    • PIM data
    • SMS, MMS
    • Mime type
    • Personal storage
    • Password, security keys
  • Negative check, user without the privilege tries to use those APIs.
P1
N/A SMACK rules All the read and execute access should be controlled by Labels marked Rules between Subjects and Objects.

Test should cover:

  • All kinds of access with matrix of subjects, objects and labels.
  • Negative test with expected failures access.
  • Test case: file access with applications that have different labels
  • Test case: file access with applications that have different labels but are configured to have access to same files
  • Test case: DBus policy
  • Test case: Network access configurations
P1

Section2. SW Distribution Security

FEA ID Feature summary Feature description and Test points Priority
4978 A Key Replacement Strategy When private signing key stops working, it needs to revoke it and change the private signing key to a new one.

Test should cover:

  • System ADMIN has the ability to replace APP key.
  • Once the new key is working, the old one should be invalid.
  • System should maintain the new keys (for revoking old ones), and automatically use them to replace the old.
P1
4977 Different Keys For Production & Non-Production Software Using a "debug key" helps ensure that non-production code does not make it into production images or repositories.

Test should cover:

  • System can support different key checking (own multiple public keys).
  • System can choose correct certification to check different signature.
P1
4976 destroy all traces of cryptographic keys Destroy all traces of cryptographic keys once the key is no longer required.

Test should cover:

  • System should provide an entrance to user to delete keys from memory.
  • Once the cryptographic keys are deleted, it should fail to use penetration test to extract a key out of memory.
P2
4975 Key Backup and Recovery MeeGo should have key backup and recovery, for if our signing keys are accidentally destroyed or inaccessible, we lose all ability to update the distribution.

Test should cover:

  • System should provide an individual GUI (or in some management APP) for Key Backup and Recovery.
  • Normal user is able to use the GUI to backup/recover keys after it pass authorization.
  • If supporting automatic backup, user can adjust the interval.
  • If it is only a manual backup, normal user can do it after authorization passes.
  • When doing recovery, user can see a backup list, sorted by time, key-numbers or some attributes else. User needs a clear backup description to restore to. (P1 priority)
P2
4974 Key Distribution & Storage MeeGo should have cryptographic key distribution and storage mechanism, in order to forbid attackers to gain access to encrypted content.

Test should cover:

  • Use ADMIN account to try to find key storage. Check if the storage is encrypted to admin.
P1

Section3. Certificate Manager

FEA ID Feature summary Feature description and Test points Priority
N/A Execution Environment Security Framework The implemented security framework SHALL be a three category model:
  • Un-trusted applications are forbidden to access any APIs.
  • Trusted Unprivileged applications (No access to privileged APIs)
  • Trusted privileged applications (access to all APIs)

Test should cover:
Fake rpm with wrong signature

  • Use RPM command to generate none signature package to install.
  • Use RPM command to generate a non-MeeGo distribution signature to install.

Trusted/untrusted check

  • If an application is not trusted, system should warn “access forbidden” when installing it.
  • If user installs the un-trusted application by force, the APP still cannot use any APIs.

To a trusted application,

  • If it is privileged, any access to API can pass smoothly.
  • If it is not privileged, it cannot access system privileged APIs, only can access unprivileged APIs. A dialog or other notice should be prompted when this illegal access happened.
P1
N/A Supports X.509 Supports all certificate types by X.509.

Test should cover:
Find or customize a Server to emulate X.509 certification process.

  • Client can be authenticated with X.509.
  • Server can be authenticated with server’s X.509
  • Server can validate the client credentials against a custom X509 Certificate Validation.
  • All the certification processes should cover X.509 V1, V2 and V3 versions.

Some negative cases can also check the X.509 certification checking:

  • Modify system time to check if the certification is out of date.
  • If the certification binds the device_ID, then copy to another machine to check if the signature can still be used.
P1
N/A Software certifications Framework provides a software certification management tool.

Test should cover:

  • All trusted certification list
  • Certification detail information show
  • Delete/Add certifications
  • Certification backup (P2, if there is)
P1
N/A Hardware certifications Framework provides hardware certification checking.

Test should cover:

  • Just check if the hardware chipset driver is enabled in kernel.
  • Just check if user could disable this hardware checking feature by BIOS or other soft switch.
P3
N/A establish authenticity of communicating parties Runtime authenticity checking for the communicating parties.

Test should cover:

  • Check the authenticity of incoming phone call
  • Check the authenticity of a network call access
  • Check the authenticity of the messages by chat app, or other plug-ins for some website.
P1

Section4. Integrity Protection Framework

FEA ID Feature summary Feature description and Test points Priority
N/A IMA support IMA (Integrity Measurement Architecture) is included in kernel-2.6.30, which can maintain a runtime measurement list of all files.

Test should cover:

  • Check hash list in hardware
  • Once the accessed file is deleted, the IMA should update itself.
  • Check that system reacts when binary is altered manually
  • Check if hash is recalculation when binary is updated
  • Check if hash is recalculated when binary starts
  • Check IMA resource cost
P1
N/A IMA-Appraisal IMA-Appraisal allows IMA to store file “known good” HASH in the security.ima extended attribute. It can verify current measurement matches “known good” value. P2
N/A IMA-Appraisal-Signature-Extension IMA-ASE can enlarge IMA-Appraisal standard to allow RSA signature in security.ima instead of HASH. P2
N/A IMA-Appraisal-Directory-Extension IMA-ADE provides local directory integrity to add HASH for directory contents.

Test should cover:

  • File names
  • Inode numbers
  • Inode appraisal value
P2
N/A EVM support EVM (Extended Verification Module) protects sensitive file system metadata not covered by other protections.

Test should cover:

  • Security.ima
  • Security.SMACK64
  • Security.capability
  • Test case: Manually alter the keyed hash to see how EVM reacts
P1
N/A Support Trusted and Encrypted Keys The kernel key ring service should support two kinds of KEYs.

Test should cover:

  • Trusted key (generated by security HW)
  • Encrypted key (generated by security software, such as entropy pool)
P1
N/A Crypto Services The kernel key ring service should support two kinds of KEYs.

Test cases

  • Test that an application data that uses libaegis-crypto is secure
  • Test accli and apscli commands
P1

Section5. Accounts and Single Sign-On

FEA ID Feature summary Feature description and Test points Priority
N/A Application password can be stored in Acct&SSO Some application (such as Buteo, a PIM synchronizer tool) may use Acct&SSO as the first password storage choice, they also support to store user password to other profile files as the second choice. So, system should let user know where the password will be stored.

Test should cover:

  • System should prompt to user, once password is stored to Acct&SSO tool.
  • If password has been stored in the Acct&SSO, user has no need to be prompted when password is changed.
  • User is able to find and delete the stored password from Acct&SSO component. (P2)
P1
N/A Password complexity check System should ensure user’s password setting with some degrees of complexity.

Test should cover: Three places to change PASSWORD.

  • Initialize the password
  • Change password from GUI
  • Change password in command line
P2
N/A Sudo ability check System could permit normal user to use sudo as root executer privilege. The configuration file is /etc/sudoer.

Test should cover:

  • If sudoer enables normal user, the user should be able to run command as root after input the root password.
  • If sudoer disables normal user, the user cannot run command with sudo.
  • With production image, normal user should not be assigned sudoer authority by default. (Based on Handset, P3)
P2

Section6. Security Basic Libraries

This component contains all the basic security libraries for encryption, hash algorithm and other technologies from the open source world. Since upstream has tested the stacks well before we integrate them, QA will take a lower test priority. Efforts can be put on right package integration check and CVE bug fix check.

So, if no special thing to be emphasized, there will be no test points in below table.

FEA ID Feature summary Feature description and Test points Priority
4973 Minimum of 2048-bit Keys For Asymmetric Cryptographic Functions Since both RSA and PGP are weak with 1028-bit keys, MeeGo needs a stronger keys cryptographic functions for asymmetric (minimum is 2048-bit). P2
4972 RSA or GPG/PGP For Digital Signatures or Key Exchange MeeGo shall only use RSA or GPG/PGP for digital signatures or key exchange. P2
4969 ECB Mode (Electronic CodeBook) On Data Sizes MeeGo shall not use Electronic Code Book (ECB) mode on data sizes of more than a single block when doing symmetric cryptography. P2
4968 AES (Advanced Encryption Standard) for Symmetric Cryptography and Minimum Key Size is 128-Bits AES is the symmetric cryptographic standard used world-wide.MeeGo shall use it for symmetric cryptography and a minimum key size of 128-bits. P2
4967 Use PRNG (pseudo-random number generators) as random number generator MeeGo shall only use pseudo-random number generators (PRNG’s) that are

conformed to NIST SP800-90

P2
4966 FIPS PUB-198 Approved HMAC Algorithms MeeGo shall only use FIPS PUB-198 approved hash based message authentication code (HMAC) algorithms to get rid of attacks. P2
4965 SHA-2 Based Cryptographic Hash Algorithms The MD5 hashing algorithm is effectively broken. SHA-1 is currently showing significant cracks and will likely fall in the lifetime of MeeGo. So, SHA-2 based cryptographic hashing algorithms are needed. P2

Section7. User Experience Security Validation

This section includes all the user executable and visible security entrances and system security modules. Validation will be done to make sure MeeGo system is safe by user actions through Browser GUI, firewall and the security related actions in other packages.

FEA ID Feature summary Feature description and Test points Priority
N/A Browser CA handling process Browser should provide an integrity mechanism to do CA process.

Test should cover:

  • Once Certification Authorization is needed, Browser should prompt user to get CA (public key) from the web site.
  • If user chooses YES, the CA should be downloaded to local.
  • If user chooses NO, the CA process fails. Website cannot be reached.
P1
N/A Browser website certification management On GUI, Web Browser should provide certification management.

Test should cover:

  • There should be local directory to store all the CA files.
  • User can get a CA list.
  • User can delete one CA file from the list, then browser should ask to fetch CA again when logon the corresponding web site.
  • Once user look up one CA, the CA train should be shown.
P1
N/A Browser Cookie management On GUI, Some cookies must reload when needed.

Test should cover:

  • Browser can record the cookies for a quick web page rendering rate.
  • Browser should provide a GUI button to delete all the stored cookies.
P2
N/A Browser web sites trusted/restricted management On GUI, browser needs to provide a mechanism for user to forbid restricted websites.

Test should cover:

  • User can add a site to restricted list.
  • User can delete restricted site or permit site once.
  • Show a restricted sites list.
P2
N/A Browser security levels management Browser should enable different levels of security restriction.

Test should cover:

  • Browser should provide a security GUI for user to choose the level.
  • Browser should also provide a button labeled “default level” for user to choose.
P2
N/A System should provide a setting for disabling auto scan on inserted DISK From GUI, user can disable the auto scan and auto mount functions when inserting a DISK, such as U-Disk, SD card, MMS card and so on.

Test should cover:

  • GUI to set auto scan enabled/disabled.
  • GUI to set auto mount enabled/disabled
P3
N/A System should provide a basic firewall setup Like other open source distributions, MeeGo needs to have a GUI to enable/disable IPTables.

Test should cover:

  • User can enable/disable iptables from some GUI.
  • In command, iptables can forbid some remote logons.
  • In command, iptables can help to do IP forwarding.
P2
N/A GPG check on software Repo The remote MeeGo package repos need to support GPG check.

Test should cover:

  • If repo file has gpgcheck=1, zypper request will need to check the key in gpg file.
  • If repo file has gpgcheck=0, zypper request will not need any gpg checking.
P2
N/A Security check for Appup System should support security checking during 3rd party application installation.

Test should cover:

  • Security system should give the ability for some 3rd party APP installation tool (e.g. Appup).
  • Once the 3rd party APP cannot be authorized to install, the security system should prompt to user if the software installation could go on.
P1

Feature not to be Tested

Some features are not tested by MeeGo Core QA team.

FEA ID Feature summary Feature description and Test points Priority
N/A Hardware Security Chipset functions supported in kernel All the hardware security functions in driver layer. Not plan to test

Test Strategy and Approach

The test plan will use following testing strategy and approaches to test MeeGo Security Framework.

Test Strategy

QA test should meet three main goals of MeeGo Security Framework development:

  • It is critical to enroll testing methods (such as fuzz testing) to detect if there are serious issues hidden behind our contributions unexpectedly when we are developing them at the same time.
  • Only after ensuring first goal is achieved, could we pay efforts to verify each feature we contribute (to the open source security foundation) in MeeGo.
  • Basic validation also covers the open source world security modules and some security packages from upstream. Priority may be middle, since those security applications have been tested well by upstream.

The Test Plan maintain Security features from user view sight, that is to say, try to map all the trivial security features to each security sub-component defined in MeeGo OS Middleware Architecture. Besides that, we also make user experience test (from GUI or other management tools in system) into an individual section to test manual actions in the real world.
The validation will focus on following aspects:

  • Functionality testing, positive function test will be done to make sure all the feature function has been integrated into MeeGo system. OS middleware test will be designed as emulating an up level application to call function APIs.
  • Negative testing, intently use some wrong actions to call security API, and check if system can handle the wrong input or border conditions well.
  • Fuzz testing, setup an internal test environment to emulate some illegal attacking. This will be done by some binary tools or hacks in some project source code.
  • Stress testing, we may do a stressful test workload in two ways. One is to put many operations or attacking at the same time. The other is to run normal workload for a long time.
  • GUI testing, to all the components and tools which are released with MeeGo system, we should ensure the GUI operation will not cause any security issues. We also treat it a help once any 3rd party APP could involve any issues to our security system.

Test Method

MSF (MeeGo Security Framework) provides a whole solution for security checking and controlling from low level to high level in MeeGo system. Security interfaces are also implemented for other applications whenever protection is needed. QA validation will be classified to four different levels, which are End Customer, Experienced User, Software Developer and Advanced Researcher. Test cases are referred by their actions. (Bold font strings are highlighted as high priority to be cared of)

  • Test as End Customer. The End Customer is a common user who does not have deep Linux background, or even knows little computer knowledge. So all the operation should be from GUI and system prompted options “Yes/No” may be chosen randomly. Based on this assumption, a basic test is planned:
    • Validate password operations from GUI --- Check system password checking and restoring mechanism.
    • Validate application install and uninstall by all software management tools from GUI --- Check application integrity checking mechanism.
  • Test as Experienced User. The Experienced user is a group of users who are familiar with some smart phone systems or even other Linux distributions. It causes more risk during they log on more websites and install various outside applications. So, they not only touch security framework by various actions from GUI or command lines, but also will do harmful validation to system intentionally or interestingly. Based on this assumption, a full test must be planned:
    • Investigate keyctl commands to check integrity.
    • Modify GPGKEY file in local repos --- RPM gpg process checking.
    • Build up HTTPS server to emulate SSL communication with MeeGo client --- check different kinds of CAs.
    • Negatively delete certificate --- Check certification manager of MSF.
    • Negatively try to touch sensitive data or limited device beyond user’s privilege. --- Check system security protection for user data and device.
    • Create shell scripts to access objects under SMACK controlling.
    • Investigate some security checking websites (such as pcflank) to do security check.
    • Some user experiences by using other Linux distribution (e.g. Meamo5, Ubuntu10.04, Fedora13, Windows7 and Andriod2.1) or tools (e.g. iTunes) will be referred for our security testing.
  • Test as Software Developer. The software creators require a secure software distribution process. In their developing process, digital signatures are made in individual environment or MeeGo published SDK. Our MSF would check the signature before software installation. Based on this assumption, a basic test is planned:
    • RPM signature and rebuild commands are needed to create a package for testing.
    • Negatively fake a packaging framework, copy RPM to disk to install --- check if MSF can handle different software certification situations.
    • Fake a RPM repo for MeeGo system to use --- check if MSF can work well during a formal software distribution process.
  • Test as Advanced Researcher. The Advanced Researchers may do much more things beyond MSF, such as code static analysis, system defects investigation and even security attacking. QA intends to reduce the unexpected security issues by some fuzz testing. Here, for some security infrastructures are worldwide used (e.g. PKI, encryption or deciphering), QA will not attempt to challenging them. Although basic test is planned, this part may need more time:
    • Run code static analysis tool to check memory leap issue.
    • Build up internal network to perform security attacks.

Test Automation

Test automation is very important in Security testing. A daily or nightly automatic test can help to avoid regression happens in the MeeGo system releases.
Automation cases will be included in MCTS tool.

Test Environment

Hardware and Lab

  • Tablet, Handset, Netbook/Nettop, and IVI platforms.
  • Host machine with MeeGo installed, some behavior may be referred from Ubuntu-10.04 and Fedora14.
  • Peripheral equipment (USB stick, SD/MMS card, external keyboard/mouse, etc.)

Software Environment

  • MeeGo-1.1 and MeeGo-1.2 system
  • Ubuntu 10.04 (desktop)
  • Fedora 14
  • MeeGo Software Stack or 3rd party APPs in app-up.
  • Opn source Security tools

Network Environment

  • LAN support and could access internet
  • Wi-Fi and Bluetooth support and could access internet
  • Serial console connection on some Handset machines

Testing Tools

  • Coverity Prevent – a static code analysis tool to find defects of C, C++, C# and Java codes.
  • Pcflank – an internet port checking tool.

QA Contact

jingke.zhang@intel.com

Developers

ryan.r.ware@intel.com
dean.e.pierce@intel.com
passion.zhao@intel.com

Referrences

Retrieved from "http://wiki.meego.com/Quality/Plans/MeeGo_Security_Test_Plan"
Personal tools