Testing Quality Characteristics
The following chapters are describing testing from different quality characteristics point of view. Note that all is not valid for every application - one should use following as a high level check list and select only the items, which are applicable. The most of examples are described from MeeGo Handheld point of view and might be not so important when application is targeting to MeeGo Netbooks.
- The application must be installable, the user must be able to remove it completely and the user must be able to reinstall the application.
- Installation of application shall be straightforward, obvious, and reasonably fast.
- Uninstall shall be straightforward and remove all application files and installed data. If there is any user content related to the application, the application verifies if the user wants to delete or keep the content.
- Core features advertised through UI or product page works as expected
- Note that application shall not attempt any unnotified communication or data transfers.
- The application should include an about with relevant information
- The application should select the device language if available
- Errors in connectivity are handled correctly. The application does not fail if connection breaks briefly. One shall check things online, offline and on situations where mode changes either from online to offline or vice versa. E.g. the application should be functional with the network connection turned off. If users try to establish a network connection, they are instructed to exit from the offline mode, if possible.
- Remember negative tests like (if applicable):
- Fill internal device flash as "user" user, e.g. by copying a file in File manager
- Use different amount of input data, e.g. load both small (1KB) and big data (10-100MB files)
- Check different states with memory card usage, e.g. memory card full, no memory card and memory card removal during operation.
- Application must not interfere with the system or other application’s function, e.g. it should not affect to browsing, voice call, music player, VoIP and messaging experience
- Check if application handles interruptions - such as phone calls, messages, battery warnings, or user pressing power key (according to the UI style).
- Sounds shall work according to the currently selected profile (if applicable).
- The application UI elements shall be large enough for a pleasant user experience.
- Application shall response in a reasonable time and does not leave the user to believe that nothing is happening. E.g. if application start-up (or closing) takes more than two seconds, a clear progress bar or animation should be displayed.
- Application should work with different themes.
- Running the application visibly shall not affects the performance and responsiveness of the system, either through specific actions or leaving the application open/running during a long period of time. Check average response times for users under different scenarios and do measurements with different CPU loads.
Resource Usage Testing
- Battery life is diminished beyond acceptable standards by having the application running in the background or just through normal use. The use case of a full day without charging shouldn't be compromised. One should not have unnecessary CPU usage (no polling), or activity when device is idle or application is on background.
- Do check for possible memory leaks (see application use cases under Valgrind/Memcheck)
- Remember to check for unsecured TCP/IP ports, network sources and bluetooth security (if applicable).
- If applicable check if application data is maintainable with backup and restore procedure.