General info
MeeGo.com IRC meetings
QA IRC meeting every Tuesday:
QA Organization
Error Management
- Error Manager Iekku Huttunen
N900 DE Blocker Bug Triage meeting minutes:
Tools
OTS setup and automated hourly testing
OTS setup
- OTS server - Ville Ilvonen/Riku Halonen, DONE
- OTS worker(s) for core tests- Ville Ilvonen/Riku Halonen/Timo Harkonen - DONE
- Reporting of hourly tests to QA-reports - Ville Ilvonen/Esa-Pekka Miettinen/Timo Harkonen - DONE
- Minimize automatic installation time to 10mins - Timo Makimattila, DONE
- OTS worker for UX tests - NOTSTARTED
- OTS Worker(s) for DE tests - ONGOING (timakima)
- Power consumption measurements - NOTSTARTED
Test automation images
- Setup hourly image building for autotest image on own setup - Riku Halonen/Timo Harkonen/Ville Ilvonen - DONE (in OTS subnet, see above)
- Move to use images from release engineering (1. download image, 2. install automation enablers, core dumping enablers etc. using mic-chroot, 3. install image to device 4. test) - NOT STARTED
If you need something else from QA please tell it to us :)
Applications for testing
- qt-demos already available from the repos
- Small applications that use Qt mobility APIs to access things like sensors to help manual testing
Test asset
- WLAN cases
- Call/SMS cases
- Audio policy framework cases (lower priority)
- Camera cases (lower priority)
- Sensor data cases (Qt Mobility, lower priority)
Crash analysis support for ARM core dumps
Core dump processing and backtraces from crashing ARM processes.
- Rich Core dumping
- Make rich-core dumping to work in MeeGo N900 DE (basic functionality) - DONE (sampos, rikhalon)
- Changes in MeeGo Gitorious (meego-n900de branch).
- Dumps are generated in /home/meego/core-dumps
- In file name, string "xxxx" is used instead of IMEI digits (privacy issue)
- Get latest packages here
- Add "-corewatcher" and "-corewatcher-applet" to .ks file to remove overlapping corewatcher.
- Fix core-reducer (Something goes wrong, when reducer processes coredump.) - DONE (sampos)
- Oopslog (and lifelog) functionality - NOT STARTED
- Back-end server
- Set up back-end server for core processing - ONGOING (rikhalon, sampos)
- Script to produce backtraces locally
- Extend rich-core-extract to proceduce backtrace from rich-core file - NOT STARTED
- Write howto guide in wiki - NOT STARTED
Boot time measurement
- Measure and optimize N900 boot time (timakima, ONGOING)
CPU load measurement during audio/video playback
Application Manager
- Implement an application manager (similar to one in N900/Fremantle) to control install/uninstall/update applications and other packages. (kyranto, ONGOING).
UX testing
Test execution schedule
- UX testing schedule: DE / Meego.com testing
| Day | Test set (status) | Release | Priority
|
| Monday
| Dataflow
| DE Weekly
| P1
|
| Monday
| Use cases
| DE Weekly
| P2
|
| Monday
| Key feature
| DE Weekly
| P3
|
| Monday
| Performance
| DE Weekly
| P5
|
| Monday
| Reliability / Iterative
| DE Weekly
| P6
|
| Monday
| Dataflow
| DE Trunk testing
| P4
|
| Tuesday
| Dataflow
| DE Trunk testing
| P1
|
| Tuesday
| Dataflow
| DE Trunk
| P2
|
| Tuesday
| Acceptance
| Meego Trunk testing
| P3
|
| Tuesday
| Key feature
| DE Tablet (N900)
| P4
|
| Wednesday
| Dataflow
| DE Trunk testing
| P1
|
| Wednesday
| Dataflow
| DE Trunk
| P2
|
| Wednesday
| Key feature
| Meego.com weekly
| P3
|
| Wednesday
| Sanity
| Meego.com weekly
| P4
|
| Thursday
| Dataflow
| DE Trunk testing
| P1
|
| Thursday
| Dataflow
| DE Trunk
| P2
|
| Thursday
| Acceptance
| Meego.com Trunk testing
| P3
|
| Friday
| Dataflow
| DE Trunk Testing
| P1
|
| Friday
| Dataflow
| DE Trunk
| P2
|
| Friday
| Acceptance
| Meego.com Trunk testing
| P3
|
Performance Testing
Performance testing results done from UI can be found here
Core Testing
QA Tasks For Developer Edition
There is a wiki article about the Developer Edition.
QA tasks for the Developer Edition differ from the usual N900 approach in that there are less features to be tested. There are currently 2 test sets for the Developer Edition, these are the Sanity Test Set and the Feature Test Set. They are described below.
Test Sets
Sanity Test Set
The sanity set should be run automatically on every image. As such it must meet the following requirements:
- 100% automated
- Testing only basic features
Feature Test Set
The feature set will be run periodically and will test the basic features as well as enablers for those features (e.g. PIM for phoning contacts). Performance will also be analysed, at the moment, this will include browser startup time and a CPU benchmark but this will be expanded later.
Suggestions are welcome.
Testing schedule
| Day | Test set (status) | Release
|
| Monday
| Feature
| DE Weekly
|
| Tuesday
| Sanity
| DE Trunk testing
|
| Wednesday
| Sanity
| DE Trunk testing
|
| Thursday
| Sanity
| DE Trunk testing
|
| Friday
| Sanity
| DE Trunk Testing
|
Core QA Team Backlog
The Core QA team intends to keep its backlog as public as possible. The limitation on this is the amount of work generated by doing this. We will attempt to keep an up-to-date lists of tasks and progress on these tasks. At the moment, the task list looks like this:
Continuous tasks
Backlog
In progress
- Plan Feature testset
- Create weekly schedule for MRT (currently in draft form)
- DE Hourly Automation improvement
- Wiki clean/update
Done
- Week 15 DE Sanity Testing
- Maturity statement of Alpha RELEASE (result in QA-report)
- Alpha RELEASE testing
QA Tasks for MeeGo.com N900
Test execution schedule
- MeeGo.com N900 Core weekly test schedule for MeeGo1.2(Tu Qingqing)
| Day | Test set (status) | Release | Priority
|
| Monday
| Acceptance test (OK)& test for changes
| MeeGo.com trunk testing
| P1
|
| Monday
| Sanity test (OK)
| MeeGo.com trunk
| P3
|
| Monday, Tuesday
| Basic feature test (OK)
| MeeGo.com pre-weekly
| P2
|
| Tuesday
| Acceptance test (OK)& test for changes
| MeeGo.com trunk testing
| P1
|
| Tuesday
| Sanity test (OK)
| MeeGo.com trunk
| P2
|
| Wednesday
| Acceptance test (OK)& test for changes
| MeeGo.com trunk testing
| P2
|
| Wednesday
| Sanity test (OK)
| MeeGo.com trunk
| P3
|
| Wednesday, Thursday
| Dataflow (OK)
| MeeGo.com weekly
| P1
|
| Thursday
| Acceptance test (OK)& test for changes
| MeeGo.com trunk testing
| P1
|
| Thursday
| Sanity test (OK)
| MeeGo.com trunk
| P2
|
| Friday
| Acceptance test (OK)& test for changes
| MeeGo.com trunk testing
| P1
|
| Friday
| Sanity test (OK)
| MeeGo.com trunk
| P2
|
META team backlog for MeeGo1.2 N900
Continuous tasks
- Bug verification
- MeeGo1.2 N900 daily validation
Backlog
- Optimize acceptance automation script
In progress
- MeeGo1.2 feature verification
- Automation testing for trunk:test and trunk image
Done
- Publish automation test result
Usefull links
Flashing tested with http://download.meego.com/testing-daily/builds/trunk/1.1.90.8.20110318.89/