Meego Wiki
Views

ARM/N900/QA/Performance

From MeeGo wiki
< ARM | N900 | QA(Difference between revisions)
Jump to: navigation, search
(→Handset Shutdown Time)
m (→Response Time)
 
(41 intermediate revisions not shown)
Line 1: Line 1:
== Performance Results ==
== Performance Results ==
-
This wiki page contains performance testing results done for MeeGo 1.2 Developer Edition for N900
+
This wiki page contains latest performance testing results done for Community Edition of MeeGo for the Nokia N900 device
=== Response Time ===
=== Response Time ===
Line 11: Line 11:
* release is run from memory card (class 6 SD card)
* release is run from memory card (class 6 SD card)
* measurements done from UI
* measurements done from UI
-
* each use case measured three times with stop watch and median result is used as a result
+
* '''each use case is measured three times with stop watch and median result is used as a result'''
 +
* '''measurements will be done for weekly release (Monday's stable image) or if specially asked'''
* currently measured applications don't contain any test data (e.g. call history, received SMS)
* currently measured applications don't contain any test data (e.g. call history, received SMS)
* same operator used all the time
* same operator used all the time
-
* used WLAN is 802.11g
 
-
Detail test case descriptions can be found from [http://meego.gitorious.org/meego-quality-assurance/handset-ux-tests gitorious] (mhts-de-performance-tests)  
+
Detail test case descriptions can be found from [http://meego.gitorious.org/meego-quality-assurance/handset-ux-tests gitorious] (mhts-de-performance-tests)
 +
 
 +
For questions or comments contact [mailto:anssi.takku@nokia.com Anssi Takku] (IRC: zotte)
==== Handset Start-up Time ====
==== Handset Start-up Time ====
[[File:Handset_startup.PNG]]
[[File:Handset_startup.PNG]]
 +
<br>
 +
==== Handset Shutdown Time ====
==== Handset Shutdown Time ====
[[File:Handset_shutdown.PNG]]
[[File:Handset_shutdown.PNG]]
-
* 1.1.99.4.20110427.5.DE: measurement N/A ([http://bugs.meego.com/show_bug.cgi?id=16196 16196] [n900]Fails to shut down device by pressing power key)
+
 
-
* Alpha (wk15) release: measurement N/A (device can't be turn off using power button bug 16196)
+
<br>
==== Application Start-up Time ====
==== Application Start-up Time ====
[[File:Application_startup.PNG]]
[[File:Application_startup.PNG]]
-
* 1.1.99.4.20110427.5.DE: Camera measurement N/A ([http://bugs.meego.com/show_bug.cgi?id=16852 16852]camera application doesn't launch)
+
 
-
* camera start-up time was measured first time with Alpha (wk15) release
+
* Note! Helium browser was first time included in Summer Release. We continue still tracking Fennec's response time. More info about browsers in Summer Release [http://forum.meego.com/showthread.php?t=3733 presentation]
 +
* Note2! Fennec start-up time is really slow after every reboot. Start-up time decreases rapidly after opening and closing application successfully few times
 +
<br>
==== Incoming Call Time ====
==== Incoming Call Time ====
Incoming call time measures how long it takes to start playing ringtone after UI notification for incoming call is visible
Incoming call time measures how long it takes to start playing ringtone after UI notification for incoming call is visible
[[File:Incoming_call.PNG]]
[[File:Incoming_call.PNG]]
-
* 1.1.99.4.20110427.5.DE: N/A ([http://bugs.meego.com/show_bug.cgi?id=15950 15950] No audio or audio routing on N900 with Pulseaudio 0.9.22)
+
 
-
* Alpha (wk15): N/A (ringtone doesn't play bug 15950)
+
<br>
==== Rotation Time ====
==== Rotation Time ====
[[File:Rotation_time.PNG]]
[[File:Rotation_time.PNG]]
 +
 +
== Core ==
 +
Core tests are run from the core level, so no UI involvement is present.
 +
 +
* HW: N900
 +
* image: latest Developers Edition image
 +
 +
=== Power measurements ===
 +
These measurements are done to keep an eye on power consumption of the device. The measurements are done on second boot when the device is in standby. Default configurations are not modified so radios that are on by default will be on. Device is registered to cellular network during the test. -lapoyhon
 +
==== Current consumption on idle device ====
 +
[[File:Current.jpg]]
 +
==== Wakeups on idle device ====
 +
[[File:Wakeups.jpg]]
 +
==== Progress from MeeGo 1.1 ====
 +
Here's a quick peek into MeeGo 1.1 handset power consumption on N900, just to give a general idea about progress on this area.
 +
<pre>
 +
MeeGo 1.1        home screen 15min after 2nd boot      470 mA
 +
MeeGo 1.1 does not blank screen when idle
 +
DE.2011-06-13.1  home screen 15min after 2nd boot      145 mA
 +
DE.2011-06-13.1  blanked screen 15min after 2nd boot    20 mA
 +
</pre>
 +
 +
=== Filesystem tests (bonnie++) ===
 +
I did a new test run with the memory cards I have available here. Also I tested eMMC from both Maemo and MeeGo system to see the difference. -lapoyhon
 +
 +
====eMMC (ext3) - Maemo Fremantle pr1.3====
 +
<pre>
 +
Version 1.03e      ------Sequential Output------ --Sequential Input- --Random-
 +
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 +
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 +
Nokia-N900-51-1  1G  1793  71  4753  7  3574  5  2577  98 20673  21 247.8  3
 +
                    ------Sequential Create------ --------Random Create--------
 +
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
 +
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 +
                16  5241  65 +++++ +++  119  1  1603  21 +++++ +++    58  0
 +
</pre>
 +
 +
====eMMC (ext3) - MeeGo Community Edition 2011-06-10.1====
 +
<pre>
 +
Version 1.03e      ------Sequential Output------ --Sequential Input- --Random-
 +
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 +
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 +
localhost.locald 1G  2186  81  5591  18  3696  14  2774  97 15423  36  98.1  3
 +
                    ------Sequential7 Create------ --------Random Create--------
 +
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
 +
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 +
                16  175  2 +++++ +++  109  2  1133  15 +++++ +++    68  1
 +
</pre>
 +
 +
====SanDisk MicroSD HC class 6 (ext4) - MeeGo Community Edition 2011-06-10.1====
 +
<pre>
 +
Version 1.03e      ------Sequential Output------ --Sequential Input- --Random-
 +
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 +
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 +
localhost.locald 1G  2684  95 14039  34  5587  20  2768  97 13227  30 283.9  10
 +
                    ------Sequential Create------ --------Random Create--------
 +
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
 +
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 +
                16  2273  39 +++++ +++  4682  47  7904  97 +++++ +++  5952  59
 +
</pre>
 +
 +
====Kingston MicroSD HC class 4 (ext4) - MeeGo Community Edition 2011-06-10.1====
 +
<pre>
 +
Version 1.03e      ------Sequential Output------ --Sequential Input- --Random-
 +
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 +
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 +
localhost.locald 1G  2404  87  7797  20  3728  12  2752  96 12813  29 198.4  7
 +
                    ------Sequential Create------ --------Random Create--------
 +
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
 +
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 +
                16  1205  32 +++++ +++ 11382  98  2041  27 +++++ +++  3040  30
 +
</pre>
[[Category:QA]]
[[Category:QA]]
[[Category:N900]]
[[Category:N900]]

Latest revision as of 05:46, 23 August 2011

Contents

Performance Results

This wiki page contains latest performance testing results done for Community Edition of MeeGo for the Nokia N900 device

Response Time

Response Time testing measures the time a system takes to react to given input by end-user; the total time it takes from a user request until response is received

Measurements will be done for every weekly release and updated here on Mondays

How measurements are done:

  • used HW is N900
  • release is run from memory card (class 6 SD card)
  • measurements done from UI
  • each use case is measured three times with stop watch and median result is used as a result
  • measurements will be done for weekly release (Monday's stable image) or if specially asked
  • currently measured applications don't contain any test data (e.g. call history, received SMS)
  • same operator used all the time

Detail test case descriptions can be found from gitorious (mhts-de-performance-tests)

For questions or comments contact Anssi Takku (IRC: zotte)

Handset Start-up Time

Handset startup.PNG

Handset Shutdown Time

Handset shutdown.PNG


Application Start-up Time

Application startup.PNG

  • Note! Helium browser was first time included in Summer Release. We continue still tracking Fennec's response time. More info about browsers in Summer Release presentation
  • Note2! Fennec start-up time is really slow after every reboot. Start-up time decreases rapidly after opening and closing application successfully few times


Incoming Call Time

Incoming call time measures how long it takes to start playing ringtone after UI notification for incoming call is visible Incoming call.PNG


Rotation Time

Rotation time.PNG

Core

Core tests are run from the core level, so no UI involvement is present.

  • HW: N900
  • image: latest Developers Edition image

Power measurements

These measurements are done to keep an eye on power consumption of the device. The measurements are done on second boot when the device is in standby. Default configurations are not modified so radios that are on by default will be on. Device is registered to cellular network during the test. -lapoyhon

Current consumption on idle device

Current.jpg

Wakeups on idle device

Wakeups.jpg

Progress from MeeGo 1.1

Here's a quick peek into MeeGo 1.1 handset power consumption on N900, just to give a general idea about progress on this area. 

MeeGo 1.1        home screen 15min after 2nd boot      470 mA
MeeGo 1.1 does not blank screen when idle
DE.2011-06-13.1  home screen 15min after 2nd boot      145 mA
DE.2011-06-13.1  blanked screen 15min after 2nd boot    20 mA

Filesystem tests (bonnie++)

I did a new test run with the memory cards I have available here. Also I tested eMMC from both Maemo and MeeGo system to see the difference. -lapoyhon

eMMC (ext3) - Maemo Fremantle pr1.3

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
Nokia-N900-51-1  1G  1793  71  4753   7  3574   5  2577  98 20673  21 247.8   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  5241  65 +++++ +++   119   1  1603  21 +++++ +++    58   0

eMMC (ext3) - MeeGo Community Edition 2011-06-10.1

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.locald 1G  2186  81  5591  18  3696  14  2774  97 15423  36  98.1   3
                    ------Sequential7 Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   175   2 +++++ +++   109   2  1133  15 +++++ +++    68   1

SanDisk MicroSD HC class 6 (ext4) - MeeGo Community Edition 2011-06-10.1

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.locald 1G  2684  95 14039  34  5587  20  2768  97 13227  30 283.9  10
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  2273  39 +++++ +++  4682  47  7904  97 +++++ +++  5952  59

Kingston MicroSD HC class 4 (ext4) - MeeGo Community Edition 2011-06-10.1

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.locald 1G  2404  87  7797  20  3728  12  2752  96 12813  29 198.4   7
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1205  32 +++++ +++ 11382  98  2041  27 +++++ +++  3040  30
Retrieved from "http://wiki.meego.com/ARM/N900/QA/Performance"
Personal tools