Quality/QA-tools/CrashAnalysisSolution

Latest revision as of 08:13, 17 January 2012

Overview

An error leading to an uncontrolled termination of a software process (i.e. a crash) is hard to locate and solve. When a crash occurs, Linux kernel generates a core dump file containing a lot of meaningful data to the developers. The challenge is that when crashes occur in automated testing or in actual use, developers have no simple means to access generated core dumps or to investigate the environment of the crashed processes.

MeeGo Crash Analysis Solution is an automated solution that brings core dumps and other useful information about the environment of the crashed process to the developers. The solution analyzes the core dumps and produces statistical data about the occurrences of different crashes which is also meaningful to testers and managers.

The figure above shows the context of the MeeGo Crash Analysis Solution. The figure depicts different kinds of users and interfaces of the system. Firstly, core dumps are collected from devices-under-test (DUTs). Core dumps can be collected by automated test environment, such as OTS, directly in the DUT by a pre-installed agent, or manually by uploading the core dumps e.g. via a PC. Secondly, the solution analyses the collected core dumps with the help of debug information available in the package repositories of the build system. Thirdly, the collected and analyzed data is consumed by users through a web interface.

System Description

The figure above shows an overview of the structure of MeeGo Crash Analysis Solution. The software consists of components deployed on the device-under-test (DUT) and Crash Analysis Service. The DUT has two components, namely Rich Core Dumper, and Crash Reporter. The Crash Analysis Service contains Upload Server, Corelysis, and Crash Reports Web UI. Also the logical data storages are illustrated in the figure. The illustration also shows the following external entities: a Test PC used for manual core dump uploads, Data users accessing the service and Build Service.

• Rich Core Dumper gathers metadata from the system and creates a compressed rich core dump from the metadata and the original core dump. The rich core dumps are stored to Unsent Core Dumps logical storage on the device.

• Crash Reporter monitors the Unsent Core Dumps storage. When a new core dump is found, it sends all unsent core dumps either automatically or after user confirmation to the Crash Analysis Service depending on a setting. The user can also disable all sending and notifications. Sent core dumps are either deleted or stored on the DUT based on another setting.

• If it is not feasible to enable Crash Reporter on the device, a tester can also send core dumps (either rich or plain) manually with a web browser on a PC. In the automated test environment, OTS uploads core dumps from Unsent Core Dumps storage to Crash Analysis Service similarly to the Web browser in the manual upload case.

• Upload Server receives core dumps and stores them to Received Core Dumps storage.

• Corelysis monitors Received Core Dumps storage and triggers analysis for new core dumps. In the analysis, the files in the rich core dump are extracted and the core dump back trace is analyzed. In order to find correct symbols for function etc. names, the debug information corresponding to the build in question must be fetched from the Package Repository of the Build Service. The results of the analysis are stored to Analyzed Core Dumps storage via the import API of Crash Reports Web UI.

• Crash Reports Web UI is accessed by the data consumers. Through the UI, the consumers can get high level statistical information about the crashes and download core dumps and related rich core files.