MeeGo wiki - User contributions [en]

ARM/N950

2011-11-17T22:12:59Z

Lbt: typos

= MeeGo 1.3 Community Edition (and [http://wiki.merproject.org/wiki/Nemo Nemo]) installation for N950 =

(Basic information as copied from https://meego.com/community/device-program/devices/nokia-n9-devkit)

The Nokia N950 is a platform available now for developers targeting the Nokia N9 and MeeGo handset apps in general. Technical details are available at http://developer.nokia.com/swipe. Questions & comments: http://forum.meego.com/showthread.php?t=3597.

IMPORTANT: commercial developers are encouraged to apply directly at http://developer.nokia.com - thank you for your understanding.

== How to use Community Edition on N950 ==

'''THIS IS IMPORTANT: Assumption: you have Nokia N950 device with the Beta2 release (firmware version *must be* at 34-2 1.2011.34-2_PR_RM680, update using the OneClickFlasher if not). You should also have the skills required, similar to what is required for using N900 CE: http://wiki.meego.com/ARM/N900/GettingStarted. These instructions also assume that you are using Linux as OS on your computer. If you have upgraded over-the-air from beta2, these instructions may not work. Also, you can't flash back to Harmattan as there is no OCF for the updated release!!!'''

'''The procedures described here will void any warranty (if there was any), and will destroy your Harmattan setup, losing any data you have there.''' However, you can always go back to a fresh Harmattan installation with the OneClickFlasher package available at: http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html (note - please login to https://www.developer.nokia.com first or you'll get redirection errors)

=== Downloading the MeeGo Community Edition release ===

{|style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0"
|-
!style="border-style: solid; border-width: 0 1px 1px 0"| Image
!style="border-style: solid; border-width: 0 1px 1px 0"| Description
!style="border-style: solid; border-width: 0 0 1px 0"| Release date
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/n900-de/archive/1.2.90.5.0.20110927.81.CE.2011-09-27.1/images/mg-handset-armv7nhl-n950-ce-testing/ Fall Release]
|style="border-style: solid; border-width: 0 1px 1px 0"| v1.3 Handset image for Fall 2011
|style="border-style: solid; border-width: 0 0 1px 0"| 28.9.2011
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/Nemo/ Weekly image]
|style="border-style: solid; border-width: 0 1px 1px 0"| From here, you can find the weekly and daily builds.
|style="border-style: solid; border-width: 0 0 1px 0"|
|}

=== Flasher for N950 ===
Download Harmattan flasher from [http://tablets-dev.nokia.com/maemo-dev-env-downloads.php here].

Before you proceed make sure that "Device lock" is not enabled in Harmattan (In Settings \ Security \ Device Lock \ Autolock: off). If you have an Mail-for-Exchange account configured, you may need to delete it before you can disable device lock.

If flasher bombs out with "Devicelock ON: cannot flash unsigned image", then you didn't, go back and disable it.

If the beta2 flasher errors with "bb5_rdc_cert_read" or [http://www.martindengler.com/proj/n950-flasher-beta1#failure a few other errors], try the [http://www.martindengler.com/proj/n950-flasher-beta1 beta1 flasher].

=== MeeGo Bootloader (Moslo) ===

In order to boot MeeGo on N950, the default OS, Harmattan, must be replaced with MOSLO, the Meego OS LOader.
Moslo can be flashed as a fiasco image, available at: http://tablets-dev.nokia.com/moslo.php. Moslo source repository is located at: https://gitorious.org/meego-developer-edition-for-n900/moslo). Current moslo fiasco image for N950 is moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin

Nokia N9 similarly will need its own Moslo.

* IMPORTANT: before installing Moslo, boot into Harmattan at least once and wait for a few minutes.
* Power off the device, disconnect usb or changer.
* Run flasher:
sudo flasher -F moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin -f
Flasher now waits for a device to connect.
* Connect your N950 and wait for flashing to finish.
* Do not disconnect. Read on.

=== Write CE to device ===

After flashing MOSLO, boot the device by disconnecting and re-connecting the USB cable to the PC. You will see a warning and a disclaimer screen for 10 seconds followed by green text based MOSLO welcome screen. Wait for the text "Rootfs now exported via USB".

If you don't get the MOSLO screen after successfully flashing MOSLO, then let the device finish booting into harmattan and reboot it. It can take several full reboots of the device to fully enable MOSLO.

If you have automount enabled in your Linux system, the device may appear as a normal USB drive to your desktop. Automount may not show the drive at first because it is in vfat format. Automount will typically use volume ID as mountpoint name.

IMPORTANT: On some systems (Ubuntu) automounting may have too restrictive options. Manual mounting is recommended.

IMPORTANT: Although the '''drive exported by device via MOSLO to the host PC it appears as /dev/sdX, inside the device it is a partition''', instead of a full drive.

At first MOSLO connection, the partition has to be reformatted. Make sure it is unmounted (if necessary umount it '''do not 'eject' it via UI!'''):
sudo umount /dev/sdX
sudo mkfs.ext4 /dev/sdX
You will see warning: "/dev/sdX is entire device, not just one partition! Proceed anyway? (y,n)". This is expected. Just make sure you got the right device node and say "y".

For mounting the MeeGo rootfs manually:
sudo mkdir -p /media/<choose mountpoint name for your N950>
sudo mount /dev/sdX /media/<mountpoint of your N950>

If you already had an MeeGo image on the partition, it can be erased by running:
sudo rm -rf /media/<mountpoint of your N950>/*

Extract the Community Edition .tar.bz2 to the mounted rootfs partition with:
sudo tar --numeric-owner -xf <path>/<CE_package>.tar.bz2 -C /media/<mountpoint of your N950>

After the extraction is ready, unmount/remove safely/eject the partition before disconnecting from usb:
sudo umount /media/<mountpoint of your N950>

=== Boot to CE ===

* The device should start booting to MeeGo right after disconnecting the USB.
* Otherwise start normally by holding power key for 2-4 seconds.
* First you should see the disclaimer screen.
* Next you see a green-on-black Moslo bootscreen.
* The last line should say: "Running Meego Kernel" (it is shown very briefly)
* Then wait patiently for desktop to come up.

Shutting down is done simply by holding the power key until either a shut down graphic or a red screen is shown.
Check that screen back light goes out. If device does not react, hold the power key for over 8 seconds.
This will force HW shut down. Note: using forceful shut down might corrupt the filesystem, leading to a non-bootable setup.

====Trouble starting?====

If device does not react, hold the power key for over 8 seconds.

In case battery is very low, the screen may stay black even if booting continues, or the device may shutdown (and likewise stay black).
* Connect usb or charger and let Moslo charge the battery at least for a few <del>minutes</del> tens of minutes.
* Then disconnect usb/charger and device should continue booting to MeeGo.
* You can reconnect usb/charger after a few seconds. Latest at the Meego splash screen.

Don't panic - try again, check the steps and finally ask for advice at #meego-arm IRC channel.

=== How to remove MeeGo/Moslo and restore Harmattan ===
In order to boot Harmattan again, MeeGo and Moslo must be removed.
This can be done by using OneClickFlasher available here:
http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html

Running the OneClickFlasher will overwrite MeeGo/Moslo with Harmattan. All personal settings will be lost.
It will take about 15-30 minutes.

== Open issues ==

* How about dual boot with Harmattan?
** Not possible with current Moslo release. Community is encouraged to find a way, with modified Moslo or some other means eg. u-boot.
* Will this work in N9?
** Not at the moment. We are working to make a N9 release as well, stay tuned for more information.

=== Reporting bugs against Community Edition ===

* File a bug report on [http://bugs.meego.com/ bugs.meego.com]
* Use '''[CE]''' in the summary
* Add the '''N950''' and '''N900CE''' keywords to the bug report
* Select from Platform '''N900''', if bug can be reproduced with N900 also.

*Notice:
** If bug is producible with MeeGo image also, remove the [CE] prefix from the summary. It's used only for the Community Edition specific bugs.
** Feel free to suggest MeeGo_N900CE_Release_Blocker.
** '''If bug is for application''', check if there's a '''upstream link''' for direct reporting in [[/AppsInCE | CE application list]]. If there's a link, please report to upstream, if not, then to the MeeGo Bugzilla.
** If you found a bug when using MeeGo 1.2 '''Harmattan''', please file the bug to the [http://www.developer.nokia.com/bugs/ http://www.developer.nokia.com/bugs/]

= More info =

More information can be found in [[N950 landing page]].

[[Category:N950]]

Release Infrastructure/IMG

2011-11-02T23:02:09Z

Lbt:

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

In order to keep the docs up-to-date they are now integrated into the code and can be found here:
http://autodoc.meego.com/mint/imager/

[[File:IMG.png]]

= Design =

* Asynchronous
:Fire and forget about it, check back later with the web client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS process or Web UI

IMG consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP using a RabbitMQ server.

= Usage =
== Example Use ==

A typical individual user would follow steps like this:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application.
# The IMG django application receives the request and saves the information
# A message containing the details for the required image is sent via BOSS to the image creation worker.
# The progress and status is noted on the WebUI
# When complete a link to the build log and the image is provided.

== Behind the scenes ==
# The webui launches a simple BOSS process with the selected ks
# The process asks build_ks to modify the ks file and prepare for image building
# Then build_image is asked to create an image
## The build_image worker that picks up the request commences image creation
## A Virtual Machine disk image containing mic2 is started (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
# Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## The worker copies the image from the virtual machine to either a local or nfs-shared www-root
## If there is an error, only a message about it is sent via BOSS.
# When mic2 has finished, the virtual machine is shut down by the worker and the virtual machine disk image is deleted

== Using IMG from BOSS ==

This is an example workflow in BOSS. The

# Prepare a ks
build_ks
# Register the job with the web ui
# image.kickstart and image.name must be set here
request_image
# Build image
# image.kickstart, image.name, image.arch etc must be set by
# this point : see http://autodoc.meego.com/mint/imager/participants/build_image.html
build_image

TO BE DONE
-->
= Installation =
Installation for IMG can be found at: http://autodoc.meego.com/mint/imager/install.html

Note that IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Whether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Release Infrastructure/IMG

2011-10-24T08:07:08Z

Lbt: add link to autodoc

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

In order to keep the docs up-to-date they are now integrated into the code and can be found here:
http://autodoc.meego.com/mint/imager/

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.4.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.4/meego-tools.repo
zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.4/devel:languages:python.repo
zypper ref

Then install the core IMG package and the amqp listener:

Worker:

zypper in img-worker

Web UI:
zypper in img-web

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Whether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Release Infrastructure/IMG

2011-10-24T08:05:12Z

Lbt: /* Installation */

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.4.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.4/meego-tools.repo
zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.4/devel:languages:python.repo
zypper ref

Then install the core IMG package and the amqp listener:

Worker:

zypper in img-worker

Web UI:
zypper in img-web

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Whether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Release Infrastructure/IMG

2011-10-24T08:04:58Z

Lbt: add python repo for suse11.4

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.4.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.4/meego-tools.repo
zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.4/devel:languages:python.repo
zypper ref

Then install the core IMG package and the amqp listener:

Worker:

zypper in img-boss

Web UI:
zypper in img-web

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Whether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Image Creation

2011-10-24T08:04:39Z

Lbt: /* Binary Package installation */

This is the main Image Creator developer's guide. For a more simplistic, step-by-step document, go to: [[Image Creation For Beginners]].

= Overview =

The tool used to create MeeGo images is called "MIC2" (to distinguish from obsolete MIC - Moblin Image Creator). MIC is composed of a series of tools to create images, convert images, chroot, etc. MIC2 is primarily based on Fedora livecd-tools and appliance-tools.

With MIC2 tools, users can create different types of images for different purposes, including live CD images, live USB images, raw images for KVM, VMDK images for Vmware, vdi images for VirtualBox, loop images for IVI platforms, NAND images for Moorestown platforms, ubi images for N900, fs image for MeeGo developers. Also, users can use MIC2 tools to manipulate images, like transforming an image from a virtual machine to a live image, and providing a chroot environment based on an existing live image. With these features, developers can do development work on a host virtual machine running MeeGo or a Meego chroot environment, and transfer the resulting new live image to a target device for final debug/verification.

= Features =

MIC2 offers these major tools:
* mic-image-creator: create images.
* mic-image-convertor: convert a raw/vmdk/vdi/live image into a live image.
* mic-chroot: provide a MeeGo environment from a live/loop image for development, it also can translate that chroot file system into a live image.
* mic-image-writer: write a MeeGo image to a USB disk. This is a safe alternative to dd.

The following support is provided:

* Supports mainstream Linux distros. MIC2 has been thoroughly tested on these distributions:
** Meego
** Fedora (Fedora 13 and above)
** Opensuse (> OpenSUSE 11.3)
** Ubuntu > 10.04
* Supports various types of images: livecd, liveusb, loop, raw, vmdk, nandmrst, vdi, fs, ubi
* Supports image coversion from virtual machine/live images to live images.
* Uses kickstart (.ks) files for image creation. Through this, users can specify which software repositories to use, which packages to install, and basic system configuration directives. Please refer to http://fedoraproject.org/wiki/Anaconda/Kickstart for more information about kickstart configuration files.

= Usage =
== Installation ==

We currently build MIC2 binary rpms/debs for many popular Linux distributions, including Fedora 13, Fedora 14, Fedora15, Ubuntu 10.04, Ubuntu 10.10, OpenSUSE 11.3, OpenSUSE 11.4, and Debian 5.0. Please go to http://repo.meego.com/tools/repos/ to get a repository URL corresponding to your Linux distribution, then add it into your repo or package source for installation and update later. If your distribution isn't in the support list, please install MIC2 from git source.

=== Installation Requirements ===

To use MIC2, your host machine (that will run mic2) must have Intel* Atom* or Intel* Core* 2 CPU (support for SSSE3), this is a hard requirement (ARM image is exceptional).

On Fedora, openSUSE and MeeGo, mic2 depends directly on the following packages (they will be automatically installed on installing mic2):

* util-linux
* coreutils
* python >= 2.5
* e2fsprogs
* dosfstools >= 2.11-8
* yum >= 3.2.24
* pykickstart >= 0.96
* python-iniparse
* syslinux >= 3.82
* curl
* kpartx
* parted
* device-mapper
* zlib
* rsync
* /usr/bin/mkisofs
* wget
* cpio
* isomd5sum
* gzip
* bzip2
* squashfs-tools >= 4.0
* btrfs-progs
* python-zypp >= 0.5.7

On Debian and Ubuntu, mic2 depends on or recommands or suggests the following packages (they will be automatically installed if possible on installing mic2):

Depends:

* bzip2
* curl
* dbus
* dmsetup
* dosfstools
* e2fsprogs (>= 1.41),
* genisoimage
* kpartx
* parted
* psmisc
* python-iniparse
* python-pykickstart (>= 0.96) | pykickstart (>= 0.96),
* python-urlgrabber
* rsync
* squashfs-tools (>= 4.0)
* syslinux (>= 3.82)
* yum (>= 3.2)

Recommends:

* binfmt-support
* btrfs-tools
* extlinux
* qemu-user-static
* udisks | hal

Suggests:

* python-zypp

For source installation, these two packages are necessary:

* zlib-devel(for compiling)
* python-devel(for installation)

You should load these modules, as well, if they're not loaded automatically by the kernel:
 (Use <code>lsmod</code> to list modules.)

* squashfs
* squashfs-tools
* dm_snapshot
* loop

Specific packages for Ubuntu 8.10:

* python-celementtree
* python-elementtree
* dmsetup

=== Installing requirements for Ubuntu 10.10 ===

sudo apt-get install yum rpm kpartx parted syslinux isomd5sum kvm zlib1g-dev squashfs-tools python2.6-dev qemu-arm-static python-urlgrabber

=== Binary Package installation ===

* '''Installation Steps For Fedora 13, Fedora 14, and Fedora 15'''

1. Add MIC2 repo as user root:

<pre>
# cat <<REPO > /etc/yum.repos.d/meego-tools.repo
[meego-tools]
name=MeeGo Tools for Fedora
baseurl=http://repo.meego.com/MeeGo/tools/repos/fedora/\$releasever
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-meego
REPO
</pre>

2. Add gpg key as user root:

<pre>
# gpg2 --keyserver subkeys.pgp.net --recv 0BC7BEC479FC1F8A
# gpg2 --export --armor 0BC7BEC479FC1F8A > /etc/pki/rpm-gpg/RPM-GPG-KEY-meego
</pre>

3. Install mic2 as user root:
<pre>
# yum install mic2
</pre>

if you get asked to import the key do so:
<pre>
warning: rpmts_HdrFromFdno: Header V3 DSA/SHA1 Signature, key ID 79fc1f8a: NOKEY
meego-fedora/gpgkey | 3.3 kB 00:00 ...
Importing GPG key 0x79FC1F8A:
Userid: "Moblin Build (Moblin Build User) <build@moblin.org>"
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-meego
Is this ok [y/N]: y
</pre>

* '''Update Steps For Fedora 13, Fedora 14, and Fedora 15'''

You can use the below command to get the latest mic2 package.

<pre>
# yum update mic2
</pre>

* '''Installation Steps For Ubuntu 10.04, Ubuntu 10.10, and Debian 5.0'''

1. Add package source

For Ubuntu 10.04, add the below line to /etc/apt/sources.list

<pre>
deb http://repo.meego.com/MeeGo/tools/repos/ubuntu/10.04/ /
</pre>

For Ubuntu 10.10, add the below line to /etc/apt/sources.list

<pre>
deb http://repo.meego.com/MeeGo/tools/repos/ubuntu/10.10/ /
</pre>

For Debian 5.0, add the below line to /etc/apt/sources.list

<pre>
deb http://repo.meego.com/MeeGo/tools/repos/debian/5.0/ /
</pre>

mic2 is also available in Debian Testing, which will become Debian 6.0.

2. sudo apt-get update

You should see the following error:
W: GPG error: http://repo.meego.com Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0BC7BEC479FC1F8A

To add the repository public key use the following command:
:gpg --keyserver subkeys.pgp.net --recv 0BC7BEC479FC1F8A
:gpg --export --armor 0BC7BEC479FC1F8A | sudo apt-key add -

Note: Often the port used by gpg is blocked in companies, so gpg will time out. You can install it manually; this example is for Ubuntu 10.10:
:wget http://repo.meego.com/MeeGo/tools/repos/ubuntu/10.10/Release.key
Open System->Administrator->Software Sources. Under the Authentication tab, import the key.

Now redo sudo apt-get update

3. sudo apt-get install mic2

* '''Installation Steps For OpenSUSE 11.3, and OpenSUSE 11.4'''

1. Add repos

For OpenSUSE 11.3:

<pre>sudo zypper addrepo http://repo.meego.com/MeeGo/tools/repos/opensuse/11.3/ meego-tools</pre>

For OpenSUSE 11.4:

<pre>sudo zypper addrepo http://repo.meego.com/MeeGo/tools/repos/opensuse/11.4/ meego-tools</pre>

2. sudo zypper install mic2

* '''Install Steps For OpenSUSE 11.3, and OpenSUSE 11.4'''

<pre>
sudo zypper install mic2
</pre>
See the #bootstrap section for opensuse - you may need to run:
<pre>
mic-create-bootstrap -n trunk -k /var/cache/mic -r http://repo.meego.com/MeeGo/builds/trunk/latest/repos/oss/ia32/packages/ -o /var/cache/meego-bootstrap
</pre>

=== From Stable Git Source Releases ===

You can get the latest stable release of MIC2 from the tag section in http://meego.gitorious.org/meego-developer-tools/image-creator. Click on the "Source tree" link on the top of the screen. There will be a list of tags on the right hand side.

You should follow the below steps to install it:

<pre>
git clone git://gitorious.org/meego-developer-tools/image-creator.git
cd image-creator
git checkout 0.17 #check Gitorious for the most recent tag
make
sudo make install
</pre>

=== From Development Git Tree ===

Note: MIC2 GIT tree has latest-and-greatest source, so stability is not guaranteed. If you run into errors, please use a 'Stable Release' instead before filing a bug.

You need to follow the below steps to git clone MIC2:

<pre>
git clone git://gitorious.org/meego-developer-tools/image-creator.git
</pre>

Build and install:

<pre>
cd image-creator
make clean
make
sudo make install
</pre>

You should add the repo for mic2 then run

<pre>
sudo ./tools/mic-check-alldeps
</pre>

to check/install all the depended packages, otherwise mic2 can't work normally.

== Proxy Setting ==

If you need to use proxy to access Internet, you must set proxy for mic2 correctly. Generally, you should export these environment variables:

<pre>
$ export http_proxy="http://proxy.yourcompany.com:8888"
$ export https_proxy="http://proxy.yourcompany.com:8888"
$ export no_proxy="127.0.0.0/8, .yourcompany.com"
</pre>

If you use sudo to run mic2, you also need to add these into /etc/sudoers in order that they still are valid after sudo.
<pre>
$ sudo vi /etc/sudoers
Add: Defaults env_keep += " no_proxy http_proxy https_proxy"
</pre>
so sudo will adhere to the no_proxy and proxy settings.

You also can set proxy in /etc/mic2/mic2.conf or in ~/.mic2.conf by adding the below two lines:

<pre>
proxy=http://proxy.yourcompany.com:8888/
no_proxy=localhost,127.0.0.0/8,.yourcompany.com
</pre>

Of course, you need to change proxy.yourcompany.com:8888 and .yourcompany.com according to your network.

From mic2 0.23.0 on, mic2 enabled zypp backend, it uses /etc/sysconfig/proxy to get proxy settings, so you also need to create this file (it is a system file in OpenSUSE, so for openSUSE, you just set correct values inside of it) if you need to use proxy to access internet. Here is a sample:

<pre>
#### /etc/sysconfig/proxy sample ####
## Path: Network/Proxy
## Description:
## Type: yesno
## Default: no
## Config: kde,profiles
#
# Enable a generation of the proxy settings to the profile.
# This setting allows to turn the proxy on and off while
# preserving the particular proxy setup.
#
PROXY_ENABLED="yes"

## Type: string
## Default: ""
#
# Some programs (e.g. lynx, arena and wget) support proxies, if set in
# the environment. SuSEconfig can add these environment variables to
# /etc/SuSEconfig/* (sourced by /etc/profile etc.) -
# See http://portal.suse.com/sdb/en/1998/01/lynx_proxy.html for more details.
# Example: HTTP_PROXY="http://proxy.provider.de:3128/"
HTTP_PROXY="http://proxy.yourcompany.com:8888/"

## Type: string
## Default: ""
#
# Some programs (e.g. lynx, arena and wget) support proxies, if set in
# the environment. SuSEconfig can add these environment variables to
# /etc/SuSEconfig/* (sourced by /etc/profile etc.) -
# this setting is for https connections
HTTPS_PROXY="http://proxy.yourcompany.com:8888/"

## Type: string
## Default: ""
#
# Example: FTP_PROXY="http://proxy.provider.de:3128/"
#
FTP_PROXY="http://proxy.yourcompany.com:8888/"

## Type: string
## Default: ""
#
# Example: GOPHER_PROXY="http://proxy.provider.de:3128/"
#
GOPHER_PROXY=""

## Type: string(localhost)
## Default: localhost
#
# Example: NO_PROXY="www.me.de, do.main, localhost"
#
NO_PROXY="localhost, 127.0.0.1, .yourcompany.com"

#### /etc/sysconfig/proxy sample ####
</pre>

You need to change proxy.yourcompany.com:8888 and .yourcompany.com according to your network. By default, zypp will be used, but you can use option --pkgmgr=yum to force mic2 to use yum, you can do so if you find zypp can't work for you, but remember to file a bug on http://bugs.meego.com/enter_bug.cgi?product=Development%20Tools&component=MIC%20(Image%20Creator) if you find any issue related to zypp.

An option --proxy for repo command in kickstart file also can set proxy, you can specify it as follows:

<pre>
repo --name=meego --baseurl=http://repo.meego.com/trunk/repo/ia32/os/ --proxy=http://proxyhost:proxyport/
</pre>

You also need to add two more options to repo if your proxy needs user authentication:

<pre>
repo --name=meego --baseurl=http://repo.meego.com/trunk/repo/ia32/os/ --proxy=http://proxyhost:proxyport/ --proxyuser=proxyusername --proxypaswd=proxyuserpassword
</pre>

== Running mic-image-creator ==

Configuration of images is based on kickstart, the format used for unattended installation in Fedora and Redhat.

Super user privileges are needed. The tool is more or less self-documented, use the --help option to see options.
<pre>
sudo mic-image-creator --help
</pre>

=== Creating Supported Image Types ===

KickStart (.ks) configuration files are passed to MIC2 to create tailored images. KickStart files specify what repos to pull from, what packages to include, what post-scripts to run and what type of images to create.

creating arm images needs 'qemu-arm' os 'qemu-arm-static' command available, and creating vdi or vmdk images needs 'vboxmanager' available, so be sure your system have installed them before creating these types of images.

To obtain the official Meego .ks files, go here: http://wiki.meego.com/Image_Creation#Official_Meego_.ks_files

'''Create ARMv7 Images'''

When creating images for ARMv7 hardware, pass ''--arch=armv7hl'' to ''mic-image-creator'' commands.

'''Create Livecd Image'''

<pre>
sudo mic-image-creator --config=default.ks --format=livecd --cache=mycache
</pre>

This tells image-creator to use the kickstart file default.ks to obtain info about which packages to download and include in the image, and --cache is the directory on your local machine which will host a cache of these packages. The cache is very useful if you are remote to the server and reduces the amount of downloaded packages next time you create an image. Next time the command is run, this cache will simply be 'updated' if there are changes, rather than re-downloading from the repositories again.

The output of this command will be a file named meego-1.2-default-XX.iso created. This ISO image is a hybrid image and can be either written to a disk device or burned onto a cd.

To burn it onto a USB stick, just run the following command, assuming the USB stick is /dev/sdb in your system:

<pre>
sudo mic-image-writer meego-1.2-default-XX.iso
</pre>

'''Create Moorestown NAND Image'''

<pre>
sudo mic-image-creator --config=default.ks --format=mrstnand

remove the first 512 bytes from the image to remove the partition information.
sudo dd bs=512 skip=1 if=meego-1.2-default-XX-sda.bin of= meego-1.2-default-XX-nand.img
</pre>

'''Create Liveusb Image'''

<pre>
sudo mic-image-creator --config=default.ks --format=liveusb --cache=mycache
</pre>

A file named meego-1.0-default-XX.usbimg will be created. To burn it onto a USB stick, run the following command, assuming the USB stick is /dev/sdb in your system:

<pre>
sudo mic-image-writer meego-1.2-default-XX.usbimg
</pre>

This image has a FAT file system and can be mounted easily on Windows and other OSes.

'''Create Liveusb Image Interactively'''

<pre>
sudo mic-image-creator --config=default.ks --format=liveusb --interactive --cache=mycache
</pre>

It directly creates a MeeGo live usb stick.

Liveusb image can be created in two different ways, interactive and non-interactive (default). When running MIC2 in a non-interactive mode, MIC2 creates a USB images file that can be copied directly into a USB stick. The non-interactive mode can be used for daily build or automated testing. The interactive mode will write the images onto a USB disk, it will detect if a USB stick is available and detects and verifies a partition is available. If two or more USB devices are present, you will be asked to select the target device. This way the contents of the existing USB stick won't be destroyed unless no appropriate partition is found, end user should use it to create the live USB.

'''Create Loop Image'''

This is the simplest image format available. Such images can be loop mounted and chrooted into, for example, to build applications or for debugging purposes.

<pre>
sudo mic-image-creator --config=default.ks --format=loop --cache=mycache
</pre>

A file named meego-1.2-default-XX.img is created. You can use below to mount it and chroot into it.

<pre>
sudo mount -o loop meego-1.2-default-XX.img /mnt
sudo chroot /mnt su -
</pre>

'''Create KVM Image'''

Can be used with QEMU or other VMM applications to launch MeeGo as a virtualized instance.

<pre>
sudo mic-image-creator --config=default.ks --format=raw --cache=mycache
</pre>

A file named meego-1.2-default-XX folder with a meego-1.2-default-XX-sda.raw image is created. For optimal results, use this feature in a machine with VT support and enable it in the BIOS.

If you use Fedora or openSUSE, run the following command to launch the image into MeeGo KVM virtual machine:

<pre>
sudo qemu-kvm -m 512 -boot c -hda meego-1.2-default-XX-sda.raw -std-vga
</pre>

If you use Ubuntu, run the following command launch image:

<pre>
sudo kvm -m 512 -boot c -hda meego-1.2-default-XX-sda.raw
</pre>

'''Create VMDK Image'''

VMDK images can be loaded into Vmware or Vmware player. MIC2 also generates a VMX file that has image configuration and can be used to launch the image in Vmware by just clicking the file. If you have vmplayer, it should open automatically.

<pre>
sudo mic-image-creator --config=default.ks --format=vmdk --cache=mycache
</pre>

A file named meego-1.2-default-XX folder with both meego-1.2-default-XX-sda.vmdk image and meego-1.2-default-XX-sda.vmx is created. Just run vmware or vmware player, and select the generated vmx file.

=== KickStart Files (configuration files used for image creation) ===

* [[Image Configurations - KickStart Files]]

=== MIC2 Configuration file ===

MIC2 options can be specified in a global configuration file ('''/etc/mic2/mic2.conf''') and local configuration file ('''$HOME/.mic2.conf'''), where you can specify:
* Repository configuration
* proxy settings
* cache directories and
* other variables
that you normally would have to re-type in your command-line.

The two configuration files have the same format, /etc/mic2/mic2.conf is global, no matter whoever is running MIC2, this configuration file will be read first, $HOME/.mic2.conf is just valid for current user, every user can have their own settings.
The final configuration is a combination of two configuration files, options in '''$HOME/.mic2.conf''' will override those from '''/etc/mic2/mic2.conf'''. Options from command line have the highest priority.
If an option is specified on the command line, the same option in both configuration files, global and local will be ignored.

Below is an example you can cut-and-paste, and reuse for your needs:

<pre>
[main]
cachedir=/home/user1/mycache
tmpdir=/home/user1/mystorage/tmp
outdir=/home/user1/mystorage
proxy=http://my.proxy.com:911/
no_proxy=localhost,127.0.0.0/8,.mysite.com,172.16.0.0/16
</pre>

''cachedir'' = directory where the cached repo(s) will reside. With this variable set, you do not need to pass the --cache flag in the command-line.

''tmpdir'' = temporary directory used by MIC2 when creating images. With this variable set, you do not need to pass the --tmpdir flag in the command-line.

''outdir'' = where your images will reside once they are created. With this variable set, you do not need to pass the --outdir flag in the command-line.

''proxy'' = specify your proxy if you're behind a behind a firewall.

''no_proxy'' = specify what domains should not sure the proxy setting.

''image_format'' = specify image format.

''default_ks'' = specify a default kickstart file, there are several kickstart files provided in repository http://repo.meego.com/MeeGo/devel/trunk/repo/ia32/os/

'''Note:''' When specifying proxy and no_proxy, you do not need to use the --proxy flag in your .ks files when referring to repos.

In configuration files, you also can specify one or multiple repositories, for example:

<pre>
[trunk]
name=trunk
baseurl=http://repo.meego.com/MeeGo/devel/trunk/repo/ia32/os/
enabled=1
</pre>

Once there is this repository info in your configuration files, and both image_format and default_ks are set, you can just run "sudo mic-image-creator" to create a image, there isn't any extra argument needed.

=== Use Bootstrap ===

MIC2 highly depends on yum and librpm, but different Linux distributions used diffrent yum and librpm versions, so MIC2 has some compatibility issues on OpenSUSE, Ubuntu and Debian, bootstrap is just for fixing these compatibility issues.

bootstrap is a minimal MeeGo file system, MIC2 can run very smoothly on it using chroot mode. You can use the below command to create a bootstrap:

<pre>
sudo mic-create-bootstrap -n trunk -k /your/repo/cache/path -r http://repo.meego.com/MeeGo/builds/trunk/latest/repos/oss/ia32/packages/ -o /your/final/bootstrap
</pre>

NOTE: The repo-url specific for your needs may differ

-n is used to specify repository name, if you have its cache before, it can reduce runtime dramatically to use it with -k option, -k is used to specify your repository cache dir, -r is used to specify MeeGo main repository.

Once you created your bootstrap, you can use this bootstrap to run MIC2 as follows

<pre>
sudo mic-image-creator --bootstrap=/your/final/bootstrap --format=livecd --config=default.ks --cache=/your/repo/cache/path
</pre>

Alternatively, you can create and use bootstrap in one mic-image-creator run, as follows

<pre>
sudo mic-image-creator --build-bootstrap --bootstrap=/your/final/bootstrap --format=livecd --config=default.ks --cache=/your/repo/cache/path
</pre>

== Running mic-image-writer ==

mic-image-writer writes a Meego image to a USB disk as an alternative to dd.

mic-image-writer can run in both console mode and GUI mode. It can decide which mode to run, according to current system environment. You can also use a given option to force it to run in some other mode. Run 'mic-image-writer --help' to get usage information:

<pre>
Usage: mic-image-writer [options] [image file]

Options:
-h, --help Show this help message and exit
-c, --console Run in console mode
-g, --gui Run in GUI mod
</pre>
Here is a run example in console mode:

<pre>
$ sudo mic-image-writer meego-xxx.img
Available usb disk:
[1] /dev/sdc: SanDisk USB Flash Drive
[2] /dev/sdb: SanDisk U3 Cruzer Micro
Please choice [1..2] ? 2
Source: /myhome/meego-xxx.img
Target: /dev/sdb
Image size: 559 MB
Estimated time: 55 seconds
Elapsed time: 64, progress: 100% 8944+0 records in
8944+0 records out
586153984 bytes (586 MB) copied, 63.0486 s, 9.3 MB/s
</pre>

mic-image-writer can estimate how long it will take to write the given image to your USB disk and reports current writing progress. It automatically unmounts your USB disk, if it is mounted. If your USB can't be unmounted, it will terminate.

If you love UI mode, you can run it in X enviroment. The command with the added -g flag:

<pre>
sudo mic-image-writer -g meego-xxx.img
</pre>

Image name is optional.

== Running mic-image-convertor ==

It's very easy to use:

<pre>
sudo mic-image-convertor --source-image=InputImage --target-format=Targegformat

sudo mic-image-convertor -I InputImage -T Targegformat
</pre>

For example, to translate a KVM raw image to livecd image, just type:

<pre>
sudo mic-image-convertor --source-image=meego-core-200902200545/meego-core-200902200545-sda.raw --target-format=livecd
</pre>

A new livecd image of meego-converted-from-raw-200902201804.iso is generated. The tool can detect the type of input image, either raw or vmdk.

This tool is very useful. For example, the developer can launch a virtual machine running MeeGo v2 and make whatever changes on the virtual system like yum install/remove a package, scp a source tarball to VM and build and try, and etc. Then with a simple "sync" or shutdown of VM, the VM image now contains his changes. With this tool, the VM image can be transformed to a live image and burned to a USB flash disk. Now developers can have final verifications in a target device, with changes in the live system.

== Running mic-chroot ==

There are two major usage models for mic-chroot for developers.

'''chroot directly into the image to use it as a development environment, then optionally create a new image based off of your changes'''
<pre>
sudo mic-chroot -c livecd meego-core-200903131337.iso
</pre>

The above command would present a chroot, using the livecd image, to developers with some bind mounts like /proc /sys /dev/pts and /parentroot. With those bind mounts, developers now can easily exchange files in chroot env with host /parentroot, and conduct yum install, yum remove and any other network related operations. After done and typing "exit", a new live ISO image is created from the chroot env with the changes developers made.

'''Unpack and modify the image's filesystem (which you can save), and create a new image based off of your changes'''

Sometimes, developers want to keep the chroot env, so that they can do multiple changes in that root file system at different times. So they want to execute following:

<pre>
sudo mic-chroot -s my-chroot-fs --unpack-only meego-core-200903131337.iso --bind-mounts=/proc:/proc;/:/parentroot;/sys:/sys;/dev/pts:/dev/pts
</pre>

The above command creates a folder called 'my-chroot-fs' in the current directory, using the chroot environment from the livecd image. Of course, if one forgets to add the --bind-mounts options, you'll need to manually do the bind mounting yourself. Don't forget copying /etc/reslov.conf into my-chroot-fs if you need to use DNS. Should you wish to create an image from the chroot, just execute following to create a livecd image

<pre>
sudo mic-chroot -c livecd --convert-only my-chroot-fs/
</pre>

== Enabling Autoinstallation ==

You need to copy a kickstart file into /root/mic2-ks.cfg which is used for
automated installation of the image. To activate autoinstall mode, boot with autoinst on the command line
or add --menus=autoinst to the bootloader directive in the kickstart file you use to
create the image.

= Development Release Processes =

http://wiki.meego.com/MIC2_Development_Release_Processes

= Known issues =

* MIC2 is not compatible with Ubuntu 8.04 (Yum package compatibility)
* MIC2 is not compatible with the initial Ubuntu 9.04. Make sure you have package libsqlite3-0 of 3.6.10-1ubuntu0.2 or later.
* MIC2 has some issues on Debian, please refer to [[MIC on Debian]] to get installation instructions

= Troubleshooting =

You can file a bug on http://bugzilla.meego.com/enter_bug.cgi?product=Development%20Tools (Note: you should select MIC for component) if you run into any MIC2-related issue, bugzilla can track your issue faster and more efficiently.

===libgcc related issues===
<pre>
libgcc_s.so.1 must be installed for pthread_cancel to work"

"Error: failed to create image : '/sbin/mksquashfs /var/tmp/imgcreate-FKCSsk/iso-u8xCPh/LiveOS/osmin /var/tmp/imgcreate-FKCSsk/iso-u8xCPh/LiveOS/osmin.img' exited with error (-6)
</pre>

This error occurs when MIC2 is unable to find the proper version of libgcc. This usually happens on x86_64 systems - MIC2 is looking for the 32 bit version of libgcc, but only the 64 bit version is installed. Installing the 32 bit version of libgcc should resolve this.

===General squashfs issues===
<pre>
"Error: failed to create image : '/sbin/mksquashfs /var/tmp/imgcreate-FKCSsk/iso-u8xCPh/LiveOS/osmin /var/tmp/imgcreate-FKCSsk/iso-u8xCPh/LiveOS/osmin.img' exited with error (-6)
</pre>

This error can also be an issue with the syslinux package. Try updating to the latest version for your distro.

===device-mapper issues===
<pre>
unable to remove open device
</pre>

This is a race condition with device-mapper in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=506644. Usually rerunning the MIC2 command will resolve this.

===python related issues===
<pre>
from urlgrabber.grabber import URLGrabber
ImportError: No module named urlgrabber.grabber
</pre>

Some people are using python they built themselves, so some python modules aren't under their python library path. For such error, only one way is not to use your own python but to use python your system installed.

===debian-based distros related issues===
On debian-based distros, we used pycentral to package mic2, but mic2 source installtion used distutils, so two kinds of installation have different installation path, if they exist there at the same time, it will result in some unpredictable errors to run mic2. To run it correctly, you can keep only one installation, source or binary.

Remove binary installation
<pre>
sudo apt-get remove mic2
</pre>

Remove source installation

Get python lib path
<pre>
$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils.sysconfig
>>> print distutils.sysconfig.get_python_lib()
<PYTHONLIBPATH>
>>>
$
</pre>

Remove mic2 files
<pre>
$ sudo rm -rf <PYTHONLIBPATH>/mic
$ sudo rm -rf <PYTHONLIBPATH>/mic-0.*
$ sudo rm -f /usr/bin/mic
$ sudo rm -f /usr/bin/mic-*
$ sudo rm -f /usr/bin/moblin-*
</pre>

Note: You must replace <PYTHONLIBPATH> with the above python lib path

If program jams after mkfs.vfat, it might be due missing ''vol_id'' program. ''vol_id'' is replaced with ''blkid'' in new ubuntu versions. Try run ''/lib/udev/vol_id'', if it's missing you have troubles.

===ARM related issues===
<pre>
Traceback (most recent call last):
File "/usr/bin/mic-image-creator", line 856, in <module>
ret = main()
File "/usr/bin/mic-image-creator", line 682, in main
run_in_bootstrap(options.bootstrap, argv, bindmounts, arch = options.arch)
File "/usr/bin/mic-image-creator", line 261, in run_in_bootstrap
ret = subprocess.call(args, preexec_fn = chroot_bootstrap)
File "/usr/lib/python2.5/subprocess.py", line 444, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.5/subprocess.py", line 594, in __init__
errread, errwrite)
File "/usr/lib/python2.5/subprocess.py", line 1149, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
</pre>

This error occurs when trying to build ARM images without passing ''--arch=armv7l'' to ''meego-image-creator''.

<pre>
Error: Requires qemu version >= 0.13 for armv7hl
</pre>

you should update the version of 'qemu-arm' or 'qemu-arm-static' to 0.13.0 or above.

Tips: if you encounter arm images creating failed, please try to run bootstrap mode, for armv7hl and armv7nhl is not supported in legacy mode.

===Proxy cache===

See http://bugs.meego.com/show_bug.cgi?id=2343

Some servers (such as with squid as a proxy) may cache more aggressively than
others, and as a result repomd.xml is invalid (old copy) and
references out of date metadata files (primary, comps, filelists, etc). This is
because the server is returning a cached/old copy of repomd.xml.

Proxy cache should be disabled in mic and yum.

= Advanced Hacking Tips =

===replace kernel in live image===

<pre>
$sudo mic-image-convertor --source-image=<your-netbook.img> --target-format=livecd --shell
</pre>

This command will give you a chroot environment of MeeGo, you can follow the below steps to replace the default kernel with a new one.

1). remove the old kernel

<pre>
# rpm -e kernel-netbook
</pre>

NOTE: execute the above command in chroot environment.

2). copy new kernel to chroot environment of MeeGo

<pre>
$sudo cp /path/to/kernel-*.rpm /var/tmp/imgcreate-*/install_root/
</pre>

NOTE: execute the above command in your host system.

/path/to/kernel-*.rpm is your new kernel to install, /var/tmp/imgcreate-*/install_root/ is the directory where chroot MeeGo mounted on.

3). install the new kernel

<pre>
# rpm -ivh /kernel-*.rpm
# rm /kernel-*.rpm
</pre>

NOTE: execute the above command in chroot environment.

After that, you can find out the new kernel in directory /boot/.

4). update contents of isolinux

<pre>
$sudo cp /var/tmp/imgcreate-*/install_root/boot/initrd-*.img /var/tmp/imgcreate-*/iso-*/isolinux/initrd0.img
$sudo cp /var/tmp/imgcreate-*/install_root/boot/vmlinuz-* /var/tmp/imgcreate-*/iso-*/isolinux/vmlinuz0
</pre>

NOTE: execute the above command in your host system.

5. create a new image
<pre>
#exit
</pre>
NOTE: execute the above command in chroot environment.

After following all the above steps, you'll have a new image.

===build btrfs image===
From version 0.20.1 on, mic2 can support btrfs, you can create btrfs image by change fstype=ext3 in your kickstart file to fstype=btrfs. A successful creation will depend on btrfs-progs (btrfs-tools on debian-based distros, btrfsprogs on openSUSE) and btrfs kernel module in your local system, the known issue is we can't create btrfs image on Fedora 11, the issue is the default btrfs kernel module has some issue which will make kernel panic, so for this case, you must make sure you have a stable and latest kernel and btrfs-progs 0.19 on your system. You'll need to build your kernel by yourself and use it to boot your system.

Btrfs-progs is a prerequisite tool, it includes /sbin/mkfs.btrfs, but its version must match btrfs kernel module, so you must ensure this, mic2 repo http://repo.meego.com/MeeGo/tools/repos/ has an issue which will install btrfs-progs 0.19 when you install mic2, this is an repo sync error, so please make sure you have your distro's btrfs-progs after mic2 is installed, if your distro's btrfs-progs version is 0.19, this isn't an issue, otherwise you must remove btrfs-progs installation introduced by mic2 and reinstall your distro's btrfs-progs.

How to build btrfs image on Fedora 11 or lower version?

# Download kernel-2.6.34 or kernel-2.6.35
# Configure btrfs as module
# make; make modules; make install; make modules_install
# Reboot your system using this built-newly kernel
# Install btrfs-progs 0.19

Done, you're ready for creating btrfs image.

===Metapackages, Patterns, and Repository Creation===
Meta-packages are discouraged in MeeGo. The preferred method is to use patterns in the repository (supported by both zypper and mic2). This is done by creating a patterns.xml file, which is a fairly simple XML file:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<pattern xmlns:rpm="http://linux.duke.edu/metadata/rpm"
xmlns="http://novell.com/package/metadata/suse/pattern">
<name>meego-core</name>
<summary>MeeGo Core</summary>
<description>Packages needed for Compliance</description>
<uservisible/>
<category lang="en">Base Group</category>
<rpm:requires>
<rpm:entry name="pam"/>
<rpm:entry name="rootfiles"/>
<rpm:entry name="bash"/>
<rpm:entry name="sysvinit"/>

</rpm:requires>
</pattern>
</pre>

The detailed documentation for this XML schema may be found on the [http://en.opensuse.org/openSUSE:Standards_Rpm_Metadata OpenSuSe Wiki]. There is also a Relax NG schema found in the source code for [http://gitorious.org/opensuse/libzypp libzypp] (buried in the folder zypp/parser/yum/schema).

In MeeGo's [http://meego.gitorious.org/meego-os-base/package-groups package-groups], you can find several utilities, style sheets, and examples for how to create patterns and convert them to groups (comps).

Once you have both a patterns.xml and comps.xml file, you can create your repository like this. (For brevity, assuming a simple 'packages' repository.)

<code>
## Copy RPM packages to a folder, repos/
$ mkdir -p repos/repodata
$ cp -f patterns.xml comps.xml repos/repodata
$ createrepo -g repos/repodata/comps.xml repos/
$ modifyrepo repos/repodata/patterns.xml repos/repodata
</code>

Now, GPG sign your repos/repodata/repomd.xml file. Note that `modifyrepo` is a part of the `createrepo` package.

Release Infrastructure/BOSS/Installation

2011-10-23T10:19:23Z

Lbt: /* Participants and SkyNET Installation */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to continuous change.

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 installation (OpenSuse 11.4 has some issues - patches welcome)

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install (and start) the boss viewer.

Then, as usual
/etc/init.d/boss-viewer {start|stop|restart|force-reload|log}

== Participants and SkyNET Installation ==

BOSS needs participants to do any real work. These can be installed on the main BOSS machine or on other machines in the network.

Participants are managed by SkyNET (ie start/stop/logging/registration etc)

You may install SkyNET by itself or it will be pulled in when you install any of the boss-participant packages. See http://autodoc.meego.com/mint/boss-standard-workflow/ for details of participants

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing
zypper ref
zypper in boss-participant-standard-workflow boss-participant-prechecks
or
zypper in boss-skynet

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-standard-workflow boss-participant-standard-workflow boss-participant-prechecks
or for an empty installation:
apt-get install --no-install-recommends boss-skynet

This will install (but not start) the boss participants.

=== SkyNET ===
To use skynet see:

skynet help

To see the participants :

skynet list

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.2/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-10-23T10:05:17Z

Lbt:

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to continuous change.

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 installation (OpenSuse 11.4 has some issues - patches welcome)

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install (and start) the boss viewer.

Then, as usual
/etc/init.d/boss-viewer {start|stop|restart|force-reload|log}

== Participants and SkyNET Installation ==

BOSS needs participants to do any real work. These can be installed on the main BOSS machine or on other machines in the network.

Participants are managed by SkyNET (ie start/stop/logging/registration etc)

You may install SkyNET by itself or it will be pulled in when you install the boss-standard-workflow package.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing
zypper ref
zypper in boss-participant-standard-workflow boss-participant-prechecks
or
zypper in boss-skynet

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-standard-workflow boss-participant-standard-workflow boss-participant-prechecks
or for an empty installation:
apt-get install --no-install-recommends boss-skynet

This will install (but not start) the boss participants.

=== SkyNET ===
To use skynet see:

skynet help

To see the participants :

skynet list

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.2/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-10-23T09:43:16Z

Lbt: /* BOSS-Viewer */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to continuous change.

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 installation (OpenSuse 11.4 has some issues - patches welcome)

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now: (this manual step should be no longer necessary in future, all participants will have a dependency on boss-skynet)

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install (and start) the boss viewer.

Then, as usual
/etc/init.d/boss-viewer {start|stop|restart|force-reload|log}

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-10-23T09:23:41Z

Lbt: /* Debian Squeeze/6.0 */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to continuous change.

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 installation (OpenSuse 11.4 has some issues - patches welcome)

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/Testing/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now: (this manual step should be no longer necessary in future, all participants will have a dependency on boss-skynet)

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-10-23T09:23:03Z

Lbt: /* Overview */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to continuous change.

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 installation (OpenSuse 11.4 has some issues - patches welcome)

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now: (this manual step should be no longer necessary in future, all participants will have a dependency on boss-skynet)

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-10-23T09:21:34Z

Lbt: remove RC

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to change. In particular the
repositories are not yet finalised (eg installing from :RC).

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 or OpenSuse 11.4 installation.

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now: (this manual step should be no longer necessary in future, all participants will have a dependency on boss-skynet)

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

This is an optional step to verify that you installation was successful so far. Experienced users can skip it. In order to
make the test works you need to install package <code>boss-skynet</code> manually. (Using the same command you have installed boss components above, it comes from the same repository.) If you choose to skip this step the package will be installed automatically as a dependency in a later step (this is not true as of Sept. '11 because some dependencies are still missing)

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

ARM/N950

2011-10-19T15:21:14Z

Lbt: /* How to use Community Edition on N950 */

= MeeGo 1.3 Community Edition installation for N950 =

(Basic information as copied from https://meego.com/community/device-program/devices/nokia-n9-devkit)

The Nokia N950 is a platform available now for developers targeting the Nokia N9 and MeeGo handset apps in general. Technical details are available at http://developer.nokia.com/swipe. Questions & comments: http://forum.meego.com/showthread.php?t=3597.

IMPORTANT: commercial developers are encouraged to apply directly at http://developer.nokia.com - thank you for your understanding.

== How to use Community Edition on N950 ==

Assumption: you have Nokia N950 device with the Beta2 release (firmware version *must be* at 34-2 1.2011.34-2_PR_RM680, update using the OneClickFlasher if not). You should also have the skills required, similar to what is required for using N900 CE: http://wiki.meego.com/ARM/N900/GettingStarted. These instructions also assume that you are using Linux as OS on your computer.

'''The procedures described here will void any warranty (if there was any), and will destroy your Harmattan setup, losing any data you have there.''' However, you can always go back to a fresh Harmattan installation with the OneClickFlasher package available at: http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html (note - please login to https://www.developer.nokia.com first or you'll get redirection errors)

=== Downloading the MeeGo Community Edition release ===

{|style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0"
|-
!style="border-style: solid; border-width: 0 1px 1px 0"| Image
!style="border-style: solid; border-width: 0 1px 1px 0"| Description
!style="border-style: solid; border-width: 0 0 1px 0"| Release date
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/n900-de/archive/1.2.90.5.0.20110927.81.CE.2011-09-27.1/images/mg-handset-armv7nhl-n950-ce-testing/ Fall Release]
|style="border-style: solid; border-width: 0 1px 1px 0"| v1.3 Handset image for Fall 2011
|style="border-style: solid; border-width: 0 0 1px 0"| 28.9.2011
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/n900-de/ Weekly image]
|style="border-style: solid; border-width: 0 1px 1px 0"| From here, you can find the weekly and daily builds.
|style="border-style: solid; border-width: 0 0 1px 0"|
|}

=== Flasher for N950 ===
Download Harmattan flasher from [http://tablets-dev.nokia.com/maemo-dev-env-downloads.php here].

Before you proceed make sure that "Device lock" is not enabled in Harmattan (In Settings \ Security \ Device Lock \ Autolock: off). If you have an Mail-for-Exchange account configured, you may need to delete it before you can disable device lock.

If flasher bombs out with "Devicelock ON: cannot flash unsigned image", then you didn't, go back and disable it.

If the beta2 flasher errors with "bb5_rdc_cert_read" or [http://www.martindengler.com/proj/n950-flasher-beta1#failure a few other errors], try the [http://www.martindengler.com/proj/n950-flasher-beta1 beta1 flasher].

=== MeeGo Bootloader (Moslo) ===

In order to boot MeeGo on N950, the default OS, Harmattan, must be replaced with MOSLO, the Meego OS LOader.
Moslo can be flashed as a fiasco image, available at: http://tablets-dev.nokia.com/moslo.php. Moslo source repository is located at: https://gitorious.org/meego-developer-edition-for-n900/moslo). Current moslo fiasco image for N950 is moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin

Nokia N9 similarly will need its own Moslo.

* IMPORTANT: before installing Moslo, boot into Harmattan at least once and wait for a few minutes.
* Power off the device, disconnect usb or changer.
* Run flasher:
sudo flasher -F moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin -f
Flasher now waits for a device to connect.
* Connect your N950 and wait for flashing to finish.
* Do not disconnect. Read on.

=== Write CE to device ===

After flashing MOSLO, boot the device by disconnecting and re-connecting the USB cable to the PC. You will see a warning and a disclaimer screen for 10 seconds followed by green text based MOSLO welcome screen. Wait for the text "Rootfs now exported via USB".

If you don't get the MOSLO screen after successfully flashing MOSLO, then let the device finish booting into harmattan and reboot it. It can take several full reboots of the device to fully enable MOSLO.

If you have automount enabled in your Linux system, the device may appear as a normal USB drive to your desktop. Automount may not show the drive at first because it is in vfat format. Automount will typically use volume ID as mountpoint name.

IMPORTANT: On some systems (Ubuntu) automounting may have too restrictive options. Manual mounting is recommended.

IMPORTANT: Although the '''drive exported by device via MOSLO to the host PC it appears as /dev/sdX, inside the device it is a partition''', instead of a full drive.

At first MOSLO connection, the partition has to be reformatted. Make sure it is unmounted (if necessary umount it '''do not 'eject' it via UI!'''):
sudo umount /dev/sdX
sudo mkfs.ext4 /dev/sdX
You will see warning: "/dev/sdX is entire device, not just one partition! Proceed anyway? (y,n)". This is expected. Just make sure you got the right device node and say "y".

For mounting the MeeGo rootfs manually:
sudo mkdir -p /media/<choose mountpoint name for your N950>
sudo mount /dev/sdX /media/<mountpoint of your N950>

If you already had an MeeGo image on the partition, it can be erased by running:
sudo rm -rf /media/<mountpoint of your N950>/*

Extract the Community Edition .tar.bz2 to the mounted rootfs partition with:
sudo tar --numeric-owner -xf <path>/<CE_package>.tar.bz2 -C /media/<mountpoint of your N950>

After the extraction is ready, unmount/remove safely/eject the partition before disconnecting from usb:
sudo umount /media/<mountpoint of your N950>

=== Boot to CE ===

* The device should start booting to MeeGo right after disconnecting the USB.
* Otherwise start normally by hosding power hoy for 2-4 seconds.
* First you should see the disclaimer screen.
* Next you see a green-on-black Moslo bootscreen.
* The last line should say: "Running Meego Kernel" (it is shown very briefly)
* Then wait patiently for desktop to come up.

Shutting down is done simply by holding the power key until either a shut down graphic or a red screen is shown.
Check that screen back light goes out. If device does not react, hold the power key for over 8 seconds.
This will force HW shut down. Note: using forceful shut down might corrupt the filesystem, leading to a non-bootable setup.

====Trouble starting?====

If device does not react, hold the power key for over 8 seconds.

In case battery is very low, the screen may stay black even if booting continues, or the device may shutdown (and likewise stay black).
* Connect usb or charger and let Moslo charge the battery at least for a few <del>minutes</del> tens of minutes.
* Then disconnect usb/charger and device should continue booting to MeeGo.
* You can reconnect usb/charger after a few seconds. Latest at the Meego splash screen.

Don't panic - try again, check the steps and finally ask for advice at #meego-arm IRC channel.

=== How to remove MeeGo/Moslo and restore Harmattan ===
In order to boot Harmattan again, MeeGo and Moslo must be removed.
This can be done by using OneClickFlasher available here:
http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html

Running the OneClickFlasher will overwrite MeeGo/Moslo with Harmattan. All personal settings will be lost.
It will take about 15-30 minutes.

== Open issues ==

* How about dual boot with Harmattan?
** Not possible with current Moslo release. Community is encouraged to find a way, with modified Moslo or some other means eg. u-boot.
* Will this work in N9?
** Not at the moment. We are working to make a N9 release as well, stay tuned for more information.

=== Reporting bugs against Community Edition ===

* File a bug report on [http://bugs.meego.com/ bugs.meego.com]
* Use '''[CE]''' in the summary
* Add the '''N950''' and '''N900CE''' keywords to the bug report
* Select from Platform '''N900''', if bug can be reproduced with N900 also.

*Notice:
** If bug is producible with MeeGo image also, remove the [CE] prefix from the summary. It's used only for the Community Edition specific bugs.
** Feel free to suggest MeeGo_N900CE_Release_Blocker.
** '''If bug is for application''', check if there's a '''upstream link''' for direct reporting in [[/AppsInCE | CE application list]]. If there's a link, please report to upstream, if not, then to the MeeGo Bugzilla.
** If you found a bug when using MeeGo 1.2 '''Harmattan''', please file the bug to the [http://www.developer.nokia.com/bugs/ http://www.developer.nokia.com/bugs/]

= More info =

More information can be found in [[N950 landing page]].

[[Category:N950]]

ARM/N950

2011-10-18T14:04:16Z

Lbt: /* How to use Community Edition on N950 */

= MeeGo 1.3 Community Edition installation for N950 =

(Basic information as copied from https://meego.com/community/device-program/devices/nokia-n9-devkit)

The Nokia N950 is a platform available now for developers targeting the Nokia N9 and MeeGo handset apps in general. Technical details are available at http://developer.nokia.com/swipe. Questions & comments: http://forum.meego.com/showthread.php?t=3597.

IMPORTANT: commercial developers are encouraged to apply directly at http://developer.nokia.com - thank you for your understanding.

== How to use Community Edition on N950 ==

Assumption: you have Nokia N950 device with the Beta2 release (firmware version 34-2 1.2011.34-2_PR_RM680). You should also have the skills required, similar to what is required for using N900 CE: http://wiki.meego.com/ARM/N900/GettingStarted. These instructions also assume that you are using Linux as OS on your computer.

'''The procedures described here will void any warranty (if there was any), and will destroy your Harmattan setup, losing any data you have there.''' However, you can always go back to a fresh Harmattan installation with the OneClickFlasher package available at: http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html (note - please login to https://www.developer.nokia.com first or you'll get redirection errors)

=== Downloading the MeeGo Community Edition release ===

{|style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0"
|-
!style="border-style: solid; border-width: 0 1px 1px 0"| Image
!style="border-style: solid; border-width: 0 1px 1px 0"| Description
!style="border-style: solid; border-width: 0 0 1px 0"| Release date
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/n900-de/archive/1.2.90.5.0.20110927.81.CE.2011-09-27.1/images/mg-handset-armv7nhl-n950-ce-testing/ Fall Release]
|style="border-style: solid; border-width: 0 1px 1px 0"| v1.3 Handset image for Fall 2011
|style="border-style: solid; border-width: 0 0 1px 0"| 28.9.2011
|-
|style="border-style: solid; border-width: 0 1px 1px 0"| [http://repository.maemo.org/meego/n900-de/ Weekly image]
|style="border-style: solid; border-width: 0 1px 1px 0"| From here, you can find the weekly and daily builds.
|style="border-style: solid; border-width: 0 0 1px 0"|
|}

=== Flasher for N950 ===
Download Harmattan flasher from [http://tablets-dev.nokia.com/maemo-dev-env-downloads.php here].

Before you proceed make sure that "Device lock" is not enabled in Harmattan (In Settings \ Security \ Device Lock \ Autolock: off). If you have an Mail-for-Exchange account configured, you may need to delete it before you can disable device lock.

If flasher bombs out with "Devicelock ON: cannot flash unsigned image", then you didn't, go back and disable it.

If the beta2 flasher errors with "bb5_rdc_cert_read" or [http://www.martindengler.com/proj/n950-flasher-beta1#failure a few other errors], try the [http://www.martindengler.com/proj/n950-flasher-beta1 beta1 flasher].

=== MeeGo Bootloader (Moslo) ===

In order to boot MeeGo on N950, the default OS, Harmattan, must be replaced with MOSLO, the Meego OS LOader.
Moslo can be flashed as a fiasco image, available at: http://tablets-dev.nokia.com/moslo.php. Moslo source repository is located at: https://gitorious.org/meego-developer-edition-for-n900/moslo). Current moslo fiasco image for N950 is moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin

Nokia N9 similarly will need its own Moslo.

* IMPORTANT: before installing Moslo, boot into Harmattan at least once and wait for a few minutes.
* Power off the device, disconnect usb or changer.
* Run flasher:
sudo flasher -F moslo-rootfs-1.2011.34-2_RM680-OEM1-916_0.0.13-12.1.bin -f
Flasher now waits for a device to connect.
* Connect your N950 and wait for flashing to finish.
* Do not disconnect. Read on.

=== Write CE to device ===

After flashing MOSLO, boot the device by disconnecting and re-connecting the USB cable to the PC. You will see a warning and a disclaimer screen for 10 seconds followed by green text based MOSLO welcome screen. Wait for the text "Rootfs now exported via USB".

If you don't get the MOSLO screen after successfully flashing MOSLO, then let the device finish booting into harmattan and reboot it. It can take several full reboots of the device to fully enable MOSLO.

If you have automount enabled in your Linux system, the device may appear as a normal USB drive to your desktop. Automount may not show the drive at first because it is in vfat format. Automount will typically use volume ID as mountpoint name.

IMPORTANT: On some systems (Ubuntu) automounting may have too restrictive options. Manual mounting is recommended.

IMPORTANT: Although the '''drive exported by device via MOSLO to the host PC it appears as /dev/sdX, inside the device it is a partition''', instead of a full drive.

At first MOSLO connection, the partition has to be reformatted. Make sure it is unmounted:
sudo umount /dev/sdX
sudo mkfs.ext4 /dev/sdX
You will see warning: "/dev/sdX is entire device, not just one partition! Proceed anyway? (y,n)". This is expected. Just make sure you got the right device node and say "y".

For mounting the MeeGo rootfs manually:
sudo mkdir -p /media/<choose mountpoint name for your N950>
sudo mount /dev/sdX /media/<mountpoint of your N950>

If you already had an MeeGo image on the partition, it can be erased by running:
sudo rm -rf /media/<mountpoint of your N950>/*

Extract the Community Edition .tar.bz2 to the mounted rootfs partition with:
sudo tar --numeric-owner -xf <path>/<CE_package>.tar.bz2 -C /media/<mountpoint of your N950>

After the extraction is ready, unmount/remove safely/eject the partition before disconnecting from usb:
sudo umount /media/<mountpoint of your N950>

=== Boot to CE ===

* The device should start booting to MeeGo right after disconnecting the USB.
* Otherwise start normally by hosding power hoy for 2-4 seconds.
* First you should see the disclaimer screen.
* Next you see a green-on-black Moslo bootscreen.
* The last line should say: "Running Meego Kernel" (it is shown very briefly)
* Then wait patiently for desktop to come up.

Shutting down is done simply by holding the power key until either a shut down graphic or a red screen is shown.
Check that screen back light goes out. If device does not react, hold the power key for over 8 seconds.
This will force HW shut down. Note: using forceful shut down might corrupt the filesystem, leading to a non-bootable setup.

====Trouble starting?====

If device does not react, hold the power key for over 8 seconds.

In case battery is very low, the screen may stay black even if booting continues.
* Connect usb or charger and let Moslo charge the battery at least for a few minutes.
* Then disconnect usb/charger and device should continue booting to MeeGo.
* You can reconnect usb/charger after a few seconds. Latest at the Meego splash screen.

Don't panic - try again, check the steps and finally ask for advice at #meego-arm IRC channel.

=== How to remove MeeGo/Moslo and restore Harmattan ===
In order to boot Harmattan again, MeeGo and Moslo must be removed.
This can be done by using OneClickFlasher available here:
http://www.developer.nokia.com/info/sw.nokia.com/id/db230178-aa63-4c73-ba7f-20930da13cad/Nokia_N950_OneClickFlashers.html

Running the OneClickFlasher will overwrite MeeGo/Moslo with Harmattan. All personal settings will be lost.
It will take about 15-30 minutes.

== Open issues ==

* How about dual boot with Harmattan?
** Not possible with current Moslo release. Community is encouraged to find a way, with modified Moslo or some other means eg. u-boot.
* Will this work in N9?
** Not at the moment. We are working to make a N9 release as well, stay tuned for more information.

=== Reporting bugs against Community Edition ===

* File a bug report on [http://bugs.meego.com/ bugs.meego.com]
* Use '''[CE]''' in the summary
* Add the '''N950''' and '''N900CE''' keywords to the bug report
* Select from Platform '''N900''', if bug can be reproduced with N900 also.

*Notice:
** If bug is producible with MeeGo image also, remove the [CE] prefix from the summary. It's used only for the Community Edition specific bugs.
** Feel free to suggest MeeGo_N900CE_Release_Blocker.
** '''If bug is for application''', check if there's a '''upstream link''' for direct reporting in [[/AppsInCE | CE application list]]. If there's a link, please report to upstream, if not, then to the MeeGo Bugzilla.
** If you found a bug when using MeeGo 1.2 '''Harmattan''', please file the bug to the [http://www.developer.nokia.com/bugs/ http://www.developer.nokia.com/bugs/]

= More info =

More information can be found in [[N950 landing page]].

[[Category:N950]]

ARM/N900/DeveloperEdition/BOSS

2011-09-23T13:47:23Z

Lbt: email subject

= Automating DE processes =

The DE workflow is (roughly) :

# Developer branches package from P:DE:Trunk
# Developer submits package to P:DE:Trunk:Testing
# Boss runs checks on package
# Boss creates branch of P:DE:Trunk
# Boss submits package(s) to new branch

Later:
# Boss rejects package
or
# Boss notifies maintainers that checks passed

Then:
# Package is accepted to P:DE:Trunk by a maintainer
# bz is updated

The actual processes can be seen in the [http://autodoc.meego.com/boss/ autodoc boss] area until the OBS plugin is available.

== BOSS Checks ==

An SR for one or more packages into the Testing project is seen by BOSS.

BOSS runs the following checks (and stops as soon as one fails)

* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_submitter_is_maintainer check_submitter_is_maintainer]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_built_at_source check_package_built_at_source]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_is_complete check_package_is_complete]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_spec check_spec]

To be done:
* changelog in proper format (present, syntactically correct)
* changelog has entry for this commit
* last changelog version matches spec file version
* tarball not changed within same package version
* executing spectacle doesn't change anything if .yaml present
* yaml must be present if spec is based on .yaml
* content of source tar ball should not change without its changing.
* check that old tarball has been removed if new one is introduced
* SR not empty (define empty?)
** No changes at all. At times people are trying rebuild packages by submitting empty change. (Sage)

Then it tries to build the package in a 'link' project.

This project should be dynamically created and retained in the event of failure for diagnostics. BOSS should then clean up the dynamic project (when told, or after a time).

An image is created using the newly built package(s) and any packages rebuilt in the link project. (Clarify - only those mentioned in the .ks or *all* rebuilt packages?)

Eventually the image will be processed by OTS.

Once all checks have been run the request will be either accepted or marked for acceptance.

== Process Testing ==

The Project:DE:BOSS:Trunk and Project:DE:BOSS:Trunk:Testing areas are used
to test the processes.

The processes to be tested are deployed to /srv/BOSS/processes/Project/DE/BOSS/Trunk/

=== DE_Trunk_SRs ===

To deploy:
* set irc_channel and final_project
* Copy to Project:DE:Trunk
* symlink: SRCSRV_REQUEST_CREATE and SRCSRV_REQUEST_STATECHANGE in Trunk and Trunk:Testing

cd Trunk
ln -s DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE
cd Testing
ln -s ../DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE

= Wish List =
List of wishes for the BOSS. Please write your nick in parentheses to the end so people know who to contact if they have more questions.

# e-mail notice of SR's especially when SR is rejected
# Check if package upstream has changed and do a report of the packages that needs to be rebased on top of the new upstream. Currently this check is done manually with https://gitorious.org/meego-developer-edition-for-n900/upstreamchecker/ script by N900 RE. (Sage)
# In case package fails to connection failure start rebuild automatically after time X. (Sage)
# Add architecture and repository to build error message. (Sage)
#* ''14:06.33 < boss> Build failure in Project:DE:Trunk:Testing: meegotouch-theme More details soon''
# Update patterns based on package-groups package. (Sage)
#* We should have package-group package in the Project:DE:Trunk[:Testing] project that contains the CE package groups. This information should be when ever repository is published pushed to the repository with ''osc meta pattern'' or similar command.
# When submit supersedes another one the handling of the old submit request should be stopped. (Sage)

# SR email subject should include Rejected/Accepted %fromproject to %toproject : %packagelist.

= Bug Lifecycle =

The DE bug lifecycle is (roughly):

* 'open' -> FIXED -> RELEASED -> VERIFIED

# Developer starts working on the bug/feature and sets to ASSIGNED
# Developer submits package to P:DE:Trunk:Testing and documents changelog and/or commit properly (fixes bmc #1234;) - if changelog doesn't have a valid bug/feature reference the request MUST fail but we do not document the failure in the bug report
# Bugzilla records the commit message with link to the repo and timestamp
# Package is accepted to P:DE:Trunk (manual for now....)
# Bugzilla records acceptance with comment and bug status is automatically changed to FIXED if bug is still in OPEN status (New, assigned, needinfo, reopened, waiting)
# If bug is already in FIXED resolution, only acceptance comment is written in the bug report
# If bug is not in OPEN status (New, assigned, needinfo, reopened, waiting) or resolved FIXED (eg. duplicate, wontfix, etc... or verified or closed) the request MUST fail but we do not document the failure in the bug report

Later:
# Release is triggered
# Bug status is automatically changed to RELEASED, a comment with a link to the image is created and the Target Build value is set to match appropriate release

Later still:
# QA review the release and for each bug marked RELEASED, either change to VERIFIED or re-open

Note that eventually, both FIXED and RELEASED statuses would require specific credentials to be modified.

= Image Building =

CE has a private installation of IMG on host img in the internal meego.com network. Speak to David Greaves / lbt for access details.
Login is via meego.com credentials.

ARM/N900/DeveloperEdition/BOSS

2011-09-18T20:10:03Z

Lbt: /* BOSS Checks */

= Automating DE processes =

The DE workflow is (roughly) :

# Developer branches package from P:DE:Trunk
# Developer submits package to P:DE:Trunk:Testing
# Boss runs checks on package
# Boss creates branch of P:DE:Trunk
# Boss submits package(s) to new branch

Later:
# Boss rejects package
or
# Boss notifies maintainers that checks passed

Then:
# Package is accepted to P:DE:Trunk by a maintainer
# bz is updated

The actual processes can be seen in the [http://autodoc.meego.com/boss/ autodoc boss] area until the OBS plugin is available.

== BOSS Checks ==

An SR for one or more packages into the Testing project is seen by BOSS.

BOSS runs the following checks (and stops as soon as one fails)

* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_submitter_is_maintainer check_submitter_is_maintainer]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_built_at_source check_package_built_at_source]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_is_complete check_package_is_complete]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_spec check_spec]

To be done:
* changelog in proper format (present, syntactically correct)
* changelog has entry for this commit
* last changelog version matches spec file version
* tarball not changed within same package version
* executing spectacle doesn't change anything if .yaml present
* yaml must be present if spec is based on .yaml
* content of source tar ball should not change without its changing.
* check that old tarball has been removed if new one is introduced
* SR not empty (define empty?)
** No changes at all. At times people are trying rebuild packages by submitting empty change. (Sage)

Then it tries to build the package in a 'link' project.

This project should be dynamically created and retained in the event of failure for diagnostics. BOSS should then clean up the dynamic project (when told, or after a time).

An image is created using the newly built package(s) and any packages rebuilt in the link project. (Clarify - only those mentioned in the .ks or *all* rebuilt packages?)

Eventually the image will be processed by OTS.

Once all checks have been run the request will be either accepted or marked for acceptance.

== Process Testing ==

The Project:DE:BOSS:Trunk and Project:DE:BOSS:Trunk:Testing areas are used
to test the processes.

The processes to be tested are deployed to /srv/BOSS/processes/Project/DE/BOSS/Trunk/

=== DE_Trunk_SRs ===

To deploy:
* set irc_channel and final_project
* Copy to Project:DE:Trunk
* symlink: SRCSRV_REQUEST_CREATE and SRCSRV_REQUEST_STATECHANGE in Trunk and Trunk:Testing

cd Trunk
ln -s DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE
cd Testing
ln -s ../DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE

= Wish List =
List of wishes for the BOSS. Please write your nick in parentheses to the end so people know who to contact if they have more questions.

# e-mail notice of SR's especially when SR is rejected
# Check if package upstream has changed and do a report of the packages that needs to be rebased on top of the new upstream. Currently this check is done manually with https://gitorious.org/meego-developer-edition-for-n900/upstreamchecker/ script by N900 RE. (Sage)
# In case package fails to connection failure start rebuild automatically after time X. (Sage)
# Add architecture and repository to build error message. (Sage)
#* ''14:06.33 < boss> Build failure in Project:DE:Trunk:Testing: meegotouch-theme More details soon''
# Update patterns based on package-groups package. (Sage)
#* We should have package-group package in the Project:DE:Trunk[:Testing] project that contains the CE package groups. This information should be when ever repository is published pushed to the repository with ''osc meta pattern'' or similar command.
# When submit supersedes another one the handling of the old submit request should be stopped. (Sage)

= Bug Lifecycle =

The DE bug lifecycle is (roughly):

* 'open' -> FIXED -> RELEASED -> VERIFIED

# Developer starts working on the bug/feature and sets to ASSIGNED
# Developer submits package to P:DE:Trunk:Testing and documents changelog and/or commit properly (fixes bmc #1234;) - if changelog doesn't have a valid bug/feature reference the request MUST fail but we do not document the failure in the bug report
# Bugzilla records the commit message with link to the repo and timestamp
# Package is accepted to P:DE:Trunk (manual for now....)
# Bugzilla records acceptance with comment and bug status is automatically changed to FIXED if bug is still in OPEN status (New, assigned, needinfo, reopened, waiting)
# If bug is already in FIXED resolution, only acceptance comment is written in the bug report
# If bug is not in OPEN status (New, assigned, needinfo, reopened, waiting) or resolved FIXED (eg. duplicate, wontfix, etc... or verified or closed) the request MUST fail but we do not document the failure in the bug report

Later:
# Release is triggered
# Bug status is automatically changed to RELEASED, a comment with a link to the image is created and the Target Build value is set to match appropriate release

Later still:
# QA review the release and for each bug marked RELEASED, either change to VERIFIED or re-open

Note that eventually, both FIXED and RELEASED statuses would require specific credentials to be modified.

= Image Building =

CE has a private installation of IMG on host img in the internal meego.com network. Speak to David Greaves / lbt for access details.
Login is via meego.com credentials.

Build Infrastructure/Community Builder/Installation

2011-09-17T14:46:47Z

Lbt: /* Installing the Workers */

== Preparing Host ==
Starting with a minimal Suse 11.2 install

Most of the commands below can be copied into a shell; so we'll define some base data in shell variables to allow you to customise your setup.
(Yes this could all be one monolithic script or a pre-made VM ... but if you're going to run one of these you should at least get your hands dirty setting it up ;) )
<pre>
ROOTFS=/data/11.2min/image-root
ROUTER_IP=10.0.0.1
VG=VM
</pre>

Based on http://en.opensuse.org/Build_Service/KIWI/Cookbook
<pre>
zypper ar http://download.opensuse.org/repositories/Virtualization:/Appliances/openSUSE_11.2/ Virtualization:Appliances
zypper refresh
</pre>

<pre>
zypper in kiwi kiwi-templates kiwi-desc-xenboot squashfs emacs
</pre>

Prepare the storage for LV usage
<pre>
parted /dev/sdb
mklabel
gpt
yes
mkpart p1 0 10%
mkpart p2 10% 20%
mkpart p3 20% 30%
mkpart p4 30% 40%
mkpart p5 40% 50%
mkpart p6 50% 60%
mkpart p7 60% 70%
mkpart p8 70% 80%
mkpart p9 80% 90%
mkpart p10 90% 100%
quit
</pre>
Then make the VG
<pre>
pvcreate /dev/sdb?*
vgcreate $VG /dev/sdb1
</pre>

Prepare an openSUSE minimal image:
<pre>
mkdir -p /data/11.2min
rm -rf /data/11.2min/image-root
kiwi --prepare suse-11.2-JeOS --root $ROOTFS --add-profile xenFlavour --add-package less --add-package iputils --add-package kernel-xen --add-package wget --add-package less --add-package iputils --add-package terminfo --add-package emacs --add-package sudo
</pre>

Update the config & modules:

<pre>
echo default $ROUTER_IP > $ROOTFS/etc/sysconfig/network/routes
echo NETCONFIG_DNS_POLICY=\"\" >> $ROOTFS/etc/sysconfig/network/config
echo nameserver 8.8.8.8 > $ROOTFS/etc/resolv.conf
echo default $ROUTER_IP > $ROOTFS/etc/sysconfig/network/routes
cat << EOF >$ROOTFS/etc/sysconfig/network/ifcfg-eth0
BOOTPROTO='static'
BROADCAST=''
STARTMODE='onboot'
EOF
echo /dev/xvda1 swap swap defaults 0 0 >> $ROOTFS/etc/fstab
</pre>

Prepare some overlay data from the main host to allow ssh into guests
etc
<pre>
# Allow user ssh to all VMs and retain sudo rights
# Should probably be done periodically somehow
mkdir -p /data/vm_overlay
mkdir -p /data/vm_overlay/etc/sysconfig/
ln /etc/passwd /data/vm_overlay/etc/
ln /etc/shadow /data/vm_overlay/etc/
ln /etc/group /data/vm_overlay/etc/
ln /etc/sudoers /data/vm_overlay/etc/
# Fix for screen/bash ctrl-arrow
ln /etc/inputrc /data/vm_overlay/etc/
# Network proxy information
ln /etc/sysconfig/proxy /data/vm_overlay/etc/sysconfig/
</pre>

(note JeOS 11.2 won't allow su - <user> or ssh when user is disabled using !
in /etc/shadow. Instead use an impossible hash.)

=== Xen Networking ===
Yast is cleverer than you so apparently you should use that... I'm sure if you ask it nicely it will work. Alternatively:

in /etc/xen/xend-config.sxp
{{{
(network-script 'network-bridge netdev=eth0')
}}}
Then
{{{
ln -s /dev/.sysconfig/network/ /dev/shm/sysconfig
}}}
and then
{{{
rcxend restart
}}}

== Useful ==

<pre>
chroot_vm() {
GUEST=$1
xm list | grep "^$GUEST " && echo "$GUEST is running" && return
mkdir /mnt/${GUEST}_chroot/
mount /dev/$VG/${GUEST}_root /mnt/${GUEST}_chroot/ &&
chroot /mnt/${GUEST}_chroot/
umount /mnt/${GUEST}_chroot/
rmdir /mnt/${GUEST}_chroot/
}
</pre>

== VM creation fn/scripts ==

The following function/scripts assume
* a VG exists defined by the env variable $VG
* there is a rootfs at /data/11.2min/image-root/

The scripts look in /etc/hosts for the IP so put appropriate lines in there.

Create Xen volumes
<pre>
mk_lv() {
GUEST=$1
lvremove /dev/$VG/${GUEST}_*
lvcreate -L 10G $VG -n ${GUEST}_root
lvcreate -L 2G $VG -n ${GUEST}_swap
mkswap -f /dev/$VG/${GUEST}_swap
}
</pre>

Copy the minimal image and overlay to the VM root disk and set an IP
<pre>
mk_fs() {
GUEST=$1
IP=$(grep " $GUEST " /etc/hosts | cut -f1 -d" ")
if ! [[ $IP =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]] ; then
echo "No IP for $GUEST in /etc/hosts" && return
fi
echo IP is $IP
mkdir -p /mnt/lvm
echo mkfs &&
mkfs -text4 /dev/$VG/${GUEST}_root &&
echo mounting &&
mount /dev/$VG/${GUEST}_root /mnt/lvm &&
echo copy rootfs &&
rsync -HAXa /data/11.2min/image-root/ /mnt/lvm/ &&
echo copy overlay &&
echo ${GUEST}.meego.com > /mnt/lvm/etc/HOSTNAME &&
echo "IPADDR='$IP/24'" >> /mnt/lvm/etc/sysconfig/network/ifcfg-eth0 &&
rsync -HAXa /data/vm_overlay/ /mnt/lvm/ &&
echo copy home &&
rsync -HAXa /home /mnt/lvm/ &&
echo setup root equivalence to allow root rsync/ssh &&
mkdir /mnt/lvm/root/.ssh &&
chmod 700 /mnt/lvm/root/.ssh &&
cp /root/.ssh/id_rsa.pub /mnt/lvm/root/.ssh/authorized_keys &&
sed -i -e's/^root:.*/root:*:::::::/' /mnt/lvm/etc/shadow &&
echo sync &&
sync &&
echo umount &&
umount /mnt/lvm
}
</pre>

Make per-machine files in /etc/xen/ with unique MACs
Additional LV space can be allocated here too
<pre>
mk_g() {
GUEST=$1
MAC=$2
cat <<EOF > /etc/xen/$GUEST.cfg
name='${GUEST}'
disk=['phy:/dev/$VG/${GUEST}_root,xvda2,w', 'phy:/dev/$VG/${GUEST}_swap,xvda1,w']
vif=['mac=$MAC, bridge=eth0']
memory='2048'

root='/dev/xvda2 rw'
kernel='/boot/vmlinuz-2.6.31.12-0.2-xen'
ramdisk='/boot/initrd-2.6.31.12-0.2-xen'
extra='clocksource=jiffies console=hvc0 xencons=tty'

on_poweroff='destroy'
on_reboot='restart'
on_crash='restart'
EOF
}
</pre>

== Create The VMs ==

On the appropriate xen host, make sure you add the machines to
/etc/hosts and setup the base data environment:
<pre>
VG=VM
</pre>

<pre>
mk_lv cfe
mk_lv cbe
mk_lv cstore
mk_lv csign
</pre>

<pre>
mk_fs cfe
mk_fs cbe
mk_fs cstore
mk_fs csign
</pre>

<pre>
mk_g cfe 00:16:3E:40:B5:FE
mk_g cbe 00:16:3E:40:B5:BE
mk_g cstore 00:16:3E:40:B5:5E
mk_g csign 00:16:3E:40:51:64
</pre>

Then start the VMs:
<pre>
xm create /etc/xen/cfe.cfg
xm create /etc/xen/cbe.cfg
</pre>

== Installing the Backend ==

On this guest we need also to setup openSUSE Tools repository:

<pre>
cd /etc/zypp/repos.d/;
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra:/OBS/Tools_Unstable_openSUSE_11.2/Maemo:MeeGo-Infra:OBS.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.2/openSUSE:Tools:Unstable.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key
<pre>
Install:
<pre>
zypper in obs-server obs-signer obs-utils createrepo dpkg lighttpd
</pre>

<pre>
nano /etc/sysconfig/obs-server
OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv5el armv7el"
</pre>

Now /usr/lib/obs/server/BSConfig.pm needs to point to correct server names corresponding to source server, where workers are going to download the source, and the repository server, where RPM repos are going to be shared to users.

What's needed here?????????
<pre>
nano /usr/lib/obs/server/BSConfig.pm
#add
our $srcserver = "http://csrc:5352";
our $reposerver = "http://cbe:5252";
our $serviceserver = "http://csrc:5152";
our $servicedir = "/usr/lib/obs/service/";
our $repodownload = "http://crepo.meego.com";
our @reposervers = ("http://cbe:5252");
our $stageserver = 'rsync://obsrun@cdownload/repo';
#
</pre>

Configure services as daemons

<pre>
chkconfig --add obsrepserver obsscheduler obsdispatcher obspublisher obswarden obssigner
</pre>

Start Services
<pre>
rcobsrepserver start
rcobsscheduler start
rcobsdispatcher start
rcobspublisher start
rcobswarden start
rcobssigner start
</pre>
Not started
<pre>
rcobsservice
</pre>

=== Lighttpd ===

lighttpd also needs to be available on backend server. This is required to provide directory listing on the repositories available on this server when an http/s request to maemo-repo is made through web ui.

Create a new file under /etc/lighttpd/vhosts.d/. It can be obs.conf as well, and add:

<pre>
nano /etc/lighttpd/vhosts.d/obs.conf

$HTTP["host"] =~ "crepo.meego.com" {
server.name = "crepo.meego.com"

server.document-root = "/srv/obs/repos/"
dir-listing.activate = "enable"
}
</pre>

To enable vhosts, remember to uncomment the following in the 'custom includes':

<pre>
nano /etc/lighttpd/lighttpd.conf
##
## custom includes like vhosts.
##
#include "conf.d/config.conf"
# following line uncommented as per
# /usr/share/doc/packages/obs-api/README.SETUP
include_shell "cat vhosts.d/*.conf"
</pre>
And disable ipv6 unless it's secured correctly
<pre>
server.use-ipv6 = "disable"
</pre>
Start lighttpd

<pre>
#first add it as deamon
chkconfig --add lighttpd
rclighttpd start
</pre>

=== Prepare repository imports ===
<pre>
mkdir -p /srv/obs/build/MeeGo:current:Handset/standard/armv5el
mkdir -p /srv/obs/build/MeeGo:current:Handset/standard/armv7el
mkdir -p /srv/obs/build/MeeGo:current:Handset/standard/i586
mkdir -p /srv/obs/build/MeeGo:current:IVI/standard/i586
mkdir -p /srv/obs/build/MeeGo:current:IVI/standard/armv5el
mkdir -p /srv/obs/build/MeeGo:current:IVI/standard/armv7el
mkdir -p /srv/obs/build/MeeGo:current:Netbook/standard/armv7el
mkdir -p /srv/obs/build/MeeGo:current:Netbook/standard/armv5el
mkdir -p /srv/obs/build/MeeGo:current:Netbook/standard/i586

chown -R obsrun:obsrun /srv/obs/build/MeeGo:current:Handset
chown -R obsrun:obsrun /srv/obs/build/MeeGo:current:IVI
chown -R obsrun:obsrun /srv/obs/build/MeeGo:current:Netbook

ln -s /data/mirror/meego-current/current\:Handset/armv5l_full/ /srv/obs/build/MeeGo\:current\:Handset/standard/armv5el/\:full
ln -s /data/mirror/meego-current/current\:Handset/armv7l_full/ /srv/obs/build/MeeGo\:current\:Handset/standard/armv7el/\:full
ln -s /data/mirror/meego-current/current\:Handset/i586_full/ /srv/obs/build/MeeGo\:current\:Handset/standard/i586/\:full

ln -s /data/mirror/meego-current/current\:IVI/armv5l_full/ /srv/obs/build/MeeGo\:current\:IVI/standard/armv5el/\:full
ln -s /data/mirror/meego-current/current\:IVI/armv7l_full/ /srv/obs/build/MeeGo\:current\:IVI/standard/armv7el/\:full
ln -s /data/mirror/meego-current/current\:IVI/i586_full/ /srv/obs/build/MeeGo\:current\:IVI/standard/i586/\:full

ln -s /data/mirror/meego-current/current\:Netbook/armv5l_full/ /srv/obs/build/MeeGo\:current\:Netbook/standard/armv5el/\:full
ln -s /data/mirror/meego-current/current\:Netbook/armv7l_full/ /srv/obs/build/MeeGo\:current\:Netbook/standard/armv7el/\:full
ln -s /data/mirror/meego-current/current\:Netbook/i586_full/ /srv/obs/build/MeeGo\:current\:Netbook/standard/i586/\:full
</pre>

== Installing the Storage node ==

On this guest we need also to setup openSUSE Tools repository:

<pre>
cd /etc/zypp/repos.d/;
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra:/OBS/Tools_Unstable_openSUSE_11.2/Maemo:MeeGo-Infra:OBS.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.2/openSUSE:Tools:Unstable.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key
<pre>
Install:
<pre>
zypper in obs-server obs-source_service
</pre>

<pre>
nano /etc/sysconfig/obs-server
OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv5el armv7el"
</pre>

Now /usr/lib/obs/server/BSConfig.pm needs to point to correct server names corresponding to source server, where workers are going to download the source, and the repository server, where RPM repos are going to be shared to users.

What's needed here?????????
<pre>
nano /usr/lib/obs/server/BSConfig.pm
#add

our $srcserver = "http://csrc:5352";
our $reposerver = "http://cbe:5252";
our $serviceserver = "http://csrc:5152";
our $servicedir = "/usr/lib/obs/service/";
our $repodownload = "http://crepo.meego.com";
our @reposervers = ("http://cbe:5252");

#
</pre>

Configure services as daemons

<pre>
chkconfig --add obssrcserver obsservice
</pre>

Start Services
<pre>
rcobssrcserver start
rcobsservice start
</pre>

== Installing the Frontend ==

On this guest we need also to setup openSUSE Tools repository:

<pre>
cd /etc/zypp/repos.d/;
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra:/OBS/Tools_Unstable_openSUSE_11.2/Maemo:MeeGo-Infra:OBS.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.2/openSUSE:Tools:Unstable.repo
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key
<pre>

Install obs-api (It's going to install lighttpd webserver by dependency for you).
<pre>
zypper in obs-api memcached
</pre>

=== Setup MySQL ===

MySQL server needs to be installed and configured to start as daemon

<pre>
chkconfig --add mysql
rcmysql start
</pre>

Setup a secure installation, if it's the first time starting MySQL

<pre>
/usr/bin/mysql_secure_installation
</pre>

<pre>
touch /root/.my.cnf
chmod 0600 /root/.my.cnf
nano /root/.my.cnf

[client]
user = root
password = <PASSWORD>
[mysqladmin]
user= root
password = <PASSWORD>
</pre>

The frontend instance holds 2 applications, the API and the webui. Each one need a database created

<pre>
mysql -u root -p
create database api_production;
create database webui_production;
</pre>

Add obs user to handle these databases
<pre>
GRANT all privileges
ON api_production.*
TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
GRANT all privileges
ON webui_production.*
TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
FLUSH PRIVILEGES;
</pre>

Now secure the passwd storing config files
<pre>
touch /srv/www/obs/api/config/database.yml
touch /srv/www/obs/webui/config/database.yml
chmod 600 /srv/www/obs/api/config/database.yml
chmod 600 /srv/www/obs/webui/config/database.yml
chown lighttpd /srv/www/obs/api/config/database.yml
chown lighttpd /srv/www/obs/webui/config/database.yml
</pre>

Configure your MySQL user and password in the "production:" section of the API config:
<pre>
nano /srv/www/obs/api/config/database.yml
#change the production section
production:
adapter: mysql
database: api_production
username: obs
password: ************
</pre>

Do the same for the webui. It's configured, by default to use SQLite, but since we're configuring the cluster for production environment, let's bind it to mysql:
<pre>
nano /srv/www/obs/webui/config/database.yml
#change the production section
production:
adapter: mysql
database: webui_production
username: obs
password: ************
</pre>

Populate the database
<pre>
mkdir -p /srv/www/obs/api/db/data/production
cd /srv/www/obs/api/
RAILS_ENV="production" rake db:migrate
chown lighttpd.lighttpd log/*

cd /srv/www/obs/webui/
RAILS_ENV="production" rake db:migrate
chown lighttpd.lighttpd log/*
</pre>

You can check the migration was successful verifying the “migrated” message at the end of each statement.

Setup and configure lighttpd for the API and webui

You need to setup the correct hostnames to where webui, API and repo server are going to point to

Edit /etc/lighttpd/vhosts.d/obs.conf
<pre>
$SERVER["socket"] == "192.168.60.100:443" {
ssl.engine = "enable"
ssl.pemfile = "certificate.pem"
$HTTP["host"] =~ "^cbuild" {
server.name = "cbuild.meego.com"

rails_app = "webui"
rails_root = "/srv/www/obs/webui"
rails_procs = 10
# production/development are typical values here
rails_mode = "production"

log_root = "/srv/www/obs/webui/log"

include "vhosts.d/rails.inc"
}
$HTTP["host"] =~ "^capi" {
server.name = "capi.meego.com"
rails_app = "api"
rails_root = "/srv/www/obs/api"
rails_procs = 10
# production/development are typical values here
rails_mode = "production"

log_root = "/srv/www/obs/api/log"

include "vhosts.d/rails.inc"
}

}

$HTTP["host"] =~ "download" {
# This should point to an rsync populated download repo
# server.name = "download.obs.maemo.org"
# server.document-root = "/srv/obs/repos/"

proxy.server = ( "" => ( (
"host" => "10.1.1.11",
"port" => 80
))
)
}
</pre>

To enable these vhosts, make sure to '''uncomment''' the following in the 'custom includes' section at the bottom of /etc/lighttpd/lighttpd.conf:
<pre>
nano /etc/lighttpd/lighttpd.conf
##
## custom includes like vhosts.
##
#include "conf.d/config.conf"
include_shell "cat /etc/lighttpd/vhosts.d/*.conf"
</pre>

Also need to disable IPv6 unkess it's secured
<pre>
server.use-ipv6 = "disable"
</pre>

Also, the modules "mod_magnet", "mod_rewrite" and FastCGI need to be enabled by uncommenting the corresponding lines in /etc/lighttpd/modules.conf:

<pre>
server.modules = (
"mod_access",
# "mod_alias",
# "mod_auth",
# "mod_evasive",
# "mod_redirect",
"mod_rewrite",
# "mod_setenv",
# "mod_usertrack",
)

##
## mod_magnet
##
include "conf.d/magnet.conf"

##
## FastCGI (mod_fastcgi)
##
include "conf.d/fastcgi.conf"
</pre>

You need also to configure /srv/www/obs/webui/config/environments/production.rb to point to correct server names:

<pre>
nano /srv/www/obs/webui/config/environments/production.rb
FRONTEND_HOST = "capi.meego.com"
FRONTEND_PORT = 80
EXTERNAL_FRONTEND_HOST = "capi.meego.com"
BUGZILLA_HOST = "http://bugs.moego.com/"
DOWNLOAD_URL = "http://cdownload.meego.com/"
</pre>

Do the same for /srv/www/obs/api/config/environments/production.rb. As soon your backend is not on the same machine as the api (frontend), change the following:

<pre>
nano /srv/www/obs/api/config/environments/production.rb
SOURCE_HOST = "csrc.meego.com"
SOURCE_PORT = 5352
DOWNLOAD_URL='http://cdownload.meego.com/'
</pre>

ligthttpd user and group need to be the owner of api and webui dirs (as well as log and tmp):

<pre>
chown -R lighttpd.lighttpd /srv/www/obs/{api,webui}
</pre>

Make sure TCP port 5352 is open on the firewall. Ensure lighttpd and obs ui helpers start:

<pre>
chkconfig --add memcached
chkconfig --add lighttpd
chkconfig --add obsapidelayed
chkconfig --add obswebuidelayed

rcmemcached start
rclighttpd start
rcobsapidelayed start
rcobswebuidelayed start
</pre>

rcobsapidelayed

= Preparing Worker Host =

<pre>
vgadd OBS /dev/sda4
vgcreate OBS /dev/sda4
</pre>

== Installing the Workers ==

The other hosts on the cluster are reserved to be used as workers,
where package builds are going to place.

The same openSUSE Tools repository addition must be done for each worker.
<pre>
cd /etc/zypp/repos.d/;
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key.
</pre>

<pre>
zypper in obs-worker qemu-devel mount-static bash-static
</pre>

(mount-static and bash-static are needed on the worker for rpm cross-compile to work)

For Xen workers we need a suitable initrd:
<pre>
export rootfstype="ext4"
mkinitrd -d /dev/null -m "ext4 binfmt_misc" -k vmlinuz-$(uname -r) -i initrd-$(uname -r)-obs_worker
</pre>
This will create an initrd for your kernel
<pre>
Kernel image: /boot/vmlinuz-2.6.31.12-0.2-xen
Initrd image: /boot/initrd-2.6.31.12-0.2-xen-obs_worker
</pre>
Make sure you link initrd-xen-worker -> initrd-2.6.31.12-0.2-xen-obs_worker
<pre>
ln -s /boot/initrd-$(uname -r)-obs_worker /boot/initrd-xen-worker
ln -s vmlinuz-$(uname -r) vmlinuz-xen-worker
</pre>

This assumes you have a VG dedicated to workers called "OBS"

Edit the file /etc/sysconfig/obs-worker in order to point to correct repository server.

<pre>
nano /etc/sysconfig/obs-worker
OBS_SRC_SERVER="csrc:5352"
OBS_REPO_SERVERS="cbe:5252"
OBS_VM_TYPE="xen"
OBS_VM_KERNEL="/boot/vmlinuz-xen-worker"
OBS_VM_INITRD="/boot/initrd-xen-worker"
OBS_VM_DISK_AUTOSETUP_ROOT_FILESIZE="8192"
OBS_VM_DISK_AUTOSETUP_SWAP_FILESIZE="2048"
OBS_INSTANCE_MEMORY="1024"
OBS_STORAGE_AUTOSETUP="yes"
OBS_SETUP_WORKER_PARTITIONS="use_obs_vg"
OBS_WORKER_ROOT_SIZE="8192"
OBS_WORKER_SWAP_SIZE="2048"
</pre>

Note that the OBS_VM_KERNEL/INITRD params are for kvm only. The Xen VM pulls in /usr/lib/build/xen.conf from the repo master (backend server, cbe). It does this

The obsstoragesetup will wipe the OBS VG and create root/swap LVs for each worker
<pre>
rcobsstoragesetup start
</pre>

Output:
<pre>
mdadm: No arrays found in config file or automatically
Waiting for udev to settle...
Scanning for LVM volume groups...
Reading all physical volumes. This may take a while...
Found volume group "OBS" using metadata type lvm2
Activating LVM volume groups...
0 logical volume(s) in volume group "OBS" now active
done
Logical volume "worker_root_1" created
Logical volume "worker_swap_1" created
Logical volume "worker_root_2" created
Logical volume "worker_swap_2" created
Logical volume "worker_root_3" created
Logical volume "worker_swap_3" created
Logical volume "worker_root_4" created
Logical volume "worker_swap_4" created
Logical volume "worker_root_5" created
Logical volume "worker_swap_5" created
Logical volume "worker_root_6" created
Logical volume "worker_swap_6" created
Logical volume "worker_root_7" created
Logical volume "worker_swap_7" created
Logical volume "worker_root_8" created
Logical volume "worker_swap_8" created
Logical volume "worker_root_9" created
Logical volume "worker_swap_9" created
Logical volume "worker_root_10" created
Logical volume "worker_swap_10" created
Logical volume "worker_root_11" created
Logical volume "worker_swap_11" created
Logical volume "worker_root_12" created
Logical volume "worker_swap_12" created
Logical volume "worker_root_13" created
Logical volume "worker_swap_13" created
Logical volume "worker_root_14" created
Logical volume "worker_swap_14" created
Logical volume "worker_root_15" created
Logical volume "worker_swap_15" created
Logical volume "worker_root_16" created
Logical volume "worker_swap_16" created
Logical volume "cache" created
mke2fs 1.41.9 (22-Aug-2009)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
3555328 inodes, 14201856 blocks
710092 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
434 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
Looking for OBS Server LVM Volume
Setup local storage
Looking for OBS Worker Cache LVM Volume
Setting up OBS Workers according to LVM Volumes
Found XEN virtualization
done
</pre>

Test?
<pre>
xm create -c /var/run/obs/worker/8/build/xen.conf name=build:root8 memory=40 disk=phy:/dev/mapper/OBS-worker_root8,hda1,w disk=phy:/dev/mapper/OBS-worker_swap8,hda2,w extra="init=/.build/initscript_qemu_vm panic=1 console=ttyS0
</pre>

Start the worker service:

<pre>
chkconfig --add obsworker
rcobsworker start
</pre>

[[Category:Build Infrastructure]]

ARM/N900/DeveloperEdition/BOSS

2011-09-16T10:36:45Z

Lbt: link to boss processes

= Automating DE processes =

The DE workflow is (roughly) :

# Developer branches package from P:DE:Trunk
# Developer submits package to P:DE:Trunk:Testing
# Boss runs checks on package
# Boss creates branch of P:DE:Trunk
# Boss submits package(s) to new branch

Later:
# Boss rejects package
or
# Boss notifies maintainers that checks passed

Then:
# Package is accepted to P:DE:Trunk by a maintainer
# bz is updated

The actual processes can be seen in the [http://autodoc.meego.com/boss/ autodoc boss] area until the OBS plugin is available.

== BOSS Checks ==

BOSS runs the following checks (and stops as soon as one fails)

* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_submitter_is_maintainer check_submitter_is_maintainer]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_built_at_source check_package_built_at_source]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_is_complete check_package_is_complete]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_spec check_spec]

To be done:
* changelog in proper format (present, syntactically correct)
* changelog has entry for this commit
* last changelog version matches spec file version
* tarball not changed within same package version
* executing spectacle doesn't change anything if .yaml present
* yaml must be present if spec is based on .yaml
* content of source tar ball should not change without its changing.
* check that old tarball has been removed if new one is introduced
* SR not empty (define empty?)
** No changes at all. At times people are trying rebuild packages by submitting empty change. (Sage)

== Process Testing ==

The Project:DE:BOSS:Trunk and Project:DE:BOSS:Trunk:Testing areas are used
to test the processes.

The processes to be tested are deployed to /srv/BOSS/processes/Project/DE/BOSS/Trunk/

=== DE_Trunk_SRs ===

To deploy:
* set irc_channel and final_project
* Copy to Project:DE:Trunk
* symlink: SRCSRV_REQUEST_CREATE and SRCSRV_REQUEST_STATECHANGE in Trunk and Trunk:Testing

cd Trunk
ln -s DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE
cd Testing
ln -s ../DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE

= Wish List =
List of wishes for the BOSS. Please write your nick in parentheses to the end so people know who to contact if they have more questions.

# e-mail notice of SR's especially when SR is rejected
# Check if package upstream has changed and do a report of the packages that needs to be rebased on top of the new upstream. Currently this check is done manually with https://gitorious.org/meego-developer-edition-for-n900/upstreamchecker/ script by N900 RE. (Sage)
# In case package fails to connection failure start rebuild automatically after time X. (Sage)
# Add architecture and repository to build error message. (Sage)
#* ''14:06.33 < boss> Build failure in Project:DE:Trunk:Testing: meegotouch-theme More details soon''
# Update patterns based on package-groups package. (Sage)
#* We should have package-group package in the Project:DE:Trunk[:Testing] project that contains the CE package groups. This information should be when ever repository is published pushed to the repository with ''osc meta pattern'' or similar command.
# When submit supersedes another one the handling of the old submit request should be stopped. (Sage)

= Bug Lifecycle =

The DE bug lifecycle is (roughly):

* 'open' -> FIXED -> RELEASED -> VERIFIED

# Developer starts working on the bug/feature and sets to ASSIGNED
# Developer submits package to P:DE:Trunk:Testing and documents changelog and/or commit properly (fixes bmc #1234;) - if changelog doesn't have a valid bug/feature reference the request MUST fail but we do not document the failure in the bug report
# Bugzilla records the commit message with link to the repo and timestamp
# Package is accepted to P:DE:Trunk (manual for now....)
# Bugzilla records acceptance with comment and bug status is automatically changed to FIXED if bug is still in OPEN status (New, assigned, needinfo, reopened, waiting)
# If bug is already in FIXED resolution, only acceptance comment is written in the bug report
# If bug is not in OPEN status (New, assigned, needinfo, reopened, waiting) or resolved FIXED (eg. duplicate, wontfix, etc... or verified or closed) the request MUST fail but we do not document the failure in the bug report

Later:
# Release is triggered
# Bug status is automatically changed to RELEASED, a comment with a link to the image is created and the Target Build value is set to match appropriate release

Later still:
# QA review the release and for each bug marked RELEASED, either change to VERIFIED or re-open

Note that eventually, both FIXED and RELEASED statuses would require specific credentials to be modified.

= Image Building =

CE has a private installation of IMG on host img in the internal meego.com network. Speak to David Greaves / lbt for access details.
Login is via meego.com credentials.

ARM/N900/DeveloperEdition/BOSS

2011-09-16T09:09:36Z

Lbt: /* Bug Lifecycle */

= Automating DE processes =

The DE workflow is (roughly) :

# Developer branches package from P:DE:Trunk
# Developer submits package to P:DE:Trunk:Testing
# Boss runs checks on package
# Boss creates branch of P:DE:Trunk
# Boss submits package(s) to new branch

Later:
# Boss rejects package
or
# Boss notifies maintainers that checks passed

Then:
# Package is accepted to P:DE:Trunk by a maintainer
# bz is updated

== BOSS Checks ==

BOSS runs the following checks (and stops as soon as one fails)

* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_submitter_is_maintainer check_submitter_is_maintainer]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_built_at_source check_package_built_at_source]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_package_is_complete check_package_is_complete]
* [http://wiki.meego.com/Release_Infrastructure/BOSS/Participants#check_spec check_spec]

To be done:
* changelog in proper format (present, syntactically correct)
* changelog has entry for this commit
* last changelog version matches spec file version
* tarball not changed within same package version
* executing spectacle doesn't change anything if .yaml present
* yaml must be present if spec is based on .yaml
* content of source tar ball should not change without its changing.
* check that old tarball has been removed if new one is introduced
* SR not empty (define empty?)
** No changes at all. At times people are trying rebuild packages by submitting empty change. (Sage)

== Process Testing ==

The Project:DE:BOSS:Trunk and Project:DE:BOSS:Trunk:Testing areas are used
to test the processes.

The processes to be tested are deployed to /srv/BOSS/processes/Project/DE/BOSS/Trunk/

=== DE_Trunk_SRs ===

To deploy:
* set irc_channel and final_project
* Copy to Project:DE:Trunk
* symlink: SRCSRV_REQUEST_CREATE and SRCSRV_REQUEST_STATECHANGE in Trunk and Trunk:Testing

cd Trunk
ln -s DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE
cd Testing
ln -s ../DE_Trunk_SRs SRCSRV_REQUEST_CREATE SRCSRV_REQUEST_STATECHANGE

= Wish List =
List of wishes for the BOSS. Please write your nick in parentheses to the end so people know who to contact if they have more questions.

# e-mail notice of SR's especially when SR is rejected
# Check if package upstream has changed and do a report of the packages that needs to be rebased on top of the new upstream. Currently this check is done manually with https://gitorious.org/meego-developer-edition-for-n900/upstreamchecker/ script by N900 RE. (Sage)
# In case package fails to connection failure start rebuild automatically after time X. (Sage)
# Add architecture and repository to build error message. (Sage)
#* ''14:06.33 < boss> Build failure in Project:DE:Trunk:Testing: meegotouch-theme More details soon''
# Update patterns based on package-groups package. (Sage)
#* We should have package-group package in the Project:DE:Trunk[:Testing] project that contains the CE package groups. This information should be when ever repository is published pushed to the repository with ''osc meta pattern'' or similar command.
# When submit supersedes another one the handling of the old submit request should be stopped. (Sage)

= Bug Lifecycle =

The DE bug lifecycle is (roughly):

* 'open' -> FIXED -> RELEASED -> VERIFIED

# Developer starts working on the bug/feature and sets to ASSIGNED
# Developer submits package to P:DE:Trunk:Testing and documents changelog and/or commit properly (fixes bmc #1234;) - if changelog doesn't have a valid bug/feature reference the request MUST fail but we do not document the failure in the bug report
# Bugzilla records the commit message with link to the repo and timestamp
# Package is accepted to P:DE:Trunk (manual for now....)
# Bugzilla records acceptance with comment and bug status is automatically changed to FIXED if bug is still in OPEN status (New, assigned, needinfo, reopened, waiting)
# If bug is already in FIXED resolution, only acceptance comment is written in the bug report
# If bug is not in OPEN status (New, assigned, needinfo, reopened, waiting) or resolved FIXED (eg. duplicate, wontfix, etc... or verified or closed) the request MUST fail but we do not document the failure in the bug report

Later:
# Release is triggered
# Bug status is automatically changed to RELEASED, a comment with a link to the image is created and the Target Build value is set to match appropriate release

Later still:
# QA review the release and for each bug marked RELEASED, either change to VERIFIED or re-open

Note that eventually, both FIXED and RELEASED statuses would require specific credentials to be modified.

= Image Building =

CE has a private installation of IMG on host img in the internal meego.com network. Speak to David Greaves / lbt for access details.
Login is via meego.com credentials.

Web infrastructure/Policy

2011-09-13T12:48:16Z

Lbt: /* SSH Keys */

The policies on this page are aimed at administrative use of the system.

== Shell Access ==

There are no firm guidelines for access to infrastructure systems. We operate a validated trust approach and require consensus from a number of members of the IT team to grant access.

There are multiple levels of trust:
* No shell access : This is the norm. Very few people have shell accounts to any systems. There must be an '''extremely''' compelling reason to provide shell access.
* Shell user : When granted, access will normally be at this level.
* VM root : In exceptional cases for highly trusted people with significant sysadmin knowledge we will provide root access to a virtual machine. Having root access means you take direct responsibility for the security of the machine.
* Superadmin : The superadmin team is the core IT team - invitation only :) These users are given access to IT Private bugs.

== SSH Keys ==
Access is provided via ssh keys and not passwords.

Users are expected to take security of ssh private keys very seriously.

* Private keys are '''strictly private to an individual'''. If they are disclosed to '''anyone''', under '''any''' circumstances please let IT know '''AT ONCE'''. It's easy to change them.
* You should have a dedicated ssh key for use for meego.com access
* ssh keys '''must''' be protected by a strong pass phrase (ssh-agent and ssh-add makes this un-noticeable)
* ssh keys use '''must''' be confirmed before every use (ssh-add -c). This is usually a simple pop-up from ssh-askpass
* ssh private keys ''must never'' be checked into a git repo which could compromise them
* ssh public keys ''must never'' be obtained from a git repo which could lead to loose access controls
* Onward access from the jump host is by tunnel - no ssh or agent-forwarding
* The comment on your public key should be descriptive, such as an email address, as much as your pet name is funny, its not to the point.
* If you use a mobile device(phone, tablet, etc) please consider using a separate key for that device and not accessing critical infrastructure with that mobile device directly.

If you have any problems with implementing any of these rules then please talk to one of the IT team - they want you to get it right and will do their best to help you. (And it may help to know that they too will have struggled with ssh once upon a time!)

If you think you need an exception to a rule (eg group access, password access, unattended/cron access etc) then we'll be glad to help solve the problem you face. It's much better to get help to implement a secure solution than to "just" do something to make it work.

The following snippet can be usefully added to your .ssh/config (note you need to change the <USER> to your meego.com shell account name)
Host *.meego.com
User <USER>
IdentityFile ~/.ssh/id_rsa_meego
ServerAliveInterval 60
ForwardAgent no

Host access.meego.com
ProxyCommand none

Host *.in.meego.com
ProxyCommand ssh -q access.meego.com netcat %h 22

Notes:
* The strong passphrase means that if your computer is stolen then the attacker won't be able to use your key to access your account.
* The askpass confirmation means that if a man-in-the-middle attack is attempted then your agent should prompt you when you didn't just establish a connection. If this happens then refuse the request.
* Agent forwarding means a compromised jump host would be able to spoof an onward connection as your account

== Bugs ==
(repeated from [[Meego_IT#Contact_points|IT team contact section]]).

The MeeGo IT team have 2 bug products: MeeGo IT Private and MeeGo IT. We use the private one for bugs which discuss secure or confidential information; sometimes we'll make a public bug private or a private bug public. The criteria are not well defined but in general we don't discuss specifics of our installation in public. Whilst we do not operate "security through obscurity", nor do we claim to be infallible - limiting information means that when we make errors, the information is less likely to leak and the window that we have to rectify the error without discovery should be larger.

Web infrastructure/Policy

2011-09-13T08:16:31Z

Lbt: Clarify IT Private bug policy - see https://bugs.meego.com/show_bug.cgi?id=22699 for debate.

The policies on this page are aimed at administrative use of the system.

== Shell Access ==

There are no firm guidelines for access to infrastructure systems. We operate a validated trust approach and require consensus from a number of members of the IT team to grant access.

There are multiple levels of trust:
* No shell access : This is the norm. Very few people have shell accounts to any systems. There must be an '''extremely''' compelling reason to provide shell access.
* Shell user : When granted, access will normally be at this level.
* VM root : In exceptional cases for highly trusted people with significant sysadmin knowledge we will provide root access to a virtual machine. Having root access means you take direct responsibility for the security of the machine.
* Superadmin : The superadmin team is the core IT team - invitation only :) These users are given access to IT Private bugs.

== SSH Keys ==
Access is provided via ssh keys and not passwords.

Users are expected to take security of ssh private keys very seriously.

* Private keys are '''strictly private to an individual'''. If they are disclosed to '''anyone''', under '''any''' circumstances please let IT know '''AT ONCE'''. It's easy to change them.
* You should have a dedicated ssh key for use for meego.com access
* ssh keys '''must''' be protected by a strong pass phrase (ssh-agent and ssh-add makes this un-noticeable)
* ssh keys use '''must''' be confirmed before every use (ssh-add -c). This is usually a simple pop-up from ssh-askpass
* ssh private keys ''must never'' be checked into a git repo which could compromise them
* ssh public keys ''must never'' be obtained from a git repo which could lead to loose access controls
* Onward access from the jump host is by tunnel - no ssh or agent-forwarding
* The comment on your public key should be descriptive, such as an email address, as much as your pet name is funny, its not to the point.
* If you use a mobile device(phone, tablet, etc) please consider using a separate key for that device and not accessing critical infrastructure with that mobile device directly.

If you have any problems with implementing any of these rules then please talk to one of the IT team - they want you to get it right and will do their best to help you. (And it may help to know that they too will have struggled with ssh once upon a time!)

If you think you need an exception to a rule (eg group access, password access, unattended/cron access etc) then we'll be glad to help solve the problem you face. It's much better to get help to implement a secure solution than to "just" do something to make it work.

The following snippet can be usefully added to your .ssh/config
Host *.meego.com
User <USER>
IdentityFile ~/.ssh/id_rsa_meego
ServerAliveInterval 60
ForwardAgent no

Host access.meego.com
ProxyCommand none

Host *.in.meego.com
ProxyCommand ssh -q access.meego.com netcat %h 22

Notes:
* The strong passphrase means that if your computer is stolen then the attacker won't be able to use your key to access your account.
* The askpass confirmation means that if a man-in-the-middle attack is attempted then your agent should prompt you when you didn't just establish a connection. If this happens then refuse the request.
* Agent forwarding means a compromised jump host would be able to spoof an onward connection as your account

== Bugs ==
(repeated from [[Meego_IT#Contact_points|IT team contact section]]).

The MeeGo IT team have 2 bug products: MeeGo IT Private and MeeGo IT. We use the private one for bugs which discuss secure or confidential information; sometimes we'll make a public bug private or a private bug public. The criteria are not well defined but in general we don't discuss specifics of our installation in public. Whilst we do not operate "security through obscurity", nor do we claim to be infallible - limiting information means that when we make errors, the information is less likely to leak and the window that we have to rectify the error without discovery should be larger.

Release Infrastructure/BOSS/Installation

2011-09-12T12:34:30Z

Lbt: Remove confusing comment about a related upstream package - this page doesn't talk about running ruote and it's not a sensible thing to do.

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to change. In particular the
repositories are not yet finalised (eg installing from :RC).

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 or OpenSuse 11.4 installation.

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Release Candidate : https://build.pub.meego.com/project/monitor?project=Project:MINT:RC (~June 2011)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now:

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register check
skynet register notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process instance ==

With a participant ready to do some work, we're ready to launch a process instance:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

TODO

= BOSS OBS Plugin =
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

== Installation ==
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

== Configuration ==

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/IMG

2011-09-11T20:49:42Z

Lbt: Update repos

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.4.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.4/meego-tools.repo
zypper ref

Then install the core IMG package and the amqp listener:

Worker:

zypper in img-boss

Web UI:
zypper in img-web

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Whether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Release Infrastructure/BOSS/Installation

2011-09-02T14:04:45Z

Lbt: /* Overview */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to change. In particular the
repositories are not yet finalised (eg installing from :RC).

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 or OpenSuse 11.4 installation.

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing (Current)

Release Candidate : https://build.pub.meego.com/project/monitor?project=Project:MINT:RC (~June 2011)

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo MINT
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== SkyNET installation for OpenSuse 11.4 ===

In order to be able to work with BOSS, please install SkyNET now:

zypper in boss-skynet

from the repository

https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar https://build.pub.meego.com/project/monitor?project=Project:MINT:RC
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

== BOSS OBS Plugin ==
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

=== Installation ===
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/Testing/openSUSE_11.4/Project:MINT:Testing.repo
zypper ref
zypper in boss-obs-plugin

=== Configuration ===

'''You *MUST* have a participant registered as "obs_event" (provided by the robogrator launcher package https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator ) running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register -n check
skynet register -n notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process ==

With a participant ready to do some work, we're ready to launch a process:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Meego IT

2011-08-19T19:38:09Z

Lbt: cross-link to policy/processes

The MeeGo IT team takes care of the infrastructure in meego.com located at Oregon State University hosting center. It's devoted to offer the best service to the Community and to the other teams so that everyone can do his/her job without worrying nor caring about the underlying layers of the stack.

== Team ==

* '''[http://meego.com/users/stezz Stefano Mosconi]''' is the MeeGo IT manager. He is responsible for the MeeGo IT team, making sure that we don't break too much stuff, the services are up, running and fast and people using them are happy.
* '''[http://meego.com/users/mshaver Michael Shaver]''' is the MeeGo webmaster, he takes care of the websites content down to the spelling of the words and to the colors of the pages. He installs weird Drupal modules in his spare time and tests them to see if they work.
* '''[http://meego.com/users/lbt David Greaves]''' is the guy who reminds us to clean up after we made some mess, document the stuff and in his spare time takes care of the community OBS infra. He does also IT architecure, dances tango and speaks with a funny british accent.
* '''[http://meego.com/users/xfade Niels Breet]''' is the guy who knows a lot of stuff about a lot of IT things but he doesn't want to be called IT engineer, sysadmin nor architect, X-Fade is enough. He's also the maemo.org webmaster and brings to Meego IT a lot of his expertise in community related matters. Taking care of community OBS and a lot of the other infrastructure.
* '''[http://meego.com/users/adamgretzinger Adam Gretzinger]''' is the Meego Sysadmin, he has the keys to our realm and does a zillion of things, from plugging/unplugging cables, to tagging servers, to installing machines, to configuring web servers, to debugging network problems... etc.
* '''[http://meego.com/users/maxivano Maxim Ivanov]''' is the other Meego Sysadmin, he also does a lot of stuff, architecture, installation, testing, monitoring, debugging but can't unplug cables as he's far away from the buttons' room.
* '''[http://meego.com/users/pierce Dean Pierce]''' is the Meego IT security engineer. He reminds us that we can trust no one, not even ourselves.

== Supported servers and services ==

You can find a complete list of services in [[Web_infrastructure |this separate page]] with some details about them

== Policy and Processes ==

You can find some of the [[Web_infrastructure/Processes | processes here]] and some [[Web_infrastructure/Policy|IT policy here]]

== Contact points ==

We have a [mailto:meego-it@lists.meego.com mailing list] where you can talk to the team in case you need us to know something or you want to know something. You can subscribe there if you want to have an idea of what we discuss in our weekly meetings.

Also in case of problems with the websites or the IT in general you can enter a bug in [https://bugs.meego.com/enter_bug.cgi?classification=MeeGo%20Community%20Infrastructure any of our components in bugs.meego.com]

The MeeGo IT team have 2 bug products: MeeGo IT Private and MeeGo IT. We use the private one for bugs which discuss secure or confidential information; sometimes we'll make a public bug private or a private bug public. The criteria are not well defined but in general we don't discuss specifics of our installation in public. Whilst we do not operate "security through obscurity", nor do we claim to be infallible - limiting information means that when we make errors, the information is less likely to leak and the window that we have to rectify the error without discovery should be larger.

Web infrastructure/Policy

2011-08-19T19:36:47Z

Lbt: Policy page .. add some light Bug policy

The policies on this page are aimed at administrative use of the system.

== Shell Access ==

There are no firm guidelines for access to infrastructure systems. We operate a validated trust approach and require consensus from a number of members of the IT team to grant access.

There are multiple levels of trust:
* No shell access : This is the norm. Very few people have shell accounts to any systems. There must be an '''extremely''' compelling reason to provide shell access.
* Shell user : When granted, access will normally be at this level. These users are given access to IT Private bugs if requested.
* VM root : In exceptional cases for highly trusted people with significant sysadmin knowledge we will provide root access to a virtual machine. Having root access means you take direct responsibility for the security of the machine.
* Superadmin : The superadmin team is the core IT team - invitation only :)

== SSH Keys ==
Access is provided via ssh keys and not passwords.

Users are expected to take security of ssh private keys very seriously.

* Private keys are '''strictly private to an individual'''. If they are disclosed to '''anyone''', under '''any''' circumstances please let IT know '''AT ONCE'''. It's easy to change them.
* You should have a dedicated ssh key for use for meego.com access
* ssh keys '''must''' be protected by a strong pass phrase (ssh-agent and ssh-add makes this un-noticeable)
* ssh keys use '''must''' be confirmed before every use (ssh-add -c). This is usually a simple pop-up from ssh-askpass
* ssh private keys ''must never'' be checked into a git repo which could compromise them
* ssh public keys ''must never'' be obtained from a git repo which could lead to loose access controls
* Onward access from the jump host is by tunnel - no ssh or agent-forwarding
* The comment on your public key should be descriptive, such as an email address, as much as your pet name is funny, its not to the point.
* If you use a mobile device(phone, tablet, etc) please consider using a separate key for that device and not accessing critical infrastructure with that mobile device directly.

If you have any problems with implementing any of these rules then please talk to one of the IT team - they want you to get it right and will do their best to help you. (And it may help to know that they too will have struggled with ssh once upon a time!)

If you think you need an exception to a rule (eg group access, password access, unattended/cron access etc) then we'll be glad to help solve the problem you face. It's much better to get help to implement a secure solution than to "just" do something to make it work.

The following snippet can be usefully added to your .ssh/config
Host *.meego.com
User <USER>
IdentityFile ~/.ssh/id_rsa_meego
ServerAliveInterval 60
ForwardAgent no

Host access.meego.com
ProxyCommand none

Host *.in.meego.com
ProxyCommand ssh -q access.meego.com netcat %h 22

Notes:
* The strong passphrase means that if your computer is stolen then the attacker won't be able to use your key to access your account.
* The askpass confirmation means that if a man-in-the-middle attack is attempted then your agent should prompt you when you didn't just establish a connection. If this happens then refuse the request.
* Agent forwarding means a compromised jump host would be able to spoof an onward connection as your account

== Bugs ==
(repeated from [[Meego_IT#Contact_points|IT team contact section]]).

The MeeGo IT team have 2 bug products: MeeGo IT Private and MeeGo IT. We use the private one for bugs which discuss secure or confidential information; sometimes we'll make a public bug private or a private bug public. The criteria are not well defined but in general we don't discuss specifics of our installation in public. Whilst we do not operate "security through obscurity", nor do we claim to be infallible - limiting information means that when we make errors, the information is less likely to leak and the window that we have to rectify the error without discovery should be larger.

Web infrastructure/Policy

2011-08-19T09:52:30Z

Lbt: /* Shell Access */

The policies on this page are aimed at administrative use of the system.

== Shell Access ==

There are no firm guidelines for access to infrastructure systems. We operate a validated trust approach and require consensus from a number of members of the IT team to grant access.

There are multiple levels of trust:
* No shell access : This is the norm. Very few people have shell accounts to any systems. There must be an '''extremely''' compelling reason to provide shell access.
* Shell user : When granted, access will normally be at this level. These users are given access to IT Private bugs if requested.
* VM root : In exceptional cases for highly trusted people with significant sysadmin knowledge we will provide root access to a virtual machine. Having root access means you take direct responsibility for the security of the machine.
* Superadmin : The superadmin team is the core IT team - invitation only :)

== SSH Keys ==
Access is provided via ssh keys and not passwords.

Users are expected to take security of ssh private keys very seriously.

* Private keys are '''strictly private to an individual'''. If they are disclosed to '''anyone''', under '''any''' circumstances please let IT know '''AT ONCE'''. It's easy to change them.
* You should have a dedicated ssh key for use for meego.com access
* ssh keys '''must''' be protected by a strong pass phrase (ssh-agent and ssh-add makes this un-noticeable)
* ssh keys use '''must''' be confirmed before every use (ssh-add -c). This is usually a simple pop-up from ssh-askpass
* Onward access from the jump host is by tunnel - no ssh or agent-forwarding
* The comment on your public key should be descriptive, such as an email address, as much as your pet name is funny, its not to the point.
* If you use a mobile device(phone, tablet, etc) please consider using a separate key for that device and not accessing critical infrastructure with that mobile device directly.
* You'll be told the

If you have any problems with implementing any of these rules then please talk to one of the IT team - they want you to get it right and will do their best to help you. (And it may help to know that they too will have struggled with ssh once upon a time!)

If you think you need an exception to a rule (eg group access, password access, unattended/cron access etc) then we'll be glad to help solve the problem you face. It's much better to get help to implement a secure solution than to "just" do something to make it work.

The following snippet can be usefully added to your .ssh/config
Host *.meego.com
User <USER>
IdentityFile ~/.ssh/id_rsa_meego
ServerAliveInterval 60
ForwardAgent no

Host access.meego.com
ProxyCommand none

Host *.in.meego.com
ProxyCommand ssh -q access.meego.com netcat %h 22

Notes:
* The strong passphrase means that if your computer is stolen then the attacker won't be able to use your key to access your account.
* The askpass confirmation means that if a man-in-the-middle attack is attempted then your agent should prompt you when you didn't just establish a connection. If this happens then refuse the request.
* Agent forwarding means a compromised jump host would be able to spoof an onward connection as your account

Web infrastructure/Policy

2011-08-17T22:51:17Z

Lbt: Add some security policy

Web infrastructure

2011-08-17T21:49:07Z

Lbt: /* Request new services */

== OSUOSL Migration ==
A plan is being drawn up for the move of all MeeGo services to OSUOSL for hosting. Look for details of the plan: [[Web_infrastructure/Planning_Security_Review|Planning and Security Review]]

== Request new services ==
[[Web_infrastructure/Processes|Here]] you can find a description about how to request new services or change requests to existing services.

== Policy ==
We have some [[Web_infrastructure/Policy|policies]] related to services.

== MeeGo Services ==
Like most open source projects, MeeGo has a vast number of services that keep the community functioning on a day to day basis. As the number of services grows and the number of people involved in administering these services grows, it's important to closely track. Below is a list of the various services with links to details about these services.

{| border="1" cellpadding="5"
!Service
!Platform
!Current Host
!Status
!Owners
!Details
|-
|http://meego.com
|Drupal 6.x
|OSU
|active
|[http://meego.com/users/mshaver Mike Shaver] [http://meego.com/users/qgil Quim Gil]
|[[Web_infrastructure/Meego.com|details]]
|-
|http://help.meego.com
|Drupal 6.x
|OSU
|active
|[http://meego.com/users/mshaver Mike Shaver] [http://meego.com/users/noel Noel Arnold]
|details
|-
|[[Main Page|http://wiki.meego.com]]
|MediaWiki 1.16.2
|OSU
|active
|??
|[[Web_infrastructure/Help.meego.com|details]]
|-
|http://bugs.meego.com
|Bugzilla 3.6x
|OSU
|active
|[http://meego.com/users/ericlr Eric Le Roux] [http://meego.com/users/may May Xie]
|[[Quality/Error_Management|details]]
|-
|http://apidocs.meego.com
|Static HTML/files
|OSU
|active
|[http://meego.com/users/bspencer Bob Spencer]
|[[SDK|details]]
|-
|http://lists.meego.com
|Mailman
|OSU
|active
|[http://meego.com/users/agretzinger Adam Gretzinger]
|[[Mailing list guidelines|details]]
|-
|http://forum.meego.com
|vBulletin
|external community hosted
|active
|[http://meego.com/users/reggie Reggie Suplido]
|[[Web_infrastructure/Forum|details]]
|-
|http://planet.meego.com
|Midgard
|OSU
|active
|[http://meego.com/users/bergie Henri Bergius]
|[[Web_infrastructure/Planet.meego.com|details]]
|-
|http://crashdb.meego.com
|Custom PHP
|??
|active
|
|details
|-
|developer.meego.com
|Drupal 6.x
|Offsite/moving to OSU or Softlayer
|In Review
|[http://meego.com/users/macron Ronan Mac Laverty/macron,maclaver(IRC)]
|[[Application developer site|details]]
|-
|[http://download.meego.com download.meego.com]
|Generic Web Server, Serves Intermediate Builds
|OSU
|active
|[http://meego.com/users/agretzinger Adam Gretzinger]
|details
|-
|Metrics Website
|??
|??
|proposed
|
|details
|-
|Conference Website http://conference2010.meego.com/
|Drupal 6.x
|none
|active/in process
|
|details
|-
|Meeting IRC Bot
|[http://wiki.debian.org/MeetBot MeetBot]
|Stskeeps' shell in DK
|active/should migrate to central location
|
|details
|-
|povbot IRC bot (#meego) + web logs
|Supybot + [http://mg.pov.lt/irclog2html/ irclog2html]
|mgedmin
|active/should migrate to central location
|
|details
|-
|merbot IRC bot (#meego-arm etc) + web logs
|irssi
|Stskeeps' shell in DK
|active/should migrate to central location/merge with meetbot
|
|details
|-
| MeeBot IRC bot
| supybot + [http://mg.pov.lt/irclog2html/ irclog2html]
| OSUOSL
| proposed, centralisation of meeting bot and logging
|
| [[Web_infrastructure/IRC|details]]
|-
|build.pub.meego.com api.pub.meego.com
|Community OBS
|OSU servers
|beta setup on spare hardware, currently setting up servers at OSU.
|
|details
|-
|Community Apps Website
|??
|??
|proposed
|
|details
|-
|http://meego.gitorious.org
|Gitorious
|http://meego.gitorious.org
|active
|
|[[Web_infrastructure/Meego.gitorious.org|details]]
|-
|http://meego.transifex.org
|Transifex
|http://meego.transifex.org
|active/in process
|
|details
|-
|http://mxr.meego.com
|??
|http://mxr.moego.org
|active
|
|details
|-
|http://tl.meego.com
|http://www.teamst.org/
|OSU
|active
|[http://meego.com/users/jennycao Jenny Cao]
|details
|-
|http://hg.meego.com
|http://mercurial.selenic.com/
|Bitbucket
|none
|
|details
|-
|Patchwork
|http://ozlabs.org/~jk/projects/patchwork/
|none
|proposed
|
|[http://lists.meego.com/pipermail/meego-dev/2010-August/005169.html details]
|-
|ots.meego.com
|http://gitorious.org/qa-tools/ots
|OSU
|Decomissioned
|
|[[Quality/QA-tools/OTS|details]]
|}

Meego IT

2011-08-17T20:41:16Z

Lbt: Add note on new IT bug policy

The MeeGo IT team takes care of the infrastructure in meego.com located at Oregon State University hosting center. It's devoted to offer the best service to the Community and to the other teams so that everyone can do his/her job without worrying nor caring about the underlying layers of the stack.

== Team ==

* '''[http://meego.com/users/stezz Stefano Mosconi]''' is the MeeGo IT manager. He is responsible for the MeeGo IT team, making sure that we don't break too much stuff, the services are up, running and fast and people using them are happy.
* '''[http://meego.com/users/mshaver Michael Shaver]''' is the MeeGo webmaster, he takes care of the websites content down to the spelling of the words and to the colors of the pages. He installs weird Drupal modules in his spare time and tests them to see if they work.
* '''[http://meego.com/users/lbt David Greaves]''' is the guy who reminds us to clean up after we made some mess, document the stuff and in his spare time takes care of the community OBS infra. He does also IT architecure, dances tango and speaks with a funny british accent.
* '''[http://meego.com/users/xfade Niels Breet]''' is the guy who knows a lot of stuff about a lot of IT things but he doesn't want to be called IT engineer, sysadmin nor architect, X-Fade is enough. He's also the maemo.org webmaster and brings to Meego IT a lot of his expertise in community related matters. Taking care of community OBS and a lot of the other infrastructure.
* '''[http://meego.com/users/adamgretzinger Adam Gretzinger]''' is the Meego Sysadmin, he has the keys to our realm and does a zillion of things, from plugging/unplugging cables, to tagging servers, to installing machines, to configuring web servers, to debugging network problems... etc.
* '''[http://meego.com/users/maxivano Maxim Ivanov]''' is the other Meego Sysadmin, he also does a lot of stuff, architecture, installation, testing, monitoring, debugging but can't unplug cables as he's far away from the buttons' room.
* '''[http://meego.com/users/pierce Dean Pierce]''' is the Meego IT security engineer. He reminds us that we can trust no one, not even ourselves.

== Supported servers and services ==

You can find a complete list of services in [[Web_infrastructure |this separate page]] with some details about them

== Processes ==

You can find some of the processes [[Web_infrastructure/Processes | here]].

== Contact points ==

We have a [mailto:meego-it@lists.meego.com mailing list] where you can talk to the team in case you need us to know something or you want to know something. You can subscribe there if you want to have an idea of what we discuss in our weekly meetings.

Also in case of problems with the websites or the IT in general you can enter a bug in [https://bugs.meego.com/enter_bug.cgi?classification=MeeGo%20Community%20Infrastructure any of our components in bugs.meego.com]

The MeeGo IT team have 2 bug products: MeeGo IT Private and MeeGo IT. We use the private one for bugs which discuss secure or confidential information; sometimes we'll make a public bug private or a private bug public. The criteria are not well defined but in general we don't discuss specifics of our installation in public. Whilst we do not operate "security through obscurity", nor do we claim to be infallible - limiting information means that when we make errors, the information is less likely to leak and the window that we have to rectify the error without discovery should be larger.

Release Infrastructure

2011-08-05T11:25:13Z

Lbt: Add team information

= MINT =
MINT (MeeGo Integration) is a set of integrated tools, that was created with the goal of creating a CI system around OBS for enabling vendors to build meego products.
It provides a business process automation system, interacting with many different systems such as bugzilla and OBS to facilitate product building.

== Overview ==
* MINT (MeeGo Integration) is a platform composed of integrated tools, that was created with the goal of creating a CI system around OBS for enabling vendors to build meego products.
* MINT provides a business process automation system, interacting with many different systems such as bugzilla and OBS to facilitate product building.
* MINT is a platform not a product, delivering a base to build a CI infrastructure to suit each customer's need.
* MINT out-of-the-box provides an end-to-end build and release infrastructure for making linux based products.
* Different deployments may have additional systems and MINT is designed to allow components to be replaced or added to fulfill different needs.

=== Features ===
* Covers all the activities involved in continuous integration
** Building through integration with OBS (http://en.opensuse.org//Portal:Build_Service)
** publishing and notifictaions
** Creation of installable/flashable images
** On-device Testing, through integration with OTS (http://wiki.meego.com/Quality/QA-tools/OTS/About)
** Error and feature management, through bugzilla integration (www.bugzilla.org)
** Reporting on releases progress
* Distributed architecture means no single point of failure and better resource management
* Massive parallelism, all the components of the system run multiple parallel jobs serving several parallel workflows
* Parallel workflows, means you can run many different workflows servicing multiple products
* Simple workflow description language
* Reliable
** based on mature, production level opensource projects
** Reliable messaging using the AMQP standard, means no events will be lost
** Workflow management based on ruote, an enterprise grade workflow engine
* Robust
** Live migration of workflows, means you don't need to bring your system down for changing your workflow. Simply deploy your workflow and it gets loaded without disruption to currently running jobs
* Extensible. The system has a minimal core and all operations and functions are implemented as plugins ([[/BOSS/Participants|Participants]])

== Architecture ==
how does it all fit together

== Deployment ==
=== Joining the Team ===
MINT is a set of open source tools and if you use it you should consider yourself a part of the team.

The primary communication tools are irc (#meego) and email (http://lists.meego.com/listinfo/meego-distribution-tools)

There are no bug components in bugzilla yet.

It's important to learn how to influence the project in order to get the best out of it and this is primarily done through participation
on the mailing list and in irc. The mailing list is also used for mutual support, to report security issues and discuss bugs.

=== Planning ===
* decide on a process, write it
* identify needed participants
=== Installation ===
=== Configuration ===

== Tools ==
* [[/BOSS|Build Orchestration BOSS]]
** [[/BOSS_Launcher|Launcher documentation]]
** [[/BOSS_Process|Process documentation]]
** [[/BOSS_Testing|Testing]]
** [[/BOSS/Standard_workflow|standard workflow]]
* [[/REVS|Release reporting REVS]]
* [[/IMG|Image Creation IMG]]
* [[/BOSS_Participant|Participant documentation]]
* [[/BOSS/Participants|Participants]]
* [[/SKYNET|Skynet]]

== [[/Packaging|Release team packaging information]] ==

== misc. ==
* [[/devroot|devroot]]
* [[/BOSS/Installation|BOSS and Skynet]]

MeeGo Apps/Problem Statement

2011-08-02T08:22:51Z

Lbt: discussion links

= Apps - options, pros and cons =
Discussion on mailing lists and [http://forum.meego.com/showthread.php?p=27975 MeeGo.com forum]
== Problem Statement ==

The Linux Foundation (LF) owns the MeeGo trademark and hosts the MeeGo.com infrastructure through an arrangement with OSU. They have zero involvement in the running of the infrastructure.

Community OBS (C.OBS) is a hosted service that compiles source packaged to binary packages and then distributes the binaries. It is located at https://build.pub.meego.com/ and has been operational since August 2010.

The MeeGo community are developing a service known as apps.meego.com

This service is a web application which indexes certain community vetted, OSI-approved-licensed applications built by individual community members at the C.OBS

LF have refused permission to :
* use the domain apps.meego.com or apps.for.meego.com to identify the service
* host the apps.meego.com service within the MeeGo infrastructure at OSU

LF have not (currently) refused:
* the continued operation of the C.OBS and the MeeGo.com infrastructure to build and distribute apps.

In private conversations the LF have justified the refusal as:
* If an app is accused of violating a patent then the LF could be at legal risk from linking to it.
* This only applies to individually contributed apps and not to corporate apps because there is a risk of 'dark corners' where an app may be published for many years without the hypothetical violation being discovered.

LF are (currently) happy to distribute all binaries produced by the C.OBS. [editors note: no, this doesn't make sense to him but it's not a typo - it may be a misunderstanding]

== Possible Solutions ==
These possible solutions are put forward for the community to consider.

=== Retain apps.meego.com and C.OBS ===
Use existing infrastructure and host apps.meego.com through LF and at OSU.

Pros:
* Expected solution
* Maintains integrity of MeeGo project
* Linux Foundation seen to support and host Opensource Apps
* Single organisational unit (resource, infrastructure, legal entity)
* Potentiallly hundreds of thousands of users will download MeeGo apps for the first production devices with the MeeGo label (the N9/950) from meego.com.
* This service will continue to be available for as long as the LF wants to support the MeeGo opensource community.

Cons:
* LF at incremental risk for linking to possibly patent-infringing apps (over and above existing risk for distributing same apps and linking to and distributing UX, wifi, bluetooth, and other OSS apps included in MeeGo)
* LF have rejected this option and have said they do not have the resource at this time to provide a statement giving their reasons.

Options:
* Authors of Apps to be indexed in apps.meego.com enter some kind of legal agreement that is compatible with whatever OSI license the App has to minimise LF's exposure to risk.

Editors Comment:
If LF is happy to continue to *distribute* apps from repo.pub.meego.com and to allow the community OBS to provide index/search/linking services then it is not clear what additional risk is taken by hosting apps.meego.com

=== Move apps.meego.com to apps.formeego.org ===
Create a new set of infrastructure in EU under the formeego.org domain.
Leave C.OBS app building and distribution at meego.com

Pros:
* LF not exposed to legal risk of search/linking from apps.formeego.org

Cons:
* LF still exposed to legal risk of distribution and search/linking from build.pub.meego.com
* LF seen as 'preventing the distribution of OSI apps and sucumbing to patent FUD'
* MeeGo project members forbidden from linking to apps in meego.com domains?
* Very unclear position for:
** any Community Build and distribution indivdual users
** any Community Build and distribution projects (N900 CE)
** home projects in core MeeGo Build and distribution
** use of meego.com identity at forapps.meego.com
** use of bugs.meego.com for community apps
* Duplicate infrastructure

Editors Comment:
No idea how this (currently favoured) solution solves the stated problem. Makes one wonder if the problem to be solved is openly stated.

=== Move apps.meego.com and Community Build and Distribution to formeego.org ===
As previous but also move Community Build and Distribution systems.

Pros:
* LF not exposed to legal risk of search/linking or distribution of apps.formeego.org

Cons:
* LF seen as unwilling to support OSS apps
* MeeGo possibly seen as unwilling to support community apps, projects and community contributions
* Duplicate infrastructure
* Very unclear position for
** use of meego.com identity at forapps.meego.com
** apps and any projects for meego

Comment:
Probably the solution most favoured by Android, Apple and Microsoft. Will almost certainly alienate the entire OSS community from MeeGo.

=== Move all MeeGo infrastructure to formeego.org and away from LF ===

This sounds more dramatic than it really is.

Establish a non-profit organisation. Fund organisation to be able to support hosting and distribution services.

Retain minimal infrastructure at LF to support: www.meego.com and repos.meego.com

Nominally move all meego.com infrastructure to formeego.org. This includes core OBS, community OBS, mailing lists, wiki, Bugzilla etc

Retain the same community organisation - including the TSG, CO, IT etc etc.

Pros:
* The LF has a clear role as a the trademark and compliance owner for MeeGo project
* MeeGo.com is the place to distribute final and compliant code
* formeego.org does not expose LF to any legal risk
* LF is not seen as succumbing to FUD
* Non-profit is not an attractive legal target
* The project and community take complete charge of their activities

Cons:
* More sleepless nights for the IT team
* Bookmarks become invalid (eventually - this could be phased)
* .... none really

Options:

* use another name (bikeshed)
* Leave infrastructure physically in OSU.
* Move infrastructure /organisation to a different legal jurisdiction

Editors Comment:

This approach is accepted in the OSS world: Fedora / RedHAT and openSuSE / SuSE being two of the better known examples. The LF do not need to get involved in the normal operations of the project and are clearly manage the trademark and compliance.

Given the LF's inability to support community software distribution
this is, IMHO, the best medium-term approach although it does sound
slightly melodramatic.

MeeGo Apps

2011-08-02T07:56:50Z

Lbt: link to apps.meego.com problem statement

MeeGo Community Apps is the repository where open source software created by the MeeGo community can be found. Developers can build their applications on the MeeGo Community OBS, a sophisticated build system. These applications can be published into the end-user facing Apps repository after certain conditions have been met. This project falls under the responsibility of the [[Community Office]]. The LF have refused to allow 'apps' in the meego.com namespace; the [[/Problem Statement|problem statement]] tries to explain more.

= Accessing Apps =

* [http://apps-beta.meego.com Apps Beta Web Interface]
* [http://apps-beta.meego.com/ocs/providers.xml Open Collaboration Services (OCS) API provider file for Apps]
* [http://gitorious.org/meego-community-extras-client Client code in Gitorious]
* [http://github.com/nemein Server code on GitHub]

= Process =

A developer will need to [[Build_Infrastructure/Community_Builder#Getting_Access|request access]] to the Community OBS. Once the account has been activated, the developer can create a 'home' project. Inside this project one can build packages against any project(repository) in the OBS. This allows the developer to build an application against the Apps repository and make sure there are no dependency issues or other problems.

Once a developer determines that the application is ready for end-users, the package can be promoted to Apps-testing. The promotion request triggers some automated tests on the package. The Apps-testing repository is intended for [[MeeGo Apps/QA|stability testing and community QA]]. If the package meets all promotion criteria, it will be moved to the Apps repository. This makes it available for every end user with a MeeGo device, provided they have enabled the repository in their garage client.

== Supporting Process References ==
* [[Build_Infrastructure/Community_Builder#Getting_Access|Getting access to the Community OBS]]
* [[Packaging/Guidelines|Packaging Guidelines]]
* [[MeeGo Apps/QA|QA Process]]
* [[MeeGo_Apps/repositories|Repository layout description]]

= Community Apps Team =

* Lead Developer: Niels Breet
* Client Application: Martin Grimme
* Web Site: Ferenc Szekely
* OCS Interface: Henri Bergius
* Build Service: David Greaves

== Community Apps Volunteer Team ==

* Graphics: Tim Samoff

== Team Communication Channels ==
* Use the [http://lists.meego.com/listinfo/meego-community MeeGo-Community Mailing List] for questions and discussion.
* File bugs in the [https://bugs.meego.com/enter_bug.cgi?product=Community%20Apps Community Apps Component]
* Ask questions in the #meego IRC Channel

= Plans and Schedule =

Working on:

W26:
* Automatic import/delete packages in MeeGo Apps by BOSS participant signaling (niels)
* OCS interface additions: commenting and voting (ferenc/henri)
* List apps in testing queue (ferenc)
* QA result signaling to BOSS (niels/david)
* Configure Apps build targets in OBS, cleanup old build targets (niels)
* Document Apps submit process (niels)

Open issues (help appreciated):

Documentation:
* Document QA process

Design:
* Front page design (Mike?/Tim)
* Icons for App categories

MeeGo Apps/Problem Statement

2011-08-02T07:45:18Z

Lbt: Describe apps.meego.com problem statement

= Apps - options, pros and cons =

== Problem Statement ==

The Linux Foundation (LF) owns the MeeGo trademark and hosts the MeeGo.com infrastructure through an arrangement with OSU. They have zero involvement in the running of the infrastructure.

Community OBS (C.OBS) is a hosted service that compiles source packaged to binary packages and then distributes the binaries. It is located at https://build.pub.meego.com/ and has been operational since August 2010.

The MeeGo community are developing a service known as apps.meego.com

This service is a web application which indexes certain community vetted, OSI-approved-licensed applications built by individual community members at the C.OBS

LF have refused permission to :
* use the domain apps.meego.com or apps.for.meego.com to identify the service
* host the apps.meego.com service within the MeeGo infrastructure at OSU

LF have not (currently) refused:
* the continued operation of the C.OBS and the MeeGo.com infrastructure to build and distribute apps.

In private conversations the LF have justified the refusal as:
* If an app is accused of violating a patent then the LF could be at legal risk from linking to it.
* This only applies to individually contributed apps and not to corporate apps because there is a risk of 'dark corners' where an app may be published for many years without the hypothetical violation being discovered.

LF are (currently) happy to distribute all binaries produced by the C.OBS. [editors note: no, this doesn't make sense to him but it's not a typo - it may be a misunderstanding]

== Possible Solutions ==
These possible solutions are put forward for the community to consider.

=== Retain apps.meego.com and C.OBS ===
Use existing infrastructure and host apps.meego.com through LF and at OSU.

Pros:
* Expected solution
* Maintains integrity of MeeGo project
* Linux Foundation seen to support and host Opensource Apps
* Single organisational unit (resource, infrastructure, legal entity)
* Potentiallly hundreds of thousands of users will download MeeGo apps for the first production devices with the MeeGo label (the N9/950) from meego.com.
* This service will continue to be available for as long as the LF wants to support the MeeGo opensource community.

Cons:
* LF at incremental risk for linking to possibly patent-infringing apps (over and above existing risk for distributing same apps and linking to and distributing UX, wifi, bluetooth, and other OSS apps included in MeeGo)
* LF have rejected this option and have said they do not have the resource at this time to provide a statement giving their reasons.

Options:
* Authors of Apps to be indexed in apps.meego.com enter some kind of legal agreement that is compatible with whatever OSI license the App has to minimise LF's exposure to risk.

Editors Comment:
If LF is happy to continue to *distribute* apps from repo.pub.meego.com and to allow the community OBS to provide index/search/linking services then it is not clear what additional risk is taken by hosting apps.meego.com

=== Move apps.meego.com to apps.formeego.org ===
Create a new set of infrastructure in EU under the formeego.org domain.
Leave C.OBS app building and distribution at meego.com

Pros:
* LF not exposed to legal risk of search/linking from apps.formeego.org

Cons:
* LF still exposed to legal risk of distribution and search/linking from build.pub.meego.com
* LF seen as 'preventing the distribution of OSI apps and sucumbing to patent FUD'
* MeeGo project members forbidden from linking to apps in meego.com domains?
* Very unclear position for:
** any Community Build and distribution indivdual users
** any Community Build and distribution projects (N900 CE)
** home projects in core MeeGo Build and distribution
** use of meego.com identity at forapps.meego.com
** use of bugs.meego.com for community apps
* Duplicate infrastructure

Editors Comment:
No idea how this (currently favoured) solution solves the stated problem. Makes one wonder if the problem to be solved is openly stated.

=== Move apps.meego.com and Community Build and Distribution to formeego.org ===
As previous but also move Community Build and Distribution systems.

Pros:
* LF not exposed to legal risk of search/linking or distribution of apps.formeego.org

Cons:
* LF seen as unwilling to support OSS apps
* MeeGo possibly seen as unwilling to support community apps, projects and community contributions
* Duplicate infrastructure
* Very unclear position for
** use of meego.com identity at forapps.meego.com
** apps and any projects for meego

Comment:
Probably the solution most favoured by Android, Apple and Microsoft. Will almost certainly alienate the entire OSS community from MeeGo.

=== Move all MeeGo infrastructure to formeego.org and away from LF ===

This sounds more dramatic than it really is.

Establish a non-profit organisation. Fund organisation to be able to support hosting and distribution services.

Retain minimal infrastructure at LF to support: www.meego.com and repos.meego.com

Nominally move all meego.com infrastructure to formeego.org. This includes core OBS, community OBS, mailing lists, wiki, Bugzilla etc

Retain the same community organisation - including the TSG, CO, IT etc etc.

Pros:
* The LF has a clear role as a the trademark and compliance owner for MeeGo project
* MeeGo.com is the place to distribute final and compliant code
* formeego.org does not expose LF to any legal risk
* LF is not seen as succumbing to FUD
* Non-profit is not an attractive legal target
* The project and community take complete charge of their activities

Cons:
* More sleepless nights for the IT team
* Bookmarks become invalid (eventually - this could be phased)
* .... none really

Options:

* use another name (bikeshed)
* Leave infrastructure physically in OSU.
* Move infrastructure /organisation to a different legal jurisdiction

Editors Comment:

This approach is accepted in the OSS world: Fedora / RedHAT and openSuSE / SuSE being two of the better known examples. The LF do not need to get involved in the normal operations of the project and are clearly manage the trademark and compliance.

Given the LF's inability to support community software distribution
this is, IMHO, the best medium-term approach although it does sound
slightly melodramatic.

Community Office/Community device program/Nokia

2011-08-01T17:56:58Z

Lbt: update lbt

= Nokia Participation Details =
* Program Contact: [[User:qgil|Quim Gil]]
Update: Nokia N950 handsets are ready! https://meego.com/community/device-program/devices/nokia-n9-devkit

== N950 Devkit Program Details ==
* Device: Nokia N950 loaded with MeeGo 1.2 Harmattan
* Quantity: 250
* Additional Criteria / Terms:
** One submission per developer please
** Device to be loaned to participant for [period unspecified].
** May not be able to ship to certain countries / locations.
** Nokia employees are not eligible.
* Timeframe: distribution active.

'''QUESTIONS / ANSWERS & UPDATES:''' http://forum.meego.com/showthread.php?t=3597

'''[[N950 landing page]]'''

== Results ==

'''WORK IN PROGRESS'''

For the sake of transparency and collaboration:
* Please link your name to a page describing your Nokia N950 related work e.g. a wiki page.
* Add here one line of text summarizing the project(s) and feature(s) you are concentrating.
* We haven't done the 'Nokia employee' check yet. If you happen to be one, contact Quim Gil.

=== Still pending ===
Participants accepted that still haven't received the N950.

==== Still haven't ordered the N950 ====
You haven't completed the first steps at Nokia Developer and therefore they can't send you an N950. Please move fast before it's too late. You risk losing your chances of getting the device at all.

* Vitaly Petrov (fruct.org/sketchit, fruct.org/gpw) '''ID Sent, Waiting for Lauhcpad account to be activated'''
* Si Howard
* Piotr Pokora (piotras) - '''ID Sent, Waiting for Launchpad account to be activated'''
** I am core developer of Midgard Content Repository which (as library) is used by different Maemo apps: Conboy, MaeCalories, Tablet of Adventure, Qaikuclient. Also I am maintainer of libgda and midgard packages (debs and rpms). From date of birth, I am interested in unified and simplified data access. And such, I am also going to develop for N950.
* Luke Bratch
* Tadej Novak '''ID sent''', '''Launchpad:Accepted(06-Jul-2011)'''
** Porting my desktop IP TV player and schedule to Meego
* Evgeny Linsky (evlinsky) '''ID Sent, Waiting for Launchpad account to be activated'''
** Small apps on QtQuick for Meego/Symbian (http://projects.forum.nokia.com/icthub)

==== N950 ordered, waiting to be sent ====
Please move your entry from the sections below if this is your case. Sort yourselves by country since this is most likely the reason why you haven't got the device yet.

* Brazil - this is a special case, issue known.
** [[User:Lizardo|Anderson Lizardo Gomes]] (user: lizardo): ID sent, Accepted for the Nokia Developer Launchpad program (06-Jul-2011–06-Jul-2012), device ordered (07-Jul), waiting for DHL tracking number.
** [[User:Briglia|Anderson Briglia]] (user: briglia), ID sent | Launchpad:Accepted | Ordered:Yes(7th-July-2011) | Received/Sent:'''No''' 
** [[User:Lfelipe|Luis Felipe Strano Moraes]] '''ID sent''', '''Launchpad accepted''' 
** [[User:Rafael2k|Rafael Diniz]] '''ID sent''', '''Launchpad for individuals account active ''' 
*** I plan to develop FM RDS applications with focus in the new standards from RadioDNS like the RadioVIS (partly based in the already existent the N900-fmvis http://code.google.com/p/n900-fmvis/).
*** I'm a member of a university radio station (Radio Muda FM, 88.5MHz) and my plan is to develop "real life" radio station applications. 
*** I'll also write one audio and one audio/video icecast2 clients. I can provide icecast2 server access for beta testers at radiolivre.org. I'll take ideas from softwares I already wrote for this purpose, like darknow (a gui for darkice, http://darksnow.radiolivre.org) and theorur, an audio/video icecast2 client (a gui for ffmpeg2theora, http://theorur.sarava.org), all using QT.
** [[User:vivijim|Rodrigo Vivi]] (user: vivijim), ID sent | Launchpad:Accepted | Ordered:Yes(7th-July-2011) | Received/Sent:'''No''' 

*Netherlands
** Willem Liu '''ID Sent, applied to Launchpad, Device ordered 2011-08-01'''

*New Zealand
**[http://wiki.meego.com/User:Mohannad Mohannad Hammadeh] '''Ordered N950 (July 8, 8:30am NZST) | N950SentEmail:No'''
*** Porting mPrayerTime to Meego-Harmattan, updating the UI and adding more features.
*** Writing new application ''Spotter'' - exercise tracking app

*Norway
**[[User:karljohang|Karl Johan Grøttum]] '''Order: OID-053015''' | '''Device: Nokia N950''' | Porting [http://trac.itek.norut.no/n4c/wiki Hiker's app] from '''EU WP7''' project [http://www.n4c.eu/N4C-open-source-code.php N4C (open source code)]. Hiker's app runs on N810 and N900, and integrates [http://sourceforge.net/projects/dtn/ DTN2] (or [http://garage.maemo.org/projects/dtn/ DTN2 for Maemo]).

*USA
** [[user:n8willis|Nathan Willis]] '''New Order'''
*** Font packaging, porting DIN 1451 designs to open font license
** [[user:reggie|Reggie Suplido]] '''New Order'''
*** Custom MeeGo web development related to meego.com and forum.meego.com.

* Finland
**[[User:Vranki|Ville Ranki]] '''ID sent, applied for Nokia Launchpad, Ordered device ''' 
[http://www.siilihai.com Siilihai web forum reader], [http://www.youtube.com/watch?v=erTAMOzdf0Y&feature=related Drone Taxi], PPCards.

* Russia
** Roman Deninberg([http://maemo.team16.ru/ Bonapart]) - ''applied for Nokia Launchpad, Launchpad for individuals (28-Jul-2011–28-Jul-2012)''','''New Order (28.7.2011)''', '''Not Received Yet''' Psx4m\PCSX-rearmed\Psx4m-gui projects basically

* England
** Damion Yates '''ID sent'''. '''Accepted in to Nokia Developer Launchpad programme, device ordered - OID-054526'''
*** Video streaming for multiple desktops to receive the video from your phone.
*** New in 2.6.32+ kernel's wifi 80211 Infrastructure AP rather than Ad-Hoc for tethering.
*** qemu x86 chroot for wine and arbitrary x86 X11 binaries
*** tun/tap network interface for dns tunnelling with iodine
*** usb-storage usage monitor with idle umount && rsync and remount
*** turn phone in to bluetooth dongle

* $country
** $entry.

==== N950 shipped, waiting to get it ====
Please list yourselves keeping the list by countries in order to see the geographical progress in ''real time''. :)

* $country
** $entry.

* Serbia (Devices were sent to Moscow by accident, reordered them to an address in Hungary)
** [[User:Blackwicked|Edvin Rab]], '''Developer ID sent''', '''Applied for Launchpad''', '''Device Ordered''', '''Waiting for Arrival'''. [http://t.co/4Os8iIh EvidenceHunt Game]
** Robert Marki - '''Developer ID sent, Applied for Launchpad, Device Ordered, Waiting for Arrival.''' Developing an application called [https://projects.developer.nokia.com/feed_reader FeedReader], it's a universal feed reader with support for podcasts. More info on the project's website. Would like to develop image processing related applications like: Image translation application, Image gallery with face recognition, Porting control software of a hexapod robot.

* Israel
** Assaf Paz (damagedspline) '''ID sent''', '''applied for Nokia Launchpad, Launchpad for individuals (06-Jul-2011–06-Jul-2012)''','''Order committed (7-Jul-2011)''', '''Device Sent (29-Jul-2011)''', '''Not Received Yet''' Adapting [http://code.google.com/p/qwazer/ Qwazer] to also work on Meego, hopefully create an Exchange Webmail client in pure QML (N900 was the initial target), Hebrew support
** Mohammad Abu-Garbeyyeh '''Device Ordered, Waiting for Arrival''' Planning a wiki page with a todo list, main project here: http://bt-messenger.com

*Maldives
**Hussain Shafiu(demonreeper) ID sent | Launchpad:Accepted | N950eMail:Yes | Ordered:Yes(7th-July-2011) | Sent:'''Yes(1st-August-2011)'''

* Switzerland
** [[User:milliams|Matt Williams]] (milliams) '''Device Sent (28/7/2011) - picking up from DHL depot tomorrow''' Creation of a Particle Physics information database (from [http://pdg.lbl.gov/ PDG]) application. Porting of [http://games.kde.org/game.php?game=ksquares KSquares] to pure Qt for MeeGo and creation of other similar simple games. Porting of [http://thermite3d.org PolyVox] to MeeGo and port games built on it when they are ready. Port the [http://falconpl.org Falcon] programming language to MeeGo.

* Australia
** [[User:Termana|Bradley Smith]] (Termana) '''ID sent, Launchpad: Accepted, N950 Email: Received, Ordered: Yes, Shipped: Yes 27/July, Received Device: No''' South Australia. As above, it has shipped and I received a tracking number on the 27th of July, I have not received the device as of yet. Developing a karaoke game with built-in pitch correction.

* New Zealand
**[[User:Kidproquo|Sam Bristow]] '''Shipped 2011-08-01. DHL Tracking number not yet working though'''

* Spain
** Daniel Martin Yerga '''ID sent''' | '''Launchpad:Accepted(05-Jul-2011)''' | '''N950eMail:Yes''' | ''' Ordered:Yes ''' | ''' Shipped:Yes (July 29) ''' | '''Received:No''' Porting my Maemo applications: [http://maemo-wordpy.garage.maemo.org/ MaStory], [http://cusl4-cservices.forja.rediris.es/ CasualServices], [http://pyrecipe.garage.maemo.org/ Pyrecipe], [http://maemo.org/downloads/product/Maemo5/copernicium/ Copernicium], [http://stockthis.garage.maemo.org/ StockThis], and developing new ones, like [https://gitorious.org/r-dmobiley R&DMobiley].

* PRC
** [[User:leafjohn|Lifu Zhang(leafjohn)]] '''ID sent''' | '''N950eMail:Yes''' | ''' Ordered:Yes ''' | ''' Shipped:Yes (July 28) ''' | '''Received:No''' Create an opensource Qt astrology app for handset, Project Page: [https://github.com/cardmaster/qastro/tree/develop qastro hosting by github] Porting apps on our company page ([http://store.ovi.com.cn/publisher/EB EB OVI Page]) to MeeGo From DHL track web page, it's very close to me now.

* Russia
** [[User:Kulakov|Kirill Kulakov]], '''ID sent''' | '''N950eMail:Yes''' | ''' Ordered:Yes ''' | ''' Shipped:Yes (July 28) ''' MySocials project - clients, libraries and plugins for frameworks and platforms for social networks

==== FIXME Unclear cases & lazy wiki editors ====

'''Let's clean this section, please! Move your entry wherever appropriate. Thanks!'''

Adam Pigg '''ID sent''', '''applied for Nokia Launchpad''', '''waiting for reply''' 
Porting my Qt/QML apps/games from maemo, and further work on Kexi and some more QML games
[http://www.piggz.co.uk My Site]

Oleg Bodnarchuk(bloody)'''ID sent''', '''applied for Nokia Launchpad''' 
Developing Wiki-based offline database.

Diego Marcos '''ID sent''' | '''Launchpad:Accepted(05-Jul-2011)''' | '''N950eMail:Yes''' | ''' Ordered:Yes ''' | '''Received:No''' 
The goal is porting to mobile devices open source data visualization tools of astronomical data aimed at outreach and science communication. I've been previously working on Qt/QML desktop applications based on stellarium.org
http://www.youtube.com/watch?v=COkwscvTnnM&feature=youtube_gdata_player

Frank Sievertsen '''ID Sent, Launchpad member now''' 
Open-Source Spideroak Mobile Client and other apps

[[User:gbraad | Gerard 'gbraad' Braad]] '''ID sent''' | '''Launchpad: waiting''' | '''N950eMail: Yes''' | ''' Ordered: Yes ''' | '''Sent: Yes (28 Jul)''' | '''Received: Not yet''' 
Porting of of Node.JS, phonegap, unhosted and a mobile org-mode editor. Aiming for good integration with the MeeGo API and Qt Mobility. Code will be published on [https://github.com/gbraad github] and described on my [http://gbraad.nl/ blog]. Small [https://github.com/gbraad/meego-pomodoro PomodoroTimer] app has been created:

Hiemanshu Sharma '''Completed ''' 
Currently working on porting [[http://forum.meego.com/showthread.php?t=3660|Komedia]]. More apps in the pipeline including Quassel (IRC Client), a Google Reader (name suggestions are welcome) and a 'Line of the day' kind of app (a glorified version of cowsay). Also working on getting an opencv port to give way for Face Detection/Facial recognition APIs.

[[User:kemargrant | kemargrant]], '''ID sent''', '''Applied for the Nokia Developer Launchpad program,(Ordered N950:Phone recieved)''' 
My goal is to bring Screen Mirroring to Meego along with playing local files
easily to a desktop. The app is called groundwork and it is opensource. Code will be shifted to Launchpad once I can begin testing on a meego device.
http://code.google.com/p/groundwork/

Laszlo Papp (Already got one earlier, thus I do not need a new one ;) )

[http://wiki.meego.com/User:Martink Martin Kolman] (MartinK) '''ID sent''', '''applied for the Nokia Launchpad''' | '''N950eMail:Yes''' | '''Ordered:Yes''' | '''Received:Yes''' 
Porting the modRana GPS navigation system and Mieru manga and comic book reader.

[[User:Seif|Seif Lotfy]], '''ID sent''', '''Device received''' 
My goal is to port the Zeitgeist to MeeGo with all the fun stuff with it. I already have a Qt port for "El Loco".

Sergey Ivanov '''Order (11.7.2011), Not Received Yet''' 
Developing software for the mobile operating system MeeGo, associated with the processing of audio and video streams.

Susanna Huhtanen

[http://wiki.meego.com/User:Kenya888 Takahiro Hashimoto(kenya888)] '''ID sent, accepted into Nokia Launchpad, device ordered''' 
porting qimsys/mozc to Harmattan/MeeGo, developing streaming multimedia player with QML

Tasuku Suzuki

Teemu Hukkanen

Thomas Cherryhomes - Lead Developer for LinuxMCE - '''ID and Launchpad ID sent'''
* LinuxMCE is a next generation smart home platform encompassing media, home automation, telecom, and security features. http://www.linuxmce.org/
* A 25 min demo of the software can be seen here: http://video.google.com/videoplay?docid=2176025602905109829
* Nokia N950 will be used as a test platform for the new QML/Qt Quick based qOrbiter we are writing to replace our existing Orbiter software, qOrbiter videos here:
** http://www.youtube.com/watch?v=NDGagn3EciA
** http://www.youtube.com/watch?v=oUHrCdBgoyQ

[[User:tlaukkanen|Tommi Laukkanen]] '''Device received'''
* Facebook client [http://kasvopus.com Kasvopus], Twitter client [http://twimgo.com TwimGo], Google Reader client [http://newsflow.mobi NewsFlow], FourSquare client [http://nelisquare.com Nelisquare]

[[User:toninikkanen|Toni Nikkanen]] '''ID sent''', '''applied for Nokia Launchpad''', '''order sent'''

Anthony Day '''ID sent''', '''Device received'''
* Porting and extending [http://talk.maemo.org/showthread.php?t=72951 inner-spin] game
* Porting and extending [http://talk.maemo.org/showthread.php?t=73942 Take it away Marco] N900 drum machine
* writing new game and realtime music Apps content for the N9/950

Eike Hein

[[User:Theonehumble|Stephan Bulgin]] '''COMPLETED''' 
- I will be porting NXEngine http://nxengine.sourceforge.net/ to MeeGo/Harmattan. My previews work for Maemo can be found here http://talk.maemo.org/showpost.php?p=971709&postcount=1
Description: A clone/engine-rewrite of the classic jump-and-run platformer Cave Story.
- Right now Im in the process of re-writing DonQt for MeeGo/Harmattan. Previews work for Maemo here http://www.forums.internettablettalk.com/showpost.php?p=976671&postcount=1 (will most likely be a name change and better code.)
Description: Don is a "SDK installer" for developers to compile on the go.
- More ports and some original stuff and looking forward to collaborations.



Moritz Mühlenhoff

Felipe Erias Morandeira '''ID sent, Applied for the Nokia Developer Launchpad program'''
* Design user interfaces using QML and collaborate with the MeeGoTouch project.

[[User:sebas|Sebastian Kügler]] (sebas) '''device has arrived...'''
* Bringing Plasma Active ( http://community.kde.org/Plasma/Active )to MeeGo

Juha Ristolainen '''ID sent, already a Launchpad member''' 
Heiaheia fitness-service application for MeeGo. Untappd.com client for MeeGo.

Shan Yafeng '''ID sent''', '''Applied for the Nokia Developer Launchpad program''' 
An education program for exchange information between students and teacher in class. And port some programs to the nokia N900/N950 device. The progress can be found here : http://cuckoohello.wordpress.com

Koos Vriezen '''Device Received''' 
Will port the popular [http://maemo.org/downloads/product/Maemo5/kmplayer/ kmplayer] application from maemo5. 
For now sources are found at [https://garage.maemo.org/plugins/scmsvn/viewcvs.php/branches/harmattan/?root=kmplayer maemo garage]

Aigars Mahinovs

Andreas Schildbach (Goonie) '''ID sent''', '''Applied for the Nokia Developer Launchpad program''' 
Porting of [http://code.google.com/p/public-transport-enabler/ Public-Transport-Enabler] and [https://market.android.com/details?id=de.schildbach.oeffi Öffi] to Meego.

Ilya Paramonov '''Device received''' 
Development of collaborative mind mapping application [http://yar.fruct.org/projects/hivemind HiveMind] for mobile and desktop platforms and sophisticated GTD-style personal time management application [http://yar.fruct.org/projects/octotask Octotask].

Philford Barrett (sevla) '''ID sent, Applied for the Nokia Developer Launchpad program'''
* Swipe Style/Swypr/SwipeMe - Allows the user to assign swiping from the outside of the screen to specific actions i.e. swiping from the top jumps to the multi-tasking view while swiping from the bottom jumps to the Feeds View. Each side of the screen (Top/Left/Bottom/Right) can have a max of 4 zones. Each zone can be assigned to an action thereby giving the user the ability to configure 16 "invisible" shortcuts. Each of which will be available at all times, regardless of what the user is doing in the current app/view.
* Drop Box Integration - Integrate downloading/uploading data to and from an existing Drop Box account. Wherever possible, existing apps will be modified to show this data. i.e. photos in the users drop box account can be (meaning this will be configurable) displayed from the n9 picture viewer.
* Audio Galaxy Integration - Enable streaming of your audio galaxy library to your device through the n9 media player.
* Feeds++ - Feeds++, an enhanced feeds view, extends the functionality of the feeds view by allowing multiple views and enabling the ability to assign specific data to each view. i.e. Show Facebook only data in View A and Twitter only data in View B. Feeds++ will also allow the user to reply directly to events without having to opening the corresponding app.

[[User:hardaker|Wes Hardaker]] '''ID sent, Launchpad application submitted'''
I'm continually developing applications for multitudes of devices, including many Qt applications at [http://www.dnssec-tools.org/ dnssec-tools] as well as personal projects, my favorite being my [http://www.hamtools.org/cutecw/ Morse Code Training Software], which is what I want to port immediately. See my [[User:hardaker|User Page]] for a more complete list.

William Stephenson (wstephenson) '''Device Received''', '''Applied for the Nokia Developer Launchpad program'''
I'm working on a high level toolkit for the creation of branded RSS based apps, in order to facilitate the creation of these simple apps.

pancake

[[User:pancake|pancake]] '''
I'm the author of radare2, a reverse engineering framework for disassembling, debugging, hexediting binaries and doing some forensics-related tasks. I already wrote a GTK frontend for Maemo (n770,n810,n900) and my plan is to write a QT/QML ui for it.
I will also port other programs of mine like tokipona language learning tools, simple games (but addictive!) to QT (from commandline).
In the future I would like to work on Vala and Gtk3/gtkaml (multitouch) support for MeeGo based devices.

[[User:lamikr|Mika Laitio]]
* kernel
* MeeGo CE edition
* VDR linux tv client

* kernel
* Meego CE edition

Christos Zamantzas ([[User:Saturn|Saturn]]) - '''ID sent, Applied for the Nokia Developer Launchpad program'''

Sivan Greenberg--> Nokia Developer Champion ID: sivang , Applied for individual Nokia Developer Launchpad Membership. Working on [[http://developer.qt.nokia.com/groups/qt_contributors_summit/wiki/pdf/CrowdQuick CrowdQuick]] and some platform stuff, as evident by the talks I had given in 2010/2011 begin_of_the_skype_highlighting 2010/2011 end_of_the_skype_highlighting begin_of_the_skype_highlighting 2010/2011 end_of_the_skype_highlighting begin_of_the_skype_highlighting 2010/2011 end_of_the_skype_highlighting MeeGo conferences.

Tapio Pyrhönen '''Device ordered 2011-07-11 and arrived on 2011-07-16!!! Getting busy now.''' 
Porting my old Nintendo DS apps/games and making new ones too.
[http://tapsa.bitmagick.com/nds My Site]

Michael Schloh von Bennewitz (MSvB) '''- Got "A Nokia N950 is waiting for you" but... after going to the order URL an error appears "Support Center, Unexpected error has occured. Please try again." This since three days now.''' Using the device for a MeeGo lecture series in the fall, giving demos. Application development includes LDAP client, and a chess clock. I've ported a number of network and security packages as well, will begin to get them over to the MeeGo repos.

Svetozar Belic ([[User:trx|trx]]) '''- Got "A Nokia N950 is waiting for you", Device ordered (2011-07-07) ''', Port TxPad, TxMySQL Explorer, libQt4Pas library, etc.. Will create a list of apps to port/create

=== Completed ===
Participants that have received the Nokia N950, sorted by meego.com nick. You know this device program is completed when we have reached 250:

# [[User:aaporantalainen|aaporantalainen]] (Aapo Rantalainen)
#*[http://www.umsic.org/jammo/ JamMo] (will need some underlying libraries, e.g. [http://www.clutter-project.org/ clutter])
# [[User:Agomez|Agomez]] (Andres Gomez)
#*Development of drondas, a personal application for the management of the payments shared with other people so you can get track of who paid which in name of whom.
# [[User:ajalkane|ajalkane]] (Arto Jalkanen)
#*Developing dynamic profile switcher, with location and day/time based rules on which profile to use.
# [[User:aklapper|aklapper]] (Andre Klapper)
#*General testing and bug hunting
# [[User:amandalam|amandalam]] (Amanda Hoi Ching Lam) '''Device received (2011-07-18)'''
#*Traditional Chinese language and utility apps for the MeeGo & Harmattan platforms, including but not limited to a Chinese character lookup app, and applications localized for the Traditional Chinese communities in Hong Kong, Macau and Taiwan. [https://sites.google.com/site/amandahoic/Home/ Amanda's Software Projects]
# [https://meego.com/users/andreagrandi Andy80] (Andrea Grandi)
#*QML native client for Soma.fm radio. Current code available here: https://github.com/andreagrandi/CuteSoma
# [[User:andrei1089|andrei1089]] (Andrei Mirestean) '''device received'''
#*Develop a step counter application based on the [http://maemo.org/downloads/product/Maemo5/pedometerhomewidget/ Pedometer desktop widget] for N900
# [[User:anidel|anidel]] (Aniello Del Sorbo) '''Device received. All is well in the world.'''
#*Porting [http://maemo.org/downloads/product/Maemo5/xournal/ Xournal] from Maemo to Harmattan/MeeGo
# [[User:antman8969|antman8969]] (Anthony Naddeo) '''device received'''
#*[http://umcs.maine.edu/~naddeoa/profile/linkedup-project.html Linkedup] - LinkedIn client for Maemo, Meego, Harmattan..... anything Qt
#*[http://umcs.maine.edu/~naddeoa/profile/qtweather-project.html QtWeather] - United States National Weather Service application
#[[User:apachelogger|apachelogger]]
#*[http://git.videolan.org/?p=QtMobileVLC.git;a=summary Porting VLC] to handsets and tablets using Qt for UI awesomeness.
#Aleix Pol (apol)
#*Porting "horaris" and "kanban" maemo applications, finally get to have a usable KAlgebra Mobile version working on MeeGo, hopefully drag other KDE applications with this effort.
# [[User:ar|ar]] (Antti Raina) '''device received'''
#*Develop a simple budgeting application + porting apps
#[[User:arfoll|Arfoll]] (Brendan Le Foll) '''device received'''
#*Porting XBMC + MeeGo TV stuff + doing audio continuums using pulseaudio.
#[[User:Avis|Avis]] (Alexander Terekhov) '''device received'''
#*[http://qt-apps.org/content/show.php/Smart+Shopper?content=139742 Smart Shopper] - porting and improving GPS-based shopping application.
#[[User:awhiemstra|awhiemstra]] (Arjen-Wander Hiemstra) 
#*Porting [http://gluon.gamingfreedom.org Gluon] to MeeGo/Harmattan.
#*Working on [http://calligra-suite.org Calligra].
# [[User:b0unc3|b0unc3]] (Daniele Maio), '''device received'''
#* Porting maemo stuff
#[[User:Bart-cerneels|Bart Cerneels]](Stecchino) 
#*Mobile UX' for Amarok using QML. [http://amarok.kde.org Amarok website]
#[[User:Bemasc/N950_Project|bemasc]] (Benjamin M. Schwartz) '''Received. Appears to be in working order.''' 
#*I will attempt to convert [http://sugarlabs.org Sugar] [http://activities.sugarlabs.org Activities] into MeeGo apps, and hopefully in the process acquire some insight into the potential for MeeGo to form the basis of future Sugar revisions
# [[User:bergie|Henri Bergius]]
#* Porting Buscatcher, Midgard and Node.js -related tools to MeeGo. However, I've withdrawn my device program application because I already got a N950 via Helsinki MeeGo Network.
# [[User:Broothy|Broothy]] (Ádám Balázs), '''Sent my Account ID to Quim, i'm already Nokia launchpad member. Awaiting any reply. Device shipped, received''' 
#* [http://store.ovi.com/content/113753 Switchboard] [http://www.youtube.com/watch?v=GdskgAfjjxc MobileMind]
# [[User:Bundyo|Bundyo]] (Kamen Bundev), '''ID sent''', '''Launchpad activated, notification email received, device ordered, device shipped, received on 18th July''' 
#* Rewriting Search Tool, porting Maemo 5 work, NodeJS, possible Tear rewrite.
#[[User:CaCO3|George Ruinelli]] '''received my device, got account for launchpad and OBS''' Porting my [http://maemo.org/packages/view/sleepanalyser/ SleepAnalyser] from MAEMO as well as other smaller apps I wrote/ported. See [http://wiki.maemo.org/User:Caco3] for details.
#[[User:captianigloo|captainigloo]] (Aguirre Nicolas) 
#* Porting [http://enna.geexbox.org Enna], [http://svn.enlightenment.org/svn/e/trunk/E-MODULES-EXTRA/elfe elfe] and all [http://www.enlightenment.org EFL/Enlightenment] libraries to Meego.
# [[User:cgrozea|cgrozea]] (Cristian Grozea) '''Device received.''' 
#* creating magnus-plus-photo: an application that combines a camera-based magnifier with more advanced image processing techniques, that would enable one to use it as a magnifier (with optional light from the camera LED), use it as a photo negatives lightbox that automatically inverts the negatives and adjusts the colors for proper display; use it as an EVF add-on to SLRs to help with manual focus, leveraging the possibility of amplifying contrast and magnifying. 
# [[User:cip|cip]] (Christian Pühringer) '''ID sent, already a Launchpad member''', '''Ordered (8-Jul-2011)''', '''Dispatched (15-Jul-2011)''', '''Device received (18-Jul-2011) 
#* [https://github.com/cip/WikiOnBoard/wiki WikiOnBoard] Offline reader for Wikipedia using [http://openzim.org zim] format.
# [https://meego.com/users/conny Conny] (Cornelius Hald) '''Device received''' 
#* [http://conboy.garage.maemo.org Conboy] [http://thp.io/2011/mong Mong aka Plonk]
# [[User:Clint|Clint Adams]] 
#* Libre.fm-related software development and porting, advocacy
# [[User:Cpscotti|cpscotti]] (Clovis Scotti) '''Device received.''' 
#* Developing the "connected snowboarding" [http://www.pushsnowboarding.com Push Snowboarding] application/project. Also, I'll be very happy to port other apps I did (mainly for Maemo) + new projects.
# [[User:Creamygoodness|Creamy Goodness]] (Lance Colton) '''Device received.''' 
#* Working on Proximus during July, I will see what we can do with Conky after that.
# [[User:crevetor|crevetor]] (Antoine Reversat) '''Device received.''' 
#* Developing a bixi app [http://forum.meego.com/showthread.php?t=3650 App thread]. Some Meego CE hacking
# [[User:dakron|dakron]] (Pawel Kurdybacha) '''Device received.''' 
#* Testing and contribution to Qt Mobility on Harmattan platform
#* Multimedia Home controller based on gUPnP
# [[User:daperl|daperl]] 
#* Porting the Sudoku clone Tomiku
# [[User:davidmaxwaterman|Max Waterman]] '''Device received.''' 
#* Porting ZouBa to MeeGo/H and QML, plust other app ideas.
# [[User:davidsansome|David Sansome]] '''Device received.''' 
#* Porting [http://www.clementine-player.org Clementine music player] to MeeGo. Clementine already uses Qt and GStreamer.
# [[User:deimos|deimos]] (Marco Bavagnoli) 
#* I'm porting [http://mediadownloader.cz.cc/?page_id=2 mediadownloader] application just ported to [http://mediadownloader.cz.cc/?p=153 maemo] and here a N900 [http://www.youtube.com/watch?v=_Dsj2piBQCw video].
# [[User:Dimitar | Dimitar]] (Dimitar Pashov) 
#* Porting pdf viewer in case the stock one is not better than the one in n900. Try the abilities of the n9/50 HW with an engineering/scientific 3D model viewer. Implement some other ideas.
# [[User:Divan|Ivan Daniluk]] ''' Device received. COMPLETED.''' 
#* Porting [[User:Divan|my Maemo5 applications]], adding full Vkontakte support, developing new apps.
# [https://meego.com/users/djarty DJArty] (Artem Sereda) 
#* Porting qutIM, openpref, arora, links, groove, microdc, Ukrainian localization.
# [[User:Dm8tbr | dm8tbr]] (Thomas B. Ruecker) 
#* MeeGo Community edition for N9(|50|00)
#* APRS application in QML to teach myself something about QML and Qt Mobility.
#* LiveView daemon/application based on code found here: http://code.google.com/p/adqmisc/source/browse/#svn%2Ftrunk%2Fliveview
#* USB host mode
# [[User:drowne | Drowne]] (Valerio Di Donato) 
#* Location-Based games and application development, mobile game design. Junomi Developer ( serious game presented at Games for Health Conference in Boston, May 2011 ) 
# [[User:druid23 | druid23]] (Dru Moore) 
#* To port / create multi-track editing and mixing software to Meego / Harmatten, and multimedia capabilities in general 
#* Additionally, to port remote controls for various networked media players (Singbird, Foobar2000, Squeeze, VLC etc).
# [[User:dwaradzyn|dwaradzyn]] (Damian Waradzyn) '''Device received. Thank you!''' 
#* Porting and further development of [http://talk.maemo.org/showthread.php?t=58402 CloudGPS]
# [[User:Eipi|eipi]] (Sanjeev Visvanatha) , '''Device Received 18/7/2011-Thanks!''' 
#*Porting MaeFlight from Maemo 5, and adding functionality for Harmattan
# [[User:Elleo|Elleo]] ([http://blog.mikeasoft.com/tag/maemo/ Michael Sheldon])
#*Creating a [http://libre.fm Libre.fm] radio client and porting [http://www.jokosher.org Jokosher] to small screen devices.
#[[User:Emocow|emocow]] (Ferdinand Mayet) ('''Device received. Thank you!''') 
#*Development of a golf GPS application
# [[User:epage|Ed Page]]
#* Updating [http://wiki.maemo.org/DialCentral DialCentral], [http://wiki.maemo.org/Gonvert Gonvert], [http://wiki.maemo.org/Ejpi ejpi] for Meego/Harmattan
#* Port all other appliations to Qt for Meego/Harmattan
#* Continue writing new applications
# [[User:fcrochik|fcrochik]] (Felipe Crochik) '''Device received''' 
#* port applications from n900: mobwebmail, macuco, geeps, wakeonlan, ...
#[[User:feri|Ferenc Székely]] (ferenc)
#* Working on [http://apps-beta.meego.com MeeGo Apps], an "app store" for open source, free apps for MeeGo
#* Will help packaging and porting Maemo -mainly location based- apps to MeeGo
# [[User:fiferboy|fiferboy]] (Andrew Olmsted) '''Device received''' 
#* [http://andrew.olmsted.ca/meego fiferboy's applications] - Personal Lexicon, Birdlist, some other ideas for new programs and a few ports
#[[User:fms|fms]] (Marat Fayzullin) ('''Device received 22.7.2011''') 
#* Porting the following: [http://fms.komkon.org/SlideRule/ SlideRule], [http://fms.komkon.org/ColEm/ ColEm], [http://fms.komkon.org/fMSX/ fMSX],[http://fms.komkon.org/Speccy/ Speccy], [http://fms.komkon.org/ATI85/ AlmostTI], [http://fms.komkon.org/MG/ MasterGear], [http://fms.komkon.org/iNES/ iNES], [http://fms.komkon.org/VGB/ VGB], [http://fms.komkon.org/VGBA/ VGBA]. Also expecting to port the FBReader and an IRC client (although most likely not XChat).
#[[User:frankusb|frankusb]] (Frank Banul) '''ID sent, Applied for the Nokia Developer Launchpad program. Device ordered. Device received.'''
#* Port TabletBridge and RadioTimeToGo
# [[User:garyd|Gary Driggs]] ('''Device received.''') Porting web apps and misc. Unix tools to MeeGo ARM devices.
#[[User:generalantilles|GeneralAntilles]] (Ryan Abel)
#* Working with fiferboy on a photographer's application suite ([http://thousandsparrows.com/meego/ SnapGo] and [https://gitorious.org/fiferboy/geosnap Gitorious], currently) to include feature like a light meter and GPS track recording.
# [[User:gri|gri]] (Christoph Keller) '''Device received'''
#* Porting [http://web2sms.garage.maemo.org Web2SMS], splitting it up into a telepathy plugin, service daemon, contacts integration and hopefully sms application integration plus new provider plugins.
# [[User:Harbaum|Harbaum]] (Till Harbaum)
#*Currently re-writing CacheMe UI in qml, working on Zeemote driver
# [[user:hawaii|hawaii]] (Simon LR) '''Device received'''
#* FOSS porting, repo priming, tool building. Platform/product evangelism.
# [http://forum.meego.com/member.php?u=9286 helex] (Michael Muth)
#* [http://talk.maemo.org/showthread.php?p=1001316 ClipMan], [http://talk.maemo.org/showthread.php?t=52589 DreamRemote], TcpKeyboard, something like [http://talk.maemo.org/showthread.php?t=72408 ConkyLayoutSwitcher] (have to see how the UI works in detail - need to create it from scratch)
#[[User:helihyv|helihyv]] (Heli Hyvättinen)
#*Porting Ghosts Overboard (a game) and Chess Clock from Maemo and adding new features to the former.
# hiemanshu (Hiemanshu Sharma)
#* Porting Komedia and OpenCV. Writing a google reader client, a cowsay GUI.
#[[User:hopbeat|hopbeat]] (Arkadiusz Stopczynski)
#*Various academic projects, including novel user interfaces, social web, BCI and portable cognitive sensors. All the crazy stuff mentioned here: http://www.milab.imm.dtu.dk
#*Some utility applications that make your everyday tasks easier, such as shortcutd or lockdaemon for Maemo
#[[User:ieatlint|ieatlint]](Jeffrey Malone)
#*Creating NextBus transit application for North America
#[[User:ivan4th|ivan4th]] (Ivan Shvedunov) '''Device received'''
#*Working on [http://github.com/ivan4th/i4checklist Shopping list/checklist] application inspired by HandyShopper for PalmOS
#*Common Lisp stuff (will try to make CCL+CommonQt etc. work on Meego/Harmattan)
# [[User:Jaffa|Jaffa]] (Andrew Flegg)
#*Porting apps from Maemo (Attitude & Hermes), developer tools, and apps.meego.com workflow. [[User:Jaffa|"Want to know more?"]]
#[[User:Javispedro|javispedro]] (Javier de San Pedro)
#*Porting my [http://wiki.maemo.org/User:Javispedro Maemo 5 applications and SDL games], and [http://gitorious.org/hsdl/pages/Home SDL] itself.
#[[User:jbos|jbos]] (Jeremias Bosch) '''Device arrived.'''
#* Bringing Peregrine Communication Client to Harmattan
#* http://www.peregrine-communicator.org
#* MeeGo CE
#[[User:jflatt|jflatt]] (Jason Flatt) '''Completed'''
#* https://gitorious.org/meego-nonograms
#JLP (Jure Repinc)
#* Creating a Thousand Parsec game client
#* Moodle client
#* Help with testing
#* Translation into Slovenian
# [[User:Joergrw|Joergrw]] (Joerg Reisenweber) '''device arrived. COMPLETED'''
#* USB hostmode. Give N9(50) access to external storage etc. (co-devels: Thomas B. Ruecker, MohammadAG)
#* Review the core functionality and find other similar fields to tackle (see *# starhash-enabler for N900). To mind comes: user profiles (refer the modest "default" & "silent" on fremantle), dialplans, location aware event triggers (cinema profile triggers automatically on entering the building), improved battery management and monitoring, theft protection and recovery...
#* cablefinder based on fast magnetometer readout detecting 50/60Hz fields (co-devel: alterego)
#* torch/flashlight app for N950 - possibly augmented to do optical data transfer, RX via a v4l2 based decoder app
#* I am contributing/associated to:
#**SnapGo / Ryan Abel [consulting on low level stuff]
#[[User:jukey|jukey]] (Uwe Kaminski)
#*General Bug reporting / Community QA processes
#*[http://meegofreeday.org MeeGo Freeday]: event organizing / presentations
#[[User:junousia|junousia]] (Jukka Nousiainen) '''- Device received - COMPLETED'''
#* Creating a tethering application for DSLR cameras, and porting necessary libraries, e.g. libgphoto2
#* Other small random applications and utilities
#[[User:jykae|jykae]] (Ville Jyrkkä) '''device arrived. COMPLETED'''
#* [http://www.cs.tut.fi/~jyrkkav/ Homesite|Ville Jyrkkä]
#* [http://jykae.blogspot.com BLOG: Hacker's Life In Finland]
#* [https://launchpad.net/m-poker MPoker]
# [[User:kdrozd|kdrozd]] (Krzysiek Drozd) - '''N950 At Home '''
#*Clients for a number of local network services, casual games. More soon, on my MeeGo wiki
# [[User:kenyoung|kenyoung]] (Ken Young) - '''device arrived. COMPLETED '''
#*I'm working on porting Orrery, maeSat and maeFat to the N9. The N950 is a beautiful device!
# [https://meego.com/users/khertan khertan] (Benoît HERVIER)
#* Currently working on KhtEditor
# [[User:Kimitake|Kimitake]] (Kimitake) '''device arrived. COMPLETED''' 
#*developing Qt-based micro blogging client for twitter, identi.ca, wassr, jp
#*porting qimsys, Japanese Input method, as maliit plugin
# [[User:kkv|kkv]] (Kirill Krinkin) '''device arrived. COMPLETED''' 
#* Developing Tracker (client for open [https://github.com/OSLL/geo2tag Location Base Platform ]). Tracker will gather user moving data, find patterns, do logistic tasks. Project tracker and progress can be found [[http://osll.spb.ru/projects/geo2tag/issues here]].
#* Mesh networks and energy consumption on device investigation
#[[User:klausr|klausr]] (Klaus Rotter) '''device arrived. COMPLETED''' 
#* Started to work with QtQuick. Nice Tool. Trying to rewrite my EasyPlayer program in QtQuick.
#[[User:kojacker|Kojacker]] (Ryan Faulkner)
#* Various applications, bits and bobs (links coming)
#[[User:kulve|kulve]] (Tuomas Kulve) '''device arrived. COMPLETED'''
#[[User:kypeli|kypeli]] (Johan Paul) '''device arrived. COMPLETED'''
#[[User:kyranto|kyranto]] (Kyösti Ranto) '''device arrived. COMPLETED'''
#* [https://gitorious.org/meego-developer-edition-for-n900/mg-package-manager mg-package-manager]
#[[User:Laasonen|Laasonen]] (Olli Laasonen)
#*Porting apps from Maemo (Who is calling?, Advanced phone lock, Sanakirja.org dictionary client).
#*Developing small handy applications.
#[[User:lardman|lardman]] (Simon Pickering) - '''Device arrived, thanks! :)'''
#*Porting mBarcode, working on Augmented Reality app (mAR), time and location event app (Proximus), additional location methods (offline cellid, magnetic field line direction)
#[[User:Lbt|lbt]] (David Greaves)
#* Mainly CE, Harmattan and Apps to start with. Hopefully Surrounds later.
#[[User:lcuk|lcuk]] (Gary Birkett) '''device arrived. COMPLETED, thanks'''
#* N9 Qt port of liqcalendar and misc liq* apps
#liang wei (foolegg), '''ID sent:Yes''' | '''Launchpad Accepted:Yes''' | ''' Ordered:Yes ''' | '''Received:Yes''' 
#*[[Cuteinputmethod]] is a Chinese Input Method, designed for handset device. 
#[[User:Lorenzph|lorenzph]] (Philip Lorenz) - '''Device arrived - thank you'''
#*Development of a hiking application supporting the user when planning and executing the trip.
#[[User:lostinmirkwood|lostinmirkwood]] (Kristopher C. Kantor) - '''Device arrived, Thank You.'''
#*Continuing Development of [http://ansela.garage.maemo.org/ Ansel-A]: Digital Darkroom for Qt Devices
# [[User:mardy|Mardy]] (Alberto Mardegan)
#* Developing QML port of [http://www.mardy.it/mappero Mappero], possibly [http://www.mardy.it/oculo Oculo]
#* [http://neverball.org Neverball and Neverputt] (currently I'm working on a N900 port).
# [[User:Martink|MartinK]] (Martin Kolman)
#* Porting the modRana GPS navigation system and Mieru manga and comic book reader.
# [[User:marxian|marxian]] (Stuart Howarth)
#* Porting my [https://garage.maemo.org/projects/qmltube cuteTube] application (QML version).
#* MythTV controller/recording scheduler (similar to the Android XBMC application)
# [[User:Masterzap|MasterZap]] (Zap Andersson) - '''Device arrived.'''
#* Porting [http://maemo.org/packages/view/zaploc/ ZapLoc] app to Qt/Meego (currently pygame/Maemo)
#* Porting game "Slightly Annoyed Rodents" (yet to be released) to Qt/Meego (currently pygame/Maemo)
#* Sadly, my unit is developing issues with the display, stripes of black pixels below halfway down the screen, and they seem to get a pixel longer every few days.... :( It's still usable for development, but it's not exactly pretty to look at.
# [[User:mattaustin|Matt Austin]] '''Device received (01/08/11)'''
#* Transperth trains live departure boards app, Player numbers AFL footy app, Amazon S3 bucket & file browser.
#[[User:mdengler|mdengler]] (Martin Dengler) '''device arrived. COMPLETED'''
#* I am working on porting a tron-like game (armegatron preferably or glTron) to the N9, and developing Ringr, a location-based ringtone management application.
# [[User:Mece|Mece]] (Marcus Wikström) '''Device received''' 
#*[http://talk.maemo.org/showthread.php?t=73490 Tweed Suit] for N9/50
#*Qlister (shopping list program)
#*Planning an location based tracking service/app.
#[[User:Metropt|Jose Xavier]], '''device arrived. COMPLETED'''
#* My goal is to port the OpenPilot Ground Control Station to the MeeGo platform and adapt the UI for a better mobile experience. You can see more #information about OpenPilot GCS here: http://wiki.openpilot.org/display/Doc/Ground+Control+Station+User+Manual
#[[User:mgedmin|mgedmin]] (Marius Gedminas) '''Device arrived'''
#*Played with bugfixing in QMLCompass in order to learn the Qt Creator IDE.
#*Ported vim, fbreader for own use.
#*Built unofficial-meego-terminal so that a newer meego-terminal can be installed side-by-side, again, for own use.
#*Put the stuff in apt repository at http://mg.pov.lt/770/ because hasn't found the time to learn OBS yet.
#*Planning to port [http://mg.pov.lt/gtimelog GTimeLog].
# [[User:mickeprag|mickeprag]] (Micke Prag) - '''Device arrived.'''
#*[https://gitorious.org/telldus/tellduscenter-light TelldusCenter Light] - Using the mobile phone as the central hub in your home automation. Control your lights, electrical appliances and curtains wirelessly from the palm of your hands.
#[[User:Mikec|Mikecy]] (Mike Choy) - '''Device arrived.'''
#*Porting svgclock, Maesynth and Maelophone from N900 Python to QML and C++. Stress testing the new [https://projects.developer.nokia.com/qtgameenabler Qt Game Enabler] to see if we finally have low latency audio support in Qt. Will also look to see if we can get midi sample support via Wild Midi or equivalent.
# [[User:mikelima|mikelima]] (Luciano Montanaro) - '''Device arrived.'''
#*Porting [http://quandoparte.garage.maemo.org Quando Parte], implementing a QML patience/puzzle game, porting and adapting KGoldrunner, and writing an OpenStreetMap survey tool, all for use with MeeGo Harmattan (and future MeeGo versions).
# [[User:mikkov|mikkov]] http://forum.meego.com/showthread.php?t=3607
# [[User:Milhouse|Milhouse]] '''Device arrived''' 
#*Develop an audio recording application with geo-location support, plus other applications to improve personal productivity utilising the Harmattan notification/event view.
# [[User:mitrandir|mitrandir]] Ilya Skriblovsky
#* Port NWTBible (Bible reader)
#*Planaris (Hierarchical Todo list) to MeeGo
# [[User:mkruisselbrink|Marijn Kruisselbrink]] '''Device arrive'''
# [[User:mmlado|mmlado]] (Mladen Milankovic) - '''Device arrived''' 
#*Develop [https://projects.developer.nokia.com/home/user/mmlado games] in QML
# [[User:MSvB|MSvB]] (Michael Schloh von Bennewitz) - '''Device arrived, thank you.''' 
#*Using the device for a MeeGo lecture series in the fall, giving demos.
#*Application development includes LDAP client, and a chess clock.
# [[User:Muresandone|MuresAndone]] (Mures Andone) - '''Device arrived''' 
#*Develop location-aware apps with Qt/QML, an enhanced e-book reader based on FBReader engine.
#*Current project: Maemo Application Launcher: http://sourceforge.net/p/maplau/code/
# [[User:netvandal|netvandal]] (Michele Tameni)
#* Luca's Mirror: It’s a simple app that transform your phone into a hand-held mirror with some other cool addictions.
#* Semantic experiment : Experiment with Notification Area mixed with the semantic information stored in tracker, reacting to user action with usefull notification
#*More info [http://michele.tameni.it/project/meego/ Here]
# [[User:Nicolai|Nicolai]] (Nicolai Hess)
#*Port my [http://maemo.org/packages/view/scout scout] application to Qt (Application to search contacts, calendar and conversations)
# [http://twitter.com/#!/mja_fin mja] (Miika Ahdesmaki)
#* Trap, Shake, Kill 'em and other multi sensor apps' development. [http://forum.meego.com/showthread.php?t=3633] '''Nokia Developer Launchpad program approved 06Jul2011, Device available for order 07Jul2011 (ordered, OID-052820), Order sent on 13.7.2011 (email 10:40am), Received 14.07.2011.'''
#[[User:Nielsmayer|Niels Mayer]]
#*[http://wiki.meego.com/Tubelet-and-cutetube-port Rewrite cutetube-qml for MeeGo tablet UX/harmattan UX.] and add automatic-cue-point detection, and social deep-linking of media podcasts. http://nielsmayer.com/meego/qml/qmltube_1_0_6_armel.deb is already usable.... Still a ways to go [http://nielsmayer.com/ts-episode-timeline.png porting my webapp] [http://nielsmayer.com/ts-episode-evnt-anls.png that I call "trainspodder"] :-)
#*Infrastructure for the above: [http://code.google.com/p/qtzibit/ qtzibit: Simile-Widgets Exhibit in QtWebKit with QML integration]. For example, here's [http://nielsmayer.com/meego/qml/qtzibit-youtube.png Timeline faceted browser for 388 YouTube Episodes] [http://qtzibit.googlecode.com/svn/trunk/exhibit/src/webapp/examples/YouTube/YouTube.html can be created with very little code].
#*[http://code.google.com/p/ytd-meego/wiki/CitizenJournalismWithYoutubeDirectForMeego YouTube Direct For MeeGo]
# [http://twitter.com/#!/gregjroberts Noobmonkey] - '''Launchpad:Accepted(06-Jul-2011) | N950eMail:07 July, 12pmGMT| Ordered:07 July 12pmGMT |Received: 14 July| '''
#*Developing/Porting [http://maemo.org/downloads/product/Maemo5/healthcheck/ Healthcheck] with many new fun things (Qt) 
#*Will Port and update [http://talk.maemo.org/showthread.php?t=65522&highlight=maecount MaeCount] (Qt) 
#*Would like to develop a new game (Some ideas, and basic code for a few - so will update shortly) 
# [[User:Nijel|Michal Čihař (Nijel)]] '''device arrived. COMPLETED'''
#* [https://gitorious.org/dofcalc DOF Calc]
#* Creating a [http://wammu.eu/ Gammu] application for phone for data synchronization and backup.
# [[user:nilchak|nilchak]] (Nilanjan Chakravorty) '''Device arrived'''
#* Finance app and Bloomberg Pricing data app
# [[User:niqt|niqt]] (Nicola De Filippo) - '''Device arrived.'''
#*Porting my maemo5 applications [http://badge.garage.maemo.org Badge] and QLshop.
#*New qml game
#*Other mail client.
# [[User:olka|Oleksandr Kachur]]
#* Developing cloud music player integrated with Google music, Amazon music and last.fm services.
# [[User:omllobet|omllobet]]
#*Port 2d puzzle board game [http://kde-apps.org/content/show.php/kMagnet?content=109111 kMagnet] or a new 2d board puzzle game
# [[User:orava|orava]] (Lasse Stenberg)
#* Porting and further developing [http://talk.maemo.org/showthread.php?t=72982 Mapsi]
# [[User:ossipena|ossipena]] (Timo Pelkonen)
#* App for measuring distances and keeping statistics, will reveal more when I get it working well
#* Willing to test others apps, contact me if needed
# [[User:ph0b|ph0b]] 
#* Writing tutorials to help other developers to step in MeeGo / Building MeeGo Paris network / Developing an audio player to access to more than 47 000 webradios referenced on AOL shoutcast (by name, genre, current track)
#[[User:Ph5|pH5]] (Philipp Zabel)
#* Integration of [https://www.torproject.org/ Tor] support
#* Porting of [http://maemo.org/downloads/product/Maemo5/frogatto/ Frogatto], pending SDL support
#* Porting of [https://garage.maemo.org/projects/beifahrer/ Beifahrer] and [https://garage.maemo.org/projects/cinaest/ Cinaest]
#[[User:philippengelhard|philippengelhard]] (Philipp Engelhard) '''Device arrived.''' 
#* Develop a maze game for children and adults
#* Develop a "Nokia Bots" like program for alarm and battery
# [[User:emanymton|Boris Pohler]]
#* porting Zeitkonto and HandsOff (not yet released) from Maemo to Meego, maybe a rewrite with QML
#* other ideas in pipeline (remote for mythtv, live sports-ticker, ...)
#* Helping other users at the german side meego.de (there known as Cermit)
# [[User:pycage|pycage]] (Martin Grimme) '''Sent back defective device. Replacement device arrived. Thanks DDP!'''
#*Doing the Community Apps installer client. Also targetting Harmattan with my OSS MeeGo apps (which are currently mostly running on the WeTab).
#[[User:Qole|qole]] [http://maemo.org/downloads/product/Maemo5/easy-deb-chroot/ Easy Debian] and other projects as they arise
#[[User:quang|quang]] (Quang Pham) '''Device arrived.''' 
#* Develop a location based services application
#* Test Vietnamese localization
# [http://maemo.org/profile/view/rambo/ rambo] (Eero af Heurlin) '''ID sent''', '''Launchpad:Accepted''' | '''N950eMail:Yes''' | ''' Ordered:Yes ''' | '''Received:Yes''' 
#* Port [http://maemo.org/downloads/product/Maemo5/maecalories/ MaeCalories], [http://maemo.org/downloads/product/Maemo5/mobilehotspot/ Mobile hotspot] (possibly, depends on many things and might not be actually neccessary), I'm also looking into some wearable computing and augmented reality stuff, I'll have to see how suitable platform the N9(50) is going to be for that.
# [[User:ravirdv|ravirdv]] (Ravi Vagadia)
#* Trip Management App
#* VLC Remote
# [[User:raydonnelly|raydonnelly]] (Ray Donnelly) '''Device arrived.''' 
#* Investigating porting TermKit
#* Ensure MeeGo works ok with Necessitas SDKs Qt Creator
# [[User:reffy|reffy]] (Alex Tyler)
#* I plan to port my Subsonic client [http://maemo.org/packages/view/aerofy/ Aerofy] to the platform. I also plan to develop a range of media related applications.
# [[User:Rlinfati|rlinfati]] (Rodrigo Linfati) ''' ID send: 30-Jun-2011, Launchpad-Applied: 30-Jun-2011, Launchpad-Accepted: 06-Jul-2011 | N950eMail: 07-Jul-2011 | Ordered: 07-Jul-2011 | Received:14-07-2011 '''
#* Upgrade GoogleLatitude to the current API
#* Find your Frient: a apps that inform you position directly to you friend without any external server.
# [http://maemo.org/profile/view/rm_you/ rm_you] (Adam Harwell)
#* Porting [http://maemo.org/downloads/product/OS2008/advanced-backlight/ Advanced Backlight] from Maemo, adding new features
#* Helping with photo utility suite project (SnapGo, with GeneralAntilles and others)
#* Will help beta test apps for people on IRC
# [[User:rnazarov|rnazarov]] (Ruslan Nazarov)
#* Porting [https://gitorious.org/titanim TitanIM] (Vkontakte instant messenger)
# [[User:roman4|roman4]] (Roman Morawek)
#* Porting [http://babyphone.garage.maemo.org/ Babyphone] to the platform.
# [[User:Rzr|RzR]] (Philippe Coval) '''thank you Nokia for n950 and supporting GNU/Linux
#* tags: ( qt4, qml, opengl, debian, emulator, pinball, neheglqt, p-uae)
#* more: http://rzr.online.fr/q/handset (diary)
#[https://meego.com/users/sandst1 sandst1] (Topi Santakivi)
#* Porting FunkeySynth, a MeeGo Tablet synthesizer to Harmattan
#* Demo clip and further info in [http://sandst1.wordpress.com/ my blog]
# [https://meego.com/users/scifiguy scifiguy] (Sudheer K.) '''ID sent'''| '''Launchpad:Accepted(07-Jul-2011)''' | '''N950eMail:Yes(07-Jul-2011)''' | ''' Ordered:Yes (07-Jul-2011)''' | '''Received:Yes(18-Jul-2011)''' 
#* Porting [https://garage.maemo.org/projects/marketstoday Markets Today], a Stock Quotes app to Harmattan. [http://forum.meego.com/showthread.php?t=3903 Released for Harmattan] (19-Jul-2011)
#* Evaluate porting of VICaR (Call router application) and new application ideas on Harmattan
# [[User:sebas|sebas]] (Sebastian Kügler) - ('''device arrived, thanks!''')
#* Bringing Plasma Active to Meego / handsets
# [[User:Sfietkonstantin|Sfietkonstantin]] (Lucien XU) - ('''thanks Qgil and Nokia for the N950''')
#* Develop a centralized public transportation system : [[TransportApp|libpublictransportation]] (first priority)
#* And also a game [http://sfietkonstantin.free.fr/blog/?p=11 Blog post about the game] (No gitorious yet, will come)
# [[User:shadymilkman|shadymilkman]] (Kyle Thomas) - ('''recieved devkit''')
#* Building Reedit, a reddit application [http://www.shadymilkman.com/p/n9-project.html] removed name from incomplete list.
# [[User:shmerl|shmerl]] (Hillel Lubman) - ('''device arrived, thanks to Nokia for open source Meego development support''')
#* Building/testing Firefox/Fennec (release, beta, aurora, nightly) on Meego
#* Porting [http://code.google.com/p/kosherjava/ Zmanim API] to C++ and preparing it for Meego as a library (libzmanim). Planned - Hebrew calendar application in Qt based on the Zmanim API.
# [[User:sjgadsby|sjgadsby]] (Stephen Gadsby)
#* writing a [[User:Sjgadsby#Preferred Shopper Card Wallet|not-yet-named wallet for store loyalty cards]]
# [[User:slaine|slaine]] (Glen Gray)
# [[User:slvr32|slvr32]] (Jason Byrne)
#* [https://garage.maemo.org/projects/nfqm nfqm] (Netflix Queue Manager) Qt/C++, targeting Symbian^3, Maemo 5, and Meego/Harmattan - discussion thread [http://forum.meego.com/showthread.php?t=3715 here]
# [[User:smoku|smoku]] (Tomasz Sterna)
#* Port my touch screen [http://tomasz.sterna.tv/maemo/ ports of games for Maemo] (Widelands, Bos Wars, Robbo) and UAE4All, PSX4All emulators
#* Port support for SIXAXIS(TM) Controller
#[[User:smurfy|smurfy]] (Philipp Andreas)
#* Porting [https://garage.maemo.org/project/fahrplan fahrplan] (Public Transportation App) for the N9
#[[User:solmis|solmis]] (Janne Mäkinen)
#* Porting/Rewriting Maemo 5 stuff
# [[User:Somnathbanik|Somnathbanik]] (Somnath Banik) 
#* Porting my existing Symbian^3 multimedia applications to MeeGo/N9 with a new and exciting UI components of Harmattan/MeeGo.
#* Creating simple and easy open source application to inspire beginner developers to work on MeeGo/N9 technology.
#[[User:sony123|sony123]] (William Su)
#* Porting Stockona to Harmattan.
#[[User:Spenap|Spenap]] (Simón Pena)
#* Porting and enhancing Maevies from Maemo 5 to Meego/Harmattan. Now tracked at [[User:Spenap/Butaca|Butaca]]
#[[User:stanokia|stanokia]] (Stani)
#* Develop RadioTouch to play online radio stations. Ready for testing: [http://forum.meego.com/showthread.php?t=3993 click here].
#* Develop a creative photo and/or camera application based on the code of the [http://www.phatch.org Phatch] project. (This code uses wxPython for the GUI. So it will take some effort to port it to PySide and QML, with which I have no previous experience yet.)
# [[User:Stskeeps|stskeeps]] (Carsten Munk) '''Device received'''
#* N950/N9 MeeGo CE work and Wayland on these devices
# [[user:summeli|summeli]] (Antti Pohjola) - '''Device arrived.'''
#* Porting [http://www.summeli.fi/?p=2453 AntSnes] and [http://www.summeli.fi/?p=2520 gpSP] from Symbian^3 to Harmattan/MeeGo.
#[[User:swinkels|swinkels]] (Sławomir Musiał)
#* Porting [http://www.swinkels.tvtom.pl/eCards eCards] - Application for creating and sending e-cards
# [[user:syrjala|syrjala]] (Ville Syrjälä)
#* Porting [https://gitorious.org/maemo-tvout-control maemo-tvout-control]
# [[user:tassu|tassu]] (Tapio Pyrhönen)
#* [http://tapsa.bitmagick.com/nds/ My site] - Porting my old DS games and making new ones.
# [[user:teo|teo]] (Teo Mrnjavac)
#* [http://teom.wordpress.com My blog]
#* [http://ur1.ca/4kkwh Porting] [http://amarok.kde.org Amarok] to tablets and handsets running MeeGo/Harmattan.
# [[User:texrat|texrat]] (Randall Arnold, Community Device Program Lead)
# [[User:thebootroo|thebootroo]] (Thomas Boutroue, Developer/Designer)
#* [https://gitorious.org/meego-community-mobile-ux-ng Gitorious] - Making a C++ Declarative-based Visual Components API Lib, alternative to plain QML, or specific platform Components
#* [http://modern-os.projects.servhome.org/mobileApps/ MobileApps] - Porting my old Maemo5 apps to my new Lib, and creating new ones
#[[User:Theonehumble|Stephan Bulgin]]
#*Porting NXEngine
#*In the process of re-writing DonQt for MeeGo/Harmattan.(will most likely be a name change and better code.)
# [[user:thp|thp]] (Thomas Perl)
#* [http://gpodder.org/ gPodder] - Integrating gPodder with Harmattan (including specific APIs)
#* Open source work on Python-related APIs (PySide, etc..) + Python tutorials
#* Get [[Games|Mong]] in shape for Harmattan
#* Port over some of my existing [http://maemo.org/profile/view/thp/ Maemo 5 apps]
# [[user:tigerite|tigerite]] (Peter Hunt)
#* Integrating the BFS CPU scheduler https://garage.maemo.org/projects/kernel-bfs/ into the N9/50 kernel, along with the Budget Fair Queueing I/O scheduler http://algo.ing.unimo.it/people/paolo/disk_sched/
#* Porting projects such as the Phoronix Test Suite http://www.phoronix-test-suite.com/ to Harmattan
#* Converting a Flash cards based learning system which I developed, loosely based on the one found at http://www.educationlabs.com/projects/flashcards/Pages/default.aspx, from C#/XAML to Qt/QML and making it standalone
# [[User:timoph|timoph]] (Timo Härkönen)
#* [http://gitorious.org/random-timoph impuzzle, etc.]
#* [http://timoph.fi timoph.fi]
#* [https://build.pub.meego.com/project/show?project=home%3Atimoph Community OBS home project]
# [[User:timsamoff|timsamoff]] (Tim Samoff)
#* [http://thp.io/2011/mong/ Plonk]
#* MeeGo Community Apps website design
#* A few other things that are brewing (games, sound generators, etc.)
# [[User:tronic|Tronic]] (Lasse Kärkkäinen)
#* [http://performous.org/ Performous] singing/band game
# [[User:tswindell|tswindell]] (Tom Swindell)
#* [http://stage.rubyx.co.uk/columbus columbus]
#[[User:twoboxen|twoboxen]] (Matt Hawkins)
#* Porting [https://code.google.com/p/hawkengine hawkengine] to Meego
#* Porting [https://sites.google.com/site/hawkorn/games games] via hawkengine
#[[User:v13|v13]] (Stefanos Harhalakis)
#* Port WifiEye from maemo to meego
#* Port MaeGirls from maemo to meego
#* Perhaps complete MaeSlap and release it for meego
#[[User:vandenoever|vandenoever]] (Jos van den Oever)
#*Porting [http://webodf.org WebODF] to MeeGo using QML and JavaScript.
#*[http://www.webodf.org/redmine/projects/webodf/wiki/WebODF_on_an_N950 WebODF on an N950]
#*Experiment with a semantic logging tool.
#*Experiment with a [http://blogs.kde.org/node/4161 metronome application] in QML.
#[https://meego.com/users/vasvlad Uladzislau Vasilyeu] (Vasvlad)
#* Porting OMWeather to Harmattan
#[[User:Asys3|Uwe Koch]] (Uwe Koch)
#*Port hopefully all of my games Lineo,Q,TwinDistress,Sokoban and Jooleem
#[[User:Venemo|Venemo]] (Timur Kristóf)
#* [http://wiki.meego.com/User:Venemo/HarmattanPlans My Harmattan Plans]
#** [http://gitorious.org/colorful-apps/puzzle-master Puzzle Master]
#** [http://forum.meego.com/showthread.php?t=3711 Public transportation app] (Click on the [http://forum.meego.com/showthread.php?t=3711 link] and post to the thread if you are interested to contribute.)
#** [https://gitorious.org/colorful-apps/memory-game Memory game]
#** Labirynth game (No code available yet)
#[[User:Vgrade|vgrade]] (Martin Brook)
#*I would plan to contnue my contributions to the N900 Community Edition of MeeGo which I assume will push right through into the N9. I am very interested in contributing to the exciting new architecture #*built on Wayland to give this device the best user experience.
#*Local Network Meetups, Cambridge, Birmingham, Koln, Dusseldorf
#[[User:vitaminj|VitaminJ]] (Stephen Spencer)
#* [http://jenkins.vitaminj.co.uk/job/meex/ Meex], a portable DJing application
#[[User:vitna|vitna]] '''COMPLETED
#*My actual project is http://forum.meego.com/showthread.php?t=3652, but i have in program to develop much more game for the Harmanattan platform
#[[User:vudentz|vudentz]] (Luiz Augusto von Dentz) '''COMPLETED
#*Porting upstream BlueZ and obexd
#[[User:w00t|w00t]] (Robin Burchell)
#* meego-ux hackery, Groovy, anything else I think of that's interesting
#[[User:wazd|wazd]] (Andrew Zhilin)
#* OMWeather, Live Wallpapers, BlueMaemo, Ati85, QML gPodder, tons of other design-related stuff
#[[User:Wicket|wicket]] (David Derby)
#*Porting [http://www.6809.org.uk/dragon/xroar.shtml XRoar - Dragon & CoCo emulator] and [http://icculus.org/avp/ Aliens versus Predator (Gold Edition) game engine].
# [[user:wonko|wonko]] (Ruediger Gad)
#* Amongst other things I'll port my existing applications for Maemo5/Fremantle to MeeGo/Harmattan: VU Meter, StultitiaSimplex, Zeecontrol, Advanced Clock Plugin (for details please see my page).
# [[User:Xando|xando]] (Sebastian Pawluś)
#* Porting [https://github.com/xando/thesis/tree/master/locit-client LocIt] application from N900 Maemo to Meego system.[https://github.com/xando/thesis/blob/master/thesis/Obrazki/UiFlowDiagram.pdf?raw=true pics]
# [[user:xawotihs|xawotihs]]
#* Porting [http://code.google.com/p/wagic/ Wagic] on Harmattan based on either Qt or SDL.
#[[user:xerxes2|xerxes2]] (Jens Persson)
#* [http://gpodder.org/panucci Panucci] - Resuming audiobook and podcast player
#* Meego CE
# [[user:xfade|X-Fade]] (Niels Breet)
#* Set up & Testing Harmanttan building on MeeGo Community OBS
# [[user:zaheerm|zaheerm]] (Zaheer Merali)
#* Porting [http://gstreamer.freedesktop.org GStreamer] plugins not shipped by Nokia to Harmattan
#* Porting [http://www.flumotion.net Flumotion] an open source streaming solution to Harmattan taking advantage of the hardware encoding and the camera
# [[user:zas|zas]] (Matti Henrik Kinen)
#* Setting up SDK
# [[user:zchydem|zchydem]] (Marko Mattila) '''Device arrived'''
#* QuickFlickr - QML based flickr client for mobile handsets.
# [[user:zeamoceq|zeamoceq]] (Olle Tränk)
#* Porting [http://qticksize.zeamoceq.net qTickSize] (interface to Swedish online stock broker)
# [https://meego.com/users/zehjotkah zehjotkah] (Cosimo Kroll)
#* [http://wiki.maemo.org/MeeGo_Coding_Competition_2011 MeeGo Coding Competition 2011]
# [[user:zeenix|zeenix]] (Zeeshan Ali)
#* Ensuring [http://www.gupnp.org GUPnP]/[http://www.rygel-project.org Rygel] works on N9.
#* Possibly porting/writing [http://spice-space.org SPICE] client for
#* N9/MeeGo.
# [[user:nowheremanmail|nowheremanmail]] David Galindo
#* N950 received Thanks!. Starting to migrate Wallet and Calculus. 
#* [http://www.nowhereman.eu]







== '''Updated Questions and answer for those people awaiting N950 Dev Kits:''' ==

1) '''If you have registered for the launchpad, please wait.'''
If you want to find out more information you can email the launchpad team, but a quick response is unlikely.
''Also, Quim will be emailing / sending all of the names and accounts across, therefore, if they have any questions / problems contacting devs they'll let Quim know.''

2) '''Timeline - Timescale'''
There is no defined deadline or timescale for this. Keep an eye on the delivered and pending sections below, as people are posting dates / times.
If things start happening and you feel you are being left out - please then email the Nokia Developer launchpad teams. But until then, not much communication if any will be received. Hold tight and please wait.
If your status is similar to someone else's, and in the same batch, and they get a device, wait a few days then fire a message to the launchpad team or here. No point asking the same questions on the forum. Most of the devs mentioned below are also on twitter, so ask there or elsewhere on the forums if really needed.

3) '''Timescale Part 2 - Patience!'''
Arranging, confirming, emailing, packaging and sending 250 devices is not a day's job.
Realistically expect a few weeks once they have started being sent out.
Be clear in all contact emails you send, to speed up the process - include account names and any other IDs requested/required. It is hard for people to swap from real names, nicknames, etc on a list of 250+ people..

4) '''People who already are registered with Launchpad'''
If you have a launchpad account (Lucky you) there is an option which allows you to select available devices, however, nothing is certain as of now, therefore that may not be the route.
Once the team start going through the list, it sounds sensible that they will start emailing / contacting the people on the list with instructions, confirmations and/or queries. (see below! - thank you Jaffa for the update)
Update - It seems the next step once the launchpad section is confirmed may be an email from '''no.reply-developer@nokia.com''', subject "A Nokia N950 is waiting for you".

5) '''Why have I heard nothing from Launchpad?'''
There is no launchpad confirmation email (But if you try to register again it says that there is already an application waiting) - therefore.... re-register if you have to. Just make sure you use the individual and not company registration. (There is however a developer registration email! - and logging in also proves that stage works!)

6) '''"I didn't realize this was happening, can I still apply for one?"'''
Answer- "Short term: register to http://developer.nokia.com and watch Nokia developer activities in your country. "
- '''This program is closed''', but as Quim says, keep your eyes on the internet, as there are other programs and similar things available, and different countries where Nokia reps do things too

7) '''Why is Nokia Developer saying the device program has been closed, and we still do not have our devices? *rant rant*'''
There are other device programs being run separately to the MeeGo DevKit program. The programs are not joined, but the team that sends out the devices is the same. Therefore, any messages you read are not exclusive to this particular set of 250 devices. Other programs may or may not appear across other Nokia sites, they are all separate from this one.

If you have been accepted, don't panic - they have not gone out yet! (As far as we all know!)
''Please do update this section if you feel other questions from the forum have been answered?''

== General thoughts on device program ==

The Nokia N950 is a platform available now for developers targeting the Nokia N9 and MeeGo handset apps in general. Technical details are available at http://developer.nokia.com/swipe

Candidates must be community developers ready to start working on new or existing open source applications, to be published in apps.meego.com and the Nokia Store. Links to your current projects are relevant! Deadline for applications: end of Tuesday, June 28th.

Questions & comments: http://forum.meego.com/showthread.php?t=3597

IMPORTANT: *commercial* developers are encouraged to apply directly at http://developer.nokia.com - thank you for your understanding.

Release Infrastructure/BOSS/Participants

2011-07-30T09:25:04Z

Lbt: Add instructions for osc installation

List of participants ported / implemented for BOSS SkyNET :

== Installation Prerequisites ==

Some partiticipants will require the osc tool to interact with OBS. This will be installed as a dependency to relevant participants.
The version included in Debian is too old and the required repository must be added:
<pre>
cat <<EOF > /etc/apt/sources.list.d/MINT-tools.list
deb http://download.opensuse.org/repositories/openSUSE:/Tools/Debian_6.0/ /
EOF
</pre>
== OBS prechecks ==

GIT : https://meego.gitorious.org/meego-infrastructure-tools/boss-participant-prechecks

Package : https://build.pub.meego.com/package/show?package=boss-participant-prechecks&project=Project:MINT:Testing

Configuration : The file at /etc/oscrc needs to be edited so that these participants can communicate with the OBS instance.

=== check_already_testing ===
Compares the checksum of the packages being submitted to packages of the same
name possibly in the Testing project. If the checksum matches it sets STATUS
= FAILED

This is for over-eager developers - or if two people in a team submit
close together.

=== check_no_changes ===
Compares the checksum of the packages being submitted to packages of the same
name possibly in the Target project. If the checksum matches it sets STATUS =
FAILED

Developers sometimes submit without doing a proper update - this
catches that situation.

=== check_multiple_destinations ===
Checks if the request tries to submit packages to multiple projects at the same
time and sets STATUS = FAILED if so

=== check_package_is_complete ===
Checks if each of the packages being submitted contains at least the following
files :
* Source tarball : *.tar.gz *.tar.bz2 or *.tgz
* Changes file : *.changes
* Spec file : *.spec
and sets STATUS = FAILED if not

Nb the presence of the .changes file is important for the changelog
participants to operate correctly

=== check_package_built_at_source ===

Prerequisite : check_has_valid_repo

Checks if the packages being submitted are built successfully against the
designated target repository for the architectures of interest and sets STATUS = FAILED if not

=== check_spec ===
Checks if the spec file of each of the packages being submitted is valid. Currently
the only validity check applied is that it shouldn't contain the %changelog tag
and sets STATUS = FAILED if it does

OBS is responsible for inserting the .changes file contents into the spec file.

=== get_submitter_email ===
Gets the request submitter email from OBS and makes sure it is not an empty string
sets STATUS = FAILED otherwise.

=== check_submitter_is_maintainer ===
Checks the request submitter is actually a maintainer in the source project from
which the request is origination, sets STATUS = FAILED otherwise.

=== check_has_valid_repo ===
Finds a repository in the source project that builds ONLY against a certain target
project / repo , sets STATUS = FAILED if it does not find one.

=== check_has_relevant_changelog ===

Prerequisite : get_relevant_changelog

Checks that each package in the request actions has a non empty relevant change entries field.

Use this if you want to enforce that new entries be added to the .changes file before allowing a request to be accepted.

=== check_is_from_devel :regexp ===

Checks that each package in the request originates from a project the matches the regexp provided as a parameter.

== Resolving Requests ==

Package : [https://build.pub.meego.com/package/show?package=boss-participant-resolverequest&project=Project:MINT boss-participant-resolverequest]

=== change_request_state ===

Used to accept or decline a submit request.

Parameters:
* :action (accept/reject) : Determines what to do with the request

Fields:
* 'msg' : The contents of the 'msg' array in the workitem is used as the action message

Config:
* See [[#OBS / BuildService Participants]]

=== build_trial ===

=== revert_trial ===

== Changelog ==

GIT : https://meego.gitorious.org/meego-infrastructure-tools/boss-participant-getchangelog

* branch skynet

=== get_changelog ===

Gets the .changes file from the project/package and puts it into the
'changelog' field. Note that for SRCSRV_REQUEST_*, project is the
_target_ and may not return the expected log.
Thus get_relevant_changelog may be more relevant for that case.

Dependencies:
* /etc/skynet/

=== get_relevant_changelog ===

Gets the .changes file for the source project/package/revision and
does a diff against the target project/package putting the 'added'
lines into a 'relevant_changelog' field for each package in the
request's actions array.

This is particularly useful for acting on external links such as bug#
or feature# mentioned in the new changelog lines.

The complexity of a diff provides for situations where multiple
changelog entries are made in one area of a project before the package
is finally accepted.

== Bugzilla ==

Interacts with bugzilla to:
* add comments to bugs
* change bug status

Links :
* GIT : https://meego.gitorious.org/meego-infrastructure-tools/boss-participant-bugzilla
* Package : https://build.pub.meego.com/package/show?package=boss-participant-bugzilla&project=Project:MINT:Testing
* Configuration : The file at /etc/skynet/bugzilla.conf needs to be edited so that the participant can communicate with the Bugzilla instance. The file has lots of comments to explain the settings.

Prerequisites :
* Bugzilla with REST api installed (https://wiki.mozilla.org/Bugzilla:REST_API)
* get_relevant_changelog

Takes the following parameters (can be different depending on the target bugzilla):
* status : UNCONFIRMED/NEW/ASSIGNED/REOPENED/RESOLVED/VERIFIED/CLOSED
* resolution : FIXED/INVALID/WONTFIX/DUPLICATE/WORKSFORME/MOVED
For the comment :
* comment : Comment text for the bug
or
* template : Cheetah template file passed the workitem hash

== Documentation ==

== Image Creation ==

== Notification ==

GIT : https://meego.gitorious.org/meego-infrastructure-tools/boss-participant-notify

NOTE: skynet branch for now

Package : https://build.pub.meego.com/package/show?package=boss-participant-notify&project=Project:MINT:Testing

=== notify ===
Email notification based on templates.

=== notify_irc :msg, :irc_channel ===
#TODO IRC notification to an IRC bot.

== OBS / BuildService Participants ==

Several participants interact with the OBS using the buildservice library.

They use the skynet configuration system to look for an 'oscrc' value
in the [obs] section.
eg
<code>
[obs]
oscrc = /etc/skynet/default.oscrc
</code>

This is a normal oscrc file (mode 600) and should contain [apiurl] sections with
an aliases section that includes the namespace value set by the OBS in
the obsEvent (see BSConfig.pm)
eg:
<code>
[https://api.pub.meego.com]
aliases=OBS
user=<user>
passx=<passx>
</code>

Migrating from N900 to N950

2011-07-28T20:35:23Z

Lbt: tips

'''Should this go as a page under [[Community Office/Community device program/Nokia]]?'''

== Introduction ==
Lots of developers will be transitioning from an N900 to an N950 as their primary device as part of the [[Community Office/Community device program/Nokia|device programme]]. This transition allows people to get a feel for Harmattan and work out what itches they need to scratch.

The following is a small list of points which helped the author(s) get started. '''It is not indicative of steps necessary on the N9, and is based on the initial steps taken by the author when he received his pre-production, developer device.'''

Other channels: IRC:freenode.net #harmattan

== Developing apps ==

If you have done this before, check out [[Porting Fremantle Applications to Harmattan]].

If you are starting from scratch, use QML, not QWidgets (unfortunately the QtCreator tutorials are outdated and show use of QWidgets). Get started reading about the [http://library.developer.nokia.com/index.jsp?topic=/MeeGo_1.2_Harmattan_API/html/guide/html/Developer_Library_Application_development_framework_ebcf.html MeeGo 1.2 Harmattan API] and be sure to digest the [http://library.developer.nokia.com/topic/MeeGo_1.2_Harmattan_API/html/guide/html/Developer_Library_Best_practices_for_application_development_fc5f.html Best Practices] section.

== Building packages in Scratchbox ==

Follow the instructions in [http://www.developer.nokia.com/Community/Wiki/Harmattan:Platform_Guide/Publishing_with_Harmattan_Platform_SDK/Packaging_your_application_with_Harmattan_Platform_SDK the official Harmattan Platform Guide page] on packaging.

A hint for the experienced: if you have a source deb for a different platform then it's just a matter of unpacking it with dpkg -x *.dsc, cd'ing inside package-version/ and running:

dpkg-buildpackage -us -uc -b -rfakeroot

Another protip: 'debuild' tends to fail with fakeroot-related problems where dpkg-buildpackage succeeds.

== Events feed ==
1.2011.22-6 is a pre-production, developer edition. It only has Facebook integration for chat and events, no Google Talk or Twitter. RSS feeds can be set, in the ''Feeds'' application, to appear here.

== Contacts ==
Some N900s are unreliable with Bluetooth connections. To transfer all the contacts from your N900 to your Harmattan device, including avatars:

* Delete any Facebook accounts on your Harmattan device
* Delete all contacts
* Do ''not'' try to use the "sync your phone" feature from the N950 (if you've tried it, delete the channel on both the N950 and N900)
* ''N900:'' go to Settings > Transfer & sync > New. Define a new send-only, contacts-only channel to the N950 (which should have Bluetooth switched on and visible)

This method does NOT transfer any IM-only contacts.

It should be added that the BT sync from the N950 can be used. At least it worked without any issues for me. YMMV it would seem. -lardman

== Email ==
Replying/quoting is FUBAR in 1.2011.22-6 (no ability to edit quote, forced top posting). With Gmail over IMAP, ''Delete'' now deletes messages, rather than archiving them (as it did in Modest). To "archive" email, move it to the ''All mail'' folder.(There's definitely an opportunity for someone to write a good Gmail client!)

=== Hotmail ===

If you want your folders & state to appear on the device, use '''Mail for Exchange''' for your Hotmail account. [http://windowsteamblog.com/windows_live/b/windowslive/archive/2010/08/30/hotmail-now-supports-push-email-calendar-and-contacts-with-exchange-activesync.aspx The settings can be found here.]

== Syncing multiple Google Calendars ==
This Harmattan build supports multiple Mail for Exchange and multiple CalDAV accounts. For Google Calendar:

* Go to Accounts > Add accounts > CalDAV
* Username should be your full Google Calendar username (e.g. jbloggs@gmail.com) and password.
* The URL is:
** For your primary calendar: <code><nowiki>https://www.google.com/calendar/dav/</nowiki></code>
** For other calendars: <code><nowiki>https://www.google.com/calendar/dav/uuid@group.calendar.google.com/</nowiki></code> 
** For some users, the primary calendar will only work with, supposing your e-mail is jbloggs@gmail.com: <code><nowiki>https://www.google.com/calendar/dav/jbloggs@gmail.com/</nowiki></code>
The ''uuid'' can be found on the calendar settings page within Google Calendar, and is a long string of characters & numbers. 
The trailing slashes are ''crucial'' for Google.

* The primary calendar claimed to sync, but no events showed up. So I deleted the CalDAV entry for my primary calendar and set up Mail for Exchange:
** Username is your full Google Calendar username and password
** Server: <code>m.google.com</code>
* I synced [[User:Vitaminj|my]] primary calendar using the syntax of the second URL (with my Google login email instead of uuid@google) and it worked great. YMMV.

== Conversations ==
Unknown at this time

== Map loader ==
Maps contains a ''Manage Maps'' feature which downloads, but doesn't expand. It is possible to do it manually, though:

# Connect your device as USB mass storage
# Open <code>cities/MapLoader/catalog-....xml</code>
# Find the region you want to download
# Take the <code>Link</code> element and append it to <code><nowiki>http://static.s2g.gate5.de/map5/</nowiki></code>. For example, the UK maps are at http://static.s2g.gate5.de/map5/maploaderzip-00.02.42.122/48210.zip
# Copy the contents of the zip file to <code>cities/diskcache/</code>

== Music ==
Hopefully this is a bug in 1.2011.22-6: ringtones and gPodder podcasts appear in the ''Music'' app; and will play on an all songs shuffle. This can be fixed by editing Tracker configuration:

== Tracker ==
Tracker config file can be found at:
/home/user/.config/tracker/tracker-miner-fs.cfg

Edit the file as regular <code>user</code> and add to the semi-colon separated <code>IgnoredDirectories</code> list.

Tracker indexes file system and applications. Haven't tried ''(could someone please confirm?)'' but re-indexing should be triggered with following command:
/usr/lib/tracker/tracker-store -r (or /usr/lib/tracker/tracker-store --force-reindex)

== Notification light ==
N950 has only single white led instead of N900's RGB LED. The led is driven with same config file:
/etc/mce/mce.ini

It seems that you can also modify the functionality of power button with <code>mce.ini</code> 
Don't know if the services you must restart are the same than in N900 though.

== root ==
The default root password is <code>rootme</code> (as in the 770 days ;-)). You can get root to change it (using <code>passwd</code>) using:

ssh root@localhost

...or:

devel-su

== Security framework ==

Aegis is the name of the platform security system. If you understand POSIX file ownership-based security, '''this is not that'''. Forget that root can read from or write to all files (it cannot on the N950).

N950s given to developers have '''Aegis turned on'''.

If you edit protected files, like some config files or some shell scripts, '''you may have to re-flash the device'''! Don't do this.

You can enable running of arbitrary binaries by running:

develsh
aegis-developer-mode --relaxed-exec

...and then '''reboot'''ing.

found useful for root (runs commands that otherwise wouldn't run):
devel-su -c bash

As a temporary workaround, liberal use of <code>chmod 777 ...</code> for a moment before <code>cp ...</code> and then <code>chmod 700 ...</code> after (as user) can work; <code>su -c user chmod</code> will work as root too.

=== Harmattan Security FAQ ===

There is an [http://library.developer.nokia.com/index.jsp?topic=/MeeGo_1.2_Harmattan_API/html/guide/html/Developer_Library_Developing_for_Harmattan_Harmattan_security_Security_guide_Harmattan_security_FAQ_3358.html Harmattan Security FAQ] that has very useful tips.

=== Further reading ===

* http://www.developer.nokia.com/Community/Discussion/showthread.php?226594-Running-console-apps-on-n950-Operation-not-permitted
* http://forum.meego.com/showthread.php?t=3827
* http://pastebin.com/C5jbvj7N
* http://lwn.net/Articles/372937/
* http://fruct.org/conf7/Kasatkin_Maemo_6_Security_Framework.pdf
* http://www.developer.nokia.com/Community/Wiki/Harmattan:Developer_Library/Developing_for_Harmattan/Harmattan_security/Security_guide#Overview_of_application_security
* http://www.slideshare.net/reshetov/maemo-platform-security-fosdem
* http://www.developer.nokia.com/Community/Wiki/Harmattan:Platform_Guide/Harmattan_platform_and_Platform_SDK_overview/Security_domain_overview

== Screenshots ==
Launch ''Boosted Widget Gallery'' from the applications list, and select ''Debug tools > Take a screenshot''. Ctrl-Shift-P also works, as on N900, but only in stock (MTF?) screens.

They can be shared from ''Gallery'' (they are stored in ~/MyDocs/.images)

== Task switching ==
=== Accesssing ===
In addition to the swipe, the up-right arrow & backspace shortcut works almost similarly to Ctrl-Backspace on the N900.

=== Rotation ===
Out of the box, applications which use Qt Components and lock themselves to landscape, still get rerendered in portrait in the task switcher if the keyboard is closed. [http://www.developer.nokia.com/bugs/show_bug.cgi?id=231 Bug #231] has been reported.

It is possible to modify the themes for the [http://forum.meego.com/showpost.php?p=24933&postcount=14 home screens] and [http://forum.meego.com/showpost.php?p=25070&postcount=19 lock screen] to support portrait, but this is obviously only a developer-only workaround.

== Keyboard layout ==
You can easily change the hardware keyboard's layout with setxkbmap:
setxkbmap -rules evdev -model nokiarm680 -layout fi

It is possible to set two layouts with Ctrl+Shift switching:
setxkbmap -model nokiarm680 -layout us,ru -variant ,cyrillic -option "grp:ctrl_shift_toggle"

== Keyboard shortcuts ==
Sh = shift, Ctrl = control, Fn = Function (lower left key), BS = backspace (upper right key), CR = CarriageReturn/Enter (right 2nd from top)
* Fn+BS : jump back to last screen - usually either taskswitcher or applauncher
(none of the following seem to work in non-MTF apps)
* Ctrl+Q : close current screen and back to last, like Fn+BS
* Ctrl+Sh+P : take ->screenshots
* Ctrl+Sh+T : switch on/off positions(?)
* Ctrl+Sh+S : switch on/off dimensions(?)
* Ctrl+Sh+L : cycle through some languages
* Ctrl+Sh+F : switch on/off frames-per-second
* Ctrl+Sh+B : switch on/off borders(?)
* Ctrl+Sh+M : switch on/off margins(?)
* Ctrl+Sh+N : switch on/off some writing that shows up when scrolling applauncher(?)
* start typing in any of the 3 basic screens : start Search

== Hardware related bits ==
The N950 comes with a quick start guide, USB cable and device. To charge, the N900 wallwart fastcharger works like a charm. Device charges from non standard (missing D+- short) USB chargers with a maximum current of 100mA. The USB receptacle seems to be pretty sturdily mounted to the PCB, should not come off like some of N900.

The screws holding the battery cover are Torx TX4, pry off the cover with your fingernails starting at speaker both directions to the other end, and "hinge" around USB and then off.

== Tricks ==
=== Harmattan home screens in landscape mode ===

Get root:
<pre>ssh root@localhost</pre>
then:
<pre>
mkdir /usr/share/themes/blanco/meegotouch/meegotouchhome/
mkdir /usr/share/themes/blanco/meegotouch/meegotouchhome/style/

echo 'MainWindowStyle {
locked-orientation: "";
}' > /usr/share/themes/blanco/meegotouch/meegotouchhome/style/meegotouchhome.css
</pre>
[http://forum.meego.com/showpost.php?p=24933&postcount=14 source]

To fix the lock-screen:
<pre>nano /usr/share/themes/blanco/meegotouch/sysuid/sysuid.css</pre>

Scroll down until you find <pre>locked-orientation: "portrait";</pre> and change it to <pre>locked-orientation: "";</pre>

Hold the Ctrl on-screen key (not the physical one) and press X on the keyboard. If it just types an 'x' then backspace and do it again. Press 'Y' to save, then 'Enter'. Reboot.

This makes the lock-screen orientation aware, but only just after boot. It seems there is something elsewhere locking its orientation once you have actually used any apps and from then on the lock-screen is fixed in the last orientation you used before locking it.
[http://forum.meego.com/showpost.php?p=25070&postcount=19 source]
[[Category:N900]]
[[Category:N950]]

Release Infrastructure/BOSS/Installation

2011-07-25T10:44:38Z

Lbt: /* Getting started */

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to change. In particular the
repositories are not yet finalised (eg installing from :RC).

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 or OpenSuse 11.4 installation.

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing

Release Candidate : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== Testing ===

In order to verify BOSS is working, please install SkyNET and follow the sample deployment.

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

== SkyNET ==
=== Overview ===

SkyNET manages long-running BOSS Participants.

It provides:

* daemonisation
* start/stop control
* consistent logging
* user-owned participants

=== SUSE ===

zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-skynet

If you installed a minimal OpenSUSE you might get a dependency error and as shown here proceed with choice 1 :

Problem: boss-skynet-0.1-5.1.noarch requires /usr/bin/python, but this requirement cannot be provided
uninstallable providers: python-base-2.7-8.4.i586[openSUSE-11.4-11.4-0]
python-base-2.7-8.4.x86_64[openSUSE-11.4-11.4-0]
Solution 1: deinstallation of patterns-openSUSE-minimal_base-11.4-6.9.1.x86_64
Solution 2: do not install boss-skynet-0.1-5.1.noarch
Solution 3: break boss-skynet by ignoring some of its dependencies
Choose from above solutions by number or cancel [1/2/3/c] (c): 1
.
.
.

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get install --no-install-recommends boss-skynet

=== Configuration ===
The /etc/skynet/skynet.conf file specifies where SkyNET stores the
running participants and where daemontools monitors them
(/var/lib/SkyNET/services/ and /var/lib/SkyNET/store/ by default)

The [boss] section specifies the boss instance used by default and the credentials
required. Individual participants can override this.

== BOSS OBS Plugin ==
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

=== Installation ===
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

=== Configuration ===

'''You *MUST* have a participant registered as "obs_event" running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Getting started and Testing the Deployment =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register -n check
skynet register -n notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process ==

With a participant ready to do some work, we're ready to launch a process:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/BOSS/Installation

2011-07-25T10:43:50Z

Lbt: add note on how to test BOSS

= Overview =

Whilst BOSS is deployed in production, this packaged version is still
in beta-testing and is subject to change. In particular the
repositories are not yet finalised (eg installing from :RC).

However BOSS and SkyNET both install and run at a basic level on a
clean Debian 6.0 or OpenSuse 11.4 installation.

= Installation =

== Preparation ==

* Proxy : On OpenSUSE you need to set the proxy in /etc/sysconfig/proxy

== BOSS ==
=== Overview ===
BOSS provides the ruote worker and the main AMQP server.

This component is the main process dispatcher; no work happens
here but all process steps are initiated from here.

Currently BOSS uses the Ruote filesystem storage system and the
worker and viewer need to run on the same system. This will change
when BOSS moves to Redis.

The stable and testing projects are at the community OBS, the state
of the packages can be monitored at :

Testing : https://build.pub.meego.com/project/monitor?project=Project:MINT:Testing

Release Candidate : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT:RC

Stable : (No release yet) https://build.pub.meego.com/project/monitor?project=Project:MINT

=== SUSE ===

(old version of BOSS on openSUSE 11.2)
zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ref
zypper in boss

(new version of BOSS on opensuse 11.4)
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss

This will install Rabbit MQ and the boss worker daemon.

To run boss:

rcboss start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss

This will install Rabbit MQ and the boss worker daemon.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

Default settings are found in /etc/default/boss

=== Configuration ===

BOSS is preconfigured with username boss and password boss on the boss vhost
(AMQP will lose vhosts soon)

BOSS stores workitem and process data in /var/spool/boss/

=== Testing ===

In order to verify BOSS is working, please install SkyNET and follow the sample deployment.

== BOSS-Viewer ==

This must be installed on the boss server when using the FS storage.

The default port is 9292, point your browser at http://127.0.0.1:9292/_ruote/ (if you are running on the same machine)

#FIXME: Make sure you go to /_ruote/ otherwise you will get a big scary error.

=== SUSE ===

zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-viewer

This will install boss-viewer.

To run boss-viewer:

rcboss-viewer start / stop

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get update
apt-get install --no-install-recommends boss-viewer

This will install the boss viewer.

Then, as usual
/etc/init.d/boss {start|stop|restart|force-reload|log}

== SkyNET ==
=== Overview ===

SkyNET manages long-running BOSS Participants.

It provides:

* daemonisation
* start/stop control
* consistent logging
* user-owned participants

=== SUSE ===

zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-skynet

If you installed a minimal OpenSUSE you might get a dependency error and as shown here proceed with choice 1 :

Problem: boss-skynet-0.1-5.1.noarch requires /usr/bin/python, but this requirement cannot be provided
uninstallable providers: python-base-2.7-8.4.i586[openSUSE-11.4-11.4-0]
python-base-2.7-8.4.x86_64[openSUSE-11.4-11.4-0]
Solution 1: deinstallation of patterns-openSUSE-minimal_base-11.4-6.9.1.x86_64
Solution 2: do not install boss-skynet-0.1-5.1.noarch
Solution 3: break boss-skynet by ignoring some of its dependencies
Choose from above solutions by number or cancel [1/2/3/c] (c): 1
.
.
.

=== Debian Squeeze/6.0 ===

cat <<EOF > /etc/apt/sources.list.d/MINT.list
deb http://repo.pub.meego.com/Project:/MINT:/RC/Debian_6.0/ /
EOF

apt-get install --no-install-recommends boss-skynet

=== Configuration ===
The /etc/skynet/skynet.conf file specifies where SkyNET stores the
running participants and where daemontools monitors them
(/var/lib/SkyNET/services/ and /var/lib/SkyNET/store/ by default)

The [boss] section specifies the boss instance used by default and the credentials
required. Individual participants can override this.

== BOSS OBS Plugin ==
The OBS plugin is designed to launch processes when a build/publish event occurs on the OBS

=== Installation ===
The plugin is installed on the OBS backend which runs the schedulers

openSUSE 11.2
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.2/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.2/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

opensuse 11.4
zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/openSUSE_11.4/devel:languages:perl.repo
zypper ar http://repo.pub.meego.com/Project:/MINT:/RC/openSUSE_11.4/Project:MINT:RC.repo
zypper ref
zypper in boss-obs-plugin

=== Configuration ===

'''You *MUST* have a participant registered as "obs_event" running before enabling this plugin or ruote will silently fill up with hundreds/thousands/millions of stalled processes.'''

Edit the file:
/usr/lib/obs/server/BSConfig.pm
and define/add:
our $notification_plugin = "notify_boss";
our $BOSS_host="boss"; # hostname for server running BOSS AMQP
our $BOSS_user="boss"; # AMQP username
our $BOSS_passwd="boss"; # AMQP password (cleartext)

You may also uncomment and set to 1:
our $multiaction_notify_support = 1;
(This allows osc requests containing multiple actions to be handled properly).

The plugin itself is installed in :
/usr/lib/obs/server/plugins/notify_boss.pm

The schedulers will need to be restarted to take effect. Note there is currently an issue that
the plugin will block if the AMQP server is not available - this will halt the OBS.

= Getting started =

Once you have installed BOSS and SkyNET you are ready to create and run some processes.

== Watch BOSS ==
On the BOSS machine:

/etc/init.d/boss log &

You'll see messages about the engine being alive and participant
registration. Further debug output is possible here.

== Setup a participant ==

On the skynet machine we need to setup some code as a participant:

skynet make_participant -n check -p /usr/share/doc/python-boss-skynet/example-check-participant
skynet make_participant -n notify -p /usr/share/doc/python-boss-skynet/example-notify-participant

This creates a daemontools/participant directory structure with a
symbolic link to the code for these participants.

Now we need to ensure they start to run under daemontools (and restart if
there's a problem):

skynet enable check
skynet enable notify

To watch the log output:

skynet log check &
skynet log notify &

Now we need to tell boss that there is something listening on these
queues and available for use in processes:

skynet register -n check
skynet register -n notify

The skynet command also supports:
* stop : shutdown the participant and don't respawn.
* reload : stop and start once
* register : can also specify the amqp queue to use if it's not the name

== Launch a process ==

With a participant ready to do some work, we're ready to launch a process:

/usr/share/boss-skynet/skynet_launch

The log output should show

2011-03-29 00:49:21.186558500 checking for /tmp/success
2011-03-29 00:49:21.186849500 Failed to read /tmp/success
2011-03-29 00:49:21.708706500 Nothing to say

Now do:

echo Hi there > /tmp/success

And relaunch:

/usr/share/boss-skynet/skynet_launch

gives
2011-03-29 00:49:31.659986500 checking for /tmp/success
2011-03-29 00:49:31.660440500 Read /tmp/success
2011-03-29 00:49:31.878978500 Send email saying : Hi there
2011-03-29 00:49:31.878980500

Of course a process can be launched from anywhere on the network using
any 'trigger' and participants also run on any machine.

== Debugging ==

= Next steps =

Do something useful with your installation using the [[../Standard workflow|standard workflow]] and the [[../Participants|participants and launchers]]

Release Infrastructure/IMG

2011-07-22T11:36:16Z

Lbt: Add sample IMG process

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.2.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.2/devel:languages:python.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.2/meego-tools.repo
zypper ref

Then install the core IMG package and the amqp listener:

zypper in img-core img-amqp img-boss

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Wether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Integration =

One valuable use of IMG is to build an image prior to accepting a package into Trunk. Just verifying that a package doesn't prevent building an image is useful; but beyond this, the image can be sent to automated test system and the results used to determine acceptance.

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

== Sample Workflow ==

The following simple workflow can be attached to the REPO_PUBLISHED OBS event like so:

ln -s /srv/BOSS/processes/build_IMG /srv/BOSS/processes/${PROJECT//:/\/}/REPO_PUBLISHED

Each time $PROJECT is published the process will run.

Note that this is a simple process that does not handle situations
like locking the project to ensure changes are not made during the
image build process.

<pre>
Ruote.process_definition :name => 'Project_Trunk_PUB' do
# This process uses:
# img_boss package:
# * build_ks
# * build_image
# optionally from: boss-participant-obsticket
# * obsticket

sequence do

set 'archs' => ['i586']
# The name of the build target (eg 'standard')
set 'targetrepo' => 'Project_Trunk'
set 'debug_dump' => 'true'
set 'image' => {'image_id' => 'latest',
'image_type' => 'livecd',
'arch' => 'i586',
'name' => 'latest',
'ksfile' => 'meego.ks'
}

echo "Start Project_PUB : ${ev.state} in ${project}"

# We could lock the Project to ensure that no SR is accepted whilst the image is building
# with_OBS_ticket do
build_ks
build_image
# end

# We are not doing acceptance based on the image so drop the lock and test it
# test_image
end

# This subprocess shows how a group of actions can be wrapped inside
# other process steps; in this case reserving the use of a build project
define 'with_OBS_ticket' do
sequence do
obsticket :action => 'get', :lock_project => '${test_project}', :debug_dump => 'TRUE'
apply
obsticket :action => 'release', :lock_project => '${test_project}', :debug_dump => 'TRUE'
end
end

end
<pre>

Release Infrastructure/IMG

2011-07-21T22:18:43Z

Lbt: /* BOSS Workitem definition */

= What is IMG =

IMG (Image Me Give) is a small python client/server application suite, its sole job is to get a kickstart file from a user, via a web interface, command line or BOSS process. Then it runs Moblin-Image-Creator, either in a host or in a VM. IMG also has the capability to run as a BOSS participant and thus be a part of a build process.

[[File:IMG.png]]

= Design =
* Asynchronous
:Fire and forget about it, check back later with the cli client for results
* Queue view with django,
:Show a nice html view of currently running or standby MIC jobs in the queue
* Submit a ks via a web interface
:Upload via a form
* No security
:No https or file filter restrictions :)
* Two ways of communication
: BOSS and pure AMQP messages

(FIXME: Which rpms provide which services and where should they be installed? What is the expected network layout - in particular does a single img-web control a pool of img workers? What about the participant? lbt 27/Dec)

== Example workflow ==

The application consists of two parts, django frontend and the actual image creation application. Both of them are connected via AMQP, RabbitMQ is used as the server.

So an example workflow, in AMQP:

# User posts an image creation request with email address and sample kickstart file (or a package overlay), using the web application or the command line client.
# Django application receives the request and imports the information on a model object and saves it (Even for the command line client? lbt 27/Dec)
# A message is sent via an AMQP broker to the image creation worker, containing the details for the required image. (Does this use BOSS? lbt 27/Dec)
## Image creation commences
## Virtual machine disk image containing mic2 is engaged (as a kvm overlay image), with ssh access.
### When a package overlay is received, a kickstart file is constructed from a ready-made template and passed to kickstart part
### When a kickstart file is received, it is passed directly to worker
## Worker starts up and connects to the virtual machine using ssh and passes the required arguments to mic2.
## Worker copies image on success from the virtual machine to well-known www-root, if there is an error, only a message about it is sent via AMQP.
## When mic2 has finished, the virtual machine is shut down by the worker.
## When image is created, the virtual machine disk image is deleted


= Installation =
Installing IMG via RPM for OpenSUSE 11.2.

The following repos are needed : MeeGo-Infra, opensuse python and MeeGo tools:

zypper ar http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra/openSUSE_11.2-plus/Maemo:MeeGo-Infra.repo
zypper ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_11.2/devel:languages:python.repo
zypper ar http://repo.meego.com/MeeGo/tools/repos/opensuse/11.2/meego-tools.repo
zypper ref

Then install the core IMG package and the amqp listener:

zypper in img-core img-amqp img-boss

IMG performance can be massively improved with caching proxy, please take a look at [[Release_Infrastructure/IMG-PROXY|Setting up IMG PROXY]].

= Worker / img-core =

The worker code is provided by img-core and is responsible for running mic2 either locally or on a virtual machine. The initialisation code takes care of creating the necessary command stubs for mic2 use for either implementations. It also handles the KVM creation command stubs.

In KVM mode, it first creates an overlay image from the base image (not distributed) and then starts it using the overlay image as the hard disk. Virtio is used as a speed optimisation method for KVM. When a specific amount of time has passed, it will copy the kickstart and mic2 config file to the guest VM. Then it runs mic2 in the VM with parameters specified in the init method. After mic2 has run, the image is copied from the guest using scp.

In local mode, only mic2 is run on the host machine. It will create the image based on the options in the init method along with the kickstart file.

A worker needs a queueing mechanism to accept jobs. The primary solution is expected to be the BOSS participant (img-boss) but there is a plain amqp client too (img-amqp).

= BOSS participant / img-boss =

The general design of the client is pretty much the same as the AMQP server, since they both get activated and then call the worker code to create the image, with or without KVM. The worker code is described in the section above.

More information about BOSS [[Infrastructure/BOSS]].

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_build_image.py boss_build_image.py] for the source code.

= BOSS Client / img-amqp =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_boss/boss_img_client.py boss_img_client.py] for example BOSS client.

== Usage ==

(Is this the img-amqp usage? lbt 27/Dec)

Usage: boss_client.py -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release <release> -a|--arch <arch >-s|--submit -k <kickstart_file.ks>

boss_client.py Sends a message (poll for result later) to the BOSS, using
<kickstart.ks> as the kickstart file.

Options:
-h, --help show this help message and exit
-s, --submit Submit to BOSS, takes no options
-k KICKSTART, --kickstart=KICKSTART
Kickstart file
-t TYPE, --type=TYPE Image type
-n NAME, --name=NAME Image name
-e EMAIL, --email=EMAIL
Author email
-r RELEASE --release Image release, goes directly to mic2
-a ARCH --arch Architecture to use

= BOSS OTS Client =

[http://gitorious.org/boss-scripts/boss-scripts/blobs/master/boss_ots_client.py BOSS OTS Client] is a AMQP launcher enabling communication with [[Quality/QA-tools/OTS|OTS Framework]]. BOSS OTS Client is used to send image URL to OTS and commence image testing.

Download from: [http://gitorious.org/boss-scripts/boss-scripts/blobs/raw/master/boss_ots_client.py boss_ots_client.py]. Usage ''boss_ots_client.py --help''.

== Configuration ==

See ''defaultconf'' section in the script or provide standalone configuration file.

= Server =

See [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/build_image.py build_image.py] for example server.

This section documents the server component of IMG.

== Functionality ==

=== Consumer functions ===

*kickstarter_callback:
:Receives messages as described [#Kickstarter here], decodes JSON data to variables and creates a kickstart file with the received list of packages and then submits them to the mic-image-creator [#MoblinImageCreator here].
*mic2_callback:
:Receives messages as described [#MoblinImageCreator here], decodes JSON data to variables and runs moblin-image-creator (NB! this has to exist in /usr/bin/moblin-image-creator, will be changeable in the future) against the supplied configuration file and image type.
:The images are saved to to a configured location, needs to be changed in the python subprocess call in order to get downloadable images (eg. to a webserver media path).
:If the image creation fails (is caught with "CalledProcessError" exception in the code, this happens with nonzero exit status), a message is sent to "error_queue" queue, with the following JSON encoded dictionary:

'error': Contains the error message of the exception received from the python subprocess call.
'id': identification string of the original message sent to "image_queue" queue
'url': Contains the URL from which one can download the log

:If the image creation succeeds, a message is queued to "link_queue" queue, with the following JSON encoded dictionary:

'url': Contains the URL from which one can download the image

'id': identification string of the original message sent to "image_queue" queue

== Exchanges ==

* image_exchange, The exchange for image creationg related queues
* django_result_exchange, Exchange for results, picked up by django

== Queues ==

Queues for exchange "image_exchange":

* image_queue
:Stores messages related to Moblin Image Creator runtime.
* kickstarter_queue
:Stores messages related to kickstarter runtime.

Queues for exchange "django_result_exchange":

* status_queue
:Contains messages in the following format (JSON encoded dictionary):
** 'status': status of the image build, compulsory
** 'id': the identification string of the original message sent to kickstarter_queue, compulsory
** 'url': Contains the URL from which one can download the image
** 'log': specifies the log file to read from
** 'error': any error messages faced

= Client =

For demonstration see [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/src/img_amqp/img_client.py img_client.py] for example client.

== Usage ==

img_client.py -p|--poll <id> -n|--name <name> -t|--type <imagetype> -e|--email <author@email> -r|--release -a|--arch s|--submit -k <kickstart_file.ks>"

where:
: -p|--poll <message_id>, poll the AMQP server for messages relating to the id, see -a|--async
: -t|--type <imagetype>, image type, can be of: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
: -n|--name Image name
: -a|--arch Image architecture
: -r|--release Image release number
: -c|--config Configuration file to use

== Documentation ==

This client version uses AMQP as the message bus to communicate with the server. The format for the messages is described in the next part of this document.

==== Messages ====

The general message format is JSON encoded dictionary.

===== Moblin Image Creator =====

To initialize the actual image building, send a JSON formatted message as follows:

{'email':email, 'id':id, 'imagetype':imagetype, 'ksfile':ksfile, 'name': name, 'arch':arch, 'release':release}

where:
* email, email address of the submitter
* id, identification string of the original message (passed on from kickstarter process)
* imagetype, image type of values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
* ksfile, the actual kickstar file to upload (in text)
* name, the name for the image
* arch, architecture to use for the image building
* release, release numbering, $VERTICAL-$ARCH-$VARIANT like in mic2

= Django client / img-web =

=== Views ===
* submit
:If this view receives a POST request, it constructs a bound form with the data from POST array. If the form is valid it extracts the data, as specified in the forms section.
:Applying the overlay is handled in this form, in the following way.
* Get the overlay text from the form field
* use python strings split() method to split the overlay text in to a list, with a comma delimiter
** construct a kickstart file using this list and append the packages to the kickstart files package list
* queue
:This view is responsible of two things. Firstly it checks wether there are any messages in the "status_queue" message queue, if there are, then it updates the !ImageJob model object (identified by the identification string from the message) to include the ready image URL.
:Secondly, it checks for possible error messages in "status_queue" queue, if there are any, it assigns a special variable to indicate that there has been an error.
:If no messages are received, it simply redirects to a template with all the !ImageJob model objects.
* job
:Job view is responsible of providing the information about the log file. The information about the logfile is formed in the server side, such as the URL to the log file. If there is a URL, it will open the URL and read its contents and return the resulting text to the template. The template renders the log message with a textarea html widget.
*index
:Index only returns a template in which one can click a link about uploading the image.

=== URLs ===
url(r'submit/$', 'meego_img.app.views.submit', name='img-app-submit'),
url(r'queue/$', 'meego_img.app.views.queue', name='img-app-queue'),
url(r'job/(?P<msgid>\S+)$', 'meego_img.app.views.job', name='img-app-job'),
url(r'images/(?P<msgid>\S+)$', 'meego_img.app.views.download',name='img-app-download'),
=== Models ===

IMG currently has only one model, !ImageJob, here is the Django code for it:
# Create your models here.
class ImageJob(models.Model):
email = models.CharField(max_length=40)
filename = models.CharField(max_length=40)
logfile = models.CharField(max_length=50)
task_id = models.CharField(max_length=30)
imagefile = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add=True)
error = models.CharField(max_length=500)
type = models.CharField(max_length=10)
status = models.CharField(max_length=30)
def delete(self, *args, **kwargs):
if self.logfile:
if os.path.exists(self.logfile):
os.remove(self.logfile)
os.remove(self.logfile.replace("-log", ""))
print "Removed %s"%self.logfile
super(ImageJob, self).delete(*args, **kwargs)

As can be seen in the delete method, this model cleans up all image creation related files, like the kickstarter file and log file.

=== Forms ===
==== UploadFileForm ====

Contains the following fields
* email, email field
* overlay, text input field
* name, text input field
* platform, select field for the platform, parsed from the default template
* release, a text input field for release
* arch, architecture selection field
* imagetype, select field for the image type, with values: livecd, liveusb, loop, raw, nand, mrstnand, vdi, vmdk
* ksfile, the raw kickstart file

= Worker configuration =

The configuration file is to be installed in /etc/imger/img.conf and a sample file is [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/img.conf here].

== Configuration file ==
base_url: A base URL that has a direct access to the base_dir via HTTP, example http://127.0.0.1
base_dir: A directory to put the finished images, example /var/www/images
num_workers: Number of workers to start, OBSOLETE, USE INITSCRIPT OR MANUALLY RUN MANY PARTICIPANTS
post_creation: Path to a script to run after the image is created
use_kvm: Wether to use a virtual machine, values are "yes" or "no"

mic_opts: Example, mic_opts = --save-kernel, --use_comps, so comma separated options

amqp_host: Host that runs BOSS
amqp_user: Username to tap into the BOSS virtual host
amqp_pwd: Password
amqp_vhost: The actual virtual host to connect to

== KVM Image ==

The KVM image must be created in such a way that it has openssh and mic2, along with having ssh-key based login for root, which uses a certain [http://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/master/ssh/id_rsa.pub ssh public key].

The format of the image is recommended to be qcow2, with preallocation and cluster_size=2M as the qemu-img options. This is because the worker code uses qcow2 to create the overlay, thus saving time if the image is in the same format.

If the host is using openSUSE, please enable the following things for zypper repositories:

http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.2/
http://download.opensuse.org/repositories/Virtualization/openSUSE_11.2/

And add kvm-intel to /etc/sysconfig/kernel.

== Virtual machine ==

Currently, IMGer can run the MIC2 jobs inside a KVM virtual machine, the only prerequisites are that image is located in /usr/share/img/base.img (configurable in the future) path and that the virtual machine has MIC2 installed and configured.

Suse studio is one way to create the images, just make sure that you have a rpm repository which contains mic2, python-xml and qemu-static-arm to make mic2 run properly on the image.

One must also confirm that the image SSH-keys in the source distribution are configured in the virtual machine and both keys exist in /usr/share/img (configurable in the future). During the runtime of IMGer, it is possible that ssh-keys inside the image may change, this is already handled by IMGer by ignoring the host key checking.

= BOSS Workitem definition =

IMG provides 2 participants that can be used in a BOSS process:
* build_ks : Modifies a template kickstart file to add specific packages or groups
* build_image : Builds an image defined by a kickstart file

== build_ks ==
This participant is used to customise a kickstart file to add specific
packages or groups; for instance to add a package to a minimal
kickstart for testing purposes.

=== Configuration ===
The build_ks participant is configured at the OS level.

This allows the repository server (reposerver) and the kickstart
template store (ksstore) to be specified.

==== Input ====

The participant documentation defines the input for the participant.

=== Output ===

The image.kickstart field is populated with the modified kickstart file.

== build_image ==
The main IMG participant. It takes a kickstart file and some values and returns URLs which point to the image and log.

=== Configuration ===
The build_image participant is configured at the OS level.
The [https://meego.gitorious.org/meego-infrastructure-tools/imger/blobs/skynet/src/img_boss/build_image.conf config file] is documented.

It mainly determines where images are stored on the worker and the base url of the webserver that has been setup to serve them.

==== Input ====

The following defines the input dictionary for the participant:
"kickstart" : contents of a kickstart file (ie not a path)
"email": email address, not parsed so can be a dummy address
"id": a simple UUID, eg. from pythons uuid library
"type": type, from values: livecd, liveusb, loop, raw, nand, mrstnand, vdi or vmdk
"name": name, a simple string for the image name
"arch": architecture, for image
"release": release

=== Output ===

The work item can contain the following dictionary entries:

Status: Status, is either "DONE" or "ERROR" in the final workitem, depending on success/failure
URL: URL to a image/log/kickstart file location on the worker
Error: An error string with details of the error, if any
Image: A direct URL to the finished image
Log: A direct URL to the log file of the mic2 output

Release Infrastructure/BOSS/Standard workflow

2011-07-19T08:15:07Z

Lbt: Add sample process

BOSS comes with a sample "standard workflow" which consists of:
* a plugin to the OBS to intercept the [[Release Infrastructure/BOSS/OBS Event List|event notifications]]
* the robogrator process launcher
* a [[Release_Infrastructure/BOSS/Participants|coherent set of participants]]

This is not a mandatory workflow but was developed at Nokia and is designed to be useful for MeeGo vendors

=== Installation and source ===

Package : https://build.pub.meego.com/package/show?package=boss-launcher-robogrator&project=Project:MINT:Testing

GIT : https://meego.gitorious.org/meego-infrastructure-tools/boss-launcher-robogrator
NOTE: skynet branch for now

=== Robogrator Process Launcher ===

Robogrator is the robotic package integrator. It reacts to the events occuring on the OBS and looks at the data inside to identify which project-specific process to launch.

Installation is as boss-launcher-robogrator:
apt-get install boss-launcher-robogrator
or
zypper in boss-launcher-robogrator

NOTE: robogrator is the package name , the participant name in processes is obs_event so register the participant with skynet by invoking the command :

skynet register -n obs_event

It uses the data sent by the obs-boss-plugin to launch a fresh process.

All processes have the fields:
* project : the project used to drive the process. Usually the project the package is in but for requests type events this is the target project.
* ev : the (undocumented) OBS Notify object parameters

For the actual process, robogrator uses the namespace/project/event-label to find a specific ruote process.

For the standard workflow the first participant is the standard_workflow_handler

It is configured in /etc/skynet/robogrator.conf

=== standard_workflow_handler ===

This participant serves two purposes:
* tracks packages that are rebuilt when a repo is published
* set some standard fields used by other participants

* REPO_PUBLISHED
** repository
** packages (packages successfully built in this build/publish cycle)
* BUILD_UNCHANGED, BUILD_SUCCESS, BUILD_FAIL
** package
* SRCSRV_CREATE_PACKAGE, SRCSRV_DELETE_PACKAGE
** package
* SRCSRV_VERSION_CHANGE
** package
** rev
** version
* SRCSRV_REQUEST_*
** actions
** who
** rid
** description
** when

For a standard installation the following script would enable Project:CE:Trunk

cat <<EOF > /srv/BOSS/processes/standard_workflow
Ruote.process_definition :name => 'BOSS_Standard_Workflow' do
sequence do
standard_workflow_handler
done
done
EOF

PROJ=Project:CE:Trunk
PDIR=/srv/BOSS/processes/${PROJ//:/\/}
mkdir -p $PDIR
for ev in SRCSRV_COMMIT REPO_PUBLISHED BUILD_UNCHANGED BUILD_FAIL BUILD_SUCCESS SRCSRV_VERSION_CHANGE SRCSRV_DELETE_PACKAGE SRCSRV_REQUEST_CREATE; do
ln -s /srv/BOSS/processes/standard_workflow $PDIR/$ev
done

skynet enable standard_workflow
skynet register -n standard_workflow

=== Example Workflow ===

Ruote.process_definition :name => 'DE_Trunk_SRs' do
sequence do

set 'irc_channel' => '#meego-arm'
set 'archs' => ['i586', 'armv8el']
set 'targetrepo' => 'standard'
set 'debug_dump' => 'true'
set 'final_project' => 'Project:DE:Trunk'
# REVS baseline
set 'baseline' => 'Trunk'
# REVS platform
set 'platform' => 'DE'

echo "Start SRCSRV_REQUEST : ${ev.state} in ${project}"
notify_irc :msg => 'lbt : New SR#${ev.id}(${ev.state}) in ${project}', :irc_channel=> '#meego-boss'

# CREATE or supersede
_if '${ev.state} == new' do
do_create
end
_if '${ev.state} == superseded' do
notify_irc :msg => 'SR#${ev.id} superseded in ${project} - FYI', :irc_channel=> '${irc_channel}'
end
_if '${ev.state} == accepted' do
do_accepted
end
_if '${ev.state} == declined' do
do_declined
end
end

define 'do_create' do
echo "${project} SRCSRV_REQUEST : do_create"
notify_irc :msg => 'New SR#${ev.id} in ${project} from ${ev.author} ${ev.actions.0.sourceproject}/${ev.actions.0.sourcepackage} to ${ev.actions.0.targetpackage}: ${ev.description} : https://build.pub.meego.com/request/show/${ev.id}', :irc_channel=> '${irc_channel}'
echo "check _if '${project} == ${final_project}'"
_if '"${project}" == "${final_project}"' do
do_final_checks
#else
do_testing_checks
end
end

define 'do_accepted' do
echo "${project} SRCSRV_REQUEST : do_accepted"
get_relevant_changelog :compare => "last_revision"

echo "check _if '${project} == ${final_project}'"
_if '"${project}" == "${final_project}"' do
sequence do
# bugzilla :status => 'RESOLVED', :resolution => 'FIXED', :relevant_bugs => "yes", :template => '/srv/BOSS/bz_comments/meego-DE'
bugzilla :relevant_bugs => "yes", :template => '/srv/BOSS/bz_comments/meego-DE'
get_changelog
revs_de_update
end
#else
bugzilla :relevant_bugs => "yes", :template => '/srv/BOSS/bz_comments/meego-DE'
end
notify_irc :msg => '${ev.who} accepted SR#${ev.id} in ${project} from ${ev.author} ${ev.actions.0.sourceproject}/${ev.actions.0.sourcepackage} to ${ev.actions.0.targetpackage}: ${ev.comment} https://build.pub.meego.com/request/show/${ev.id}', :irc_channel=> '${irc_channel}'
end

define 'do_declined' do
echo "${project} SRCSRV_REQUEST : do_declined"
notify_irc :msg => '${ev.who} declined SR#${ev.id} in ${project} from ${ev.author} ${ev.actions.0.sourceproject}/${ev.actions.0.sourcepackage} to ${ev.actions.0.targetpackage}: ${ev.comment} https://build.pub.meego.com/request/show/${ev.id}', :irc_channel=> '${irc_channel}'
end

define 'do_testing_checks' do
echo "${project} SRCSRV_REQUEST : do_testing_checks"
cursor :break_if => '${__result__} == false' do
check_submitter_maintainer
check_package_built_at_source
# check_package_is_complete
# check_spec
end
_if '${__result__} == false' do
do_decline
#else
do_accept
end
end

define 'do_final_checks' do
echo "${project} SRCSRV_REQUEST : do_final_checks"
cursor :break_if => '${__result__} == false' do
check_submitter_maintainer
# check_package_built_at_source
# check_package_is_complete
# check_spec
end
_if '${__result__} == false' do
do_decline
#else
do_accept
end
end

define 'do_accept' do
echo "${project} SRCSRV_REQUEST : do_accept"
# notify_irc :msg => 'Tests pass for SR#${ev.id} in ${project} from ${ev.author}', :irc_channel=> '${irc_channel}'
end
define 'do_decline' do
echo "${project} SRCSRV_REQUEST : do_decline"
# notify_irc :msg => 'BOSS suggests decline: Tests fail for SR#${ev.id} in ${project} from ${ev.author}', :irc_channel => '${irc_channel}'
end

end

Release Infrastructure/BOSS/Standard workflow

2011-07-18T18:15:14Z

Lbt: Installing the standard_workflow_handler

Community Office/Task Forces/MeeGo Surrounds and Apps

2011-07-14T22:52:24Z

Lbt: Clarified with Dawn. This project is continuing (and is related to Apps)

== MISSION ==
* To ensure a vibrant and quality area for community open source software.
* To have at least one repository which contains software of such quality that vendors (such as Nokia[2]) who would be willing to ship it enabled on devices out-of-the-box.
* To work towards having applications within the community repositories be able to be classed as "MeeGo Compliant".

== SCOPE ==
* Management of the community OBS, the repositories it hosts, and the packages contained within it.
* Policies and procedures (including QA) governing the repositories.
* Align policies where possible, to ease the path for packages from the community repositories to the MeeGo trunk as well as application stores such as Ovi or AppUp.
* Work with the compliance folks on an approach acceptable to all on third-party software compliance[3]

== TEAM ==
* Niels Breet (X-Fade)
* David Greaves (lbt)
* TBC (a MeeGo architect)
* TBC (a developer, like Graham Cobb/Shane Bryan/so many others)
* TBC (someone to help with the work ;-))

== Initial Steps ==

[http://mer-l-in.blogspot.com/2011/01/meego-community-development-apps.html This blog post] attempts to summarise the position as at Jan 2011. The content is intended to be used as a basis for discussion and is being wikified.

* [[Community Office/Task Forces/MeeGo Surrounds and Apps/Apps|Apps]] covers the area around the MeeGo Apps store
* [[Community Office/Task Forces/MeeGo Surrounds and Apps/OBS|OBS]] is about the OBS and policies and technology there
* [[Community Office/Task Forces/MeeGo Surrounds and Apps/Teams|Teams]] is for collaborative team areas
* [[Community Office/Task Forces/MeeGo Surrounds and Apps/Surrounds|Surrounds]] discusses the idea of how typical opensource code sharing can be managed in a distro-like manner around MeeGo.

Release Infrastructure/BOSS/Standard workflow

2011-07-12T22:05:26Z

Lbt: /* Robogrator Process Launcer */

International

2011-07-12T21:07:41Z

Lbt: Proposed policy for language support

MeeGo is an international project and that is reflected in the MeeGo wiki where we will have pages in many languages.

However a wiki should be managed - pages go out-of-date and content sometimes needs to be removed. That's hard to do if you don't understand the language.

== Wiki Language Support Policy ==

The default language for the site is English.

For a new language to be accepted into the wiki the community should have at least 2 (?) active community members to take responsibility for those pages.

For pages with non-English content the page should be categorised with <nowiki>[[Category:Language_French]]</nowiki>
(Use the English word for the language in the category please.)

This page (?) should contain a list of the members responsible for each language and should contain all Language_* categories.

* French : ???
* German :

[[Category:Language_French]]
[[Category:Language_German]]

Packaging/Guidelines

2011-07-05T14:21:19Z

Lbt: clarify "The %changelog% macro must not be used in the .spec"

= Packaging Guidelines =

Guidelines below were adapted for MeeGo from Moblin, OpenSUSE, Fedora and other distributions.

== Maintaining a Package ==
Every package in MeeGo needs a maintainer (AKA owner, bug owner). Any package without an owner will automatically be nominated for deletion from MeeGo. A package maintainer is responsible for making sure that
* packages are up to date with latest upstream
* packages consistently build in the MeeGo build system and fix build failures when they occur
* package meta data in the RPM spec file is accurate
* the license of the package is correct
* she/he follow upstream for any critical security issues and fix them ASAP
* she/he Provides information about major changes to other packagers and maintainer to allow enough time for fixing compatibility issues

Since the data about ownership of packages is not maintained anywhere right now we are starting to use available meta data fields in the build system to track ownership. This will be better integrated and managed at a later point, but to be able to start somewhere MeeGo will use the bugowner key available for every package. We will start adding the metadata about maintainers (bugowners) in the build system and we will have a grace period for this data to be supplied and added to the build system. After the grace period, packages without a maintainer will be reviewed and any packages without a maintainer will be nominated for deletion.

To add yourself as a bugowner of a package, please follow the steps below:
* Update to the most recent osc version (0.131) from the MeeGo tools repository. Note: this is essential, since the needed options are not released upstream yet. Tools: http://repo.meego.com/MeeGo/tools/repos/
* Identify the packages of which you are the ultimate maintainer
* Do the following for every package you maintain in the Trunk:* projects:
<code>
osc reqms --role bugowner <project> <package> -m "I want to own this because I love this package"
</code>
* Your request will be sent and someone in release engineering will approve it

The current status can be seen here: [[Packaging/Maintainers]]

== Package Naming ==
* Dash '-' must be used as the delimiter for name parts.
* Do NOT use an underscore '_', a plus '+', or a period '.' as a delimiter.
* The spec file should be named using the %{name}.spec scheme which should also correspond to the package name within a project in the build system.

== Version and Release ==

Package Versions look like : X.Y.Z-R.B
* X.Y.Z is the 'Version' number - determined by the source package.
* R is the 'Release' number which is automatically incremented by OBS whenever a source/packaging changes (eg a check-in or request acceptance)
* B is the build number which is incremented when the package is rebuilt due to a dependency change.

=== Version ===

The Version field in the spec file is where you should put the current version of the software being packaged.
There are four cases where the version contains non-numeric characters:

* Pre-release packages: Packages released as "pre-release" versions, prior to a "final" version. Example tags include "alpha", "beta", "rc", "cvs", "git", "svn", etc... Details can be found below: Non-Numeric Version.
* Post-release packages: Packages released after a "final" version. These packages contain the same numeric version as the "final" version, but have an additional non-numeric identifier. This mechanism may also be used for packaging only changes to an upstream package.
* Snapshot packages: Packages built from SCM snapshots. These packages could be either "pre" or "post" release packages.

==== Non-Numeric Version ====

We can use letters and tilde into the version tag. We do not use the Release field for this.

Example:
<pre>
Let's assume the following Qt versions:
Qt 4.7.0~beta1
Qt 4.7.0~beta1+git1
Qt 4.7.0~beta2
Qt 4.7.0

Version comparison results:
$ rpmdev-vercmp 4.7.0~beta1 4.7.0~beta1+git1
0:4.7.0~beta1+git1-None is newer

$ rpmdev-vercmp 4.7.0~beta1+git2 4.7.0~beta2
0:4.7.0~beta2-None is newer

$ rpmdev-vercmp 4.7.0~beta2 4.7.0
0:4.7.0-None is newer

Conclusion:
4.7.0~beta1 < 4.7.0~beta1+git1 < 4.7.0~beta2 < 4.7.0
</pre>

Note that the ~ comparison order is specific to MeeGo rpm (http://rpm.org/ticket/56).

=== Release ===

This field is handled by the build system to be able to manage automated builds. The initial setting in the spec file is used by the build system but in many cases it does not need to be changed.

There is no need for the %{dist} macro in the release field. This is also handled directly by the build system.

The release number is set to zero with any version update. It is increased by one with any change in the package.

We can put letters into the version tag, so we do not use the Release field for this. Details can be found above.

If you build the package outside of the OBS or if you copy a package then you will of course not get the correct Release or Build values.

== Tags ==
*The ''Packager'' tag should not be used in spec files. The identities of the packagers are evident from the changelog entries. By not using the ''Packager'' tag, you also avoid seeing bad binaries rebuilt by someone else with your name in the header. See also the '''Maximum RPM definition of the Packager tag''' at [http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html#S3-RPM-INSIDE-PACKAGER-TAG www.rpm.org] . If you need to include information about the packager in the rpms ''you'' built, use <code>%packager</code> in your <code>~/.rpmmacros</code> instead.
*The ''Vendor'' tag should not be used. It is set automatically by the build system.

*Usually, the ''Pre<code></code>Req'' tag should be replaced by plain ''Requires''. For more info, see Maximum RPM snapshot's [http://www.rpm.org/max-rpm-snapshot/s1-rpm-depend-manual-dependencies.html#S3-RPM-DEPEND-FINE-GRAINED fine grained dependencies chapter] .
* The ''Source'' tag documents where to find the upstream sources for the rpm. In most cases this should be a complete URL to the upstream tarball.
=== Summary Tag ===
The summary is a single line string describing the package. The maximum length is 80 characters. It should fit all standard situations and not assume any special context. It should be helpful alone, in alphabetically sorted or unsorted lists of some selected packages, and in alphabetically sorted or unsorted lists of all packages.

It should describe the package's main function and point out any special properties of the package to support the user comparing similar packages. For example, the two words "Web Browser" summarize any web browser, but using additional adjectives (like minimalistic, complex, GNOME, KDE, text-based, fast, or author's) helps characterize a specific package.

The RPM spec file contains only the English version to keep the RPM database small.

*The ''Summary'' tag value should not end in a period. If this bothers you from a grammatical point of view, sit down, take a deep breath, and get over it.

=== Group Tag ===

Valid RPM Groups are:
<pre>
Amusements/Games
Amusements/Graphics
Applications/Archiving
Applications/Communications
Applications/Databases
Applications/Editors
Applications/Emulators
Applications/Engineering
Applications/File
Applications/Internet
Applications/Multimedia
Applications/Productivity
Applications/Publishing
Applications/System
Applications/Text
Development/Debuggers
Development/Languages
Development/Libraries
Development/System
Development/Tools
Documentation
System/Boot
System/Console
System/I18n/Chinese
System/I18n/Japanese
System/I18n/Korean
System/Packages
System/Base
System/Daemons
System/Kernel
System/Libraries
System/Shells
System/X11
System/X11/Fonts
System/X11/Icons
System/GUI/XFCE
System/GUI/Other
System/GUI/GNOME
</pre>

==== Domain/Subsystem based RPM Groups ====

Following the new architecture and the new [http://meego.com/developers/meego-architecture/meego-architecture-domain-view domain view], RPM groups (The Group tag in the RPM) for core packages will be changed to match the domains and any package in the core that is part of one of the domain will have a corresponding group that matches the architecture.

{|
! Domain
! Subsystem
!Groupname
|-
| Communications || Cellular Adaptation || Communications/Cellular Adaptation
|-
| Communications || Cellular Framework || Communications/Cellular Framework
|-
| Communications || Telephony and IM || Communications/Telephony and IM
|-
| Communications || Bluetooth || Communications/Bluetooth
|-
| Communications || Connectivity Adaptation || Communications/Connectivity Adaptation
|-
| Communications || ConnMan || Communications/ConnMan
|-
| Data Management || Content Framework || Data Management/Content Framework
|-
| Development Platform || Platform SDK || Development Platform/Platform SDK
|-
| Essentials || Base Essentials || Essentials/Base Essentials
|-
| Graphics || Font Management || Graphics/Font Management
|-
| Graphics || Display and Graphics Adaptation || Graphics/Display and Graphics Adaptation
|-
| Graphics || Input Adaptation || Graphics/Input Adaptation
|-
| Graphics || Open GL ES || Graphics/Open GL ES
|-
| Graphics || X11 || Graphics/X11
|-
| Kernel || Linux Kernel || Kernel/Linux Kernel
|-
| Location || Location Framework || Location/Location Framework
|-
| Location || Location Adaptation || Location/Location Adaptation
|-
| Multimedia || Audio Adaptation || Multimedia/Audio Adaptation
|-
| Multimedia || Camera Adaptation || Multimedia/Camera Adaptation
|-
| Multimedia || Gstreamer || Multimedia/Gstreamer
|-
| Multimedia || Imaging and Video Adaptation || Multimedia/Imaging and Video Adaptation
|-
| Multimedia || Imaging Codecs || Multimedia/Imaging Codecs
|-
| Multimedia || PulseAudio || Multimedia/PulseAudio
|-
| Multimedia || Sharing || Multimedia/Sharing
|-
| Multimedia || UPnP || Multimedia/UPnP
|-
| Personal Information Management || Backup Framework || Personal Information Management/Backup Framework
|-
| Personal Information Management || Calendar Engine || Personal Information Management/Calendar Engine
|-
| Personal Information Management || Contacts Engine || Personal Information Management/Contacts Engine
|-
| Personal Information Management || Email Engine || Personal Information Management/Email Engine
|-
| Personal Information Management || Synchronization Framework || Personal Information Management/Synchronization Framework
|-
| Qt || Qt || Qt/Qt
|-
| Qt || Qt Mobility || Qt/Qt Mobility
|-
| Qt || Qt WebKit || Qt/Qt WebKit
|-
| Security || Accounts || Security/Accounts
|-
| Security || Certificate Manager || Security/Certificate Manager
|-
| Security || Integrity Protection Framework || Security/Integrity Protection Framework
|-
| Security || Access Control Framework || Security/Access Control Framework
|-
| Security || Single Sign-On || Security/Single Sign-On
|-
| Security || SW Distribution Security || Security/SW Distribution Security
|-
| Security || Security Adaptation || Security/Security Adaptation
|-
| Software Management || Package Manager || Software Management/Package Manager
|-
| System || Context Framework || System/Context Framework
|-
| System || NGF || System/NGF
|-
| System || Resource Policy || System/Resource Policy
|-
| System || Sensor Adaptation || System/Sensor Adaptation
|-
| System || Sensor Framework || System/Sensor Framework
|-
| System || Startup Services || System/Startup Services
|-
| System || System Control || System/System Control
|-
| System || Device Mode Adaptation || System/Device Mode Adaptation
|-
| System || Vibra and Haptics Adaptation || System/Vibra and Haptics Adaptation
|-

|}

=== BuildRoot tag ===
The ''BuildRoot'' value MUST be below <code>%{_tmppath}/</code> and MUST contain at least <code>%{name}</code>, <code>%{version}</code> and <code>%{release}</code>.

The ''recommended'' values for the ''BuildRoot'' tag are (in descending order of preference) :
<pre>
%{_tmppath}/%{name}-%{version}-%{release}-root
</pre>

The BuildRoot tag can be omitted in packages targeting MeeGo only and is handled directly by rpm in MeeGo, for packages that need to run on other distros with older rpm it should be added for backward compatibility.

=== PreReq ===

Packages should not use the PreReq tag. Once upon a time, in dependency loops PreReq used to "win" over the conventional Requires when RPM determined the installation order in a transaction. This is no longer the case.

=== Explicit Requires ===
Packages must not contain explicit ''Requires'' on libraries except when absolutely
necessary. When explicit library ''Requires'' are necessary, there should be a spec file comment justifying it.

We generally rely on rpmbuild to automatically add dependencies on library SONAMEs.
Modern package management tools are capable of resolving such dependencies to determine
the required packages. Explicit dependencies on specific package names may aid the
inexperienced user, who attempts at installing RPM packages manually, however, history
has shown that such dependencies add confusion when library/files are moved from one
package to another, when packages get renamed, when one out of multiple alternative
packages would suffice, and when versioned explicit dependencies become out-of-date and
inaccurate. Additionally, in some cases, old explicit dependencies on package names
require unnecessary updates/rebuilds.

Exemplary rationale for a versioned explicit dependency:
<pre>
# The automatic dependency on libfubar.so.1 is insufficient,
# as we strictly need at least the release that fixes two segfaults.
Requires: libfubar >= 0:1.2.3-7
</pre>

Packagers should revisit an explicit dependency as appropriate to avoid
it becoming inaccurate and superfluous.

=== BuildRequires ===

In package development and testing, please verify that your package is not missing any necessary build dependencies. Having proper build requirements saves the time of all developers and testers as well as build systems because they will not need to search for missing build requirements manually. It is also a safety feature that prevents builds with that would not otherwise fail, but would be missing crucial features. For example, a graphical application may exclude PNG support after its '''configure''' script detects that libpng is not installed.

Before adding Build<code></code>Requires to any package, please be comfortable with [[#Explicit Requires| Requires]] .

=== Patches ===
Each problem should be solved in a separate patch. To allow easy maintenance of patches, every patch should have a header providing the following information:

* Authors' names
* [http://meego.com/about/contribution-guidelines/signed-process Signed-off-by] tag
* Detailed description of the fixed problem
* URL of the original source of the patch if any

The name of a patch file consists of:

* The name and version of the source tarball from which the patched file is derived
* Some words that characterize the patch content
* The filename suffix <code class="filename">.patch</code>

Patches are in the unified format ('''diff -u''') and should be applied with 1 strip level in the spec file ('''%patch -p1'''). The only exceptions are the patches obtained from an another primary source site. The original name, suffix, and format is preserved in this case.

Each patch should be compressed with '''bzip2''' if its size is greater than 100kB. The macros <code class="systemitem">%name</code> and <code class="systemitem">%version</code> should be used whenever possible.

Example:

Source: %{name}-%{version}.tar.bz2
Patch0: %{name}-%{version}-autoconf.patch
Patch1: %{name}-%{version}-gcc31.patch

For the patches to be applied, the patches should be mentioned under %setup. For the above example, this could be done as

%setup -q
%patch0 -p1
%patch1 -p1

Patches have to be marked as such in the spec file and should be applied using the internal patch routines available in rpm. Use of alternate patch management system not supported by rpm is not allowed.

=== %clean ===

The %clean section is not required for MeeGo 1.1 and above. Each package for MeeGo 1.0 MUST have a %clean section, which contains rm -rf %{buildroot} (or $RPM_BUILD_ROOT).

== Documentation ==
Any relevant documentation included in the source distribution should be included in the package. Irrelevant documentation include build instructions, the omnipresent ''INSTALL'' file containing generic build instructions, for example, and documentation for non-Linux systems, e.g. ''README.MSDOS''. Pay also attention about which subpackage you include documentation in, for example API documentation belongs in the -devel subpackage, not the main one. Or if there's a lot of documentation, consider putting it into a subpackage. In this case, it is recommended to use <code>*-doc</code> as the subpackage name, and <code>Documentation</code> as the value of the <code>Group</code> tag.

Also, if a package includes something as <code>%doc</code>, it must not affect the runtime of the application. To summarize: If it is in <code>%doc</code>, the program must run properly if it is not present.

== Devel Packages ==
If the software being packaged contains files intended solely for development, those files should be put in a -devel subpackage. The following are examples of file types which should be in -devel:
* Header files (such as .h files)
* Unversioned shared libraries (such as libfoo.so). Versioned shared libraries (such as libfoo.so.3, libfoo.so.3.0.0) should not be in -devel.

A good rule of thumb is if the file is used for development and not needed for the base package to run properly, it should go in -devel.

=== Requiring Base Package ===
Devel packages must require the base package using a fully versioned dependency: <code>Requires: %{name} = %{version}-%{release}</code>.
Usually, subpackages other than -devel should also require the base package using a fully versioned dependency.

=== Pkgconfig Files ===
The placement of pkgconfig(.pc) files depends on their usecase. Since they are almost always used for development purposes, they should be placed in a -devel package.
A reasonable exception is when the main package itself is a development tool not installed in a user runtime, such as gcc or gdb.

== Test Packages ==
Tests should be included in -test subpackage or separate package according to the following guidelines.

[http://wiki.meego.com/Test_Packaging Test Packaging Guidelines]

== Shared Libraries ==
Whenever possible (and feasible), MeeGo Packages containing libraries should build them as shared libraries. In addition, every binary RPM package which contains shared library files (not just symlinks) in any of the dynamic linker's default paths, must call ldconfig in <code>%post</code> and <code>%postun</code>. If the package has multiple subpackages with libraries, each subpackage should also have a <code>%post/%postun</code> section that calls <code>/sbin/ldconfig</code>. An example of the correct syntax for this is:
<pre>
%post -p /sbin/ldconfig

%postun -p /sbin/ldconfig
</pre>
Note that this specific syntax only works if <code>/sbin/ldconfig</code> is the only call in <code>%post</code> and <code>%postun</code>. If you have additional commands to run during the scriptlet, call <code>/sbin/ldconfig</code> at the beginning of the scriptlet, like this:
<pre>
%post
/sbin/ldconfig
/usr/bin/foo --add

%postun
/sbin/ldconfig
/usr/bin/foo --remove
</pre>

== Configuration files ==

Configuration files must be marked as such in packages.

As a rule of thumb, use <code>%config(noreplace)</code> instead of plain <code>%config</code> unless your best, educated guess is that doing so will break things. In other words, think hard before overwriting local changes in configuration files on package upgrades. An example case when /not/ to use <code>noreplace</code> is when a package's configuration file changes so that the new package revision wouldn't work with the config file from the previous package revision. Whenever plain <code>%config</code> is used, add a brief comment to the specfile explaining why.

Don't use %config or %config(noreplace) under /usr. /usr is deemed to not contain configuration files in MeeGo.

== Initscripts ==

Currently, only SystemV-style initscripts are supported in MeeGo.

== Desktop files ==

If a package contains a GUI application, then it needs to also include a properly installed .desktop file. For the purposes of these guidelines, a GUI application is defined as any application which draws an X window and runs from within that window. Installed .desktop files MUST follow the [http://standards.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html desktop-entry-spec] , paying particular attention to validating correct usage of Name, GenericName, [http://standards.freedesktop.org/menu-spec/latest/apa.html Categories] ,
[http://www.freedesktop.org/Standards/startup-notification-spec StartupNotify]
entries.

=== Icon tag in Desktop Files ===
The icon tag can be specified in two ways:

* Full path to specific icon file:
<code> Icon=/usr/share/pixmaps/comical.png </code>

* Short name without file extension:
<code> Icon=comical </code>

The short name without file extension is preferred, because it allows for icon theming (it assumes .png by default, then tries .svg and finally .xpm), but either method is acceptable.

=== .desktop file creation ===
If the package doesn't already include and install its own .desktop file, you need to make your own. You can do this by including a .desktop file you create as a Source: (such as Source3: %{name}.desktop) or generating it in the spec file. Here are the contents of a sample .desktop file (comical.desktop):

<pre>
[Desktop Entry]
Name=Comical
GenericName=Comic Archive Reader
Comment=Open .cbr & .cbz files
Exec=comical
Icon=comical
Terminal=false
Type=Application
Categories=Graphics;
</pre>

=== Localizing .desktop files ===
The values of Name or GenericName are displayed as captions to the graphical desktop icon, so they should be localized according to the [http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s04.html Desktop Entry Specification]. Most of the time, only ''language'' codes or ''language/country'' codes are needed to select the intended system locale. For example:

<pre>
[Desktop Entry]
Type=Application
Name=Clocks
Name[de]=Uhrzeit
Name[es]=Relojes
Name[fr]=Horloges
Name[pt_BR]=Relógios
Name[zh_CN]=时钟
Icon=meego-app-clocks
Exec=meego-qml-launcher --opengl --fullscreen --app meego-app-clocks
</pre>

In the above .desktop file, <tt>[de]</tt> specifies the German language locale, covering any German-speaking locale, such as <tt>de_DE</tt> or <tt>de_AT</tt>.

'''Note:''' Since a ''language/country'' code (e.g. <tt>pt_BR</tt>) is more specific than a ''language'' code (e.g. <tt>pt</tt>), a string for the <tt>pt_BR</tt> locale will '''not''' be used for the Portuguese language locale (<tt>pt</tt>). If a string is appropriate for every Portuguese locale, you can use "<tt>Name[pt]</tt>" instead.

=== desktop-file-install usage ===
It is not simply enough to just include the .desktop file in the package, one MUST run <code>desktop-file-install</code> OR <code>desktop-file-validate</code> in <code>%install</code> (and have <code>BuildRequires: desktop-file-utils</code>), to help ensure .desktop file safety and spec-compliance. <code>desktop-file-install</code> MUST be used if the package does not install the file or there are changes desired to the .desktop file (such as add/removing categories, etc). <code>desktop-file-validate</code> MAY be used instead if the .desktop file's content/location does not need modification. Here are some examples of
usage:

<pre>
desktop-file-install \
--dir=${RPM_BUILD_ROOT}%{_datadir}/applications \
%{SOURCE3}
</pre>

<pre>
desktop-file-install \
--add-category="AudioVideo" \
--delete-original \
--dir=%{buildroot}%{_datadir}/applications \
%{buildroot}/%{_datadir}/foo.desktop
</pre>

<pre>
desktop-file-validate %{buildroot}/%{_datadir}/applications/foo.desktop
</pre>

=== .desktop file post-install ===

After installing a .desktop file, it's typical to touch the file and the folder so that the UX will detect the change. Otherwise, the new file will not be detected until reboot. I.e.

%post
touch %{_datadir}/applications/foo.desktop
touch %{_datadir}/applications

...will update the time-stamp of the folder and the .desktop file.

== Macros ==
=== Using %{buildroot} and %{optflags} vs $RPM_BUILD_ROOT and $RPM_OPT_FLAGS ===
There are two styles of defining the rpm Build Root and Optimization Flags in a spec file:

{| border="1"
|-
| ||macro style || variable style
|-
|Build Root||%{buildroot}||$RPM_BUILD_ROOT
|-
|Opt. Flags||%{optflags}||$RPM_OPT_FLAGS
|}

There is very little value in choosing one style over the other, since they will resolve to the same values in all scenarios. You should pick a style and use it consistently throughout your packaging.

Mixing the two styles, while valid, is bad from a QA and usability point of view, and should not be done in MeeGo packages.

== Handling Locale Files ==

If the package includes translations, add
<pre>
BuildRequires: gettext
</pre>
If you don't, your package could fail to generate translation files in the buildroot.

MeeGo includes an rpm macro called <code>%find_lang</code>. This macro will locate all of the locale files that belong to your package (by name), and put this list in a file. You can then use that file to include all of the locales. <code>%find_lang</code> should be run in the %install section of your spec file, after all of the files have been installed into the buildroot. The correct syntax for <code>%find_lang</code> is usually:
<pre>
%find_lang %{name}
</pre>
In some cases, the application may use a different "name" for its locales. You may have to look at the locale files and see what they are named. If they are named <code>myapp.mo</code>, then you will need to pass <code>myapp</code> to <code>%find_lang</code> instead of <code>%{name</code>}.
After <code>%find_lang</code> is run, it will generate a file in the active directory (by default, the top level of the source dir). This file will be named based on what you passed as the option to the <code>%find_lang</code> macro. Usually, it will be named <code>%{name}.lang</code>. You should then use this file in the <code>%files</code> list to include the locales detected by <code>%find_lang</code>. To do this, you should include it with the -f parameter to <code>%files</code>.
<pre>
%files -f %{name}.lang
%defattr(-,root,root,-)
%{_bindir}/foobar
...
</pre>
If you are already using the -f parameter for the <code>%files</code> section where the locales should live, just append the contents of <code>%{name}.lang</code> to the end of the file that you are already using with -f. (Note that only one file may be used with <code>%files -f</code>.)

Here is an example of proper usage of <code>%find_lang</code>, in <code>foo.spec</code>:

<pre>
...
%prep
%setup -q

%build
%configure --with-cheese
make %{?_smp_mflags}

%install
make DESTDIR=%{buildroot} install
%find_lang %{name}

%clean
rm -rf %{buildroot}

%files -f %{name}.lang
%defattr(-,root,root,-)
%doc LICENSE README
%{_bindir}/foobar

</pre>

=== Why do we need to use %find_lang? ===

Using <code>%find_lang</code> helps keep the spec file simple, and helps avoid several other packaging mistakes.

* Packages that use <code>%{_datadir}/*</code> to grab all the locale files in one line also grab ownership of the locale directories, which is not permitted.
* Most packages that have locales have lots of locales. Using <code>%find_lang</code> is much easier in the spec file than having to do:
<pre>
%{_datadir}/locale/ar/LC_MESSAGES/%{name}.mo
%{_datadir}/locale/be/LC_MESSAGES/%{name}.mo
%{_datadir}/locale/cs/LC_MESSAGES/%{name}.mo
%{_datadir}/locale/de/LC_MESSAGES/%{name}.mo
%{_datadir}/locale/es/LC_MESSAGES/%{name}.mo
...
</pre>
* As new locale files appear in later package revisions, <code>%find_lang</code> will automatically include them when it is run, preventing you from having to update the spec any more than is necessary.

Keep in mind that usage of <code>%find_lang</code> in packages containing locales is a MUST.

== Scriptlets ==
Great care should be taken when using scriptlets in MeeGo packages. If scriptlets are used, those scriptlets must be sane.

=== Scriptlets requirements ===
Do not use the <code>Requires(pre,post)</code> style notation for scriptlet dependencies, because of two bugs in RPM. Instead, they should be split like this:
<pre>
Requires(pre): ...
Requires(post): ...
</pre>
For more information, see [http://www.redhat.com/archives/fedora-devel-list/2004-April/msg00674.html www.redhat.com] .

=== Running scriptlets only in certain situations ===
When the rpm command executes the scriptlets in a package it indicates if the action preformed is an install, erase, upgrade or reinstall by passing an integer argument to the script in question according to the following:
<pre>
install erase upgrade reinstall
%pre 1 - 2 2
%post 1 - 2 2
%preun - 0 1 -
%postun - 0 1 -
</pre>
This means that for example a package that installs an init script with the <code>chkconfig</code> command should uninstall it only on erase and not upgrade with the following snippet:
<pre>
%preun
if [ $1 -eq 0 ] ; then
/sbin/chkconfig --del %{name}
fi
</pre>
See also <code>/usr/share/doc/rpm-*/triggers</code>, which gives a more formal, generalized definition about the integer value(s) passed to various scripts.

=== Scriplets are only allowed to write in certain directories ===
Build scripts of packages (%prep, %build, %install, %check and %clean) may only alter files (create, modify, delete) under %{buildroot}, %{_builddir} and valid temporary locations like /tmp, /var/tmp (or $TMPDIR or %{_tmppath} as set by the rpmbuild process) according to the following matrix

{| border="1"
|-
| || /tmp, /var/tmp, $TMPDIR, %{_tmppath} || %{_builddir} || %{buildroot}
|-
|%prep || yes || yes || no
|-
|%build || yes || yes || no
|-
|%install || yes || yes || yes
|-
|%check || yes || yes || no
|-
|%clean || yes || yes || yes
|}

Further clarification: That should hold true irrespective of the builder's uid.

== Use of Epochs ==
The Epoch tag in RPM is to be used only as a last resort, and should be avoided whenever possible. However, it is sometimes necessary to use an Epoch to handle upstream versioning changes or to ease transition from third party repositories.

== Writing a package from scratch ==

See [[Spectacle]]

Spectacle is a great tool for straightforward packages, and we have many of those, hundreds, many of those packages already have been using spectacle happily for a while now. Generally, the 80/20 rule applies here, almost 80% of packages in MeeGo can be converted to this format, probably around 20% will need to stay as is for various reasons.

Spectacle in general helps a lot when you have a package that does:
* configure
* make
* make install

and especially useful when for example you have to manage many build dependencies and patches or for common packaging of perl/python/X packages that usually follows the same packaging work flow. We have plans to add lots of nice features to make packaging easier and more fun with spectacle.

While spectacle has many advanced options to cover all kind of corner cases, it should not be used for complex packages that would require lots of customization, especially now that we support multiple architectures and where we need to apply code and custom scripts to support different scenarios.

Spectacle provides scripts to convert spec files to spectacle, those try to do their best but you SHOULD never just take the output as is and rely on the script, a review of the output is necessary, otherwise you might end up with lots of duplication in the spec file. This is the most common mistake, developers are relying on the output of the conversion script, basically picking some spec file from another distro and converting it. This can lead to major disasters in some cases.

So to summarize:
* It is NOT mandatory to use spectacle
* If you try to convert and find yourself spending more than a few minutes on a package, then probably there is something wrong and you should not be using that or you should RTFM.
* Use it with care, especially when you first import the data from existing spec files or when you first create your YAML file
* Your distro maintainer might send you a note that certain packages you are maintaining could be converted to spectacle easily, but she/he should not reject your package because it does not use spectacle.
* If you find yourself forced to edit the spec file manually for some reason, then either:
** your package is not suitable to be used with spectacle
** or you might want to ask for a feature to support that special case
* packager should not change packaging format randomly.
** You need to be the main maintainer
** If a package is already using the yaml format, you need to have a valid reason why not to use the yaml format

== Modifying existing Packages ==

If you base a new package on an existing non-MeeGo package, make sure you verify its correctness of the package and the spec file and to understand exactly what has been done to package the software exactly. Do not submit a package without knowing what those strange, but innocent-looking commands do.

In particular, you should

* ensure that original tarballs are self-contained pristine tarballs. The tarball should not contain symlinks that reference outside the tarball root directory
* verify any sources and patches and remove patches or sources that:
** are related to platforms we do not support (example: sparc, ia64, ppc, ...)
** Implement features we do not support (example: selinux)
** Read every patch and understand what it does, if it is needed, put an explanation in the header justifying why the patch is needed.
* verify that the license stated in the spec file matches the actual license of the software,
* skim the summary and description for typos and oddities (see Summary and description ),
* make sure that the correct build root is used,
* ensure that macro usage is consistent and that the macros are available in MeeGo (see Macros ).

Keep old changelog entries to credit the original authors. Entries that are several years old or refer to ancient versions of the software may be erased. If you end up doing radical changes and re-write most of the spec file anyway, feel free to start the changelog from scratch. In other words, use your best judgement.

== Changelogs ==

This section describes the MeeGo policy for RPM changelogs. (Original changelogs included in the original source are not affected by this policy.)

Please consider that a "normal end user with some technical skills" should be able to read and understand an RPM changelog. Changelog entries have to be in reverse chronological order: newer change log entries are listed above older entries, with the first entry being the most recent.

Please bear in mind that MeeGo changelogs will be automatically parsed to prepare distribution release notes and to report on bugs and CVEs and malformed entries may not be read correctly.

=== General information ===

* MeeGo uses a separate file for package changes which is similar to a debian changelog file. This file is named as the spec file, but ends in *.changes instead of *.spec. The %changelog% section must not be used in the .spec file.
* Entries in the changes file should have the following structure:
<pre>
* dow mmm dd yyyy Name Goes Here <your@email.com> - [version]
- comment
- comment

</pre>

Note the following exceptions are noted through observation:
* version is often omitted (such as: alsa-utils 1.0.15-1 -> 1.0.16-1)
* there are multiple changelog entries per version (such as: alsa-utils 1.0.19)
* the hyphen between email and version is often omitted (such as: alsa-utils)
* spaces between name and <email> are omitted (such as: Zhang, Qiang Z<qiang.z.zhang@intel.com> nano)
* name is sometimes omitted (such as: bitstream-vera-fonts nicolas.mailhot at laposte.net)
* <email> is sometimes omitted (such as: binutils Jim Kingdon)
This wide variation in formats makes automation tasks harder than they should be. Please use the correct format.

=== External References ===

Each external reference (bug numbers etc) should be of the form:
"(" + external reference code + bug number +")"

Currently defined:
* MeeGo Bugs : BMC#
* MeeGo Features: FEA#
* Common Vulnerability / Exposure : CVE

==== Bug Numbers in the change log ====

During maintenance of a distribution, every change has to be marked with the correct bug number. Normally this has to be an number from https://bugzilla.meego.com/. Add an entry with bugzilla number and a short description of the bug-summary. For example:
- Removed invalid desktop Category "Application" (BMC#4654).
- Symlink icon to pixmaps dir (BMC#2108)
- Added gnome-ui-properties to control-center (BMC#1960).

New packages related to new features will refer to the corresponding bug number in bugzilla, preceded with an FEA. For example:
- Adding Qt Contacts support FEA#8011

==== CVE numbers in change log ====

As with bug numbers: Add a short description (normally the CVE summary should be enough), the Bugzilla and the CVE number to the changelog entry. Examples:
- Add gdk-pixbuf-226710.patch (BMC#226710), and (CVE-2007-0010).
- More XPM fixes: (CVE-2005-2975) xpm too many colors DoS (BMC#129642)
- fix ~/.dmrc symlink attack (BMC#180704), (CVE-2006-2449)

=== Spec File changes ===
Be as precise as possible! This is especially important if you remove something from the spec file.

* Removing original source code must be declared in spec file with a comment ("useful for FreeBSD only" for example) - not necessary to repeat the comment in specfile.
* Extra commands (for example during %install) can be illustrated with a short comment in spec file
* Adding/Removing packages from Requires/Provides must be described in the changelog

=== Source Code changes ===

Document the most important changes but limit verbosity.

* look into the source changelog and pick up the most important changes for the distribution (changes for other operation systems are not important). What has changed in the new version, usually in the level of detail of a NEWS file, the change log files are usually too long. More than '''10-15 lines''' shouldn't be necessary to describe the most important changes.
* arrange the original changes behind the version update information. Example:
- Update to 1.3.2:
+ fixes memory leak in import function
+ new API command: unlock_client()
+ the following bugs are closed by this new upstream release:
++ ............ [MGN:332]
++ .............[MGN:337]
- split of devel package
* If upstream does not provide a meaningful change log, then only do best effort. Don't waste too much time over it.
* If the upstream tarball really has not changed except for the version number, just the version number in the change log would be fine. Same goes for packages just containing some graphics or theming (unless upstream already provides something that fits). If the upstream changes just consists of "updated translation" or "several bug fixes" even that can be sufficient for a changelog entry (unless these bug fixes contain something you find worth mentioning).

== Packaging Static Libraries ==
Packages including libraries should exclude static libs as far as possible (eg by configuring with ''--disable-static''). Static libraries should only be included in exceptional circumstances. Applications linking against libraries should as far as possible link against shared libraries not static versions.

Libtool archives, ''foo.la'' files, should not be included. Packages using libtool will install these by default even if you configure with ''--disable-static'', so they may need to be removed before packaging. Due to bugs in older versions of libtool or bugs in programs that use it, there are times when it is not always possible to remove *.la files without modifying the program. In most cases it is fairly easy to work with upstream to fix these issues. Note that if you are updating a library in a stable release (not devel) and the package already contains *.la files, removing the *.la files should be treated as an API/ABI change -- ie: Removing them changes the interface that the library gives to the rest of the world and should not be undertaken lightly.

=== Packaging Static Libraries ===
* In general, packagers are strongly encouraged not to ship static libs unless a compelling reason exists.

* We want to be able to track which packages are using static libraries (so we can find which packages need to be rebuilt if a security flaw in a static library is fixed, for instance). There are two scenarios in which static libraries are packaged:
# '''Static libraries and shared libraries.''' In this case, the static libraries must be placed in a ''*-static'' subpackage. Separating the static libraries from the other development files in ''*-devel'' allow us to track this usage by checking which packages <code>BuildRequire</code> the ''*-static'' package. The intent is that whenever possible, packages will move away from using these static libraries, to the shared libraries.
# '''Static libraries only.''' When a package only provides static libraries you can place all the static library files in the ''*-devel'' subpackage. When doing this you also must have a virtual Provide for the ''*-static'' package:
<pre>%package devel
Provides: foo-static = %{version}-%{release}</pre>

Packages which explicitly need to link against the static version must <code>BuildRequire: foo-static</code>, so that the usage can be tracked.

* If (and only if) a package has shared libraries which require static libraries to be functional, the static libraries can be included in the ''*-devel'' subpackage. The devel subpackage must have a virtual Provide for the ''*-static'' package, and packages dependent on it must <code>BuildRequire</code> the ''*-static'' package.

=== Staticly Linking Executables ===
* Static linkage is a special exception and should be decided on a case-by-case basis. The packager must provide rationale for linking statically, including precedences where available, to release engineering for approval.

[[Category:Packaging]]

Vendor Landing Page/Continuous Integration

2011-07-05T10:49:57Z

Lbt: Created page with "= Vendor Continuous Integration = Continuous Integration against MeeGo follows two basic patterns: * Track an active baseline * Build on a stable baseline The overall CI proces..."

= Vendor Continuous Integration =

Continuous Integration against MeeGo follows two basic patterns:
* Track an active baseline
* Build on a stable baseline

The overall CI process looks like [[File:MeeGo_CI_Process_outline.pdf]]

File:MeeGo CI Process outline.pdf

2011-07-05T10:46:13Z

Lbt:

Community Office/Community device program/Nokia

2011-06-30T19:50:17Z

Lbt: Update for lbt

User:Lbt

2011-06-30T19:47:18Z

Lbt:

==Me==
I'm a longtime and passionate linux user and developer:
* My TV runs linux (MythTV)
* My desktop is linux (Debian)
* My '''wife's''' desktop is linux (Debian)
* My dom0 is linux and so are my domUs
* My car stereos run linux (Empegs)
* My PDAs run linux (Zaurus, N800)
* My hifi runs off linux (Squeezebox, Rio Receiver)
* One day my pictureframe will run linux
* My N800 runs Diablo and Mer
* My SmartQ7 runs Mer
* My N900 runs Fremantle and will run MeeGo
* My Joggler will run MeeGo
* My Wifi routers run linux
I think I have 21 discrete linux machines around the house - and no Windows or Mac

== Projects ==

Things I do:
* manage the [[OBS/Community OBS|Community OBS]] with [[User:Xfade|Niels]]
* proud to be a part of the openSuSE OBS development team
* designed (and develop) [[Release_Infrastructure/BOSS|BOSS]], the process automation system for MeeGo building
* designed (and develop) [[Release_Infrastructure/REVS|REVS]], the package database and RE reporting tool
* support the [[ARM/N900|CE project]]
* work on [[MeeGo Apps]] and [[Community_Office/Task_Forces/MeeGo_Surrounds_and_Extras|MeeGo Surrounds and Extras]]
* part of the [[Meego IT]] team
* working on the [[Vendor Landing Page]]
* Nokia's MeeGo Devices Build Systems Architect
* [http://maemo.org/downloads/product/OS2008/shopper/ Shopper] - A very useful shopping list application that's badly in need of some TLC :)

* I am currently advising Teleca how to setup their OBS and product build processes for LGE.

Things I've done elsewhere:
* I used to be the Mer Build Tools Mentor too. I worked on a lot of the Mer build system including doing cross-compilation in the OBS for Mer and Fremantle.
* I help run the http://raid.wiki.kernel.org/
* I started the git documentation
* I was a lead designer/architect at BT for 9 years -

I'm a strong believer in the social and economic values of OpenSource to the community and to businesses.

contact me at david at dgreaves dot com