Latest revision as of 10:18, 25 July 2011

This page provides some technical scope for the core BOSS components.

BOSS Core covers:

• BOSS (A ruote worker daemon and RabbitMQ integration)
• SkyNET (daemontools based participant pool manager)
• BOSS-viewer (System management Web UI)
• Standard workflow
• OBS integration (Robotic integrator)

Additional systems such as REVS and IMG are handled independently.

BOSS Scope / Technical Roadmap

Out-of-the Box Goals

• (x) Install guide for : BOSS, SkyNET, BOSS-Viewer, OBS plugin
• (x) Sample Process
• (x) Sample Participant
• (x) Sample Launcher

BOSS Installable Components

BOSS engine

• (x) Packaged for Suse
• (x) Packaged for Debian
• (x) Rewrite to use daemontools to manage ruote worker instead of daemon-kit
• ( ) Move to Redis based storage (for replication and general HA)
• (x) Support participant registration process
• (x) Support start/stop
• (x) Support boss.config ini file

Web based monitor/control

• (x) Packaged for Suse
• (x) Packaged for Debian
• ( ) Logging view
• ( ) Anonymous view
• (x) admin user control
• ( ) normal user control
• ( ) Rails based to allow OBS integration

SkyNET Features:

Document design and functionality

• ( ) System overview
• (x) Installation
• (.) Access control behaviour (given user-owned participants)

Feature complete CLI for:

• (x) Packaged for Suse
• (x) Packaged for Debian
• (x) Install Participant
  • (x) support multiple instances
• (x) Uninstall Participant
  • (x) support multiple instances
• (x) Start / Stop Participant
  • (x) support multiple instances
• (x) System start/shutdown
• (x) Register participant
• (x) List of participants with 'active/registered'
• ( ) Info command (location, installation package, yaml derived description)
• ( ) Improved upgrade support (reload if running, reinstall, rebuild templates)
• ( ) Ruby client to make it more relevant upstream

maybe rewrite skynet in python and provide a REST api or something, which would allow a ruby client as well as a webui

• ( ) Skynet unix group so normal users can activate participants

Python Participant (Exo) library

• (x) Packaged for Suse
• (x) Packaged for Debian
• (x) Workitem handling
• ( ) Workitem Control handling (cancel/status)
• ( ) Lifecycle handling
• ( ) Provide a Exo.Verify() class with assertNotNull, assertIsList(v, default), assertHasRun(particpant) (now in change_request_state)
• ( ) ProcessException hierarchy (replace RuntimeError("Missing mandatory field")

Python ruote client library

• (x) Ruote workitem support
• (x) Simple workitem useage
• ( ) Migration to Pika for AMQP
• ( ) Use SSL with RabbitMQ http://www.rabbitmq.com/ssl.html
• ( ) Support cancel/pause/resume
• ( ) Complete documentation
• ( ) 100% test coverage

REST API to participant status

Note combining 'install' and 'run' and 'root' is a security issue.

• ( ) Packaged for Suse
• ( ) Packaged for Debian
• ( ) GET : status
• ( ) PUT : install
• ( ) POST : Running=True/False
• ( ) DELETE : uninstall

Clustering ability (ie see multiple SkyNETs in one view)

• ( ) Follows on from REST

Process Management:

Ruote On Rails:

An web framework for workflow interaction

• ( ) Packaged for Suse
• ( ) Packaged for Debian
• ( ) Integrate with OBS

Standard Workflow

• (x) Develop and document CI workflow
• (x) Integrate robogrator to OBS (per project workflows)
• ( ) Extend robogrator process/event manager
• (.) Store workflows in database (Redis) for remote access
• ( ) Allow multiple projects to share process components
• ( ) Improve documentation
• ( ) Best practice guidelines
• ( ) Participant libraries :
• (.) Package QA checks
• ( ) git
• (.) OBS Project
• (.) OBS Requests
• (.) REVS
• ( ) OTS
• (.) IMG
• (.) Bugzilla

User Interaction Participant

• ( ) Packaged for Suse
• ( ) Packaged for Debian
• ( ) Simple standalone web query
• ( ) Integrate with OBS
• (.) Extend to irc/jabber

Test and debugging framework

• ( ) Policy / best practice
• ( ) Trace and profile support

Notification Framework

• ( ) Possibly use Hermes
• (x) irc notifier
• ( ) jabber notifier
• ( ) email notifier
• ( ) link to backend user preferences
• ( ) configure per-project (tree?)

OBS

UI

• ( ) Integrate BOSSS workflow into OBS UI

State model

• ( ) Define state for projects/repos/packages
• ( ) Define state for requests
• ( ) ensure OBS events api is supported upstream

High Level Outstanding Areas

Security

• (x) Run participants as configurable users
• (x) Run participants on remote machines
• ( ) Granular AMQP participant / access control
• ( ) Enable SSL AMQP connections
• ( ) Use client certs for authentication
• ( ) Ruote message signing system

Management

• ( ) Monitoring processes
• ( ) Monitoring participants
• ( ) Upgrading

Documentation

• ( ) Best practice
• ( ) Wiki
• ( ) Presentation
• ( ) Reference (BOSS process driven from src)

Dashboard

The entire BOSS area needs to have a coherent UI view.

• ( ) Project status summary
• ( ) Package status

BOSS Management

Setting Direction

The BOSS roadmap is, of course, driven by customer needs. The technical scope and current statment of capabilities outlined above were identified during work on earlier projects within Nokia; but the order of implementation depends on prioritisation - and that will be driven by upcoming users.

Support MeeGo N900 DE Project

The DE project is an active customer whose needs are representative of general CI and release engineering BOSS customers. By focusing on them we ensure that BOSS development is both driven and driven in the right direction.

Support MeeGo:Apps

MeeGo Apps is a community project which has a more user-facing process. This allows BOSS development to integrate into other UI systems.

MeeGo.com Deployment

Supporting BOSS on MeeGo.com allows us to manage system administration issues such as reliability and visibility.

Immediate Roadmap

Bugzilla integration (Target: Pre SF2011)

• (.) Process definition
• (.) Identity relevant changelog data
• (.) Extract bugs and comment
• (.) Change bug state

Repo publish process (Target Mid-June 2011)

• (.) Triggered by time
• ( ) Triggered by irc bot
• ( ) Freeze repo building
• ( ) Rsync
• ( ) Publish
• ( ) Notify

Monitoring (Target End-June 2011)

• (.) Process viewing
• ( ) Alerts on process failure

Image Generation (Target Mid-June 2011)

• ( ) DE Image generation

VCS integration (Target End-June 2011)

• ( ) VCS monitor for gitorious and github

BOSS UI Process Integration (Target July 2011)

• ( ) Depends on Robogrator v2 for network DB
• ( ) Tab to view current process definitions active in a project

Author: David Greaves

Date: 2011-05-08 13:54:12 BST

HTML generated by org-mode 7.3 in emacs 23