Meego Wiki
Views

Build Infrastructure/Community Builder/Installation

From MeeGo wiki
(Difference between revisions)
Jump to: navigation, search
(Add FE)
Line 85: Line 85:
(note JeOS 11.2 won't allow su - <user> or ssh when user is disabled using !
(note JeOS 11.2 won't allow su - <user> or ssh when user is disabled using !
in /etc/shadow. Instead use an impossible hash.)
in /etc/shadow. Instead use an impossible hash.)
 +
 +
== Useful ==
 +
 +
<pre>
 +
chroot_vm() {
 +
GUEST=$1
 +
xm list | grep "^$GUEST " && echo "$GUEST is running" && return
 +
mkdir /mnt/${GUEST}_chroot/
 +
mount /dev/$VG/${GUEST}_root /mnt/${GUEST}_chroot/ &&
 +
chroot /mnt/${GUEST}_chroot/
 +
umount /mnt/${GUEST}_chroot/
 +
rmdir /mnt/${GUEST}_chroot/
 +
}
 +
</pre>
== Make VMs - per VM ==
== Make VMs - per VM ==
Line 299: Line 313:
 +
Install obs-api (It's going to install lighttpd webserver by dependency for you).
 +
<pre>
 +
zypper in obs-api memcached
 +
</pre>
-
== Useful ==
+
=== Setup MySQL ===
 +
 
 +
MySQL server needs to be installed and configured to start as daemon
<pre>
<pre>
-
chroot_vm() {
+
chkconfig --add mysql
-
GUEST=$1
+
rcmysql start
-
xm list | grep "^$GUEST " && echo "$GUEST is running" && return
+
</pre>
-
mkdir /mnt/${GUEST}_chroot/
+
 
-
mount /dev/$VG/${GUEST}_root /mnt/${GUEST}_chroot/ &&
+
Setup a secure installation, if it's the first time starting MySQL
-
chroot /mnt/${GUEST}_chroot/
+
 
-
umount /mnt/${GUEST}_chroot/
+
<pre>
-
rmdir /mnt/${GUEST}_chroot/
+
/usr/bin/mysql_secure_installation
 +
</pre>
 +
 
 +
<pre>
 +
touch /root/.my.cnf
 +
chmod 0600 /root/.my.cnf
 +
vi /root/.my.cnf
 +
 
 +
[client]
 +
user = root
 +
password = <PASSWORD>
 +
[mysqladmin]
 +
user= root
 +
password = <PASSWORD>
 +
</pre>
 +
 
 +
 
 +
The frontend instance holds 2 applications, the API and the webui. Each one need a database created
 +
 
 +
<pre>
 +
mysql -u root -p
 +
create database api_production;
 +
create database webui_production;
 +
</pre>
 +
 
 +
Add obs user to handle these databases
 +
<pre>
 +
GRANT all privileges
 +
      ON api_production.*
 +
      TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
 +
GRANT all privileges
 +
      ON webui_production.*
 +
      TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
 +
FLUSH PRIVILEGES;
 +
</pre>
 +
 
 +
Now secure the passwd storing config files
 +
<pre>
 +
touch /srv/www/obs/api/config/database.yml
 +
touch /srv/www/obs/webui/config/database.yml
 +
chmod 600 /srv/www/obs/api/config/database.yml
 +
chmod 600 /srv/www/obs/webui/config/database.yml
 +
chown lighttpd /srv/www/obs/api/config/database.yml
 +
chown lighttpd /srv/www/obs/webui/config/database.yml
 +
</pre>
 +
 
 +
Configure your MySQL user and password in the "production:" section of the API config:
 +
<pre>
 +
vi /srv/www/obs/api/config/database.yml
 +
#change the production section
 +
production:
 +
  adapter: mysql
 +
  database: api_production
 +
  username: obs
 +
  password: ************
 +
</pre>
 +
 
 +
Do the same for the webui. It's configured, by default to use SQLite, but since we're configuring the cluster for production environment, let's bind it to mysql:
 +
<pre>
 +
vi /srv/www/obs/webui/config/database.yml
 +
#change the production section
 +
production:
 +
  adapter: mysql
 +
  database: webui_production
 +
  username: obs
 +
  password: ************
 +
</pre>
 +
 
 +
Populate the database
 +
<pre>
 +
mkdir -p /srv/www/obs/api/db/data/production
 +
cd /srv/www/obs/api/
 +
RAILS_ENV="production" rake db:migrate
 +
chown lighttpd.lighttpd log/*
 +
 
 +
cd /srv/www/obs/webui/
 +
RAILS_ENV="production" rake db:migrate
 +
chown lighttpd.lighttpd log/*
 +
</pre>
 +
 
 +
You can check the migration was successful verifying the “migrated” message at the end of each statement.
 +
 
 +
Setup and configure lighttpd for the API and webui
 +
 
 +
You need to setup the correct hostnames to where webui, API and repo server are going to point to
 +
 
 +
Edit /etc/lighttpd/vhosts.d/obs.conf
 +
<pre>
 +
$HTTP["host"] =~ "^cbuild" {
 +
  rails_app  = "webui"
 +
  rails_root  = "/srv/www/obs/webui"
 +
  rails_procs = 10
 +
  # production/development are typical values here
 +
  rails_mode  = "production"
 +
 +
  log_root = "/srv/www/obs/webui/log"
 +
 +
  include "vhosts.d/rails.inc"
 +
}
 +
$HTTP["host"] =~ "^capi" {
 +
  rails_app  = "api"
 +
  rails_root  = "/srv/www/obs/api"
 +
  rails_procs = 3
 +
  # production/development are typical values here
 +
  rails_mode  = "production"
 +
 +
  log_root = "/srv/www/obs/api/log"
 +
 +
  include "vhosts.d/rails.inc"
 +
}
 +
$HTTP["host"] =~ "download" {
 +
# This should point to an rsync populated download repo
 +
#  server.name = "download.obs.maemo.org"
 +
#  server.document-root = "/srv/obs/repos/"
 +
 +
  proxy.server = ( "" => ( (
 +
        "host" => "10.1.1.11",
 +
        "port" => 80
 +
      ))
 +
  )
}
}
 +
</pre>
 +
 +
To enable these vhosts, make sure to '''uncomment''' the following in the 'custom includes' section at the bottom of /etc/lighttpd/lighttpd.conf:
 +
<pre>
 +
vi /etc/lighttpd/lighttpd.conf
 +
##
 +
## custom includes like vhosts.
 +
##
 +
#include "conf.d/config.conf"
 +
include_shell "cat /etc/lighttpd/vhosts.d/*.conf"
</pre>
</pre>
 +
Also need to disable IPv6 unkess it's secured
<pre>
<pre>
-
zypper in obs-api memcached
+
server.use-ipv6 = "disable"
</pre>
</pre>
 +
 +
Also, the modules "mod_magnet", "mod_rewrite" and FastCGI need to be enabled by uncommenting the corresponding lines in /etc/lighttpd/modules.conf:
 +
 +
<pre>
 +
server.modules = (
 +
  "mod_access",
 +
#  "mod_alias",
 +
#  "mod_auth",
 +
#  "mod_evasive",
 +
#  "mod_redirect",
 +
  "mod_rewrite",
 +
#  "mod_setenv",
 +
#  "mod_usertrack",
 +
)
 +
 +
##
 +
## mod_magnet
 +
##
 +
include "conf.d/magnet.conf"
 +
 +
##
 +
## FastCGI (mod_fastcgi)
 +
##
 +
include "conf.d/fastcgi.conf"
 +
</pre>
 +
 +
You need also to configure /srv/www/obs/webui/config/environments/production.rb to point to correct server names:
 +
 +
<pre>
 +
vi /srv/www/obs/webui/config/environments/production.rb
 +
FRONTEND_HOST = "capi.meego.com"
 +
FRONTEND_PORT = 80
 +
EXTERNAL_FRONTEND_HOST = "capi.meego.com"
 +
BUGZILLA_HOST = "http://bugs.moego.com/"
 +
DOWNLOAD_URL = "http://cdownload.meego.com/"
 +
</pre>
 +
 +
Do the same for /srv/www/obs/api/config/environments/production.rb. As soon your backend is not on the same machine as the api (frontend), change the following:
 +
 +
<pre>
 +
vi /srv/www/obs/api/config/environments/production.rb
 +
SOURCE_HOST = "cbe.meego.com"
 +
SOURCE_PORT = 5352
 +
DOWNLOAD_URL='http://cdownload.meego.com/'
 +
</pre>
 +
 +
 +
ligthttpd user and group need to be the owner of api and webui dirs (as well as log and tmp):
 +
 +
<pre>
 +
chown -R lighttpd.lighttpd /srv/www/obs/{api,webui}
 +
</pre>
 +
 +
Make sure TCP port 5352 is open on the firewall. Ensure lighttpd and obs ui helpers start:
 +
 +
<pre>
 +
chkconfig --add memcached
 +
chkconfig --add lighttpd
 +
chkconfig --add obsapidelayed
 +
chkconfig --add obswebuidelayed
 +
 +
rcmemcached start
 +
rclighttpd start
 +
rcobsapidelayed start
 +
rcobswebuidelayed start
 +
</pre>
 +
 +
rcobsapidelayed

Revision as of 07:32, 2 August 2010

Contents

Preparing Host

Starting with a minimal Suse 11.2 install

Define some base data 

ROOTFS=/data/11.2min/image-root
ROUTER_IP=10.0.0.1
VG=VM

Based on http://en.opensuse.org/Build_Service/KIWI/Cookbook 

zypper ar http://download.opensuse.org/repositories/Virtualization:/Appliances/openSUSE_11.2/ Virtualization:Appliances
zypper refresh

zypper in kiwi kiwi-templates kiwi-desc-xenboot squashfs emacs

Prepare the storage for LV usage 

parted /ddev/sdb
 mklabel
 gpt
 mkpart p1 0 10%
 mkpart p2 10% 20%
 mkpart p3 20% 30%
 mkpart p4 30% 40%
 mkpart p5 40% 50%
 mkpart p6 50% 60%
 mkpart p7 60% 70%
 mkpart p8 70% 80%
 mkpart p9 80% 90%
 mkpart p10 90% 100%
 quit

Then make the VG 

pvcreate /dev/sdb?*
vgcreate $VG /dev/sdb1

Prepare an openSUSE minimal image: 

mkdir -p /data/11.2min
rm -rf /data/11.2min/image-root
kiwi --prepare suse-11.2-JeOS --root $ROOTFS --add-profile xenFlavour --add-package less --add-package iputils --add-package kernel-xen --add-package wget --add-package less --add-package iputils --add-package terminfo --add-package emacs --add-package sudo

Update the config & modules: 

echo default $ROUTER_IP > $ROOTFS/etc/sysconfig/network/routes
echo NETCONFIG_DNS_POLICY=\"\" >> $ROOTFS/etc/sysconfig/network/config
echo nameserver 8.8.8.8 > $ROOTFS/etc/resolv.conf
echo default $ROUTER_IP > $ROOTFS/etc/sysconfig/network/routes
cat << EOF >$ROOTFS/etc/sysconfig/network/ifcfg-eth0
BOOTPROTO='static'
BROADCAST=''
STARTMODE='onboot'
EOF
echo /dev/xvda1 swap swap defaults 0 0 >> $ROOTFS/etc/fstab


Prepare some overlay data from the main host to allow ssh into guests etc 

# Allow user ssh to all VMs and retain sudo rights
# Should probably be done periodically somehow
mkdir /data/vm_overlay/home
cp -ar /home /data/vm_overlay/home
mkdir -p /data/vm_overlay/etc/sysconfig/
ln /etc/passwd /data/vm_overlay/etc/
ln /etc/shadow /data/vm_overlay/etc/
ln /etc/group /data/vm_overlay/etc/
ln /etc/sudoers /data/vm_overlay/etc/
# Fix for screen/bash ctrl-arrow
ln /etc/inputrc /data/vm_overlay/etc/
# Network proxy information
ln /etc/sysconfig/proxy /data/vm_overlay/etc/sysconfig/

(note JeOS 11.2 won't allow su - <user> or ssh when user is disabled using ! in /etc/shadow. Instead use an impossible hash.)

Useful 

chroot_vm() {
GUEST=$1
xm list | grep "^$GUEST " && echo "$GUEST is running" && return
mkdir /mnt/${GUEST}_chroot/
mount /dev/$VG/${GUEST}_root /mnt/${GUEST}_chroot/ &&
chroot /mnt/${GUEST}_chroot/
umount /mnt/${GUEST}_chroot/
rmdir /mnt/${GUEST}_chroot/
}

Make VMs - per VM

Make sure you setup the base data environment 

VG=VM
FE_IP=10.0.0.10
BE_IP=10.0.0.11

Create Xen volumes 

mk_lv() {
GUEST=$1
lvremove /dev/$VG/${GUEST}_*
lvcreate -L 10G $VG -n ${GUEST}_root
lvcreate -L 2G  $VG -n ${GUEST}_swap
mkswap -f /dev/$VG/${GUEST}_swap
}

For cfe & cbe 

mk_lv cfe
mk_lv cbe

Copy the minimal image and overlay to the VM root disk and set an IP 

mk_fs() {
GUEST=$1
IP=$2
mkdir /mnt/lvm
echo mkfs &&
mkfs -text4 /dev/$VG/${GUEST}_root &&
echo mounting &&
mount /dev/$VG/${GUEST}_root /mnt/lvm &&
echo copy rootfs &&
rsync -HAXa /data/11.2min/image-root/ /mnt/lvm/ &&
echo copy overlay &&
echo ${GUEST}.meego.com > /mnt/lvm/etc/HOSTNAME &&
echo "IPADDR='$IP/24'" >> /mnt/lvm/etc/sysconfig/network/ifcfg-eth0 &&
rsync -HAXa /data/vm_overlay/ /mnt/lvm/ &&
echo sync &&
sync &&
echo umount &&
umount /mnt/lvm
}

For cfe & cbe 

mk_fs cfe $IP_FE
mk_fs cbe $IP_BE

Make per-machine files in /etc/xen/ with unique MACs Additional LV space can be allocated here too 

mk_g() {
GUEST=$1
MAC=$2
cat <<EOF > /etc/xen/$GUEST.cfg
name='${GUEST}'
disk=['phy:/dev/$VG/${GUEST}_root,xvda2,w', 'phy:/dev/$VG/${GUEST}_swap,xvda1,w']
vif=['mac=$MAC, bridge=eth0']
memory='2048'

root='/dev/xvda2 rw'
kernel='/boot/vmlinuz-2.6.31.12-0.2-xen'
ramdisk='/boot/initrd-2.6.31.12-0.2-xen'
extra='clocksource=jiffies console=hvc0 xencons=tty'

on_poweroff='destroy'
on_reboot='restart'
on_crash='restart'
EOF
}

For cfe & cbe 

mk_g cfe 00:16:3E:40:B5:FE
mk_g cbe 00:16:3E:40:B5:BE

Then start the VMs: 

xm create /etc/xen/cfe.cfg
xm create /etc/xen/cbe.cfg

Installing the Backend

On this host we need also to setup openSUSE Tools repository: 

cd /etc/zypp/repos.d/;
wget http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra:/OBS/Tools_Unstable_openSUSE_11.2/Maemo:MeeGo-Infra:OBS.repo
wget http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.2/openSUSE:Tools:Unstable.repo
wget http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key
<pre>
Install:
<pre>
zypper in obs-server obs-source_service obs-signer obs-utils createrepo dpkg lighttpd



vi /etc/sysconfig/obs-server
OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv5el armv7el"

Now /usr/lib/obs/server/BSConfig.pm needs to point to correct server names corresponding to source server, where workers are going to download the source, and the repository server, where RPM repos are going to be shared to users.

What's needed here????????? 

vi /usr/lib/obs/server/BSConfig.pm
#add
$hostname="cbuild.meego.com";

our $srcserver = "http://$hostname:5352";
our $reposerver = "http://$hostname:5252";
our $serviceserver = "http://$hostname:5152";
our $servicedir = "/usr/lib/obs/service/";
#

Configure services as daemons 

chkconfig --add obsrepserver obssrcserver obsscheduler obsdispatcher obspublisher obswarden obssigner

Start Services 

rcobsrepserver start
rcobssrcserver start
rcobsscheduler start
rcobsdispatcher start
rcobspublisher start
rcobswarden start
rcobssigner start

Not started 

rcobsservice

Lighttpd

lighttpd also needs to be available on backend server. This is required to provide directory listing on the repositories available on this server when an http/s request to maemo-repo is made through web ui.

Create a new file under /etc/lighttpd/vhosts.d/. It can be obs.conf as well, and add: 

vi /etc/lighttpd/vhosts.d/obs.conf

$HTTP["host"] =~ "crepo.meego.com" {
  server.name = "crepo.meego.com"

  server.document-root = "/srv/obs/repos/"
  dir-listing.activate = "enable"
}

To enable vhosts, remember to uncomment the following in the 'custom includes': 

vi /etc/lighttpd/lighttpd.conf
##
  ## custom includes like vhosts.
  ##
  #include "conf.d/config.conf"
  # following line uncommented as per
  # /usr/share/doc/packages/obs-api/README.SETUP
  include_shell "cat vhosts.d/*.conf"

And disable ipv6 unless it's secured correctly 

server.use-ipv6 = "disable"

Start lighttpd 

#first add it as deamon
chkconfig --add lighttpd
rclighttpd start






Installing the Frontend

On this host we need also to setup openSUSE Tools repository: 

cd /etc/zypp/repos.d/;
wget http://download.opensuse.org/repositories/Maemo:/MeeGo-Infra:/OBS/Tools_Unstable_openSUSE_11.2/Maemo:MeeGo-Infra:OBS.repo
wget http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.2/openSUSE:Tools:Unstable.repo
wget http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/openSUSE:Tools.repo
zypper ref
# Accept the trust key
<pre>


Install obs-api (It's going to install lighttpd webserver by dependency for you). 
<pre>
zypper in obs-api memcached

Setup MySQL

MySQL server needs to be installed and configured to start as daemon 

chkconfig --add mysql
rcmysql start

Setup a secure installation, if it's the first time starting MySQL 

/usr/bin/mysql_secure_installation

touch /root/.my.cnf
chmod 0600 /root/.my.cnf
vi /root/.my.cnf

[client]
user = root
password = <PASSWORD>
[mysqladmin]
user= root
password = <PASSWORD>


The frontend instance holds 2 applications, the API and the webui. Each one need a database created 

mysql -u root -p
 create database api_production;
 create database webui_production;

Add obs user to handle these databases 

GRANT all privileges
      ON api_production.* 
      TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
GRANT all privileges
      ON webui_production.* 
      TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
FLUSH PRIVILEGES;

Now secure the passwd storing config files 

touch /srv/www/obs/api/config/database.yml
touch /srv/www/obs/webui/config/database.yml
chmod 600 /srv/www/obs/api/config/database.yml
chmod 600 /srv/www/obs/webui/config/database.yml
chown lighttpd /srv/www/obs/api/config/database.yml
chown lighttpd /srv/www/obs/webui/config/database.yml

Configure your MySQL user and password in the "production:" section of the API config: 

vi /srv/www/obs/api/config/database.yml
#change the production section
production:
  adapter: mysql
  database: api_production
  username: obs
  password: ************

Do the same for the webui. It's configured, by default to use SQLite, but since we're configuring the cluster for production environment, let's bind it to mysql: 

vi /srv/www/obs/webui/config/database.yml
#change the production section
production:
  adapter: mysql
  database: webui_production
  username: obs
  password: ************

Populate the database 

mkdir -p /srv/www/obs/api/db/data/production
cd /srv/www/obs/api/
RAILS_ENV="production" rake db:migrate
chown lighttpd.lighttpd log/*

cd /srv/www/obs/webui/
RAILS_ENV="production" rake db:migrate
chown lighttpd.lighttpd log/*

You can check the migration was successful verifying the “migrated” message at the end of each statement.

Setup and configure lighttpd for the API and webui

You need to setup the correct hostnames to where webui, API and repo server are going to point to

Edit /etc/lighttpd/vhosts.d/obs.conf 

$HTTP["host"] =~ "^cbuild" {
  rails_app   = "webui"
  rails_root  = "/srv/www/obs/webui"
  rails_procs = 10
  # production/development are typical values here
  rails_mode  = "production"
 
  log_root = "/srv/www/obs/webui/log"
 
  include "vhosts.d/rails.inc"
}
$HTTP["host"] =~ "^capi" {
  rails_app   = "api"
  rails_root  = "/srv/www/obs/api"
  rails_procs = 3
  # production/development are typical values here
  rails_mode  = "production"
 
  log_root = "/srv/www/obs/api/log"
 
  include "vhosts.d/rails.inc"
}
$HTTP["host"] =~ "download" {
# This should point to an rsync populated download repo
#  server.name = "download.obs.maemo.org"
#  server.document-root = "/srv/obs/repos/"
 
  proxy.server = ( "" => ( (
        "host" => "10.1.1.11",
        "port" => 80
      ))
  )
}

To enable these vhosts, make sure to uncomment the following in the 'custom includes' section at the bottom of /etc/lighttpd/lighttpd.conf: 

vi /etc/lighttpd/lighttpd.conf
##
## custom includes like vhosts.
##
#include "conf.d/config.conf"
include_shell "cat /etc/lighttpd/vhosts.d/*.conf"

Also need to disable IPv6 unkess it's secured 

server.use-ipv6 = "disable"

Also, the modules "mod_magnet", "mod_rewrite" and FastCGI need to be enabled by uncommenting the corresponding lines in /etc/lighttpd/modules.conf: 

server.modules = (
  "mod_access",
#  "mod_alias",
#  "mod_auth",
#  "mod_evasive",
#  "mod_redirect",
  "mod_rewrite",
#  "mod_setenv",
#  "mod_usertrack",
)

##
## mod_magnet
##
include "conf.d/magnet.conf"

##
## FastCGI (mod_fastcgi)
##
include "conf.d/fastcgi.conf"

You need also to configure /srv/www/obs/webui/config/environments/production.rb to point to correct server names: 

vi /srv/www/obs/webui/config/environments/production.rb
FRONTEND_HOST = "capi.meego.com"
FRONTEND_PORT = 80
EXTERNAL_FRONTEND_HOST = "capi.meego.com"
BUGZILLA_HOST = "http://bugs.moego.com/"
DOWNLOAD_URL = "http://cdownload.meego.com/"

Do the same for /srv/www/obs/api/config/environments/production.rb. As soon your backend is not on the same machine as the api (frontend), change the following: 

vi /srv/www/obs/api/config/environments/production.rb
SOURCE_HOST = "cbe.meego.com"
SOURCE_PORT = 5352
DOWNLOAD_URL='http://cdownload.meego.com/'


ligthttpd user and group need to be the owner of api and webui dirs (as well as log and tmp): 

chown -R lighttpd.lighttpd /srv/www/obs/{api,webui}

Make sure TCP port 5352 is open on the firewall. Ensure lighttpd and obs ui helpers start: 

chkconfig --add memcached
chkconfig --add lighttpd
chkconfig --add obsapidelayed
chkconfig --add obswebuidelayed
 
rcmemcached start
rclighttpd start
rcobsapidelayed start
rcobswebuidelayed start

rcobsapidelayed

Retrieved from "http://wiki.meego.com/Build_Infrastructure/Community_Builder/Installation"
Personal tools