Build Infrastructure/Sysadmin Distro/OBS setup openSUSE114

Revision as of 15:53, 12 August 2011

Please check http://gitorious.org/opensuse/build-service/blobs/master/dist/README.SETUP
for the latest setup instructions!

These steps show how to setup Open Build Service (OBS) on an openSuSE 11.4 installation. The OBS installation will consist of a main OBS server and an OBS worker. This document describes how to setup an evaluation/test setup and is not meant as production setup.

Most users won't need to install their own OBS and can just use an account on a publicly accessible OBS.

There are cases where vendors may rather have full control of their builds and keep source private due to licensing restrictions, etc. These instructions are for these special cases.

Add Repositories

First, we have to add the openSUSE:Tools repository to both OBS server and OBS worker:

     # zypper addrepo http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.4/openSUSE:Tools.repo
     # zypper refresh

When zypper asks, accept the trust key.

Setup OBS Server

Install the Server packages

     # zypper install obs-server obs-signd obs-utils createrepo nfs-client obs-api memcached lighttpd

If you installed a minimal openSUSE installation then python packages may conflict with the minimal configuration package patterns-openSUSE-minimal_base. This package can be safely removed if it causes dependency problems.

MySQL Setup

Start daemon by default:

     # chkconfig --add mysql
     # rcmysql start

Secure the installation:

     # /usr/bin/mysql_secure_installation

Create the databases:

    # mysql -u root -p
    mysql> create database api_production;
    mysql> create database webui_production;

Add the obs user for these databases (substitute ********** with your password):

     mysql> GRANT all privileges
            ON api_production.* 
            TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
     mysql> GRANT all privileges
            ON webui_production.* 
            TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
     mysql> FLUSH PRIVILEGES;
     mysql> quit

Now, configure OBS to use MySQL. First the API server:

     # vi /srv/www/obs/api/config/database.yml
     production:
       adapter: mysql
       database: api_production
       username: obs
       password: ************

And setup the webui to use MySQL:

     # vi /srv/www/obs/webui/config/database.yml
     production:
       adapter: mysql
       database: webui_production
       username: obs
       password: ************

Populate the database:

     # cd /srv/www/obs/api/
     # RAILS_ENV="production"  rake db:setup
     # cd /srv/www/obs/webui/
     # RAILS_ENV="production" rake db:setup

If this fails, check /srv/www/obs/api/config/environments/production.rb and /srv/www/obs/webui/config/environments/production.rb for configuration of the SOURCE_HOST and FRONTEND_HOST.

Setup lighttpd for webui

Make sure vhosts are enabled by uncommenting the following line:

      # vi /etc/lighttpd/lighttpd.conf
      include_shell "cat /etc/lighttpd/vhosts.d/*.conf"

The installed /etc/lighttpd/vhosts.d/obs.conf should be usable for this setup but it can be changed later on.

lighttpd requires some extra modules to be enabled: mod_access, mod_rewrite, mod_magnet, and mod_fastcgi. Do so by editing /etc/lighttpd/modules.conf:

     # vi /etc/lighttpd/modules.conf
   server.modules = (
     "mod_access",
   #  "mod_alias",
   #  "mod_auth",
   #  "mod_evasive",
   #  "mod_redirect",
     "mod_rewrite",
   #  "mod_setenv",
   #  "mod_usertrack",
   )

   ##
   ## mod_magnet
   ##
   include "conf.d/magnet.conf"

   ##
   ## FastCGI (mod_fastcgi)
   ##
   include "conf.d/fastcgi.conf"

You need also to configure /srv/www/obs/webui/config/environments/production.rb to point to correct server names:

     # vi /srv/www/obs/webui/config/environments/production.rb
     FRONTEND_HOST = "192.168.1.1"
     FRONTEND_PORT = 81
     FRONTEND_PROTOCOL = 'http'
     BUGZILLA_HOST = nil
     DOWNLOAD_URL = "http://192.168.1.1:82"
     ICHAIN_MODE = "off"
     BASE_NAMESPACE = nil

Do the same for /srv/www/obs/api/config/environments/production.rb. As soon your backend is not on the same machine as the api (frontend), change the following:

     # vi /srv/www/obs/api/config/environments/production.rb
     SOURCE_HOST = "192.168.1.1"

Add the server to start by default.

     # chkconfig --add memcached
     # chkconfig --add lighttpd
     # chkconfig --add obsapidelayed
     # chkconfig --add obswebuidelayed

ligthttpd user and group need to be the onwer of api and webui dirs (as well as log and tmp):

     # chown -R lighttpd.lighttpd /srv/www/obs/{api,webui}

BSConfig.pm

The file /usr/lib/obs/server/BSConfig.pm is used to configure several settings for the Build Service. One of the major components is setting up the $hostname variable to the correct value. Currently I have this set to the IP address of the server.

      # vi /usr/lib/obs/server/BSConfig.pm
      my $hostname = '192.168.1.1';

The list of directories in /srv/obs/repos/ is the repositories, so I changed the $repodownload:

      # vi /usr/lib/obs/server/BSConfig.pm
      our $repodownload = "http://$hostname:82/";

OBS API custom logger seems to need a small fix to get going:

--- /srv/www/obs/api/lib/custom_logger.rb-orig  2011-08-12 03:12:25.195998605 +0200
+++ /srv/www/obs/api/lib/custom_logger.rb       2011-08-12 03:41:34.277000004 +0200
@@ -4,9 +4,11 @@ class NiceLogger < Logger
 
   def format_message(severity, timestamp, progname, msg)
     out=""
-    while msg[0] == 13 or msg[0] == 10
-      out = out.concat(msg[0])
-      msg = msg[1..-1]
+    if msg.kind_of?(Array)
+      while msg[0] == 13 or msg[0] == 10
+        out = out.concat(msg[0])
+        msg = msg[1..-1]
+      end
     end
    
     out + "[%s|#%5d] %s\n" % [severity[0..0], $$, msg]

Verification

Start the server.

# rcmemcached start
# rclighttpd start
# rcobsapidelayed start
# rcobswebuidelayed start 

At this point you should be able to open a browser and go to your server and login to OBS. Port 80 should be the webui, port 81 should be the api, and port 82 should be the repository directory. The default username/password is Admin/opensuse.

Start By Default

The OBS should start automatically, so they should be added to chkconfig to do so:

     # chkconfig --add obsrepserver obssrcserver obsscheduler obsdispatcher obspublisher obswarden obssigner

Setup OBS Worker

Install the Worker Packages

     # zypper install obs-worker

Configure OBS Worker

Edit the file /etc/sysconfig/obs-worker in order to point to correct repository server which is one of the main OBS server services. In this example the server is 192.168.1.1.

     # vi /etc/sysconfig/obs-worker
     OBS_SRC_SERVER="192.168.1.1:5352"
     OBS_REPO_SERVERS="192.168.1.1:5252"

Start OBS Worker

Start the worker service, did not start maybe main server needs to run first?

     chkconfig --add obsworker
     rcobsworker start

Setting up Cross Compile

Configure build types

To setup cross compile, add the appropriate machine types to /etc/sysconfig/obs-server. For example, the following will enable ARM builds.

      # vi /etc/sysconfig/obs-server
      OBS_SCHEDULER_ARCHITECTURES="i586 armv5el armv7el“

Cloning Repositories

prj and prjconf

The project settings and the project config is an important part of the project and should be copied to the local project setup.

     osc -A <remote API URL> meta prj <repository name> > prj.txt
     osc -A <remote API URL> meta prjconf <repository name> > prjconf.txt

Modify the prj.txt file to name valid users in the userid fields. Once modified, the settings and configuration can be applied to the local project.

     osc -A <local API URL> meta prj <repository name> -F prj.txt
     osc -A <local API URL> meta prjconf <repository name> -F prjconf.txt

Mirror

obs_mirror_project (/usr/sbin/obs_mirror_project) is used to copy an external OBS project. The script needed to be modified to use build.meego.com as apposed to the hard coded download.opensuse.org site. You need an login to build.meego.com to execute this script.

/usr/sbin/obs_mirror_project <repository name> <type> <arch>

Rescan Repository

Once the download of the project is complete, you have to rescan the local repository.

/usr/lib/obs/server/bs_admin --rescan-repository <repository name> <type> <arch>

Troubleshooting

Here are common issues that have been seen and how to resolve them.

HTTP Error 500

Check /srv/www/obs/api/config/environments/production.rb and /srv/www/obs/webui/config/environments/production.rb for configuration of the SOURCE_HOST and FRONTEND_HOST.

Cannot load person data for Admin in application_helper

Make sure you set the permissions on the api and webui directories.

     chown -R lighttpd.lighttpd /srv/www/obs/{api,webui}

If this doesn't work then try to recreate the databases:

     # mysql -u root -p
     mysql> drop database api_production;
     mysql> drop database webui_production;

Rerun the mysql database setup from above and re-run rake.

External Links

• Maemo OBS Installation
• Maemo Fremantel Setup

Liam

Dl9pf