Release Infrastructure/Packaging

Revision as of 17:39, 23 July 2010

The infrastructure team has to package various scripts and applications for our target distributions: openSuse and Debian.

Dependencies

When 'standard' modules and libraries are used they should be packaged for deployment

Ruby

openSuse

Should be done in an openSuse VM.

Tools:

 zypper in rubygem-gem2rpm rake ruby-bundler meld dbus-1-x11

Create an OBS project targeting devel:languages:ruby:extensions eg:

<project name="home:lbt:ruby">  
  <title>ruby</title>  
  <description>Packages intended for d:l:r:e
  
</description>  
  <person role="maintainer" userid="lbt"/>  
  <person role="bugowner" userid="lbt"/>  
  <repository name="devel_languages_ruby_extensions_devel_languages_ruby_openSUSE_11.2">  
    <path project="devel:languages:ruby:extensions" repository="devel_languages_ruby_openSUSE_11.2"/>  
    <arch>x86_64</arch>  
    <arch>i586</arch>  
  </repository>  
  <repository name="devel_languages_ruby_extensions_devel_languages_ruby_openSUSE_11.3">  
    <path project="devel:languages:ruby:extensions" repository="devel_languages_ruby_openSUSE_11.3"/>  
    <arch>x86_64</arch>  
    <arch>i586</arch>  
  </repository>  
</project>

Use this script to fetch and prepare a gem package

Note that some validation is done and changes to the gems may be needed.

The gem can be manipulated by using:

gem unpack TheGem.gem
# edit data and change version
gem build TheGem.gemspec

Python

Generic

Well written setup.py for setup-tools (python-setuptools in debian) :

   #!/usr/bin/python2
   from distutils.core import setup
   import os, sys
   setup(name = 'package_name',
     version = VERSION,
     description = 'short description',
     author = 'your name',
     author_email = 'your email',
     url = 'upstream url',
     scripts = ['script1', 'script2'],
     packages = ['package1', 'package2', 'package1.package3'],
           package_dir={'mypkg': 'src/mypkg'},
           package_data={'mypkg': ['data/*.dat']},
           data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                 ('config', ['cfg/data.cfg']),
                 ('/etc/init.d', ['init-script'])]
         )

RPM based distro

generate spec file from the setup.py file and modify to your needs :

   python setup.py bdist_rpm --spec-only

DEB based distro

Simply using dh_auto_tools to build the deb, you need to have the following :

• debian/rules template :

   #!/usr/bin/make -f
   # Uncomment this to turn on verbose mode.
   #export DH_VERBOSE=1
   %:
       dh $@ --buildsystem=python_distutils
   #override steps if needed
   #override_dh_install:
   #	dh_install

• debian/control template:

   Source: package_name
   Section: devel
   Priority: extra
   Maintainer: Your name <your email>
   Build-Depends: debhelper (>= 4.0.0), python-dev, python-support
   Standards-Version: 3.7.2
   Homepage: upstream url
   
   Package: package_name
   Architecture: all
   Depends: ${shlibs:Depends},
        ${misc:Depends},
        ${python:Depends}
   Description: short description
       long description

• Debian changelog can be generated using dch --create and incremented using dch -i. To simply refresh for actual release, use dch -r.

Django applications / projects WIP

On top of generic python you need to address the following issues :

• An ini style config file /etc/packagename/packagename.conf should contain reusable constants. These can be made available by opening and parsing it in settings.py and can be made available in templates using a custom context processor.

1. FIXME add context processor

• Static resources like images and js should be in /usr/share/package_name , and that path should be defined in the configuration file.

• The application should not care what URL prefix it is running under.

• lighttpd fcgi config

      var.namebasedir = "/name"
      $HTTP["url"] =~ "^" + namebasedir {
               dir-listing.activate = "disable"
       }
       alias.url += (
         namebasedir + "/static" => "/usr/share/name/static/",
       )
       url.redirect += (
         "^" + namebasedir + "$" => namebasedir + "/"
       )
       fastcgi.server += (
           "/name.fcgi" => (
               "main" => (
                   "socket" => "/var/run" + namebasedir + ".socket",
               )
           ),
       )
       url.rewrite-if-not-file += (
          "^(" + namebasedir + "/.*)$" => "/name.fcgi$1"
       )

• init script ( WORK IN PROGRESS ) #FIXME WON'T WORK IN DEBIAN

   #! /bin/sh
   ### BEGIN INIT INFO
   # Provides:          name
   # Required-Start:    lighttpd mysql
   # Required-Stop:     lighttpd
   # Default-Start:     2 3 4 5
   # Default-Stop:      S 0 1 6
   # Short-Description: Start FastCGI servers with Django.
   # Description:       Django, in order to operate with FastCGI, must be started
   #                    in a very specific way with manage.py. This must be done
   #                    for each DJango web server that has to run.
   ### END INIT INFO
   #
   # Author:  Guillermo Fernandez Castellanos
   # Version: @(#)fastcgi 0.1 11-Jan-2007 guillermo.fernandez.castellanos AT gmail.com
   # Altered by David Reynolds for Debian Stable (http://davidreynolds.me.uk/blog/2007/mar/16/django-fcgi-init-script/)
   # ...and altered further for img
   #### SERVER SPECIFIC CONFIGURATION
   DJANGO_SITES="img"
   SITES_PATH=`python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`
   RUNFILES_PATH=/var/run/
   RUN_AS=www-data
   #### DO NOT CHANGE ANYTHING AFTER THIS LINE!
   set -e
   PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
   DESC="FastCGI servers"
   NAME=$0
   SCRIPTNAME=/etc/init.d/$NAME
   #
   #       Function that starts the daemon/service.
   #
   d_start()
   {
       # Starting all Django FastCGI processes
       for SITE in $DJANGO_SITES
       do
           echo -n ", $SITE"
           if [ -f $RUNFILES_PATH/$SITE.pid ]; then
               echo -n " already running"
           else
               start-stop-daemon --start \
               --exec /usr/bin/django-admin runfcgi --settings=$SITES_PATH/$SITE/settings.py \
               method=threaded socket=$RUNFILES_PATH/$SITE.socket \
               pidfile=$RUNFILES_PATH/$SITE.pid --pidfile $RUNFILES_PATH/$SITE.pid
               chmod 400 $RUNFILES_PATH/$SITE.pid
           fi
       done
   }
   #
   #       Function that stops the daemon/service.
   #
   d_stop() {
           for SITE in $DJANGO_SITES
           do
           echo -n ", $SITE"
           start-stop-daemon --stop --quiet --pidfile $RUNFILES_PATH/$SITE.pid \
                             || echo -n " not running"
           if [ -f $RUNFILES_PATH/$SITE.pid ]; then
              rm $RUNFILES_PATH/$SITE.pid
           fi
           if [ -f $RUNFILES_PATH/$SITE.socket ]; then
              rm $RUNFILES_PATH/$SITE.socket
           fi
           done
   }
   ACTION="$1"
   case "$ACTION" in
       start)
           echo -n "Starting $DESC: $NAME"
           d_start
           echo "."
           ;;
       stop)
           echo -n "Stopping $DESC: $NAME"
           d_stop
           echo "."
           ;;
       restart|force-reload)
           echo -n "Restarting $DESC: $NAME"
           d_stop
           sleep 1
           d_start
           echo "."
           ;;
       *)
           echo "Usage: $NAME {start|stop|restart|force-reload}" >&2
           exit 3
           ;;
   esac
   exit 0