Revision as of 18:51, 25 March 2011

Of Frameworks, Webservers, Testing, Development and Deployment

The Problem Domain

Agile evangelists will tell us that deployment is the first thing we should work on.

yeah yeah

So it is perhaps fitting that this is the probably the last thing I'll do on OTS - in the current arrangement anyway. Given where we are (fixing it on at the end) it is not something that I can resolve at this stage, but process is a journey anyway... blah blah

1. Tight Coupling

I have seen consistent errors on the introduction of the Django Plugins when trying to run the unittests straight for a git checkout. The workaround is to go back and uninstall the Developer eggs in the Django plugins

The development workflow *should* be simple

1. Download OTS
2. Run tests OTS
3. Start work

OTS is a large project now <#FIXME how many?> LOC. A significant proportion of this is backend.

In fact the front end (Django) work are supposed to be Plugins so they really shouldn't be causing any issues with the backend.

2. Unittested - but only on development

We have <#FIXME how many> unittests. Wouldn't it be great if we harness these on deployment the system?

For getting new developers up an running and installing OTS the first response to any problem should be the same:-

"Do the tests run?"

3. Platform Specific

4. Meta Packaging

With all the components of OTS we need some kind of metapackaging to bring it together?

But we have meta packaging with RPMs / Debian don't we?

RPM and Debian are deployment domain tools and not really suited to development.

Installing development software at system level is impolite and limiting.

5. Consistent Unit Test Harvesting

• Running the unittests through django was messy and lossy.
• Nose gave us various problems.

It would be good if we had a consistent and reliable unittest interface across all components.

6. Legacy and Future

We started from a privileged position. I have never worked on a project which has only targeted one OS. Let alone a subset of that OS which has one package management system. Debian was OK under Maemo, OTS serve Meego means that we are no longer there that has been addressed with RPMs which is good and fine since we have good people happy to maintain two package management systems.

If OTS is success grows then the requirement to install on other OS is pretty much an inevitability.

7. Installation is not reproducible

Currently a smaller user base making custom installs on a regular basis. I am aware of at least 6 variants.

That's fine and quite normal but.. it might be that everyone has their own hack to get it working in their set up... how are these being tracked? Hopefully I won't need to convince test engineers the value of reproducibility.

Some notes on repeatability of buildout cf virtualenv here

8. Shell scripts

I wrote the setup.sh and nose.sh early in the OTS development for personal convenience. They were shared and they have started to evolve. These scripts solve basic and common problems and we have to maintain them rather than investing in these scrappy shell scripts I'd suggest it would be better to invest the time understanding more elegant and powerful solutions.

The Goal

1. Loose Coupling between Coarse Grained Components 2. Consistent and Easy to setup Development Environment 3. Consistent and Easy to setup Deployment Environments 3. Verification of Deployment set up with unittests 4. A reproducable and improvable process

Lofty aims...

The Technologies

1. Setuptools

Helped us with our decoupling with Publishers in the journey from 0.1 -> 0.8

2. Buildout

There is plenty of propaganda documentation that makes claims that Buildout can solve many of the issues.

Buildout is designed to work with setuptools so far so food. However it is from the Zope / Plone world, concerns about interaction with Django where dispelled when I saw this post in which the Django creator says that

it’s an exceedingly civilized way to develop an app.

As with many of the commentators I have found the documentation patchy and there have been things that have taken a while to workout. Hopefully this wiki will help.

However the overall experience (as you would expect from a third generation tool) is something that lives up to its promises.

The Proof of Concept

So to work...

I have taken the approach of modelling our system on a basic shell with the same layout as OTS. My experience of deployment is the same as others in that you need to start to start with something then refine it over a number of installs before it works cleanly on a number of systems. That isn't achievable for me but hopefully I've done the groundwork that allows this knowledge to be applied. I think it is achievable over the next few releases with OTS.

You can pull the code from here

Unittest runner for Django

python setup.py test

I have added this to the top level django plugin to give consistency with the rest of the code. This wiki makes the point that the Perl guys have the ability to run tests on installation - and we can't have them having one-up on us can we ;-). Also makes us PyPi compatible if the need arises.

Tight coupling of plugins

The Django plugins have hardcoded each app. Django hasn't got a reputation for being the best framework for doing these things (interacting with other large and complex systems) more aimed at a RoR / php digital media thing -but that is what we are using so what to do?

The settings are in Python of course so we can apply the extension point ideas in theory:

It also works in practice with the model, but given the fundamental problem (using Django in a way that wasn't intended) you might prefer to put up with the hardcoding and the tight coupling.

Working with Buildout

Buildout claims to tick the boxes on a lot of the problems above reproducability, testing and deployment are it's raison d'etre.

Let's run through the workflow set out above for the buildout I've wrapped around the example.

Development

1. Download OTS

First

$python bootstrap

Then

$bin/buildout

If you check the bin directory now you'll see the parts all there.

2. Run tests OTS

$/bin/test
Creating test database 'default'...
Creating table auth_permission
<snip>
Simple Test
..
----------------------------------------------------------------------
Ran 2 tests in 0.004s

OK
Destroying test database 'default'...

3. Start work

Now you can have a sandboxed environment within which you can work straight away

$bin/python

>>> foo.bar.__file__
'/path/to/your/buildout/foo.bar.baz/foo/bar/__init__.py'

$bin/django runserver
Validating models...
0 errors found

Django version 1.2, using settings 'bar.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Pretty neat hey? It gets better...

Deployment

This is took me a while to get my head around and judging from IRC histories in the Zope channel I'm not alone.

This is how I got it to work. It should be pointed out at this point that Buildout is very much designed with PyPI in mind so it is expecting packages to be built to and come from there, naturally the open source aims of OTS are in line with that but we aren't there yet ;-) So I've used file paths.

You might've noticed that there was a source-release in the bin directory. We use this to build the release from *another* buildout.cfg file.

$/bin/buildout-source-release file:///path/to/your/buildout/deploy deploy.cfg

$tar xvzf deploy.tgz  
$cd deploy

You'll see that there is an install file there. Run it

$python install.py

So can we test our deployment? Naturally

$bin/test

The component tests and even the system tests could be incorporated to - just imagine a one click buildout of OTS that runs a self test - now that *would* be nice.