Informatiq (Talk | contribs) m (formatting fixes) |
|||
| (One intermediate revision not shown) | |||
| Line 1: | Line 1: | ||
== story == | == story == | ||
As a developer I need to be able to | As a developer I need to be able to | ||
| - | * | + | * '''quickly''' build, deploy, test my changes |
* with possibility to work offline | * with possibility to work offline | ||
* have similar build results than the releasing system so my test results are reliable | * have similar build results than the releasing system so my test results are reliable | ||
| Line 9: | Line 9: | ||
== use case == | == use case == | ||
| + | '''case 1''' | ||
# modify one package | # modify one package | ||
# build it | # build it | ||
# update target device | # update target device | ||
# run test cases | # run test cases | ||
| - | + | '''case 2''' | |
# modify code across the stack (multiple pkgs) | # modify code across the stack (multiple pkgs) | ||
# recursively build all modified pkgs | # recursively build all modified pkgs | ||
| Line 66: | Line 67: | ||
==== target-test(test_package) ==== | ==== target-test(test_package) ==== | ||
connect to device over usb-network and run the test cases provided. a good tool for that would be test-runner-lite part of OTS testing suite | connect to device over usb-network and run the test cases provided. a good tool for that would be test-runner-lite part of OTS testing suite | ||
| + | |||
| + | === Implementation notes === | ||
| + | |||
| + | ==== Chroot setup, step 1 ==== | ||
| + | |||
| + | We can use both osc or build to setup the initial chroot -- osc uses build itself, though uses OBS for dependency calculations. Both methods allow offline operation once the initial chroot got built. | ||
| + | |||
| + | For using build the best method would be having a complete offline copy of the distribution packages | ||
| + | |||
| + | For using osc we just download the packages needed. For a multiple package build a quick hack without changing too much in osc would be constructing and building a dummy package containing all the build dependencies of the packages we want to build. | ||
| + | |||
| + | Most likely it would be best to extend osc, though we currently risk having to do work again with osc2. As much of the functionality as possible should be done in osc-plugins. | ||
| + | |||
| + | ==== Chroot setup, step 2 ==== | ||
| + | |||
| + | One problem we have with the rebuild-test cycle is speed. Doing a complete package rebuild is slow, so we need to keep the buildroot around. Additionally, we need to be able to tell build to use an existing buildroot. (As a side effect, this will easily allow us building a single package from a local git source tree, too, by just specifying the git source tree as buildroot) | ||
| + | |||
| + | TODO here: | ||
| + | |||
| + | * patch build to be able to use an existing buildroot (rpmbuild already can: --short-circuit with -bc and -bi) | ||
| + | * patch osc to use the additional build functionality. This is probably the only thing we should try to implement directly in osc, not in osc plugins | ||
| + | * put the buildroot in place; currently manual bind mount, will be replaced by a custom fuse filesystem | ||
| + | |||
| + | Another problem to solve here is unpacking the source packages; it is currently unclear how we'll find all source packages we need to unpack. | ||
| + | |||
| + | The sources will end up in a git repository, transparently masked by fuse. Patches will probably go into a separate branch (gbp-pq). This allows us to: | ||
| + | |||
| + | * easily remove patches from the buildroot | ||
| + | * easily (and quickly) clean the buildroot (just a git clean) | ||
| + | * easily determine which packages need to be rebuilt by looking at the repository state on multiple package builds | ||
| + | * keep a history of changes, and produce a patch with the changes | ||
| + | |||
| + | The functionality here can be implemented partly in the filesystem, partly in an osc plugin. | ||
| + | |||
| + | |||
| + | ==== Building ==== | ||
| + | |||
| + | * (for multiple packages) determine which packages to rebuild | ||
| + | * make sure the buildroot is available | ||
| + | * commit snapshot to git (ignoring files generated by build process -- we need to generate proper gitignore here) | ||
| + | * build with patched osc/build in an existing buildroot | ||
| + | |||
| + | |||
| + | ==== Publishing ==== | ||
| + | |||
| + | We can assume that the device for testing will be reachable by network (usually: usb network), so the easiest way to get the packets to the device would be by using zypper over http. | ||
| + | |||
| + | A simple httpd to expose a local directory is trivial to implement, so we just have to | ||
| + | |||
| + | * write the packages to some directory where they'll stay available (osc cleans old packages on build) | ||
| + | * remove old versions of the same package | ||
| + | * rebuild the catalog | ||
| + | * trigger the device to refresh the catalogs, and install/update the package | ||
Contents |
As a developer I need to be able to
In order to speed up the development process, and be more productive
case 1
case 2
these requirements can be met easily with manual work, there is no missing tools, the real problem is in simplifying and automating
a tool that wraps all the commands to run that process, automating many manual steps
takes as input the list of packages the developer will work on (FIXME: more details needed) create a chroot will all operations will run, it is necessary to isolate the build env to not get reliable results fetch all needed sources and extract them in the specified location e.g /home/user/chroot/usr/share/devtree/ chroot, will contain a bootstrap of the desired distro, all the build tools, and the build dependencies of the packages specified open items:
update the chroot env, and the sources provide a merge possibility of the newer code and the modifications done locally
this should perform
this is what rpmbuild does, but it would require that we archive the source tree and then rpmbuild would unpack it, also rpm would always flush the buildroot and start building from scratch again. this way is good for releasing but too slow for our purpose. rpmbuild has oprtions we can use to control all those and reach the desired process build in place (no archive) and keep old build data, so rebuilds are much faster (depending on how make detects changes) the build script of OBS should be used as it will produce a similar result as what happens on OBS this covers the build and package, publish is needed to make that package available to the target device this can be done through filesystem or http if build method is called from inside a certain pkg tree e.g /home/user/chroot/usr/share/devtree/pkg1, then it would only build this package if called from the devtree root /home/user/chroot/usr/share/devtree/ the it should detect whcih pkgs have modifications and only build those
create an image using mic, and flash it to device, tools and details are device specific. we might skip this method
connect to device over usb-network and update the device through the published repo e.g. zypper ref;zypper up
connect to device over usb-network and run the test cases provided. a good tool for that would be test-runner-lite part of OTS testing suite
We can use both osc or build to setup the initial chroot -- osc uses build itself, though uses OBS for dependency calculations. Both methods allow offline operation once the initial chroot got built.
For using build the best method would be having a complete offline copy of the distribution packages
For using osc we just download the packages needed. For a multiple package build a quick hack without changing too much in osc would be constructing and building a dummy package containing all the build dependencies of the packages we want to build.
Most likely it would be best to extend osc, though we currently risk having to do work again with osc2. As much of the functionality as possible should be done in osc-plugins.
One problem we have with the rebuild-test cycle is speed. Doing a complete package rebuild is slow, so we need to keep the buildroot around. Additionally, we need to be able to tell build to use an existing buildroot. (As a side effect, this will easily allow us building a single package from a local git source tree, too, by just specifying the git source tree as buildroot)
TODO here:
Another problem to solve here is unpacking the source packages; it is currently unclear how we'll find all source packages we need to unpack.
The sources will end up in a git repository, transparently masked by fuse. Patches will probably go into a separate branch (gbp-pq). This allows us to:
The functionality here can be implemented partly in the filesystem, partly in an osc plugin.
We can assume that the device for testing will be reachable by network (usually: usb network), so the easiest way to get the packets to the device would be by using zypper over http.
A simple httpd to expose a local directory is trivial to implement, so we just have to