Skip to content

Home

Jump to bottom
bahamat edited this page Oct 2, 2014 · 29 revisions

Mission Statement

To produce a high quality, current, Debian native CFEngine package suitable for inclusion directly in Debian stable release.

Team members

The debian-team consists of an international (everything in alfabetical order) group of volunteers working for companies like Amazon. CFEngine, Google, Normation, SURFsara and Webhuis.

Debian repositories

Both repositories contain the same release:

  • git://git.debian.org/git/collab-maint/cfengine3.git
  • git://anonscm.debian.org/collab-maint/cfengine3.git

We are in the process of merging the existing Debian code with the code of CFengine 3.5.2. We have to determine whether it is still useful to do the merge because of the extent of the changes to the code, in fact we skipped CFengine versions 3.1 thru 3.4. It might be beyond the point where it adds to the process if we add CFengine 3.5 to this repository.

We aim at building a current CFengine release for Debian stable, but it would be helpful to the process if we decide to have a testing branch, or daily build branch, as well. We would add newer versions to the Debian repository each time CFengine releases a version we want to Debianize. Having this approach would prevent us to be in the lagging position like we are now.

Debian-team repository

The Debian-team agree on using a github repository to support our development process. We will develop the github way thus developers create branches themselves and the Debian-team decide on the merge process.

Getting set up

Before you can build you'll need to get a copy of the repository and the requisite branches.

git clone https://github.com/Webhuis/Cfengine-debian.git
cd Cfengine-debian
git checkout -b pristine-tar origin/pristine-tar
git checkout -b upstream origin/upstream
git checkout master

Building the package

The way it works now is that we have two branches:

  • master: this contains only the cfengine3 code, this can be overwritten with the new version without problems.
  • debian: this contains the debian/ dir and everything related to debian.

There are a few tools that to be used to import source packages, devtools should be installed. Issuing git-import-*, just use that with the --pristine-tar option and it will import the newest source version to the master branch and then it will merge the master branch into debian.

Build Prerequisites

This version includes some new build dependencies. You will need the following packages in order to build.

apt-get install build-essential autotools-dev libssl-dev flex bison \
libpcre3-dev dh-autoreconf liblmdb-dev libpam0g-dev libvirt-dev libacl1-dev \
libmysqlclient-dev libpq-dev libxml2-dev quilt git-buildpackage

Building

To build the deb, first make sure you're on the master branch then build.

git checkout master
git buildpackage -uc -us

Importing a new upstream version

  1. Download the tar ball from cfengine.com
  2. Import a pristine tar
  3. Update the changelog
git import-orig --pristine-tar ~/cfengine-3.6.0.tar.gz
debchange -v 3.6.0

You're now ready to build.

Bugs

We are tracking our issues here: https://github.com/Webhuis/Cfengine-debian/issues?state=open

Issue label guidelines

  • jessie tag is for bugs that only exist in jessie.
  • upstream tag is for bugs that need to be filed in cfengine/core as well as here.
  • filed-up means the bug has been filed upstream. File upstream before marking an issue as resolved!! We don't want to lose them!
  • packaging tag is for bugs that only affect packaging (i.e., in the debian directory, excluding debian/patches).
  • bug is for all bugs so they can be clearly called out.
  • build is for any bug that breaks the build and are the highest priority bugs. Repeat after me, "I will not break the build."

Details

Decisions made

Starting point

The starting point seems so have been moving. At the start (20130728) of the Debian-team initiative the offical Cfengine Debian stable release was related to CFengine 3.0.5. Recently a new version was introduced, Cfengine3 3.2.4-2+nmu1, this is a Non Maintainer Upload.

The official Debian point of view with regards to NMU is:

A non-maintainer upload (NMU) is an upload of a package to the debian archive by a developer who is not the maintainer of the package. This should usually not be the case, but in special cases (such as for RC bugs, when the maintainer does not respond to the bug report) it is allowed. See [https://wiki.debian.org/NonMaintainerUpload Non Maitainer Upload].

Integrating 3.5.2

CFengine is at 3.5.2 these days and has major changes and improvements with regards to stability and functionailty. We the Debian-team decided to build the native CFengine 3.5.2 Debian stable Wheezy package and are still looking for ways with the Debian maintainer.

The Cfengine project have been working very hard and a lot has changed in the code. There is a huge delta between the two lines of code, 3.2.4-2+nmu1 and 3.5.2.

One of our first goals is the integration of the new code line into the main line.

Process

This is the break down of the integration process:

  • Create a 3.2.4-2-NMU1 branch
  • Update changelog and copyright files, leave debian/source/local-options
  • Relocate the source files
  • Copy 3.5.2 sources file over the relocated 3.2.4-2+NMU1 source files

3.2.4-2+NMU1

The script carries out the following tasks

  • Separate orig and debian branches
  • Move orig code to the orig branch
#!/bin/bash
#
# First step to integrated Cfengine 3.5.2 in debian
#
# To be carried out in the top level git directory
#
#       Create the NMU branch

git branch 3.2.4-2+nmu1

#       Make NMU current

git checkout 3.2.4-2+nmu1

#       First apply the NMU fix
#       Whereas ${1} is the debian directory in the cfengine3_3.2.4-2+nmu1 source,
#       and ${2} is the debian directory in de git repository

cp -p ${1}copyright ${2}/debian/
cp -p ${1}changelog ${2}/debian/

mkdir orig

git mv aclocal.m4 orig
git mv AUTHORS orig
git mv autogen.sh orig
git mv ChangeLog orig
git mv config.guess orig
git mv config.sub orig
git mv configure.ac orig
git mv configure orig
git mv depcomp orig
git mv docs orig
git mv examples orig
git mv INSTALL orig
git mv install-sh orig
git mv LICENSE orig
git mv ltmain.sh orig
git mv m4 orig
git mv Makefile.am orig
git mv Makefile.in orig
git mv masterfiles orig
git mv missing orig
git mv pub orig
git mv README orig
git mv src orig
git mv tests orig
git mv ylwrap orig

Proposed update of changelog in debian:

cfengine3 (3.2.4-2+NMU1 git) unstable; urgency=low

  * Branch 3.2.4-2+NMU1
  * Put NMU updates in debian directory, left debian/source/local-options
  * Separated debian from orig
  * Upload to central repository

Merging NMU

#!/bin/bash

#       Back to master branch

git checkout master

#       Merge NMU into master

git merge 3.2.4-2+nmu1

#       Commit the NMU branch

git commit -am "Merge 3.2.4-2+NMU1 into master"

#       Push the NMU branch

# git push origin master

The push is commented out, it will not pass unless a trick is played. Checkout a branch on the master and then push the master from the client. Switch back to master after the push.

Intermediate repository

The Debian-team are looking for a way to set up a code review process pre-pushing. The best option is to have a Debian process for this, if not we will work out a flow using github.

In the mean time we need an intermediate solution in order to get to a 3.5.2 starting point. The Debian-team have access by LDAP authentication for http.

Useful links

  • [http://qa.debian.org/developer.php?login=antonio%40debian.org][Cfengine3 QA page]
  • [http://packages.qa.debian.org/c/cfengine3.html][Cfengine3 Packages QA page]
  • [http://packages.debian.org/stable/admin/cfengine3][Debian Cfengine3 Project page]

Patching

The Cfengine 3.2.4-2+nmu1 release contains the following patches in this sequence defined by the series file:

  • remove-man-errors.patch
  • fhs-compliance.patch
  • fix-examples-paths.patch
  • 626647-kfreebsd-support.patch The package needs to be patched predominantly for reasons of compliancy with Debian policies and standards.

Building the package

The cfengine source package comes in a tar ball, cfengine-3.5.2.tar.gz. Issue the following commands to start the build proces:

martin@wbhs-pkg:/usr/local/src$ tar xzf cfengine-3.5.2.tar.gz
martin@wbhs-pkg:/usr/local/src$ cd $ cfengine-3.5.2
martin@wbhs-pkg:/usr/local/src/cfengine-3.5.2$ dh_make -f ../cfengine-3.5.2.tar.gz -r -c gpl2 -e "Martin Simons [email protected]"
martin@wbhs-pkg:/usr/local/src/cfengine-3.5.2$ cp -p /home/martin/development/cfengine3/debian/patches debian/
martin@wbhs-pkg:/usr/local/src/cfengine-3.5.2$ dpkg-buildpackage -rfakeroot -uc -us

Cfengine planning

Cfengine is developing quite rapidly these days. The Masterfiles now are an integral part of the installation, as of version 3.6 we will see a split of the Masterfiles with regards to the core system. Cfengine 3.6 is due end of 2013.

Plan

  • preconditions
  • scope
  • risks
  • gouvernance
  • relationships to the commercial edtion and third party solutions
  • Planning and milestones to be obeserved

tenants

  • As much as possible cfengine-core should support Debian natively ** Prefer patching cfengine rather than Debian ** Use IFDEF where necessary
  • Strictly adhere to Debian packaging policies ** Debian FHS layout

Creating a buildable debian package should be as easy as obtaining the cfengine source and dropping in the debian directory.

Notable directory relocations wishlist

  • /var/log/cfagent.NODENAME.log and /var/log/cf3.HOSTNAME.runlog

  • /var/run/cfengine3/COMPONENT for PID files

  • all the examples should point to $(sys.inputdir) and $(sys.masterdir) and so on

  • executables under /usr/sbin or /usr/bin (need to adjust sys.bindir; not certain)

  • sys.masterdir = /var/lib/cfengine3/masterfiles

  • sys.inputdir = /etc/cfengine3

  • links: etc/cfengine3 to var/lib/cfengine3/inputs and usr/sbin to var/lib/cfengine3/bin

  • configure line:

      ./configure --enable-shared=yes \
                          --prefix=/usr \
                          --with-sql=yes \
                          --with-mysql \
                          --with-postgresql \
                          --with-libvirt \
                          --with-tokyocabinet \
                          --with-libxml2 \
                          --with-libacl \
                          --with-workdir=/var/lib/cfengine3 \
                          --mandir=\$${prefix}/share/man \
                          --infodir=\$${prefix}/share/info \
                          --htmldir=\$${prefix}/share/doc/cfengine3/html \
                          CFLAGS="$(CFLAGS)" LDFLAGS="-Wl,-z,defs"

  • daemons not started on install, need to edit /etc/default/cfengine3

Clone this wiki locally