Building rpms for p4dti

1. Introduction

The Perforce defect tracker is distributed with an rpm which allows it to be cleanly installed and uninstalled for users of the Bugzilla defect tracker integration.

This document is intended for maintainers of the integration who may not have a knowledge of package management using rpm.

2. What is an rpm?

An .rpm file (often called simply an "rpm") is a binary file which contains all the information needed to install a software package -- including information about files and scripts that need to be run on installation or de-installation.

An rpm is normally constructed from a .spec file which is a text file describing:

1. How to construct the package from a "pristine" distribution of its source -- this is intended to allow faithful recompilation of many packages on different architectures by recording everything that needs to be done, perhaps including the patching of the source, to build the software.
2. Instructions on how to install the package, including a list of files that constitute the package, their locations and properties and anything that needs to be done to the system during installation or de-installation.

Since the defect tracker is written in Python (an interpreted language) there is no need to "compile" the package in any way. However a number of additional files are needed (including a startup script) which at the moment are supplied in an additional source file, though I recommend that they are added to the main distribution, which will simplify the .spec file.

The rpm program can also package up the source(s) and .spec file into a so-called "source distribution" or .srpm file. It is normal practice to distribute these alongside any .rpm being distributed. Since .srpm's are usually automatically built with the .rpm all you need to know about is their existence. They would allow someone external to reconstruct the .rpm independently.

3. Building the rpm

The program rpm is used for all manipulations of .rpm files (building, installing, de-installing and so on). It uses a number of directories in the .rpm build process, by default they live under /usr/src/redhat, where you may find the following:

SOURCES

Is where it looks for source tarballs

SRPMS

Is where it will put any built .srpm's

BUILD

Underneath which it unpacks each package's distribution

RPMS/i386

Is where it will put any .rpm's built for i386 systems

In addition it is usual to copy the .spec file under another sibling directory called SPECS.

Unless you are happy developing the rpm under a system wide directory, you will need to give rpm an alternative location for these directories and make sure that they exist. The simplest way to do this is to create a file called .rpmmacros in your home directory. Mine looks like this:

%_topdir /home/fjmd/usr/src/redhat
%_packager fjmd1@hotmail.com

where the first line gives it a top level directory under which it will expect to find directories called SOURCES, SRPMS, BUILD and RPMS/i386. The Second line allows me to specify that every rpm I create should have my email address given as the "packager".

Now copy the p4dti.spec file into the SPECS subdirectory and the tared and gzip source of p4dti into the SOURCES subdirectory, along with the tared and gziped source of p4dti-packaging.

Change directory to the SPECS directory and issue the command

rpm -ba p4dti.spec

This should successively unpack the sources into a subdirectory of the BUILD directory and build an rpm and srpm, the paths to which it will report.

4. Things to change before final release

There are a few changes that should be made to the .spec file before final release that need to be done by people within the p4dti team.

• The line in the .spec file begining Summary: needs to have a short (one-line) description of the p4dti product.
• The section begining %description may need to be edited. It should contain a longer description of the product.
• The license line (begining License: ) needs to be checked -- I am not sure what Ravenbrook's policy on this will be.

5. Maintenance of the spec file

Each rpm has a version and a release number. The version number should be equal to the version number of the software being packaged; the release number acts as a version number on the .rpm. These will need to be changed in the following circumstances:

• When the version of the software being released is changed, the line begining Version: should have the old version number replaced by the new. The line begining Release: should be set to Release: 1
• If the .spec file needs to be changed in any important way; for example if file permissions need to be changed or the installation scripts need to be altered, then the number on the line begining Release: needs to be incremented.

A. References

B. Document History

C. The .spec file

In this appendix I will work through the p4dti.spec file in detail and explain what each section means and why I have decided to implement it in that fashion.

Name: p4dti

Gives the name of the package, which is used to identify the package in a database of installed rpms. It also causes that macro %{name} to be defined.

Version: 0.5.1

Gives the version number of the packaged software. Used by rpm to decide whether an upgrade is needed and to give a default name to the new rpm (which is name-version-architecture.rpm).

Group: Development/Tools

Lastly every rpm has to be assigned a group, which is used by various automatic package management tools to give a (supposed) user friendly, menu driven organisation of the packages. Development/Tools is the group assigned to cvs.

Source: http://www.ravenbrook.com/project/p4dti/release/%{version}/p4dti-bugzilla-%{version}.tar.gz
Source1: p4dti-packaging-0.1.tar.gz


These two lines tell rpm that it needs to unzip and untar two source files. A source can be a simple file name, or a URL. In the latter case it ignores everything except the filename. In both cases it looks in the SOURCES directory for the relevant files. The rest of the URL is used by some automatic package searching engines but is mainly for a human readers benefit.

I have used two source files, to keep my own extra material separate from the ravenbrook distribution, they should probably be amalgamated for the final distribution.

Prefix: /opt/p4dti
Prefix: /var/lock/subsys
Prefix: /var/run
Prefix: /etc/rc.d/init.d
Prefix: /usr/doc/p4dti-%{version}


Rpm has a mechanism for declaring some paths to be "relocatable". Very few packagers seem to use this facility, which is most annoying. For a system administrator it is very useful to be able to ask rpm to install files in non-standard places. I have tried to build the p4dti rpm so that it is fully relocatable. These lines declare that the 5 paths shown can be relocated by the system administrator.

License: (C) 2000 Ravenbrook Limited (see sources)
Buildroot: /var/tmp/%{name}-buildroot
Release: 1


The License: line should be self-explanatory, it is reported to a system administrator if they ask for general information about a package. The Buildroot: points to a temporary file for rpm to use while it is building (not installing) the .rpm file and the Release: line acts as the version number of the .spec file and thus of the .rpm built from it.

%description
This is a beta release of the Perforce Defect Tracking Integration
(P4DTI).

The primary purpose of this release is to test the software in realistic
conditions at real customer sites, to find defects and fix them before
the general release in 2001-02.

The Perforce Defect Tracking Integration version 0.5 is beta test
software, only intended for use during the beta program of the project.
The software _will_ be defective. We recommend that you do not rely on
the integration in your organization, but we would like you to try it
out and let us know about any problems. We are very interested in your
feedback on the beta. Please write to
<p4dti-beta-feedback@ravenbrook.com>.


The %description line introduces a section of the rpm which will be printed out in response to the -qi option on rpm. I have lifted the text from [insert reference here], which will need to be altered on final release.

%prep
%setup -n p4dti-bugzilla-0.5.1
%setup -T -D -a 1 -n p4dti-bugzilla-0.5.1


Somewhat confusingly %prep introduces a new section, which contains two instances of the %setup macro. This section is executed before the building of the .rpm and usually contains code to unpack the source and make sure it is put into the correct directory structure. In this the first %setup creates a p4dti-bugzilla-0.5.1 directory under the BUILD directory, into which it unpacks the source named on the Source: line. The second %setup refers to the source named on the Source1: line (indicated by the -a 1 option, it is instructed not to interfere with the top level directory -T -D but to unpack the source under the named directory.

This will only need to be changed if the number of sources being unpacked changes. If the packaging auxiliary files are included in the main source, the second %setup line can be deleted.

%build
rm -fr $RPM_BUILD_ROOT


The build section usually contains code to compile a package. Since p4dti doesn't need any compilation this section is rather simple. $RPM_BUILD_ROOT is an environment variable defined when the %build script is run and is set to the directory defined on the Buildroot: line.

%install

mkdir -p $RPM_BUILD_ROOT/opt/p4dti/bin
cp *.py $RPM_BUILD_ROOT/opt/p4dti/bin
mkdir -p $RPM_BUILD_ROOT/etc/rc.d/init.d
install p4dti-packaging/p4dti $RPM_BUILD_ROOT/etc/rc.d/init.d/p4dti


This script is run after the build script, during the creation of the .rpm. This script should create the target directory structure of files, from which rpm will take all files that should be installed and package them into the .rpm file. This install section creates needed target directories and then copies all python files from the distribution into /opt/p4dti/bin and the startupt script into /etc/rc.d/init.d.

%clean
rm -fr $RPM_BUILD_ROOT


The last section to be run while building the .rpm is %clean. It is run after the .rpm file is constructed. Here it simply removes the temporary build structure, so as not to clutter up /tmp.

The next part of the .spec file defines scripts to be run at installation or de-installation time. As it happens all four are needed. An rpm installer can turn off the running of these using the --noscripts option.

%pre
/usr/sbin/groupadd -r -f p4dti && /usr/sbin/useradd -c 'Perforce
Defect Tracking
Integrator' -d $RPM_INSTALL_PREFIX0 -g p4dti -r -M p4dti || printf "A p4dti
use
r has not been created\n"


This is run before package installation. It creates a group and a user named p4dti with fresh gid and uid's. $RPM_INSTALL_PREFIX0 will point to the logical directory /opt/p4dti which can be relocated by the installer.

%post

# If the package is relocated, the start script will have to be altered
# so that it uses the correct directories.

printf "/^P4DTIDIR/c\nP4DTIDIR=%s\n.\nw\n" $RPM_INSTALL_PREFIX0 | ed -s
$RPM_INS
TALL_PREFIX3/p4dti
printf "/^LOCKFILE/c\nLOCKFILE=%s\n.\nw\n" $RPM_INSTALL_PREFIX1/p4dti | ed
-s $R
PM_INSTALL_PREFIX3/p4dti
printf "/^PIDFILE/c\nPIDFILE=%s\n.\nw\n" $RPM_INSTALL_PREFIX2/p4dti.pid | ed
-s
$RPM_INSTALL_PREFIX3/p4dti

# These are only needed for package relocation, but should have no effect
# otherwise.

mkdir -p $RPM_INSTALL_PREFIX1
mkdir -p $RPM_INSTALL_PREFIX2



This script is run after installation. It is needed to make sure that the p4dti installation is correctly relocatable. Nothing in the python files has any hard wired paths internal to it, but the startup script p4dti needs to. The effect of the printf ... | ed ... lines should be to change three lines in p4dti that begin P4DTIDIR=, LOCKFILE= and PIDFILE= respectively. If further path dependencies are added to the distribution in any way, further patches will need to be applied here.

The last mkdir lines make sure that the paths that /var/lock/subsys and /var/run are relocated to exist.

%preun
for i in $RPM_INSTALL_PREFIX0/bin/p4dti.log $(ls
$RPM_INSTALL_PREFIX0/bin/*.pyc 2>/dev/null)
do
[ -f $i ] && rm $i
done


This script runs before the package is uninstalled. It removes all .pyc files that may have been created and the og file (if any). If we didn't do this, rpm would report an error on uninstallation of the package.

%postun
if id p4dti 2>/dev/null
then
/usr/sbin/userdel p4dti
fi


This is run after the uninstallation and removes the p4dti user. Redhat's userdel command takes it upon itself to also remove the p4dti group at the same time.

%files
%defattr(-,p4dti,p4dti)
%doc ug ig mg ag
/opt/p4dti/bin
%attr(-,root,root) /etc/rc.d/init.d/p4dti


Lastly the %files section tells rpm what should go into the .rpm file. It indicates that the files ug, ig, mg and ag are documentation files and should be marked as such and installed in the system documentation directory (by default /usr/doc under a subdirectory for that package. The %defattr command sets default file ownership for files, unless otherwise stated. This section should contain a list of all files to be installed, possibly with annotations setting their attributes and ownership. In fact we want all files in /opt/p4dti/bin to be installed along with the installation script in /etc/rc.d/init.d/p4dti.

D. p4dti startup

Redhat Linux uses a system V based startup. Startup scripts live in /etc/rc.d/init.d and should accept the arguments start and stop. There is also a convention that status should cause the script to report the status of any running daemon it has created and restart should cause it to stop and then start again. I have written a suggested p4dti startup script, which I will document in this section.

#!/bin/sh
#
# Startup script for the p4dti Server
#
# chkconfig: 345 95 10
# description: p4dti replicates information between a perforce repository \
# and a defect tracking system.


Some comments in a startup script are used by the redhat utility chkconfig to manipulate the script. chkconfig can be asked to "switch on" or "switch off" a startup script (which it does by creating suitable symbolic links in the /etc/rc.? directories). The 3 numbers after chkconfig: are:

• 345 are the 3 run levels (3, 4 and 5) in which p4dti should run.
• 95 is the order in which it should be started (a number from 0-99). 95 means that p4dti will only be started after most other services
• 10 is the position when it should be stopped. 10 means that p4dti should be stopped before most other services

The description: line is non-optional (chkconfig will complain if it does not find it).

# Source function library.
. /etc/rc.d/init.d/functions


Is probably obsolete. In development redhat's standard suite of utility functions for startup scripts was used, these turned out to be inappropriate for the kind of software that it is.

# The following line is detected by rpm and corrected.
P4DTIDIR=/opt/p4dti
LOCKFILE=/var/lock/susbsys/p4dti
PIDFILE=/var/run/p4dti.pid

start_p4dti()
{
python run_bugzilla.py >/dev/null&
printf "%s" $! > $PIDFILE
}


The three variable definition lines are manipulated by the rpm installer, so any reference to those directories should be made via the variable, rather than directly. The start_p4dti() function is not used.

The remaining script should be self-explanatory.

Copyright © 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.