P4DTI training course for TeamShare support

Gareth Rees, 2001-01-24


1. Introduction to the course

Name, position, relationship between Perforce, Ravenbrook, TeamShare.
The purpose of the course is to give the TeamShare support staff an
introduction to the P4DTI, and to enable them to answer basic support
questions about it.

Make sure to ask everyone what they want from the course.  What
knowledge, what detail?  What do other training courses cover?


2. Outline of the course:

history and goals of the P4DTI project
releases and schedule
introduction to Perforce
introduction to P4DTI
  -- from the users' perspective
  -- from the P4DTI administrator's perspective
  -- from the manager's perspective
support arrangements
resources
questions


3. History and goals of P4DTI project

Perforce Software sells a software confifiguration management system,
also called Perforce.  Their product has strong appeal for developers
and so is popular with small software firms.  Although Perforce as an
SCM system scales well as an organization grows, larger organizations
have other requirements.

In 1999, several queries from customers who were unhappy about the
integration of Perforce with defect tracking.  In 2000-01, Perforce
Software engaged Ravenbrook Limited to investigate the problem and
propose solutions.

We talked to Perforce's customers and found these major concerns:

-- Developers have to enter information multiple times.  For example,
   they fix a problem in the source code and write up what they've done
   in a change comment.  Then they have to switch to the defect tracker,
   find the issue they've been working on, and enter the same details in
   the fix description field.  Developers don't like to do this, and
   don't do it consistently or accurately.  This results in the
   information in the defect tracking system getting out of date with
   what is actually in the product.

   Some people said that they only wanted to use the defect tracker.
   Others only want to use Perforce.

-- Perforce provides no assistance in process implementation.  (It
   allows many processes but supports none.)

-- Perforce provides no control over development activity.  There's no
   workflow enforcement.

-- An organization using Perforce together with a defect tracker can't
   easily produce reports combining information from both systems.  Such
   queries include:

   a. Which issues were fixed in release X.Y?  (For the release note.)

   b. Which issues are known or suspected to be present in release X.Y?

   c. What changes did developers make in order to fix issue N (for
   regression testing or other SQA activity).

   d. Why was this change made to this piece of code?  (i.e., in
   response to which issue?)

   e. Were any changes made that weren't in response to any issue?
   (These are likely to be suspect.)

So the goals of the P4DTI are:

-- Make Perforce suitable for more organizations (by meeting these
   requirements)

-- Make existing Perforce customers happier.

We agreed with Perforce Software to integrate Perforce with defect
tracking systems.  We would (a) pick two defect trackers that were
widely used by Perforce customers and implement integrations; (b)
release an integration kit that would allow third parties to implement
their own integrations.

For (a) we picked TeamTrack and Bugzilla; these are representative of
two ends of the spectrum of defect trackers.

For (b) we expect the third parties to be a mixture of other defect
tracking vendors and ordinary Perforce customers.


4. Introduction to Perforce

Bear in mind the requirements while we look at Perforce in detail (enter
information once, workflow support, workflow control, reporting).

Perforce concepts and use:

-- Client/Server operation.  Runs over a network.  A number of clients:
   command-line, GUI, plug-in.

-- Repository (also known as depot).

-- Client (aka workspace).  A view that maps (part of) the repository
   onto a local disk.  (Set up two clients, one of which maps a small
   portion of the repository.)

-- Filespecs.  A way of referring to a group of files either on the
   repository or on the client.

-- Changelists.  Additions, edits, deletions.  Are atomic.  (Edit, add
   files, with change comments.  Show history.  Show pending
   changelists.)

-- Working in parallel.  (Check out same file in both clients, edit,
   resolve.)

-- Branching.  How this can be used to support release branches,
   development branches.  (Whiteboard best for this.)

-- Jobs.  Rudimentary defect tracking system.  No workflow, no control.
   Anyone can edit anything.  Features to note: customizable; filterable
   (open jobs assigned to me)

-- Fixes.  What they are ('fixes' is a historical name).  What they can
   be used for.  How to query them.



5. Introduction to the P4DTI

The P4DTI works by replication. The replicator is a process that copies
(replicates) data between a defect tracker and a Perforce server in
order to keep each one up to date with changes made in the
other. Replication of data between the defect tracker and Perforce
allows developers to do their routine defect resolution work entirely
from their Perforce client, without needing to use the defect tracker's
interface. It also allows developers to relate their changes to defect
tracking issues.

The replicator maintains a one-to-one relationship between issues in the
defect tracker's database and jobs in the Perforce repository. In other
words, each issue has a corresponding job, and vice versa. The
replicator keeps the contents of a configurable set of fields in the
defect tracker's issues the same as the contents of the corresponding
Perforce job, so that editing one edits the other.

The replicator also copies Perforce's links between jobs and changelists
(called "fixes") to the defect tracker's database.

The replicator polls the defect tracking server and the Perforce server
at regular intervals to get a list of recent changes, and attempts to
propagate these changes to the other system.

Most defect trackers have an idea of workflow -- a set of rules that
control who can do what to which issues. The replicator enforces the
defect tracker's workflow by rejecting changes to jobs in Perforce that
are illegal in the defect tracker. When it comes across such a change,
it undoes the change and sends an e-mail message to the users involved.

You can think of the defect tracker as the master of the defect tracker
records (and therefore the job contents), while Perforce is the master
of the changelists. Neither side is really master of the "fixes"
relationship.

TeamShare have made some changes to TeamTrack to support the P4DTI,
notably making the fixes relation appear in the case description pane.


6. The P4DTI from the user's perspective

See the P4DTI user guide.

If you are a developer, the P4DTI enables you to: 

-- Enter data about your changes and update the status of the issues
   you're working on directly in Perforce.   (How transitions work.)

-- Easily link changes to issues and keep them up to date. 

If you are a member of the SQA team, the P4DTI enables you to: 

-- Easily find out what has changed to resolve an issue.

-- Easily find out which parts of the system have been affected, so that
   you can direct your testing time effectively. 

-- Find out which fixes have made it into a particular codeline or
   release. 

(Show how to carry out these tasks.  TeamTrack tasks are most important.)


7. From the administrator's perspective

Most admin queries should be dealt with by Perforce support.

Installation -- configuration -- running.

Configuration involves editing variable assignments in a file of Python
source code: hostnames, userids, passwords.

Need to create a user in TeamTrack to represent the replicator.

What happens when it starts up (only replicates issues from today).
Why we did this (cite Quokka experience and need to test).

Separate script to refresh all Perforce jobs.  Separate script to check
consistency of databases.

Troubleshooting: what to try (see AG, section 12).

a. Look in the P4DTI log. Is there an error message? If so, see if the
error message is listed in Section 12.2, "Error messages".

b. Check your configuration carefully against the instructions in Section
5.2, "P4DTI configuration". Are the hostnames, userids and passwords
correct? Most problems with the P4DTI are caused by incorrect or
inconsistent configuration.

c. If you still can't solve the problem, contact Perforce support (see
http://www.perforce.com/perforce/support.html for details). It will be
helpful if you can provide the following:
 
Common problems (see jobs in Perforce).

a. Can't see fixes

-- Have you set the Version Control checkbbox in your user settings?

-- Has the admin turned on VC Integration in the registry?

How do we share support knowledge?

Writing database queries... see the schema extensions.

What if the P4DTI doesn't support your TeamTrack system?  (We don't
support all field types, e.g. MULTIPLE_SELECTION; we don't support
elapsed times like ESTTIMETOFIX.  This is because Perforce doesn't have
these kinds of fields, so they'd have to be emulated.)

What the admin can do:

-- Add the support themselves (using the Integrator's Guide for help).

-- Ask Perforce to add the support; Perforce can engage Ravenbrook if
   they think it'll be worth it for their (Perforce's) customers in
   general.

-- Engage a third party to do it.  Ravenbrook will quote for any work.
   TeamShare PSG may be able to do this kind of development.


8. The P4DTI from the manager's perspective

There are a number of management decisions that will need to be made:

-- Who will install, configure and maintain the P4DTI.

-- What to do about licences?  The P4DTI can't be used to work around
   licencing arrangements.  In particular, you need a licence in
   TeamTrack in order to be able to transition issues via Perforce.

-- Does the workflow need to be changed to accommodate the integration?
   (Difficulties that can arise: different issue types, multiple
   transitions between states, different states with the same name;
   complex workflow at the Perforce stage).

-- Training.  Perforce consultants?  Ravenbrook?  TeamShare?


9. About the project

It's an open project: source, manuals, design documents and other
supporting materials are all downloadable from Ravenbrook's web site.
(We recommended this to give people confidence in the project.)

It will be free of charge.  But licences are needed for people to use
TeamTrack and Perforce, as usual.

Daemon licences are needed for both Perforce and TeamTrack (the
replicator needs to connect to both server, so it needs a licence).


10. Support arrangements

Here's how I understand the support arrangements will work at TeamShare:

a. TeamShare support will be able to recognize whether a problem
   reported by someone using the P4DTI is to do with TeamTrack or with
   the P4DTI.

b. TeamShare support can handle basic queries about the P4DTI.

c. Queries can be escalated to the local integration expert.

d. The local expert may recognize that the problem is to do with
   Perforce, or is beyond his expertise.  In which case, he should
   advise the caller to contact Perforce support directly.

e. Alternatively, he can continue to deal with the query himself, using
   Perforce support as a resource.  I'll be training Perforce support on
   2001-01-29 and telling them to expect calls of this kind.

We don't recommend trying to pass support calls back and forth, because
this will be confusing.

Perforce support may need to draw on TeamShare support as a resource
when a query requires specialist TeamTrack knowledge.


11. Resources

You can contact me directly, or see the references.


A. References

Perforce Software; <http://www.perforce.com/>.

P4DTI project; <http://www.ravenbrook.com/project/p4dti/>.

P4DTI manuals; <http://www.ravenbrook.com/project/p4dti/master/manual/>.


B. Document history

2001-01-24  GDR  Created.