Perforce Defect Tracking Integration Project


Perforce Defect Tracking Integration Advanced Administrator's Guide

Gareth Rees, Ravenbrook Limited, 2001-11-14

Contents

1. Introduction

This manual is the Perforce Defect Tracking Integration Advanced Administrator's Guide. It describes various advanced methods of administering the P4DTI that are outside the scope of the Perforce Defect Tracking Integration Administrator's Guide because they require a higher level of expertise (see section 1.1, "Required experience").

This document is intended for P4DTI administrators. Ordinary users of the defect tracker or Perforce should read the Perforce Defect Tracking Integration User's Guide.

This guide does not describe how to install, maintain, or perform ordinary configuration and administration of the P4DTI. Read the Perforce Defect Tracking Integration Administrator's Guide.

1.1. Required experience

To do the things described in this guide, you should have the following experience:

  1. Perforce: You must be expert with jobspecs and jobs, confident about backing up and restoring from checkpoints, able to set up a duplicate or test Perforce server. You need good working knowledge of the relevant sections of the Perforce System Administrator's Guide.
  2. The Python programming language: You must be reasonably competent with Python; in particular, able to write functions that transform dictionaries.
  3. The defect tracker: You must know the defect tracker's database schema. You must have detailed knowledge of your workflow and how the workflow puts constraints on the issue data. You must able to make backups, and able to set up a duplicate or test defect tracker.

2. Choosing which issues to replicate

The P4DTI has a general way to select which issues get replicated from the defect tracker: the replicate_p function. To select which issues get replicated you need to edit this function in the file config.py in the installation directory.

Normally, the P4DTI replicates all issues created or modified after the start_date, but you can modify this function to further restrict the issues. See section 5.1, "P4DTI configuration", of the Perforce Defect Tracking Integration Administrator's Guide for more details of the start_date parameter.

Note: once an issue starts being replicated it remains replicated, even if it no longer matches the criteria.

The function is passed one argument, the issue, in the form of a Python dictionary mapping field name (string) to field value. The function must return 1 if the issue should be replicated, or 0 if it should not.

The replicate_p function does not have to check the start_date configuration parameter. The start_date is always honored by the P4DTI, in addition to any test made by replicate_p.

2.1. TeamTrack examples

  1. This example restricts replication to issues belong to the project whose ID is 6:

    
    def replicate_p(issue):
        return issue["PROJECTID"] == 6
    
  2. Suppose you want to replicate only those issues where the "Issue type" is BUG. The issue type is stored in the TS_ISSUETYPE field in the issue table, and the value of that field is the TS_ID field of the appropriate record in the TS_SELECTIONS table [TeamShare 2001-06-19a]. You need to specify a replicate_p function like this:

    
    def replicate_p(issue):
         return issue["ISSUETYPE"] == 1
    

    This example assumes the value 1 is the value for the issue type BUG. Yours might be different, so you need to look in the TS_SELECTIONS table, find the record where TS_PREFIX is BUG, and use the TS_ID field from that record.

2.2. Bugzilla examples

  1. This example restricts replication to unresolved issues in the "horseshoe" product:

    
    def replicate_p(issue):
        return (issue["product"] == "horseshoe"
                and issue["resolution"] == "")
    

3. Allowing users to create issues in Perforce

Normally, the P4DTI requires users to create new issues using the defect tracker. It ignores new jobs created in Perforce. The P4DTI can be configured to transform new jobs created in Perforce into new issues in the defect tracker.

You need to add two functions to the end of config.py: replicate_job_p and prepare_issue.

3.1. The replicate_job_p function

The replicate_job_p function selects which newly-created jobs in Perforce should be translated and added as new issues in the defect tracker. It takes the newly-created job as its argument, in the form of a Python dictionary mapping field name (string) to field value (string). It must return 1 if the job should be translated and added to the defect tracker; 0 if not.

Note: once a job starts being replicated it remains replicated, even if it no longer matches the criteria.

If you want all newly-created jobs to be translated and added to the defect tracker, you should write a replicate_job_p function as follows:


def replicate_job_p(job):
    return 1

3.1.1. TeamTrack example

Suppose you want to replicate newly-created jobs for projects called "horseshoe" and "nail".

In TeamTrack, you add the PROJECTID field to the replicated_fields configuration parameter; this gets replicated to the field "Project" in Perforce; you write a replicate_job_p function as follows:


def replicate_job_p(job):
    return job["Project"] in ["horseshoe", "nail"]

3.1.2. Bugzilla example

Suppose you want to replicate newly-created jobs for products called "horseshoe" and "nail".

In Bugzilla, you add the product field to the replicated_fields configuration parameter; this gets replicated to the field "Product" in Perforce; you write a replicate_job_p function as follows:


def replicate_job_p(job):
    return job["Product"] in ["horseshoe", "nail"]

3.2. The prepare_issue function

The P4DTI calls prepare_issue to complete the transformation of a new job in Perforce to an issue in the defect tracker. The P4DTI does its best to build the new issue, but the defect tracker may require fields that can't automatically be deduced from fields in the job. This function must supply values for these fields.

The function takes two arguments (issue, job) as follows:

  1. issue is the partly-constructed issue, in the form of a Python dictionary mapping field name (string) to field value.
  2. job is the newly-created job, in the form of a Python dictionary mapping field name (string) to field value (string).

The function must modify issue so that it is a valid issue for creation in the defect tracker. Its return value is ignored.

3.2.1. Preparing issues for TeamTrack

In the TeamTrack integration, new issues must have a valid value for these fields:

ISSUETYPE

This must be the TS_ID of a record in the TS_SELECTIONS table in the TeamTrack database (look in the TS_PREFIX field for the name of the issue type). If the ISSUETYPE field is not one of the replicated_fields, or if the value in the job was not the name of an issue type, then the P4DTI picks 0 for issue['ISSUETYPE']. This means that the issue will have no type, just a number. If this is not appropriate, you must supply a suitable value.

PROJECTID

This must be the TS_ID of a record in the TS_PROJECTS table in the TeamTrack database (look in the TS_NAME field for the project name). If the PROJECTID field is not one of the replicated_fields, or if the value in the job was not the name of a project, then the P4DTI sets issue['PROJECTID'] to 0, which is not a valid project, so the issue will fail to be created. You must supply a suitable value.

You can use this function to supply default values for other fields in TeamTrack. See the TeamTrack database schema [TeamShare 2001-06-19] for the full details.

If PROJECTID and ISSUETYPE are both in the replicated_fields, then you may be able to omit this function altogether.

3.2.2. TeamTrack example

This function supplies a default project and issue type if necessary.


def prepare_issue(issue, job):
    if issue["PROJECTID"] == 0:
        issue["PROJECTID"] = 17 # My default project
    if issue["ISSUETYPE"] == 0:
        issue["ISSUETYPE"] = 1 # BUG

3.2.3. Preparing issues for Bugzilla

In the Bugzilla integration, new issues must have a valid value for these fields:

  1. product. This must be the name of a product in the Bugzilla products table. If the product field is not one of the replicated_fields, then the P4DTI picks the empty string for issue["product"] (except when there's only one product, in which case the P4DTI picks that product).
  2. component. This must be the name of a component of the product to which the issue belongs. If the component field is not one of the replicated_fields, then the P4DTI picks the empty string for issue["component"] (except when there's only one component of the product, in which case the P4DTI picks that component).
  3. version. This must be the name of a version of the product to which the issue belongs. If the version field is not one of the replicated_fields, then the P4DTI picks the empty string for issue["version"] (except when there's only one version of the product, in which case the P4DTI picks that version).

You can use this function to supply default values for other fields in Bugzilla. See the Bugzilla database schema [NB 2000-11-14a] for details.

3.2.4. Bugzilla example

This function supplies a default product, component and version if necessary.


def prepare_issue(issue, job):
    default_component = {
        "horseshoe": "user interface",
        "nail": "manufacturing",
        }
    default_version = {
        "horseshoe": "1.7",
        "nail": "2.3beta",
        }
    if issue["product"] == "":
        issue["product"] = "horseshoe"
    product = issue["product"]
    if issue["component"] == "":
        issue["component"] = default_component[product]
    if issue["version"] == "":
        issue["version"] = default_version[product]

4. Migrating to the defect tracker from Perforce jobs

You must delete all jobs from your Perforce installation before running the P4DTI for the first time. The P4DTI takes over the jobs subsystem of Perforce and rewrites the Perforce jobspec.

Migration allows you to keep existing issues that are stored in Perforce jobs. Migration submits all the Perforce job data to the defect tracker, so that the issues can be replicated back to Perforce, and so appear in both systems.

4.1. Planning for migration

You will need a plan for installing and testing the P4DTI and migrating your Perforce jobs. In particular, you will need to have configured the P4DTI replicator before you migrate, and to have a good understanding of what it does. This makes the migration process quite complicated.

This section describes a plan which will allow your users to continue to work with Perforce and your defect tracker until you're ready to go live.

4.1.1. Configuring and testing the P4DTI and migration

The first stage is to configure and test both the P4DTI replicator and migration.

  1. Create an empty Perforce server for testing.

  2. Create a defect tracker for testing. If you have existing defect tracking data, this should be a copy that includes your workflow and at least a representative sample of your existing issues.

  3. Set up and test the P4DTI replicator (without migration) on the test Perforce server and test defect tracker, following the instructions in the Perforce Defect Tracking Integration Administrator's Guide. Ask your users to try it out, and make sure you are happy with the workflow and the P4DTI configuration.

  4. Make a note of the jobspec created by the P4DTI replicator. This will be used in configuring the translation step of the migrator. (See section 4.7, "Translating to the new jobspec".)

  5. Replace the test Perforce server with one which has a copy of all your jobs and fixes. (You can do this by creating a checkpoint of your main Perforce server and restoring it onto a new empty Perforce server. See chapter 2, "Supporting Perforce: Backup and Recovery", of the Perforce 2001.1 System Administrator's Guide.)

  6. Set up and test migration from the test Perforce server to the test defect tracker, as described in section 4.2, "How to migrate".

  7. Apply any workflow changes you made on your test defect tracker to your main defect tracker server.

When you are confident that you have the correct configuration, proceed to section 4.1.2, "Going live with the P4DTI with migration".

4.1.2. Going live with the P4DTI with migration

Once you have configured and tested the P4DTI with migration, follow these steps to make the system available to your users:

  1. Ask your users to stop using the defect tracker and Perforce server.

  2. Make a backup up of your Perforce repository and your defect tracker database, in case there are any serious problems.

  3. Edit your config.py file to point to your main Perforce and defect tracker servers, but don't start the P4DTI replicator yet.

  4. Run the migration and check the results (4.10).

  5. Start the P4DTI replicator.

You should now have a functioning integration with your old issue data intact.

4.2. How to migrate

Migration cannot be done automatically by the P4DTI, because:

  1. The data in Perforce is in a very different format from the defect tracker.
  2. The way the your workflow is implemented in Perforce is likely to be different from the way it's implemented in the defect tracker.

Your existing issues must therefore be translated in order to migrate them from Perforce jobs to the defect tracker.

Here is an overview of preparing for migration. The details of each step are covered in the following sections.

  1. Prepare the Perforce data for migration:

    1. Ensure that users in Perforce match users in the defect tracker (4.3 and 4.4).

    2. Make sure that all Perforce jobs are consistent with the current jobspec so that the P4DTI can transform them systematically (4.5).

  2. Write Python functions to your config.py to do the following:

    1. Choose which jobs to migrate (4.6).

    2. Translate these jobs so that they match jobs that would be replicated from the defect tracker (4.7).

    3. Fix up the issues transformed from jobs so they satisfy any defect tracker constraints on submitted issues (4.8).

  3. Run the migration to transform all the jobs and submit them to the defect tracker (4.10).

To migrate your jobs, work through these subsections.

4.3. Providing names and e-mail addresses for Perforce users

Make sure that every Perforce user has a name and e-mail address.

If you're migrating to TeamTrack, this will allow the P4DTI to correctly match users in Perforce to users in TeamTrack (see section 3.5, "User accounts" of the Perforce Defect Tracking Integration Administrator's Guide).

If you're migrating to Bugzilla, this allows you to automatically make user accounts in Bugzilla for each Perforce user (see section 4.4, "Making defect tracker user accounts").

4.4. Making defect tracker user accounts

Make a defect tracker user account for each Perforce user who needs to do defect tracking.

For TeamTrack, you must create users one by one using the TeamTrack Administrator application. See the TeamTrack Administrator Manual for your version of TeamTrack.

For Bugzilla, you can automatically create a Bugzilla user for each Perforce user. Follow these steps:

  1. Ensure that config.py has correct values for dbms_host, dbms_port, dbms_user and dbms_password.

  2. Add values for the following configuration parameters to config.py (unless you're happy with the default values):

    1. migrated_user_groups must be a list of Bugzilla groups. Newly created users will belong to all the groups in this list. For example, if you set migrated_user_groups = ["editbugs", "canconfirm"] then the newly created users will be able to edit and confirm bugs. The default value is [] (that is, newly created users will belong to no groups).

      To see the Bugzilla groups and their meaning, log into Bugzilla as an administrator and select "edit groups" from the options at the bottom right.

    2. migrated_user_password must be a string. Newly created users will have this password. For example, if you set migrated_user_password = "spong" then the newly created users will have password "spong". The default value is "password".

  3. Go to the P4DTI installation directory.

  4. Run the command python migrate_users.py.

Note that even after going through this process, you may still have jobs in Perforce that can't be translated to bugs in Bugzilla because they have fields containing non-existent users. This can happen easily because you can delete a user in Perforce but still leave them owning jobs.

When you run the migratioon script, you'll see these non-existent users come up with errors like this:

(P4DTI-921X) Migrating job 'job000174'... ... Bugzilla module error (P4DTI-514X) There is no Bugzilla user corresponding to Perforce user 'spong'.

When this happens, choose one of these three solutions:

  1. Add the user to Perforce and migrate it to Bugzilla as described above. (You'll need a spare Perforce licence to do this.)

  2. Edit the job and replace the non-existent user with a real user.

  3. Add the user to Bugzilla and edit the job so that the user's e-mail address appears instead of the user name. The P4DTI can match this address to the user in Bugzilla without the user needing to exist in Perforce.

4.5. Making Perforce jobs consistent with the jobspec

When you change the Perforce jobspec Perforce doesn't update its jobs to match, so if you've changed jobspecs in the past your jobs may be inconsistent with the current jobspec. For example, it may be that you have a "required" field which is missing from some jobs, or a "select" field for which some jobs have an illegal value. These problems will prevent the P4DTI from being able to read the jobs.

To find inconsistencies, go to the P4DTI installation directory and run the command python check_jobs.py (it takes about a minute for every thousand jobs on a typical system). For example:

$ python check_jobs.py
(P4DTI-10081)  Job 'job000024' doesn't match the jobspec:
(P4DTI-7087)  Error detected at line 11.
Value for field 'Status' must be one of open/suspended/closed.

Fix these jobs so that they match the jobspec. When this is done, you'll get the following output:

$ python check_jobs.py
(P4DTI-10092)  All jobs match the jobspec.

4.6. Choosing which jobs to migrate

Choose which jobs to migrate by adding the function migrate_p to the end of config.py.

The migrate_p(job) function determines whether to migrate a job to the defect tracker. The job argument is the job being considered for migration (in the old jobspec), in the form of a dictionary mapping field name to field value. The function must return 1 if the job should be migrated, 0 otherwise.

4.6.1. Examples of migrate_p

  1. To migrate all jobs:
    
    def migrate_p(job):
        return 1
    
  2. To migrate open jobs only:
    
    def migrate_p(job):
        return job["Status"] == "open"
    
  3. To migrate jobs in the "horseshoe" and "nail" projects only:
    
    def migrate_p(job):
        return job["Project"] in ["horseshoe", "nail"]
    

4.7. Translating jobs to the new jobspec

Write a function to translate jobs from the old jobspec to the new jobspec. Add the function translate_jobspec to the end of config.py.

The translate_jobspec function translates a job from the old jobspec (before migration) to the new jobspec (after migration). The old job is passed as the single argument, in the form of a dictionary mapping field name to field value. The new job must be returned in the same form (it's OK to update the argument and return that).

The P4DTI replicator takes over the jobs subsystem of Perforce and rewrites the Perforce jobspec according to the replicated_fields configuration parameter. You should have a copy of the new jobspec, made after testing the P4DTI replicator as described in section 4.1.1, "Configuring and testing the P4DTI and migration".

4.7.1. TeamTrack example

You develop a workflow in TeamTrack and decide to replicate the "Description", "Priority" and "Project" fields (in addition to the system fields "Title", "State" and "Owner" which are always replicated). So your replicated_fields configuration parameter has the value ["DESCRIPTION", "PRIORITY", "FILENAME"] and your jobs look like this under the new jobspec:

Job: job000123
Status: verified
User: gdr
Date: 2001/10/11
Title:
        The widgets are specified in imperial units (ft/lb/s).
Description:
        But 80% of customers use ISO 31, so we should use m/kg/s.
Priority: high
Project: widget

Suppose that you need to make the following changes:

  1. The old jobspec has a single "Description" field, but the new jobspec has both a "Title" (one line) and a "Description" (many lines). You want to split the old description so that its first line becomes the new title.

  2. You need to change the "Status" field to a "State" field and the "User" field to "Owner".

  3. In the old jobspec, status is "open", "closed" or "suspended". In the new jobspec, it is "_new", "assigned", "resolved" or "verified". You want to map "open" to "assigned", and both "closed" and "suspended" to "verified".

Here's how to write a translate_jobspec function that implements all the above:


def translate_jobspec(job):
    # Description -> Title + Description
    import string
    desc = job.get("Description", "")
    newline = string.find(desc, "\n")
    job["Title"] = desc[:newline]
    job["Description"] = desc[newline+1:]

    # User -> Owner
    job["Owner"] = job.get("User", "(None)")

    # Status -> State
    status = job.get("Status", "open")
    if status == "open":
        job["State"] = "assigned"
    else:
        job["State"] = "verified"

    return job

4.7.2. Bugzilla example

You consider the Bugzilla fields and workflow and decide to replicate the resolution, long description, priority and product (in addition to the summary, status and assignee which are always replicated). So your replicated_fields configuration parameter has the value ["resolution", "short_desc", "priority", "product"] and your jobs look like this:

Job: job000123
Status: closed
Assigned_To: gdr
Date: 2001/10/11
Summary:
        The widgets are specified in imperial units (ft/lb/s).
Description:
        But 80% of customers use ISO 31, so we should use m/kg/s.
Resolution: FIXED
Priority: P1
Product: widget

Suppose that you need to make the following changes:

  1. The old jobspec has a single "Description" field, but the new jobspec has both a "Summary" (one line) and a "Description" (many lines). You want to split the old description so that its first line becomes the new summary.

  2. You need to change the "User" field to "Assigned_To".

  3. In the old jobspec, the status is "open", "closed" or "suspended". In the new jobspec, the status is represented by a combination of "Status" and "Resolution" fields. You want to translate status as shown in the table below.

    Old status New status Resolution
    "open" "assigned" ""
    "closed" "closed" "FIXED"
    "suspended" "closed" "LATER"

Here's how to write a translate_jobspec function that implements all the above:


def translate_jobspec(job):
    # Description -> Summary + Description
    import string
    desc = job.get("Description", "")
    newline = string.find(desc, "\n")
    job["Summary"] = desc[:newline]
    job["Description"] = desc[newline+1:]

    # User -> Assigned_To
    job["Assigned_To"] = job.get("User", "")

    # Translate Status
    status_map = {
        "open": ("assigned", ""),
        "closed": ("closed", "FIXED"),
        "suspended": ("closed", "LATER"),
        }
    (status, resolution) = status_map[job.get("Status", "open")]
    job["Status"] = status
    job["Resolution"] = resolution

    return job

4.8. Transforming jobs to issues

Jobs translated to the new jobspec may still not yet be suitable for submission to the defect tracker. The defect tracker may have additional constraints, or require fields that can't be translated from fields in the job. You must therefore write a prepare_issue function as documented in section 3.2, "The prepare_issue function".

4.9. Summary of translation, transformation, and preparation steps

Figure 1 shows a summary of the translate, transform, and prepare steps of the migration process:

  1. The job is translated from the old jobspec to the new jobspec by the translate_jobspec function.

  2. The P4DTI transforms the job to an issue by converting each field in replicated_fields and any system fields.

  3. The issue is prepared for submission to the defect tracker by the prepare_issue function.

Figure 1. The translate, transform, and prepare steps. The example data in the figure is for TeamTrack.

Diagram of the translate, transform, and prepare steps during migration Step 1:
translate from old jobspec to new jobspec Step 2:
translate job to issue. Step 3:
prepare issue for submission to the defect tracker.

4.10. Running migration

Make sure that you have migrated users as described in section 4.4.

Make sure you have the functions migrate_p, translate_jobspec, and prepare_issue in your config.py, as described in the sections above.

Make a checkpoint of the Perforce server as described in chapter 2, "Supporting Perforce: Backup and Recovery", of the Perforce 2001.1 System Administrator's Guide.

Go to the P4DTI installation directory and run the command python migrate.py. If it succeeds, you'll see output like this. (If you see no output, then your migrate_p function isn't matching any jobs.)

2001-11-14 12:34:13  (P4DTI-8920)  Migrated job 'job000001' to issue 'BUG00001'.
2001-11-14 12:34:13  (P4DTI-8920)  Migrated job 'job000002' to issue 'BUG00002'.
2001-11-14 12:34:13  (P4DTI-8206)  -- Added fix for change 3326 with status closed.
...
2001-11-14 12:35:02  (P4DTI-8953)  Migration completed.

Notes for Bugzilla users:

  1. The P4DTI locks the Bugzilla database while migrating, which means that no-one can use Bugzilla until migration completes.

  2. If you've specified a value for bugzilla_directory, you'll see the message "Running processmail for %d bugs..." after the last job is migrated and before the "Migration completed" message appears. This is because Bugzilla's processmail script is being run for each migrated issue. (If you don't want notifications to be sent when migrating, set bugzilla_directory=None while migrating and reset it afterwards.)

If migration fails, you'll typically get one of these kinds of error:

  1. The replicator can't transform a job to an issue; for example:

    (P4DTI-6288)  No TeamTrack state in project 'Horseshoe' corresponding to Perforce state 'suspended'.

    This means that the translate_jobspec function needs to consider more possibilities (in the example, it needs to translate the Perforce status suspended into a state in TeamTrack.

  2. The replicator can't create an issue in the defect tracker; for example:

    (P4DTI-522X)  Can't create Bugzilla bug without component field.

    This means that the prepare_issue function has failed to meet all the defect tracker constraints (in the example, it needs to provide a value for the component field).

When you've fixed the problem, run python migrate.py again. Provide the command-line option -s JOBNAME to start migrating at a particular job. For example, if migration failed because of a problem job000123, then you might continue migrating with python migrate.py -s job000123.

Check the migrated issues. Are they as you expected? If not, fix the problem and run the migrator again.

Replace the existing jobs in Perforce with jobs replicated from the defect tracker's database by refreshing them as described in section 9.2, "Refreshing jobs in Perforce", of the Perforce Defect Tracking Integration Administrator's Guide.

Check the consistency of the replicated data by running the consistency checker as described in section 7.3, "Checking data consistency", of the Perforce Defect Tracking Integration Administrator's Guide. If this fails you may be able to fix up problems by hand. If not, you can restore your Perforce server from the checkpoint you made earlier.

Both the defect tracker and the Perforce server now contain transformed issues, and the system is prepared for starting the P4DTI replicator.

5. Integrating with multiple Perforce servers

The P4DTI can be set up to replicate defects from a single defect tracker to multiple Perforce servers. For example, if you have several development groups, each with their own Perforce server, you can set up the P4DTI to replicate the issues for each group to the appropriate Perforce server.

To do so:

  1. Install a copy of the P4DTI for each Perforce server, as described in readme.txt.

  2. Configure each copy of the P4DTI. These configuration parameters must be different for each copy:

    1. rid, so that the replicators can distinguish the issues they are responsible for.
    2. sid and p4_server_description, so that the defect tracker can report which Perforce server the issue is replicated to.
    3. p4_port, so each replicate can find its Perforce server (depending on how you have the Perforce servers set up, you may need to change p4_user and p4_password as well).
    4. replicate_p, so that the replicators don't try to replicate the same issues.
  3. Start and manage each replicator as described in the Perforce Defect Tracking Integration Administrator's Guide.

Example. You have Perforce servers for your development groups in France and Germany and a single TeamTrack server. Projects 4, 7 and 11 are worked on in France and projects 5, 6, 9 and 10 are worked on in Germany. So you set up one P4DTI copy with the following in config.py:

rid = "replicator_fr"
sid = "france"
p4_port = "perforce.company.fr:1666"
p4_server_description = "French development group"
def replicate_p(issue):
    return issue["PROJECTID"] in [4,7,11]

and the other with the following:

rid = "replicator_de"
sid = "germany"
p4_port = "perforce.company.de:1666"
p4_server_description = "German development group"
def replicate_p(issue):
    return issue["PROJECTID"] in [5,6,9,10]

Notes:

  1. The P4DTI doesn't support integration between one Perforce server and multiple defect tracking servers.

  2. There's no way to move issues from one Perforce server to another. Once an issue is replicated to one Perforce server then it will always be replicated to that server, even if it later matches the replicate_p function for another replicator. If it's going to be doubtful which development group is going to work on an issue, then you need to design your workflow so that issues don't get replicated until it's clear who's going to work on them.

    For example, you might have a replicate_p function like this:

    def replicate_p(issue):
        return (issue["PROJECTID"] in [4,7,11]
                and issue["STATE"] == 19)
    

    (where state 19 is "Assigned"), so that you get an opportunity to take a decision on the issue and possibly move it to a different project, before it gets replicated.

6. Querying the TeamTrack database

When integrating with TeamTrack, in order to write replicate_p and prepare_issue functions, you need to know the key values for the tables in the TeamTrack database.

You can do this by looking directly at the TeamTrack database using your database tool. Or you can use the teamtrack_query.py script that comes with the P4DTI.

Use it as follows:

  1. Provide values in config.py for the teamtrack_server, teamtrack_user and teamtrack_password configuration parameters.

  2. Run the command python teamtrack_query.py TABLE to query a table and show all fields for all records.

  3. Run the command python teamtrack_query.py TABLE FIELD1 FIELD2 ... to find the contents of a table and restrict the output to the named fields.

  4. Add the option -q QUERY to restrict the output to the rows matching the query.

Use the TeamTrack database schema [TeamShare 2001-06-19a] to work out which table you need to query and how to understand the output.

Example 1. You want to replicate issues from projects called "Horseshoe" and "Nail". The TeamTrack database schema says that the value in an issue's PROJECTID field is the ID field of a record in the PROJECTS table. So you run this command:

> python teamtrack_query.py PROJECTS ID NAME
+------+----------------+
|  ID  |  NAME          |
+------+----------------+
|  1   |  Base project  |
|  2   |  Horseshoe     |
|  3   |  Cart          |
|  4   |  Nail          |
|  ... |  ...           |
+------+----------------+

So you can write:

def replicate_p(issue):
    return issue["PROJECTID"] in [2,4]

Example 2. When a new job is created in Perforce, you want to create an issue in TeamTrack with issue type "BUG". The TeamTrack database schema says that an issue type is the ID field of a record in the SELECTIONS table where the PREFIX field gives the prefix for the issue id. So you run the command:

> python teamtrack_query.py SELECTIONS ID PREFIX NAME
+------+----------+------------------------+
|  ID  |  PREFIX  |  NAME                  |
+------+----------+------------------------+
|  0   |          |  (None)                |
|  1   |  BUG     |  Bug report            |
|  2   |  ENH     |  Enhancement proposal  |
|  ... |  ...     |  ...                   |
+------+----------+------------------------+

So you can write:

def prepare_issue(issue):
    issue["ISSUETYPE"] = 1

A. References

[GDR 2000-09-04] "TeamTrack database schema extensions for integration with Perforce"; Gareth Rees; Ravenbrook Limited; 2000-09-04; <http://www.ravenbrook.com/ project/p4dti/version/1.3/ design/teamtrack-p4dti-schema/>.
[NB 2000-11-14a] "Bugzilla database schema"; Nick Barnes; Ravenbrook Limited; 2000-11-14.
[Perforce 2001-06-18b] "Perforce 2001.1 System Administrator's Guide"; Perforce Software; 2001-06-18; <http://www.perforce.com/perforce/doc.011/manuals/p4sag/>, <ftp://ftp.perforce.com/pub/perforce/r01.1/doc/manuals/p4sag/p4sag.pdf>.
[RB 2000-08-10a] "Perforce Defect Tracking Integration Administrator's Guide"; Richard Brooksby; Ravenbrook Limited; 2000-08-10.
[TeamShare 2001-06-19a] "TeamTrack Database Schema (Database Version: 514)"; TeamShare; 2001-04-30.
[TeamShare 2001-06-19b] "TeamTrack Administrator Manual 5.0"; TeamShare; 2001-06-19.

B. Document History

2001-11-14 GDR Branched from Administrator's Guide.
2001-11-21 GDR Note the password for migrated Bugzilla users.
2001-11-22 GDR Documented the need to refresh Perforce jobs after migrating if you want to move to an integrated system.
2001-11-23 RB Substantial revision, editing for clarity and adding details where necessary. Devised and added a section giving a plan for installing with migration.
2001-11-25 GDR Added material on replicating to multiple Perforce servers and on querying the TeamTrack database.
2001-11-27 GDR Added instructions for setting migrated_user_groups and migrated_user_password. Documented the -s option to the migration script.

This document is copyright © 2001 Perforce Software, Inc. All rights reserved.

Redistribution and use of this document in any form, with or without modification, is permitted provided that redistributions of this document retain the above copyright notice, this condition and the following disclaimer.

This document is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this document, even if advised of the possibility of such damage.