Figure 1. The replication architecture
Perforce Defect Tracking Integration Project
This manual is the Perforce Defect Tracking Integration Administrator's Guide. It explains how to install, configure, maintain, and administer the Perforce Defect Tracking Integration (P4DTI).
This document is intended for P4DTI administrators. Ordinary users of the defect tracker or Perforce should read the Perforce Defect Tracking Integration User's Guide. (For ideas on how to train your users on the P4DTI, see section 8, "Training and documentation".)
This guide does not describe the basics of using the P4DTI, Perforce, or the defect tracker. Read the Perforce Defect Tracking Integration User's Guide to understand the P4DTI from a user's perspective.
To install and run the P4DTI, you must:
The P4DTI works by taking over the job tracking system of Perforce and making the defect tracker's records appear as Perforce jobs. Perforce users can work with jobs more or less as described in the Perforce manuals, and their changes are reflected in the defect tracker. For more information on how Perforce handles jobs, see the Perforce Command Line User's Guide.
Perforce has a mechanism for linking jobs to changelists (the p4 fix command), to enable you to record the work done for a particular reason. The P4DTI makes these links appear in the defect tracker, making it easy to see what was done or is currently being done to resolve a defect.
The P4DTI replicator is a process that copies data between a defect tracker and a Perforce server to keep each one up to date with changes made in the other. This approach allows developers to do their routine defect resolution work entirely from their Perforce client, without using the defect tracker's interface. It also allows developers to relate their changes to defect tracking issues.
Figure 1 shows how the replicator communicates with the defect tracking server and the Perforce server.
The replicator maintains a one-to-one relationship between issues in the defect tracker's database and jobs in the Perforce repository. (An issue is a unit of work that the defect tracker tracks; some examples are bugs, change requests, and enhancement requests.) In other words, each issue has a corresponding job, and vice versa. The replicator keeps the contents of a configurable set of fields in the defect tracker's issues the same as the contents of the corresponding Perforce job, so that editing one edits the other.
The replicator also copies Perforce's links between jobs and changelists (called "fixes") to the defect tracker's database, and makes them visible in the defect tracker's user interface. Replication of links from Perforce to the defect tracker makes it possible to track, record, and check a number of things; in particular, it makes it possible to track and record the changes made for each issue, and find out why a change was made in terms of issues.
The replicator polls the defect tracking server and the Perforce server at regular intervals to get a list of recent changes, and attempts to propagate these changes to the other system. If a defect tracker issue is changed at the same time as the corresponding Perforce job, the replicator sends an e-mail with the overwritten Perforce job data to the following people:
Most defect trackers have an idea of workflow--a set of rules that control who can do what to which issues. The replicator enforces the defect tracker's workflow by rejecting changes to jobs in Perforce that are illegal in the defect tracker. When it comes across such a change, it undoes the change and sends an e-mail message to the user.
The defect tracker manages the defect tracker records (and therefore the job contents), while Perforce manages the changelists. Neither side controls the "fixes" relationship--the links between jobs and changelists.
Figure 1 shows how the replicator connects to the Perforce and defect tracker servers.
Figure 1. The replication architecture
The P4DTI won't work well for every organization. In particular, it has the following limitations:
If you're using TeamTrack and your workflow is very complex then the P4DTI may make mistakes when translating actions in Perforce into actions in TeamTrack.
(This is because TeamTrack supports operations that can't be represented in Perforce, such as a "transition" between two states in a workflow. When someone edits a job in Perforce, the P4DTI tries to guess which TeamTrack transition was meant by looking at the state you started with and the state you ended up in. However, if you have multiple transitions between the same two states, then it may guess wrongly. See section 3.3.2, item 6 for details of workflows that the P4DTI may have trouble with.)
If you have automation that makes very frequent changes to records in the defect tracker, then users may be unable to edit jobs from Perforce.
(This is because the P4DTI works by polling the databases every 10 seconds (by default). If there are many updates to the defect tracker in this period then the data in Perforce will be out of date; any attempt to edit out-of-date data is rejected by the P4DTI because that would introduce inconsistencies.)
To administer the P4DTI, you must have the following experience:
Before installing the P4DTI, you must obtain and install the following software:
Before installing the P4DTI, you must do the following:
It is possible to keep existing issues that are stored in Perforce jobs. Migration submits all the Perforce job data to the defect tracker, so that the issues can be replicated back to Perforce, and so appear in both systems. This requires more experience of Perforce and some Python programming. See section 4, "Migrating to the defect tracker from Perforce jobs", of the Perforce Defect Tracking Integration Advanced Administrator's Guide.
Before installing the P4DTI, you must obtain and install the following software:
use_windows_event_log configuration parameter Before installing the P4DTI, you must do the following:
http:, not https:).
The P4DTI does
not support TeamTrack running on a secure web server. Make sure you don't have two states with the same name in a project. For example, when using the workflow in figure 2, there's no way for a developer using Perforce to resolve the issue and assign it to the tester. The developer would normally resolve the issue by changing the status field, but in this workflow the status field doesn't change. Rename states so that the workflow has unique state names. For example, in figure 2, name the second "Assigned" state "Resolved".
Figure 2. Workflow with two states with the same name
Make sure you don't have two transitions between the same two states, in the same direction. For example, when using the workflow in figure 3, when the developer using Perforce changes the state of the job from "assigned" to "resolved", the P4DTI has no way to work out which transition to apply. Simplify the workflow so that the transition can be deduced from the start state and the end state. For example, in figure 3 you could have a single transition from "Assigned" to "Resolved" and require developers to specify how they resolved the problem in a field in the defect.
Figure 3. Workflow with two transitions between the same two states (in the same direction)
Make sure you don't have "Update" transitions (transitions from a state to itself) that are necessary in your workflow. For example, when using the workflow in figure 4, there's no way for the developer to cause the "Assign to tester" transition using Perforce. The developer would normally resolve the issue by changing the status field, but in this workflow the status field doesn't change. Simplify the workflow so that it doesn't rely on "Update" transitions.
Figure 4. Workflow with a necessary "Update" transition
In fact these problems only matter when the problematic part of the workflow needs to be carried out in Perforce. As long as you know that the problematic part of workflow is only carried out in TeamTrack, then the P4DTI will work fine.
Before installing the P4DTI, you must obtain and install the following software:
syslog module. An RPM of Python 1.5.2 for RedHat Linux 6.2 is available from <http://www.ravenbrook.com/project/p4dti/import/1999-04-13/python-1.5.2/python-1.5.2-13.i386.rpm>. An RPM of Python 1.5.2 for RedHat Linux 7.0 is available from <http://www.ravenbrook.com/project/p4dti/import/1999-04-13/python-1.5.2/python-1.5.2-27.i386.rpm>. Before installing the P4DTI, you must do the following:
You must ensure that the P4DTI can work out how users in the defect tracker correspond to users in Perforce, as follows:
Create an account in each system for each user who needs to use the integrated system.
If you are using Bugzilla, create a Bugzilla account for every user in Perforce.
If you are using TeamTrack, then you must make sure that every Perforce user who needs to edit or fix defects has an account in TeamTrack. (If you're short of licenses, then there's no need for other TeamTrack users to have accounts in Perforce, or for Perforce users who don't do defect resolution to have accounts in TeamTrack.)
Make sure that each user has the same e-mail address in the defect tracker and in Perforce. The P4DTI uses e-mail address to match up users between the two systems.
(This design is necessary in the Bugzilla integration where your e-mail address is used as your userid. In the TeamTrack integration this allows the P4DTI to work in organizations where users have been assigned different userids in the two systems.)
There are three problems that can occur if the P4DTI can't match up users properly:
To help you prevent these problems, each time you start the P4DTI it sends an e-mail message to the administrator containing a report listing the unmatched and duplicate users. An example report is shown in figure 5. Read reports you receive and fix the problems.
Figure 5. Example e-mail sent when the P4DTI starts
|
Notes on figure 5:
Section 1 lists Perforce users that couldn't be matched to a user in the defect tracker (here TeamTrack). Here the user hasn't set their e-mail address in Perforce: they still have the default.
Section 2 lists defect tracker users that couldn't be matched to a user in Perforce.
Section 3 lists Perforce users with duplicate e-mail addresses. In this case, if a defect tracker user had the address <ndl@company.domain> then the P4DTI may have matched them wrongly. Give each Perforce user a distinct e-mail address.
Section 4 lists defect tracker users with duplicate e-mail addresses. Give each defect tracker user a distinct e-mail address.
Note: You might want to practice installing and configuring the P4DTI using a test Perforce repository and a test defect tracking database before you try it with your real data. A copy of your real Perforce repository would be ideal; for instructions on how to make a copy of your repository, see the Perforce System Administrator's Guide.
The P4DTI can be installed on any machine that can communicate with the defect tracker's server and the Perforce server. To keep administration simple and reduce network traffic, install and run the P4DTI on the same machine as the defect tracker's server. The rest of this manual assumes that you do this.
For instructions on how to upgrade from an earlier version of the P4DTI, see the readme.txt file.
The P4DTI is distributed as a self-extracting executable called p4dti-DT-RELEASE.exe (where DT is the defect tracker, such as "teamtrack", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, run this executable on the machine where the defect tracker server is installed. The installer unpacks the P4DTI into C:\Program Files\P4DTI-RELEASE\ by default.
The P4DTI is distributed as an RPM called p4dti-RELEASE-1.i386.rpm where DT is the defect tracker, such as "bugzilla", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, run the following command as root on the defect tracker server machine:
rpm -i p4dti-RELEASE-1.i386.rpm This installs the P4DTI files into /opt/p4dti and a startup script in the /etc/rc.d/init.d directory.
If you prefer not to use RPMs, you can follow the procedure in section 4.4, "Solaris installation".
The P4DTI is distributed as a gzipped tar file called p4dti-DT-RELEASE.tar.gz (where DT is the defect tracker, such as "bugzilla", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, unpack this tar file on the defect tracker server machine, using the following command:
gunzip -c p4dti-DT-RELEASE.tar.gz | tar xvf -You must determine where to put the files. You can put the files wherever you want.
Work through the subsections in the order in which they appear. Do not attempt to run the P4DTI until you have reached the end of this section, or you might end up with a non-working installation.
To configure the P4DTI with Perforce and your defect tracker, you must:
To configure the P4DTI, you edit definitions of Python variables in the file config.py in the installation directory. Edit these definitions according to the notes below. All variables in the file must have a value.
dt_name Description: The name of the defect tracking system you're integrating with. Either "TeamTrack" or "Bugzilla".
Example: "TeamTrack"
Make sure that this variable is set to the appropriate value for your defect tracker.
administrator_address Description: The e-mail address of the P4DTI administrator.
Example: "p4dti-admin@company.domain"
The replicator sends error reports to this address. If this is
None, then the replicator never sends
e-mail.
p4_port Description: The address and port of the Perforce server with which the replicator communicates.
Example: "perforce.company.domain:1666"
p4_user Description: The userid that the replicator uses to log in to the Perforce server.
Example: "p4dti-replicator0"
For information about how the replicator logs in to Perforce, see section 5.2, "Perforce configuration". If you want to add more replicators later, incorporate the replicator identifier (rid) into this userid.
p4_password Description: The password the replicator uses to log in to the Perforce server. If there is no password, specify "" (empty quotes).
Example: ""
For information about how the replicator logs in to Perforce, see section 5.2, "Perforce configuration".
replicator_address Description: The e-mail address from which the replicator sends e-mail. This address is used in the "From" field of e-mail that the replicator sends.
Example: "p4dti-replicator0@company.domain"
To make it easier for users to get assistance, make this address an alias for the administrator e-mail address (administrator_address). If you are using Bugzilla, this e-mail address is also used for the replicator's Bugzilla account; see section 5.4.2, "Creating a Bugzilla user for the replicator".
smtp_server Description: The address of the SMTP server that the replicator uses to send e-mail.
Example: "smtp.company.domain"
If this is None, then the replicator
never sends e-mail.
If you need to run the P4DTI without being connected to a network (for
example, if you want to set it up on a laptop so that you can give a
demonstration), set smtp_server=None so
that the replicator doesn't try to send e-mail.
start_date Description: The starting point in time for replication.
Example: "2001-02-10 00:00:00"
Issues modified after this date are replicated; issues unchanged
after this date are ignored. Must be a string in the form "YYYY-MM-DD HH:MM:SS".
teamtrack_version Description: The major version of your TeamTrack server. Specify
"4.5" for TeamTrack 4.5, or "5.0" for TeamTrack 5.0 or later.
Example: "5.0"
closed_state Description: The TeamTrack that maps to the "closed" state in
Perforce. Specify None if you want the
ordinary state mapping rules to apply.
(Note that you must write None
literally, not the string "None", which
would mean the state called "None").
Example: "Resolved".
Mapping the TeamTrack state that developers use most often to the
"closed" state in Perforce makes using the P4DTI easier for the developers, because
the Perforce user interfaces make it easier to fix a job to "closed"
than any other state. However, if your workflow already has a state
called "Closed", then you can't use this feature; set closed_state = None.
replicated_fields Description: A list of the database names of TeamTrack fields that are replicated in Perforce. The fields STATE, OWNER, and TITLE are always replicated, so omit those fields when setting this variable.
Example: ["DESCRIPTION", "PRIORITY", "SEVERITY"]
For advice on which fields to replicate, and how to find out their database names, see section 5.1.5, "Choosing which fields to replicate".
teamtrack_server Description: The TeamTrack server hostname and (optionally) port with which the replicator communicates.
Example: "teamtrack.company.domain"
If your TeamTrack server is reached on a path other than /tmtrack/tmtrack.dll? then you must specify the
full path to the server in this variable. For example, "http://server.company.domain/infosystem/teamtrack/tmtrack.dll?".
(Note that "localhost" won't work, even
if the TeamTrack server is on the local host.)
teamtrack_user Description: The user name that the replicator uses to log into TeamTrack.
Example: "P4DTI-replicator0"
See section 5.3.2, "Creating a TeamTrack user for the replicator".
teamtrack_password Description: The password that the replicator uses to log into TeamTrack. If there is no password, specify "" (empty quotes).
Example: ""
See section 5.3.2, "Creating a TeamTrack user for the replicator".
use_windows_event_log Description: The replicator logs activity to the Windows event log if (and only if) this is 1.
Example: 1
If you set this to 1, you must make sure to install the Python interface to Windows (item 5 in section 3.3.1, "Software prerequisites").
Regardless of the setting of this parameter, the replicator also
logs activity to to the standard output and to the log file (log_file).
The replicator can generate very many log messages. So if you set
this parameter to 1, either specify "Overwrite events as needed" in the
Windows Event Viewer on the machine running the replicator, or else set
the log_level to a restrictive value like message.LOG_WARNING.
closed_state Description: The Bugzilla state that maps to the "closed" state in Perforce. Specify None if you want the ordinary state mapping rules to apply.
(Note that you must write None
literally, not the string "None", which
would mean the state called "None").
Example: "RESOLVED".
Mapping the defect tracker state that developers use most often to
the "closed" state in Perforce makes using the P4DTI easier for the developers,
because the Perforce user interfaces make it easier to fix a job to
"closed" than any other state. If you specify a closed_state then the "CLOSED" state in Bugzilla
maps to "bugzilla_closed" in Perforce.
replicated_fieldsDescription: A list of the names of Bugzilla fields that are replicated in Perforce. The fields "bug_status", "short_desc", "assigned_to" and "resolution" are always replicated, so omit those fields when setting this variable.
Example: ["longdesc", "priority", "bug_severity", "product"]
For advice on which fields to replicate, see section 5.1.5, "Choosing which fields to replicate".
dbms_host Description: The host on which the Bugzilla MySQL server is running.
Example: "localhost"
Set this value to "localhost" if the P4DTI and the Bugzilla MySQL server run on the same machine.
dbms_port Description: The port number on the database host (dbms_host), on which the Bugzilla MySQL server
listens.
Example: 3306
MySQL normally listens on port 3306. Change this setting only if you have set up MySQL differently. Note that this parameter is expressed as a number, not as a string.
dbms_database Description: The name of the MySQL database in which Bugzilla stores its data.
Example: "bugs"
Normally set to "bugs" during Bugzilla installation (see the Bugzilla documentation for 2.10, 2.12, 2.14). Change this setting only if you have set up Bugzilla differently.
dbms_user Description: The user name that the replicator uses to log in to MySQL to use the Bugzilla database.
Example: "bugs"
Bugzilla normally logs in to MySQL as user "bugs" (see the Bugzilla documentation for 2.10, 2.12, 2.14). Change this setting only if you have configured Bugzilla differently, or if you want to set up the replicator to log in as a different user.
dbms_password Description: The password that the replicator uses to log in to MySQL to use the Bugzilla database.
Example: ""
Bugzilla normally logs in with no password (see the Bugzilla documentation for 2.10, 2.12, 2.14). Change this setting if you have configured Bugzilla differently, or you want to set up the replicator to log in as a different user and use a password.
bugzilla_directory Description: The directory in which Bugzilla is installed, or None if you don't want e-mail processed.
Example: "/home/httpd/html/bugzilla"
Bugzilla sends e-mail to its users when it notices that a bug has been changed. If the P4DTI is running on the Bugzilla server, it is able to use Bugzilla's processmail script to promptly send e-mail in the same way. This configuration parameter allows the P4DTI to locate processmail. Set it to None if the P4DTI is not running on the Bugzilla server or if you don't want the P4DTI to send these e-mail messages.
These parameters support advanced or rarely-used features. Most organizations can leave these parameters at their default values, at least to start with, and then set them later if necessary.
changelist_url Description: A format string used to build a URL for a changelist.
Specify None if there are no URLs for
changelists.
This is used by the defect tracker to provide a link from a fix to a web page providing more information about the changelist that fixed the issue. Figure 6 shows how this works in Bugzilla.
The value must be a format string valid for passing to sprintf(); it must have one %d format specifier, for which the change number
is substituted. (Note that because the value gets passed to sprintf(), you must double other percent signs.)
In order to use this feature, you must have a web application that can provide information about changelists. Applications suitable for this include:
P4Web. To allow any user to follow the link from the defect tracker to P4Web, you need to run it in "browse mode". See the P4Web documentation for details.
For P4Web, set changelist_url to
something like "http://info.company.domain:8080/%d?ac=10".
Perfbrowse.
For Perfbrowse, set changelist_url to
something like "http://info.company.domain/cgi/perfbrowse.cgi?@describe+%d".
Figure 6. Effect of changelist_url and job_url
job_url Description: A format string used to build a URL for job descriptions. Specify None if there is no URL for job descriptions.
This is used by the defect tracker to provide a link from an issue to a web page providing more information about the job that corresponds to the issue. Figure 6 shows how this works in Bugzilla.
Example: "http://info.company.domain/cgi/perfbrowse.cgi?@job+%s"
The string is a format string valid for passing to sprintf(); it must have one %s format specifier, for which the job name is substituted. (Note that because it gets passed to sprintf(), you must double other percent signs.)
log_file Description: The name of the replicator's log file. If None, messages aren't logged to any file. (Note
that you must write None literally, not the
string "None", which would mean the file
called "None").
Example: "C:\\Program Files\\P4DTI-RELEASE\\p4dti.log"
The replicator generates log messages to record its actions. These log messages are sent to all of the following locations:
use_windows_event_log is 1). log_file configuration parameter (unless it's None). log_level Description: The minimum priority level of messages to log. Messages with this priority or a higher priority appear in the replicator's log.
Example: message.INFO
This parameter must be one of these constants:
message.ERR |
Errors. |
message.WARNING |
Warnings; that is, features of your system that the replicator can work around, but which you should pay attention to. For example, "Table 'PROJECTS' has two entries called 'Compiler'.". |
message.NOTICE |
Significant but expected events. For example, "Job 'BUG00001' overwritten by issue 'BUG00001'.". |
message.INFO |
Informational messages. For example, "Replicating issue '123' to job 'BUG000123'." |
message.DEBUG |
Debugging messages. For example, "Perforce command: 'p4 -G -u p4dti-replicator0 -p perforce:1666 job -o BUG00001'." |
p4_client_executable Description: The location of the Perforce client executable.
Example: "C:\\Program Files\\Perforce\\p4.exe"
This setting doesn't need to be an absolute path name if the directory is on the replicator user's path. On Windows this setting might be "C:\\Program Files\\Perforce\\p4.exe". On UNIX it might be just "p4".
The client executable named by this parameter must be of version
2000.2 or later (run the command p4 -V to
check the client version), and it must be the same version as the
Perforce server you are connecting to. If there's a mismatch between
the Perforce client executable and the Perforce server, then you might
see the error message (P4DTI-7087)
Value for field 'Options' must be one of ....
p4_server_description Description: A description of the Perforce server. This might be used by the defect tracker to show which Perforce server an issue is replicated to.
Example: "Hardware development group Perforce server"
poll_period Description: The period of time between the end of one poll of the servers and the start of the next, in seconds.
Example: 10
prepare_issue(issue, job) Description: A function that prepares a new issue for submission to the defect tracker by providing values for all the required fields.
See section 3, "Allowing users to create issues in Perforce" in the Perforce Defect Tracking Advanced Administrator's Guide for the full details.
replicate_p(issue) Description: A function that selects which issues to start
replicating. Normally, the P4DTI replicates all issues created or modified
after the start_date, but you can modify this
function to further restrict the issues.
See section 2, "Select the issues to replicate" in the Perforce Defect Tracking Advanced Administrator's Guide for the full details.
replicate_job_p(job) Description: A function that selects which jobs in Perforce to replicate. Normally, the P4DTI ignores jobs created in Perforce, but you can provide this function to allow users to create jobs in Perforce and have them replicated to the defect tracker.
See section 3, "Allowing users to create issues in Perforce" in the Perforce Defect Tracking Advanced Administrator's Guide for the full details.
rid Description: The replicator identifier.
Example: "replicator0"
Must be 32 characters or less, start with a letter or underscore, and consist only of letters, numbers, and underscores.
The replicator identifier is used to distinguish between replicators when multiple replicators are being used to replicate issues from a defect tracker to different Perforce servers. If you have only one replicator, it doesn't matter what you use for the replicator identifier; "replicator0" is a good choice since it allows you to add more replicators later.
If you change the replicator identifier then your currently replicated defect tracker issues stop being replicated. The replicator believes they are being handled by another replicator.
sid Description: The Perforce server identifier.
Example: "perforce0"
Must be 32 characters or less, start with a letter or underscore, and consist only of letters, numbers and underscores. You might want to use the hostname of your Perforce server, if it is stable.
use_deleted_selections Description: If this is 1, deleted TeamTrack states and selections
appear as options for corresponding fields in Perforce. If this is
0, they don't appear in the Perforce jobspec
or in drop-down menus in P4Win.
Example: 0
Warning: It is risky to set this to 0 because TeamTrack
never really deletes a state or selection; it just marks it as deleted.
So you may have issues in TeamTrack that use those deleted states and
selections. If use_deleted_selections is
0 then these issues can't be replicated to Perforce. If
this happens, you'll see errors like these:
(P4DTI-6255) No Perforce state corresponding to TeamTrack state 'spong'.
If this happens you can undelete the state in TeamTrack and restart the replicator. Or you can change the state of the issue in TeamTrack so that it can be replicated.
(P4DTI-7087) Value for field 'Severity' must be one of Critical/High/Medium/Low/(None).
If this happens you can undelete the selection in the Severity field in TeamTrack and restart the replicator. Or you can change the severity of the issue in TeamTrack so that it can be replicated.
use_perforce_jobnames Description: Determines whether the replicator uses Perforce-style jobnames.
If this parameter is 1, the P4DTI lets Perforce choose the names of the jobs it creates when replicating issues from the defect tracker (so jobs will be named job000001, job000002 and so on). This means that the job name won't match the name of the corresponding issue in the defect tracker.
If this parameter is 0 (the default), the P4DTI tries to match the defect tracker's names for the issues it replicates. In the TeamTrack integration, jobs are called BUG00001, ENH00002, and so on. In the Bugzilla integration, jobs are called bug1, bug2, and so on.
Example: 1
If you change this setting, the P4DTI doesn't rename existing jobs, but new jobs get the style of name you requested.
Here's some advice on which fields to replicate:
FIX_DESCRIPTION field for developers to explain what they did. Now that you're running the P4DTI, you don't need that field--you can look in the change comments of the associated changelists to find out what the developer did--so leave it out. If you're using TeamTrack's sample database, you might want to replicate the following fields:
DESCRIPTION, so that developers can understand what the problem is. SEVERITY, so that developers can prioritize their work. PRIORITY, so that developers can prioritize their work. To find out the database name of a TeamTrack field, follow these steps:
Table 1 shows the field types in TeamTrack and indicates which ones may be replicated by the P4DTI.
Table 1. Supported field types in TeamTrack
| Field type | Field contents | Replicable by P4DTI? |
|---|---|---|
NUMERIC |
Numeric field, integer or floating-point | Yes |
TEXT |
Text field up to 255 characters | Yes |
MEMO |
Memo field | Yes |
DATETIME |
Date/Time field | Yes |
SELECTION |
Drop down selection field, one selectable | Yes |
BINARY |
Binary (two-state) field | Yes |
STATE |
The system-defined state field | Yes |
USER |
A drop down selection field containing user names | Yes |
PROJECT |
System-defined project field | Yes |
SUMMATION |
Calculated summation fields | No |
MULTIPLE_SELECTION |
Multi-select selection field | No |
CONTACT |
Contact selection field | No |
COMPANY |
Company selection field | Yes |
INCIDENT |
Incident selection field | No |
PRODUCT |
Product selection field | Yes |
SERVICEAGREEMENT |
Service Agreement | Yes |
FOLDER |
Folder link selection field | No |
KEYWORDLIST |
Keyword multi-select field | No |
PRODUCTLIST |
Product multi-select field | No |
PROBLEM |
Problem selection field. | No |
RESOLUTION |
Resolution selection field | No |
MERCHANDISE |
Merchandise selection field | No |
RELATIONAL |
Relational selection field | No |
SUBRELATIONAL |
Sub-relational selection field | No |
SYSTEM |
System field | No |
MULTIPLE_RELATIONAL |
Multiple relational selection field. | No |
Attachments such as notes are stored in separate tables in TeamTrack and are not replicated by the P4DTI. If you need to have supplementary information replicated to Perforce, use a memo field like "Additional Notes". In TeamTrack you can turn any memo field into a "journal" field which consists of a list of entries, each headed with the date of the entry and the name of user who added it.
If you're using Bugzilla, you might want to replicate the following fields:
If you're using Bugzilla, the replicator rejects the following types of changes from within Perforce:
The following table lists the field names for Bugzilla 2.10, 2.12, and 2.14. If you have modified Bugzilla, your field names may differ. To display the set of Bugzilla field names, type mysqlshow bugs bugs at a shell prompt.
Table 2. Bugzilla field names
| Field name | Name on Bugzilla form | Replication policy |
|---|---|---|
| bug_id | Bug # | always, read only |
| bug_status | Status | always, read/write |
| assigned_to | Assigned To | always, read/write, user |
| short_desc | Summary | always, read/write |
| resolution | Resolution | always, read/write |
| bug_file_loc | URL | read/write |
| bug_severity | Severity | read/write |
| op_sys | OS | read/write |
| priority | Priority | read/write |
| rep_platform | Platform | read/write |
| reporter | Reporter | read/write, user |
| qa_contact | - | read/write, user |
| status_whiteboard | Status Whiteboard | read/write |
| longdesc | Description | append only |
| groupset | - | read only |
| creation_ts | Opened | read only |
| delta_ts | - | read only |
| product | Product | read only |
| version | Version | read only |
| component | Component | read only |
| target_milestone | Target Milestone | read only |
| votes | Votes | read only |
| keywords | Keywords | read only |
| lastdiffed | - | read only |
| everconfirmed | - | read only |
The following fields are displayed on the Bugzilla bug form but are kept in separate database tables and cannot be replicated:
If you need to change the list of replicated fields after you've started using the P4DTI, see section 9, "Maintaining the P4DTI".
To configure Perforce, you must:
Create a user in Perforce for the replicator; for instructions, see the Perforce System Administrator's Guide. The replicator user must have the following properties:
The userid must be the same as the replicator Perforce userid (p4_user) that you specified in the P4DTI configuration.
The e-mail address must match the replicator e-mail address (replicator_address).
If you're using the Perforce protections, make the replicator a super user so that it can set the jobspec. For instructions, see the Perforce System Administrator's Guide. You'll need to add a line like
super user p4dti-replicator0 * //...to the protections list.
For information on getting a license from Perforce Software for this extra user, see section 3.2, "Perforce prerequisites".
You can use the P4DTI in combination with a Perforce trigger to enforce extra workflow restrictions. For example, if your organization assigns priorities to issues, you can prevent changes being made to areas of the repository unless they resolve at least one defect of priority 3 or higher.
The P4DTI comes with an example trigger script that you can adapt for your needs, installed as example_trigger.py in the default installation directory.
To enforce workflow restrictions, follow these steps:
replicated_fields configuration parameter (for Bugzilla or TeamTrack). You must delete all jobs from your Perforce installation. The P4DTI takes over the jobs subsystem of Perforce and rewrites the Perforce jobspec.
For instructions, see the Perforce Command Line User's Guide.
It is possible to keep existing issues that are stored in Perforce jobs. Migration submits all the Perforce job data to the defect tracker, so that the issues can be replicated back to Perforce, and so appear in both systems. This requires more experience of Perforce and some Python programming. See section 4, "Migrating to the defect tracker from Perforce jobs", of the Perforce Defect Tracking Integration Advanced Administrator's Guide.
If you already use Perforce jobs and have significant tools that depend on your jobspec, the configuration options described in section 5.1, "P4DTI configuration", might not be flexible enough to support your requirements. However, you might be able to write your own configuration and use your own jobspec. To write your own configuration, you must understand the P4DTI configuration architecture and be fluent in the Python programming language. See the Perforce Defect Tracking Integration Integrator's Guide for details of how to configure the P4DTI and guidance on developing your own configuration. Note that neither Perforce nor the manufacturer of your defect tracker can support a configuration that you write yourself.
To configure TeamTrack, you must:
You need to add a TeamTrack value to the Windows Registry to tell TeamTrack that the P4DTI is present. To do this, double-click the p4dti.reg file that comes with the P4DTI (it's installed in c:\program files\p4dti\p4dti.reg by default).
You need to create a TeamTrack user for the replicator. This user corresponds to the replicator TeamTrack userid (teamtrack_user) parameter you set in section 5.1, "P4DTI configuration".
To create a TeamTrack user for the replicator, follow these steps:
teamtrack_user) parameter). For information on getting a license from TeamShare for this extra user, see section 3.3, "TeamTrack prerequisites".
Figure 7. New user: General tab
Figure 8. New user: Privileges tab, System tab
Figure 9. New user: Privileges tab, Item tab
The replicator uses TeamTrack issue field descriptions as the source for the Perforce job field descriptions. These job field descriptions appear in comments in every job form (if you're using the Perforce command line) and as tooltips for the fields in the job editing dialog (if you're using P4Win, the Perforce Windows GUI).
TeamTrack leaves field descriptions blank when you create a database, so you must provide descriptions of fields that your developers edit. For example, you might describe the TITLE field as "A one-sentence statement of the problem from the user's perspective", and the DESCRIPTION field as "A detailed description of the problem from the user's perspective, including how to reproduce it."
To enter field descriptions, follow these steps:
To configure Bugzilla, you must:
You need to make some minor modifications to the Bugzilla code so that users can see Perforce information on Bugzilla bug forms, and so that the P4DTI can access the values of Bugzilla configuration parameters. These modifications are distributed as patchfiles for versions 2.10, 2.12, and 2.14 of Bugzilla.
If you have modified Bugzilla at your site, you might still be able to apply the patch successfully. Changes to the database schema, the permissions rules, or the workflow rules are likely to cause the P4DTI to malfunction. You might need to modify the P4DTI if you have changed these parts of Bugzilla.
The patch for all Bugzilla versions changes the following Bugzilla files:
bug_form.pl, adding a Perforce
section to the bug form. defparams.pl, adding a parameter to
control whether or not the Perforce section appears. doeditparams.pl, creating and maintaining
a database table containing Bugzilla's configuration parameters. The patch for Bugzilla 2.14 also changes this file:
globals.pl, deleting unsafe
environment variables so that the processmail script can be run by the P4DTI. These changes are small and self-contained. If your changes do not affect these files or only affect them in minor ways, the patch should operate correctly. If the patch program fails because of your Bugzilla modifications, it might still be possible to introduce the changes by hand. If you cannot apply the patch, the replicator might still work, but: the Bugzilla bug form will not show Perforce-specific information (for example, changelists that are linked to the bug by a "fix"); also the values of Bugzilla configuration parameters (such as "emailsuffix") will not be available to the replicator. The operation of the replicator itself is affected only if you have made drastic changes to Bugzilla (for example, if you have completely removed the "bug_status" column from the "bugs" table).
To apply the patch, follow these steps:
patch < p4dti-install-dir/bugzilla-bugzilla-version-patch
(where p4dti-install-dir is your P4DTI installation directory) and bugzilla-version is your version of Bugzilla (2.10, 2.12, or 2.14).You need to create a Bugzilla user for the replicator. The replicator uses e-mail addresses to work out which Perforce user corresponds to which Bugzilla user. A Perforce user that does not correspond to a Bugzilla user is translated to the replicator's Bugzilla user, except for user fields (for example, "AssignedTo") in jobs. The replicator rejects a change when there is no Bugzilla user corresponding to a changed user field.
To create a Bugzilla user for the replicator, follow these steps:
replicator_address). After patching the Bugzilla code, you need to enable the P4DTI extensions. To enable the extensions, follow these steps:
To disable use of the P4DTI from the Bugzilla user interface, switch the extensions off.
To start the replicator, follow these steps from the operating system command line:
python run.py. The first time you start the replicator, it displays log output explaining how the replicator is setting up the defect tracker schema extensions, as shown in the following figure:
Figure 10. Example replicator log output on startup (TeamTrack integration)
2001-03-12 20:05:45 UTC (P4DTI-6018) Installing field 'P4DTI_RID' in the TS_CASES table.
2001-03-12 20:05:45 UTC (P4DTI-6018) Installing field 'P4DTI_SID' in the TS_CASES table.
2001-03-12 20:05:45 UTC (P4DTI-6018) Installing field 'P4DTI_JOBNAME' in the TS_CASES table.
2001-03-12 20:05:46 UTC (P4DTI-603X) Installed all new fields in the TS_CASES table.
2001-03-12 20:05:47 UTC (P4DTI-6040) Put 'LAST_CHANGE' parameter in replicator configuration with value '0'.
2001-03-12 20:05:47 UTC (P4DTI-6040) Put 'SERVER' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': 'Perforce server on sandpiper'}"'.
2001-03-12 20:05:47 UTC (P4DTI-6040) Put 'STATUS_VALUES' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': '_new/assigned/closed/verified/deferred'}"'.
2001-03-12 20:05:47 UTC (P4DTI-6040) Put 'CHANGELIST_URL' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': 'http://sandpiper.ravenbrook.com:8080/%d?ac=10'}"'.
2001-03-12 20:05:48 UTC (P4DTI-8002) Mailing 'P4DTI administrator <gdr+admin@ravenbrook.com>' re: '(P4DTI-8669) The P4DTI replicator has started.'.
... |
Each log entry consists of the date of the entry, a message identifier, and the message text. You can use the message identifier of an error message to look it up in section 11.2, "Error messages by identifier".
During its startup sequence, the replicator creates Perforce jobs corresponding to every defect tracker issue created or modified after the (start_date). It then polls for changes every poll_period seconds and replicates those changes. Figure 11 shows typical replicator log output when it is replicating a change.
Figure 11. Example replicator log output on replication (TeamTrack integration)
2001-03-12 19:59:29 UTC (P4DTI-8057) Replicating job 'CHG00003' to issue 'CHG00003'.
2001-03-12 19:59:30 UTC (P4DTI-824X) -- Changed fields: {'SEVERITY': 46, 'VERSION': 53, 'STATE': 2, 'PRIORITY': 17}.
2001-03-12 19:59:30 UTC (P4DTI-6007) -- Transition: 3; User: rb.
2001-03-12 19:59:30 UTC (P4DTI-8261) -- Defect tracker made changes as a result of the update: {'Owner': 'gdr'}. |
To stop the replicator on Windows, follow these steps:
poll_period seconds). To stop the replicator on Unix systems, kill the
replicator process. If it's running in a shell, bring it to the
foreground and type Control-C. If not, find out the process id of the
replicator process and run the command kill -TERM
replicator-process-id.
The P4DTI can be run as a daemon on Unix and as an NT service on Windows. Check that the replicator starts manually and runs correctly, before leaving it to run automatically.
If you installed the P4DTI using the Linux RPM as described in section 4.3, "Linux installation", a startup script is automatically created in /etc/rc.d/init.d directory, so that the replicator starts as a daemon when the machine is booted. Alternatively you can start the P4DTI daemon manually by calling the startup script yourself:
/etc/rc.d/init.d/p4dti startThe replicator halts automatically when the system is shut down. You can stop the replicator daemon manually using the stop script:
/etc/rc.d/init.d/p4dti stop On Solaris or other Unixes (and on Linux if you did not use the RPM installer), you might want to adapt the Linux startup script. It is in the file named startup-script in the installation directory.
On Windows, you can choose to install the P4DTI as a service. The replicator then starts when the machine is booted. You need not be logged on to the machine for the service to run or to stay running.
To install the service, follow these steps from the operating system command line:
python service.pyOnce the service has been installed, it can be started in any of the following ways:
python service.py start.Once the service is running, it can be halted in any of the following ways. Note that you need not halt the service the same way that you started it.
python service.py stop.To uninstall the service, go to the P4DTI installation directory and run the command:
python service.py removeNot all of the flexibility of the P4DTI is available using the configuration options described in this section. Advanced configuration of the P4DTI is possible, but beyond the scope of this manual. Here are some of the things that are possible with advanced configuration:
Contact Perforce technical support if you need any of these facilities.
You do not need to take any special action to migrate defect tracking data from your defect tracker to the integrated system. The replicator starts replicating defect tracker issues as soon as it starts up. Only issues that are created or modified after the start_date are replicated to Perforce.
See section 3, "Allowing users to create issues in Perforce" in the Perforce Defect Tracking Advanced Administrator's Guide.
See section 4, "Migrating to the defect tracker from Perforce jobs" in the Perforce Defect Tracking Advanced Administrator's Guide.
When you're testing your P4DTI configuration, you might need to tell the P4DTI take a single step; that is, to poll the defect tracker and Perforce for changes, replicate those changes, then stop. If you need to do this:
python poll.py. Test the P4DTI configuration by creating a test issue and taking it through a complete life-cycle (that is, through the workflow) as described in the Perforce Defect Tracking Integration User's Guide. You might need to adapt the use cases described in the user's guide to your organization's workflow.
Test the P4DTI from both Perforce and the defect tracker. In Perforce, test the P4DTI using the interface that your developers are most likely to use. The main Perforce interfaces are:
To run the consistency checker and manage its output, follow these steps:
python check.py. You can also examine the database using a database application (for example, Microsoft Access or the mysql command) to ensure the Perforce data is in there.
You might want to provide training for Perforce and defect tracker users before they adopt the P4DTI for everyday use. If so, consider preparing training materials that walk them through the workflow for an issue, using the procedures that are documented in the Perforce Defect Tracking Integration User's Guide.
Even if you don't have a formal training session for your users, ensure that they:
You must stop and restart the replicator as described in section 5.5, "Starting the replicator manually" after changing any of the configuration parameters described in section 5.1, "P4DTI configuration".
You must also refresh Perforce jobs, as described in section 9.2, "Refreshing jobs in Perforce", after changing either:
replicated_fields for Bugzilla or TeamTrack), or start_date). Perforce uses the field number in the jobspec to find data, not the field name (for more information, see the Perforce System Administrator's Guide). If you change the list of replicated fields, then the field numbers change, which means that the fields of existing jobs in Perforce will be mixed up. Refreshing the jobs re-creates them from the defect tracker with the correct fields.
Refreshing jobs updates all jobs in Perforce by replicating them from the defect tracker's database. This procedure is necessary if:
To refresh the Perforce jobs, follow these steps from the operating system command line:
python refresh.py. python run.py. To uninstall the P4DTI, follow these steps:
/etc/rc.d, and so on. rpm -e p4dtiTo troubleshoot a problem with the P4DTI, follow these steps:
Look in the P4DTI log. If you find an error message, see if it is listed in section 11.2, "Error messages".
Check your configuration against section 5.1, "P4DTI configuration". Are the hostnames, userids, and passwords correct? Most problems with the P4DTI are caused by incorrect or inconsistent configuration.
See if there is any online support for your problem. Visit the P4DTI issue reports page <http://www.ravenbrook.com/project/p4dti/issue/>, choose your release and select the "Support information" report.
If you can't solve the problem, contact Perforce support (for details, see <http://www.perforce.com/perforce/support.html>). Provide the following information:
readme.txt that came with your P4DTI distribution to identify the release). p4 info" at the operating system command line. config.py file. This isn't a complete list, but it covers the errors that have been seen in testing, or which are reasonably likely to come up, or which need some explanation. If you see a message not covered in this section or section 11.3, "Other error messages" and which is not self-explanatory, please contact Perforce support (see section 11.1, "Troubleshooting").
The replicator has had an unexpected difficulty in accessing the Bugzilla database. Possibly there is a problem with MySQL or MySQLdb. Possibly you are running a version of Bugzilla which is incompatible with the P4DTI, or have customized Bugzilla in such a way that the P4DTI has become confused. Please contact Perforce support (see section 11.1, "Troubleshooting").
It looks as though you've been running with a later release of the P4DTI and then downgraded to an older release. We don't support downgrading; use the most recent release.
The P4DTI doesn't support your version of Bugzilla. Upgrade to a supported release (see section 3.4.1).
Preliminary checking of the parameters set in config.py has found a problem. Correct the named parameter and start the P4DTI again.
You are running a version of Bugzilla with different bug statuses from those in Bugzilla 2.10, 2.12, or 2.14. The P4DTI has attempted to choose a sensible translation of these bug statuses to Perforce job states, but has failed. You may be able to fix this by changing the closed_state parameter. Otherwise you must modify your Bugzilla configuration.
The P4DTI chooses the names of states of Perforce jobs based on the status names in Bugzilla. It uses the following translation system:
closed_state parameter is not None, the P4DTI translates this status to "closed" and translates "CLOSED" to "bugzilla_closed".For instance, if the closed_state parameter is "RESOLVED", the