PERFORCE DEFECT TRACKING INTEGRATION README FOR RELEASE 1.0.0 Richard Brooksby, Ravenbrook Limited, 2000-10-30 CONTENTS 1. Introduction 2. Supported configurations 3. Installation 3.1. Installing the TeamTrack integration 3.2. Installing the Bugzilla integration on Linux 3.3. Installing the Bugzilla integration on Solaris 4. What's new 5. Known defects 6. Getting support 7. Project contacts A. References B. Document history 1. INTRODUCTION This is release 1.0.0 of the Perforce Defect Tracking Integration (P4DTI). The P4DTI connects your defect tracking system to Perforce, so that you don't have to switch between them and enter duplicate information about your work. It also links changes made in Perforce with defect tracker issues, making it easy to find out why a change was made, find the work that was done to resolve an issue, or generate reports relating issues to files or codelines. For up-to-date information about releases of the P4DTI, see the product information page . From there you will find links to the latest releases, including reports of defects found. If you want to adapt or extend the P4DTI, please go to the product information page and download the Integration Kit. It contains full source code and documentation to help you. The readership of this document is anyone who wants to download and use the Perforce Defect Tracking Integration. This document is not confidential. 2. SUPPORTED CONFIGURATIONS This release supports: - Perforce 2000.2 on any platform; - TeamTrack 4.5 running on Windows NT 4 or Windows 2000; - Bugzilla 2.10 on Red Hat Linux 6.2 or Solaris using MySQL The Bugzilla integration may work on other operating systems, but has not been tested. 3. INSTALLATION 3.1. Installing the TeamTrack integration On the TeamTrack server machine, run the "p4dti-teamtrack-1.0.0.exe" installer program that came with this document. This will unpack the integration. We recommend installing the integration in "C:\Program Files\P4DTI\" but you can ask the installer to put it somewhere else. Then consult the Administrator's Guide (installed by default in "C:\Program Files\P4DTI\ag\index.html") for further instructions. 3.2. Installing the Bugzilla integration on Linux The integration is distributed as an RPM called "p4dti-bugzilla-1.0.0-1.i386.rpm". Transfer this to the Bugzilla server machine, then run the following command as root: rpm -i p4dti-bugzilla-1.0.0-1.i386.rpm This will install the P4DTI files into /opt/p4dti and a startup script in the /etc/rc.d/init.d directory. If you prefer not to use RPMs, you can follow the procedure in section 3.3. 3.3. Installing the Bugzilla integration on Solaris The integration is distributed as a gzipped tar file called "p4dti-bugzilla-1.0.0.tar.gz". Unpack this tarball on the Bugzilla server machine, using the command gunzip -c p4dti-bugzilla-1.0.0.tar.gz | tar xvf - Then consult the Administrator's Guide (installed as "p4dti-bugzilla-1.0.0/ag/index.html") for further instructions. 4. WHAT'S NEW The following defects were fixed between version 0.5 and version 1.0. CRITICAL job000019: Licences on manuals and software too restrictive in release 0.3.0. Licences on manuals and software too restrictive in release 0.3.0. job000041: Configuration is too hard Configuring the P4DTI is much too hard, failing requirement 63 and requirement 75. job000051: There's no Bugzilla integration We must integrate with Bugzilla (requirement 18). job000091: Resolution fields aren't handled by Bugzilla integration In Bugzilla there is a "resolution" field as well as a "status" field. When the status becomes "RESOLVED", the resolution field has to acquire a value (one of a small set). When the status becomes "REOPENED", the resolution field has to be cleared. Neither of these things happens if the status is changed by the replicator. The resolution field is not currently replicated to Perforce. Note that one possible resolution is "DUPLICATE", of another bug whose number is given in a separate field. job000143: Document histories missing from sources The source licence asks people to append to the document history, but they can't (and won't) if there isn't one, so we'll lose valuable records if other people modify the code. job000150: The replicator can't recover from server failures If the replicator fails to poll then it stops with the following error message: NameError: There is no variable named 'formatted_exception'. job000181: Assertion failure in translate_1_to_0 If an optional field in Perforce has no value, then a field translator may fail with an assertion error. job000190: Connection to TeamTrack hangs for several minutes When release 0.4.2 attempts to connect to TeamTrack 4407, it hangs for several minutes before connecting. ESSENTIAL job000006: TeamShare API error reporting is inadequate. When TeamTrack refuses to update or transition an issue because a user doesn't have the privilege to do so, it raises an error but provides only an empty error message. The replicator can't provide any information to the user to explain why the operation failed. job000033: Incompatible with other TeamShare API programs The P4DTI is incompatible with other TeamShare API programs because changes made by other TeamShare API programs are not replicated. The replicator has no reliable way of distinguishing its changes to TeamTrack from other users' changes in TeamTrack; this is because all API users show up as user 0 in the CHANGES table. So changes made in TeamTrack by other API users (e.g. other integrations or scripts) will not be replicated accurately. job000037: Consistency checker script is inadequate The consistency checker described in section 7.1 of the Administrator's Guide doesn't check configuration parameters, that the P4DTI_* fields are present in the defect tracker and Perforce, or changelists. job000038: TSServer::UpdateRecord doesn't let you specify a user This is a loophole that provides a means for a user to circumvent access control in TeamTrack. The user makes a change in Perforce that they wouldn't be allowed to do in TeamTrack. When the replicator replicates that change, TeamTrack check's the permissions for the replicator user, not the user who made the change. So the illegal action is not detected. job000048: Consistency checking of the configuration is inadequate Consistency checking of the configuration needs to be much stronger, to catch problems so that they don't affect the replicator when it's in use. job000050: There's no way to re-start replication If things get in a mess or the configuration is changed then the administrator may want to re-replicate everything as if he were installing from scratch. We don't provide anything like that. job000057: Special characters in single-select keywords make the replicator barf Special characters, such as spaces and slashes, in single-select field values aren't legal in a Perforce jobspec. job000075: No automatic check of configuration How do you tell if the configuration is correct? There should be automatic checking of the configuration. See e-mail "Checking the configuration automatically" for some ideas. job000087: Replicator doesn't enforce licences in TeamTrack Only licenced TeamTrack users should be able to update issues in TeamTrack. I think this isn't working. job000092: Long descriptions aren't replicated by Bugzilla integration In Bugzilla, bugs have short descriptions (called "Summary" in the web interface) and long descriptions. Each bug may have many long descriptions; they carry a timestamp and are displayed in date order. They also carry user names. Currently these long descriptions are not replicated. Perhaps they should be; the short description can be too short and/or misleading. Note that it is not possible to edit or delete long descriptions from the Bugzilla web interface; only to submit new descriptions to be appended. job000093: Can't replicate other Bugzilla fields Bugzilla bugs have loads of fields. We only replicate a small number of them. We don't support the "replicated_fields" parameter in the Bugzilla configuration. job000095: Bugzilla integration is verbose to stdout but doesn't log The Bugzilla integration reports all SQL statements and results to stdout, but doesn't use the logging mechanism at all. job000103: Can't easily add to replicated_fields list The configurator doesn't cope if you change the replicated_fields. The P4DTI administrator can't add a new field to the replicated_fields list (except at the end of that list) because the new field will get a Perforce field number that was previously used by another field. So the data that was previously in that other field gets captured by the new field. If you have used the automatic configuration generator and you edit the replicated_fields list, fields probably get the wrong values in Perforce, because the generated jobspec's field numbers probably don't match the old values. job000112: Can't easily replicate by project It's hard to set the replicator to replicate only those issues in a particular project or group of projects. job000115: Bugzilla configuration doesn't match AG The bugzilla configuration in config_bugzilla.py doesn't match the description of the configuration parameters in section 5.2 of the Administrator's Guide. job000116: Bugzilla integration doesn't do enough checking The Bugzilla integration does hardly any checking, either of the configuration or of data encountered during replication. This is in contrast to the TeamTrack integration, which now does quite a lot of checking. job000117: Jobview and job filter confused in the UG The UG says that the list of jobs displayed in the GUI dialogs is affected by the users jobview. This isn't true. It's the job filter that's used. The jobview only affects the command line interface. job000138: "Add Job Fix" always sends jobs to "closed" The "Add Job Fix" command in the Perforce Windows GUI always sends a job to status "closed". Users will want to link changes using other keywords. job000141: Can't add to replicated_fields list The P4DTI administrator can't add a new field to the replicated_fields list (except at the end of that list) because the new field will get a Perforce field number that was previously used by another field. So the data that was previously in that other field gets captured by the new field. job000144: "ignore" can't be a job status Perforce's "p4 change -s" and "p4 submit -s" interfaces use "ignore" as a special keyword meaning "don't make a fix". This means that we can't really allow "ignore" as a job status when we create a jobspec from the defect tracker's list of statuses. job000146: Empty date fields in TeamTrack give the replicator a headache When a date/time field in TeamTrack is empty, the replicator fails to replicate it. job000147: TeamTrack licence available but not mentioned TeamShare have made a TeamTrack licence for beta customers which they're happy for us to publish, but we don't mention it anywhere yet. job000155: Bugzilla integration doesn't send mail Bugzilla sends lots of email, when bugs change. We don't do that. Bugzilla has a Perl script called "processmail", which it runs whenever a bug changes. It analyses the bug activity and may send mail to one or more Bugzilla users describing that activity. We don't do this, or emulate it, so Bugzilla bugs will be changed by the replicator without email being sent out. job000156: Pending changelists aren't clearly indicated as such in Bugzilla You can't tell the difference in the P4DTI section of the Bugzilla bug form between a pending changelist and a submitted one.Ę This means that you might mistakenly think that work has been done when in fact it is only pending and may later be reverted. job000157: Improper installation on Linux The integration is distributed as a simple tarball on Linux and doesn't install anywhere in particular. The AG says to put it in the administrator's home directory for the moment, which isn't good enough. job000158: Obscure error if Perforce can't be reached or p4_client_executable is wrong The replicator produces an obscure error message if it can't reach the Perforce server. This might be because the server is down or because the configuration is wrong. When it starts, the replicator says "P4DTI Replicator error: p4 info didn't report a recognisable version". job000172: Replicating some fields can break Bugzilla Some fields can not be changed using the Bugzilla interface, or only changed in highly restricted ways. Now that these fields can be replicated into Perforce, we should restrict changing these fields in Perforce. For instance, the 'votes' field can only be changed by voting, which involves checking and updating other fields in other tables and may have other side-effects. For another instance, the 'product', 'version', and 'component' fields must be keys into other tables. For a third instance, the created_ts field is never updated. job000173: Wrong Perforce server version causes installation to fail mysteriously If you have the wrong Perforce version (2000.1 or earlier) then installation will fail, with an error like this: "Perforce error: Error in job spec specification. Missing required field 'Values-State'." It should instead say something like "Perforce release not supported; changelevel NNNNN required." job000178: AG doesn't give advice on making problem reports The AG doesn't give advice on how to get effective support for the P4DTI. Typically someone e-mails a section of their P4DTI log -- for example, the error message at the end. This is rarely enough information to track down the problem. job000180: bugzilla_user is confusing. Many people find the bugzilla_user configuration parameter confusing. We should get rid of it and use the replicator_address parameter in its place. job000182: Elapsed time fields aren't replicated properly Date fields in TeamTrack have a "display type" which can be "Date only", "Date and time", "Time only" and "Elapsed time". The repicator copes well with the first two, but poorly with the second two (an elasped time of 10 hours in TeamTrack becomes a date of 1971/01/01 10:00:00 in Perforce). job000184: Bugzilla integration doesn't work if your database is not called 'bugs' If your Bugzilla database is not called 'bugs', then P4DTI tries to connect to 'bugs' anyway. So it doesn't work! job000185: python RPMs no good The Python 1.5.2 RPMs are no good for building the MySQLdb interface. In the AG we offer the Python 1.5.2 RPM. This is insufficient on its own to build the interface (because it doesn't have the header files). There is a python-devel RPM which does include header files, but for some reason the MySQLdb 0.2.2 and 0.3.0 interfaces don't compile out of the box with it. Building python 1.5.2 from the tarball distribution works OK, as long as you include the optional syslog module (by uncommenting the syslog line from Modules/Setup before running make). job000186: AG suggests python RPM which is insufficient MySQLdb can't be built using just the python RPM linked to from the AG. Our users need to build and install MySQLdb. To build it they need the python 1.5.2 header files. The python RPM linked to from the AG doesn't have any header files. They are contained in a python-devel RPM. However, the python-devel-1.5-2 RPM which we installed on swan (which may have downloaded from ftp.python.org; www.python.org does not now link to any RPMs for python 1.5.2) provides header files which are not quite right for building MySQLdb. I haven't dug very deeply here. I managed to build MySQLdb 0.2.2 by editing its source code. This is not very satisfactory. For MySQLdb 0.3.0 I also had problems. I resolved these by uninstalling the python and python-devel RPMs and building python from the sources (python-1.5.2.tar.gz). The optional syslog module is required by the P4DTI, so I had to edit Modules/Setup before running make. We need to fix the prerequisites section of the AG so that users get a python environment on which they can build MySQLdb and run the P4DTI. job000187: Creating Bugzilla tables twice We call bugzilla.create_p4dti_tables() twice on startup; once when making a bugzilla.bugzilla in configure_bugzilla and once when making a dt_bugzilla.dt_bugzilla in init.py. We can remove the call from inside dt_bugzilla.__init__. job000189: replicated_fields can't be changed without replicating all jobs In section 10 of the prepared manuals we tell people to run the refresh script if they change the replicated_fields parameter, but this forces all issues to be replicated, which they may not want. job000198: We don't check that bugzilla_directory works The configuration parameter bugzilla_directory names a directory in which the P4DTI will find a script called 'processmail'. The P4DTI runs this script after it changes a Bugzilla bug, and it sends email to Bugzilla users just as if the bug was changed through the Bugzilla web interface. We do not currently check that this parameter names a directory, or that the directory it names contains a processmail script. If either of these things is false, the P4DTI will fail to invoke the processmail script. job000199: Auxiliary scripts send e-mail The auxiliary scripts, check.py and refresh_perforce.py, send e-mail to the administrator saying that the replicator is starting up. They shouldn't do this, because it's unnecessary and the message is not true. job000202: Errors from Perforce not reported well Some Perforce errors are reported only as "The Perforce client exited with error code %d. Is the server down or the server address incorrect?" This provides little help in tracking down the problem. job000204: Issue owned by non-existent Perforce user causes crash If the Perforce server is running without any spare licences, then when the replicator tries to overwrite a job whose owner is a user that doesn't exist in Perforce, then you get the error message "Can't create a new user - over licence quota". job000212: TeamTrack 4.5 not supported The meaning of the arguments to TSServer::AddField() have changed in TeamTrack 4.5, which means that the new fields aren't be properly added to the CASES table. job000214: No way of controlling the poll period There's no easy way for the administrator to control the poll period. If 10 seconds gives performance problems then there should be a way of increasing the delay. OPTIONAL job000024: "\012" in fixes table in TeamTrack Change descriptions in the "Perforce fixes" section of a TeamTrack case description display the octal escape "\012" as those four characters, rather than as a newline (ASCII 10). job000032: Deletion of fixes and filespecs in TeamTrack may cease to work in future releases. Deletion of fixes and filespecs from the TeamTrack interface is correctly replicated to Perforce at present. But this depends on an undocumented feature of TeamTrack and so may cease to work when new releases of TeamTrack appear. job000101: Different transitions for different issue types may confuse the replicator In TeamTrack, different issue types (bugs, enhancements, etc) may have different workflows: that is, certain transitions may be enabled for some issue types but not for others. The replicator doesn't take this into account and so may pick the wrong transition, which means that the transition may fail. It may happen that the replicator can't transition any issues for this reason. job000160: 'closed' doesn't map to 'RESOLVED' in the Bugzilla integration The 'closed' status for Perforce jobs is special: p4 fix without a -s flag will cause a transition to 'closed'. The natural status for this to translate to in Bugzilla is 'RESOLVED': engineers should make bugs RESOLVED when they resolve them. But currently 'closed' in Perforce translates to 'CLOSED' in Bugzilla: to move a bug to 'RESOLVED', Perforce users must do 'p4 fix -s resolved'. Perforce users will want to move jobs to 'RESOLVED' often but to 'CLOSED' rarely. Using 'p4 fix' without '-s' will usually cause the replicator to reject the update because there are few legal transitions to 'closed'. job000163: jobspec fields too small for Bugzilla. The replicator makes a new Perforce jobspec. The "assigned_to" field has size 32. This must in fact be large enough to hold a Bugzilla login_name (from the "profiles" table in Bugzilla), which is varchar(255). The same may apply to other jobspec fields. job000165: Bugzilla configuration parameters are not checked. We don't check the Bugzilla configuration parameters. For instance, we don't check that dbms_port is a number, or dbms_host is a string, or that bugzilla_user is an email address. The result is that if a user sets a configuration parameter incorrectly, or leaves it unset, they may get a very obscure internal error from, for instance, MySQLdb. job000169: Change numbers are links in TeamTrack even when no changelist URL has been specified If changelist_url is None in the configuration, then changelist numbers are still links in the table of fixes in TeamTrack, but they don't link to anywhere sensible. job000170: Replicator may be unable to send e-mail if the default replicator_address is unchanged The file config_bugzilla.py says replicator_address = 'p4dti-%s' % rid. Ravenbrook's SMTP server allows the replicator to send e-mail from this address, but not all SMTP servers allow this. job000193: Bugzilla integration fails if DateTime module is installed The Bugzilla integration breaks if the Python DateTime module is installed on the P4DTI server system. The Python DateTime module is a third-party module for creating and manipulating dates and times. If it is present, MySQLdb uses DateTime types for time fields (date, time, datetime, and timestamp). However, this MySQLdb/DateTime combination breaks when a MySQL field contains a zero value (e.g. "0000-00-00 00:00:00"). This happens in Bugzilla (e.g. the 'lastdiffed' field of a new bug). We have not tested the P4DTI with the DateTime module at all; it is possible that using it causes other problems even if zero datetimes are not present. I have reported this problem to the author of MySQLdb, Andy Dustman . NICE job000140: Logical field name "code" not allowed in TeamTrack We can't allow a field whose logical field name is "code" in TeamTrack because Perforce uses the Python key "code" to return status information. job000168: Easy to set dbms_port to a string. It's easy to set the configuration parameter dbms_port to a string (e.g. '3306', 'localhost:3306', etc) rather than an int. Especially easy given that p4_port has to be a string. In release 0.5.1 it was especially unclear that this had to be an integer (config_bugzilla.py has it set to ????). config.py now has a sensible value here (i.e. 3306), but this requirement should be documented in the comment and in the AG, and tested in configure_bugzilla.py (see job000165). 5. KNOWN DEFECTS IMPORTANT: Read the defect reports available from the product information page . You will find information about defects found _after_ release. There could be vital information there which we couldn't know when we wrote this document. If you don't read the document you might meet the same problems, waste a lot of time, and possibly DESTROY ALL YOUR DATA. This is a list of major defects which were known at the time of release. CRITICAL job000200: No supported TeamTrack release works with integration There's no officially supported TeamTrack release that works with the integration. job000205: Configuration is still too difficult The Administrator's Guide is too hard to use. Administrators are lazy and don't read documentation, not even the very nice documentation that we've put so much effort into. ESSENTIAL job000016: Double replication causes many conflicts There is no reliable way for the replicator to distinguish its changes to jobs from changes made by other Perforce users. This means that everything replicated from the DT to Perforce is then replicated back again, and conflicts are very likely when making edits in the DT in quick succession. We observed this at Quokka Sports. RB 2000-11-28 job000022: No migration from Perforce jobs The Administrator's Guide talks about migrating from using Perforce jobs to using the integration with a DT. We don't provide any method of doing this. The software assumes complete control of the jobs subsystem of Perforce, and will damage any existing jobs data. job000042: Rapid changes in the DT cause conflicts The replicator thinks there's a conflict if a DT issue is rapidly changed more than once. It replicates the issue to the job, then wakes up and replicates it back (see job job000033), but by now it's changed again in the DT, so the replicator believes there's a conflict. In version 0.4 there will be a "defect tracker wins" policy by default, so this will not stop replication, but it will send annoying e-mails reporting conflicts where there are none, and thus increase the burden of the system on its users. job000054: There are few reports There are no reports to help with analysis of the information gathered by the integration. This is a big reason for having the integration in the first place. (See requirement 5, requirement 60.) job000064: There's no product information page or sheet There's no product information sheet to tell people (esp. managers) at a high level what the integration does. In particular, we don't tell people that the integration enforces DT workflow and security in Perforce. job000065: Not enough logging control There's only one level of logging at the moment. We have to edit the code to get more debugging output in the log. This won't be possible or easy when beta testing or supporting the integration. job000066: We don't explain which jobspecs won't work "p4 -G job -o" applies much more stringent tests than "p4 job -o". It won't even give you a form unless all the values are correct. This means that tricks like ones in Perforce's jobspec, where an illegal value like "setme" is specified as the default value for a select field, can't be used with the integration. job000069: No resolver documentation We must write documentation for the resolver. For example, there are two ways of resolving conflicts: either set to "discard" or fix up the problem and set to "keep". Need to document both of these. job000076: Deleting records gives the replicator a headache The replicator doesn't cope well when a defect tracker record is deleted. When you delete an issue, then the next time (and every subsequent time) you edit the corresponding job it will fail to fetch the issue from the DT and send e-mail to the administrator. job000077: UG contains no version or release related documentation There's no documentation corresponding to use cases 6.12 and 6.15 dealing with finding issues related to versions or releases and vice verse. job000078: Replication failure can cause virtually irrecoverable database records If the initial replication goes wrong (e.g. fails to create a Perforce job because the server is down) then the DT issue is marked with P4DTI action "wait" but there's no corresponding job to set to "keep" in order to overwrite it. You can't fix the problem without going into the database directly using SQL or our own Python interface. job000082: AG has no training and documentation section The AG has no guidance on training and documentation for users. job000086: Users can "fix" issues that they don't have permission to change A user can create a fix for any job, and the corresponding defect tracker issue will change in status, even if the user doesn't own or have permission to change it. This hasn't been confirmed by testing as of 2000-11-28. job000122: Server failures aren't handled gracefully If the Perforce server or the defect tracker is down, the replicator produces unhelpful error messages and e-mails to the administrator. job000124: Changes by disabled Bugzilla users may be replicated Bugzilla has a mechanism for disabling users. We don't check this as part of checking user permissions. We should. job000128: Can't replicate fix to deleted changelist The replicator can't replicate a fix to a deleted changelist, and this gets it into a right tangle: it can't continue replicating. Luckily this is a rare occurrence: normally the deletion of a changelist is replicated to the defect tracker, so the situation doesn't arise. It's only if you delete a changelist and simultaneously cause the replicator to replicate a fix for that changelist back from the defect tracker that this can happen. job000134: The replicator log grows without limit The replicator log just keeps on growing. job000135: Replicator makes no more progress if replication to Perforce fails If replication from the defect tracker to Perforce fails for whatever reason, the replicator gets into a loop where it keeps e-mailing the administrator. job000137: You can fix a job that you haven't been assigned A user can add a fix for job that they don't have permission to transition or update. job000148: Too long to replicate because of reading USERS, STATES, etc. many times The replicator may read auxiliary tables in TeamTrack (like the USERS table) many times in the course of replicating a single job or issue. This slows replication down and increases the possibility of conflicts. job000149: We don't use system logging facilities. We don't use system logging facilities (syslog/Windows Event Log); instead we log to a file. We should use system logging. Managing applications which don't is irritating. job000159: Fixes which the replicator rejects still get replicated If a Perforce user makes a fix which the defect tracker disallows for some reason (e.g. it's an illegal transition in the defect tracker, the user doesn't have the necessary permissions to make the transition, etc), then the replicator will correctly refuse to replicate the change to the issue's status and will undo the change to the job's status, in accordance with the "defect tracker wins" policy. However, the fix record itself will be replicated, and will show up in the defect tracker's interface. This is misleading. Better behaviour would be for the replicator to remove the fix (and not replicate it). job000166: May fail with later MySQLdb versions We may fail with later MySQLdb versions, in particular 0.3.0 which I believe is gaining popularity. We haven't tested this. We should test and make any appropriate fixes. If there are serious incompatibilities, we should at least test the version (MySQLdb.__version__). It may not be appropriate to require 0.2.2, as the MySQLdb version is a system-wide property and users may have other requirements forcing them to use 0.3.0. Also, operating systems and Python or MySQL packages are likely to be distributed with 0.3.0 (or above) in future. job000174: The AG doesn't tell you how to upgrade from the beta A small number of sites will be running a beta release like 0.4.2 or 0.5.1. When release 1.0.x comes out they will naturally want to upgrade. But the AG doesn't have any instructions telling them how to do this. job000183: Mysterious problem with elapsed time fields in Perforce Jean Pierre Keller reported this error: Perforce error: Error in job specification. Error detected at line 18. Invalid date '1970/01/01 00:30:00'. This is mysterious because this date is valid in Perforce. job000188: tTrack integration fails with non-ASCII characters The tTrack integration does not cope with non-ASCII characters. For instance, "é" (character 0xe9). If this character is placed in a tTrack field, it gets into Perforce as "˙ffffffffffe9". Note that the first character is 0xff. Note also that the number of f's seems to depend on the exact platform: it's four on sandpiper (suggesting a 24-bit encoding) and ten on Jean Pierre Keller's machine (suggesting a 48-bit encoding ?!). This encoding also appears in the log output; I don't think the P4DTI is doing it. Going the other way, putting "é" into a Perforce job causes the P4DTI to fail when trying to update tTrack (no message from the TeamShare API). job000191: Admin can't easily find release when contacting support In the AG we tell the administrator to report which release of P4DTI they are using. But how do they find this out? The documentation isn't clear and the program doesn't report it. job000192: tTrack triggers not run A beta tester found that a case transition caused by the P4DTI did not run the tTrack triggers associated with the transition. job000194: tTrack states include tSupport states tSupport and tTrack both have states. When replicating from tTrack, the jobspec status field includes tSupport states, which are not possible for tTrack cases. This is not ideal. job000195: keyword translation is too conservative We translate keywords (items for Perforce job fields of type 'word') by using a hexadecimal escape %xx for characters other than [a-zA-Z0-9(),.?!-]. This is very conservative, and rules out a number of characters commonly used in defect trackers. We also translate the space character ' ' into the underscore character '_'. This has the problem that we do not escape underscores, so underscores in a defect tracker become underscores in Perforce and then translate back to spaces. We need a translator which is (a) bijective, (b) less conservative, and hopefully (c) less obfuscating. job000215: The replicator can send out mail bombs The replicator can get stuck in a loop where it sends e-mail to the administrator every 10 seconds (or whatever the poll_period is). This is very unfriendly. OPTIONAL job000020: If the replicator detects a conflict and there's no job then it fails If the replicator detects a conflict between a DT issue and a job, and the job doesn't exist, it fails mysteriously. This can happen if there's a bug in the replicator or the Perforce jobspec that means that the initial replication from the DT fails to make the job. It's hard for a program to tell if a job exists in Perforce, so this is a problem of robustness. Most likely to result from a problem with the configuration, or perhaps if the Perforce server is down. job000030: Users can't get help based on messages The P4DTI produces a lot of messages. It would be good to allow users to follow some sort of link from these to more information and experience. job000036: New jobs in Perforce don't get replicated to TeamTrack. New jobs in Perforce don't get replicated to TeamTrack. job000040: You can't fix the default changelist There's no way to associate a job fix with the default changelist, so that when it is submitted, the job is fixed. Instead, you have to make a numbered changelist first. job000044: There's no way to prevent a transition without a fix Users would like to prevent issues transitioning [sic] to certain states unless there's a fix that links the issue to a change. job000046: The replicator process is hard to manage The replicator process is hard to manage, especially on Windows NT, where one has to launch it using a non-trivial command line at a prompt. job000052: Likely problems not in troubleshooting section Likely errors: 1. can't login: did you set the user name in TeamTrack to match the replicator id? 2. run out of users in Perforce? have you changed the replicator id recently? If so, make sure you delete the old Perforce user for the replicator. job000062: Fixes not replicated until another change is made When we added a fix to a job, the replicator doesn't actually manage to replicate the fix until or unless you change a field in the job. We've not been able to reproduce this problem since alpha testing, and it only occurred at one site. job000084: No glossary in the UG There's no glossary in the user's guide. job000098: Perforce UIs don't link to defect tracker UIs p4win and/or p4web should link jobs to defect tracker issues. We could help with this by putting URLs into the job (e.g. in the form recommended by the appendix to RFC 1738). Then it would be up to p4win and/or p4web to turn those into live links. job000099: The p4 module has a security hole Any user could cause the replicator to run arbitrary shell commands by putting appropriate shell meta-characters in a jobname or Perforce user name. job000110: Replicator falls over if you copy an issue in TeamTrack If you do a copy transition in TeamTrack, then the replicator can't cope with the new issue. job000114: There's no description of the limitations of the integration People thinking of installing the integration need to know what its limitations are. There needs to be a "will this work for your organization?" document. job000125: Bugzilla version not checked We don't check the Bugzilla version (which should be 2.10 for the integration to work). job000139: Trigger scripts duplicate replicator code The example trigger script we provide has its own interface to Perforce, duplicating the "p4" class in the replicator. It would be better to share the code, for maintainability and robustness. job000145: Deleted TeamTrack states appear in Perforce jobspec If you delete an issue state in TeamTrack, it still appears among the values for the State field in the Perforce jobspec. job000151: Source code documentation is inadequate The documentation in comments in the source code is generally inadequate. Each source code file should conform to the general rules. There needs to be more cross-referencing to design documents. The Bugzilla patches need to be cross-referenced to the Python sources. RB 2000-12-13 job000162: Replicator needs restarting when you add new users to Perforce or the defect tracker When you add a new user to Perforce or the defect tracker or change a user's e-mail address, you have to restart the replicator. job000167: Released manuals should be on the website. The released manuals are currently not available on the website (except by downloading and installing a release). This has several undesirable consequences. For instance, users browse the master or version manuals, which do not reflect the release which they have downloaded. Also it makes it impossible to give a URL into a released manual. job000196: Illegal transitions should be prevented by Perforce clients. Perforce users can make job transitions which are prohibited by the DT workflow. The P4DTI reports and undoes these illegal transitions, but it would be better if the P4 clients did not allow them in the first place. job000197: P4DTI doesn't do ESMTP authentication Some ESMTP servers (e.g. Microsoft Exchange) may be configured to require authentication (RFC 2554). The P4DTI doesn't do this, so can not send email via these servers. Currently the P4DTI will not work if it can't send email. job000201: Empty assigned_to field can't be replicated If the "Assigned to" field in Bugzilla is empty, then it can't be replicated. You get the error "Error in job specification. Missing required field 'Assigned_To'." job000207: Deleted jobs aren't restored immediately If you delete a job in Perforce, then it gets restored when the issue is next updated in the defect tracker. It would be better if it were restored immediately. job000208: TeamTrack integration doesn't provide a .reg file It's error prone to tell people to use regedit to set the "VC Integration" registry key. It would be better to provide a .reg file which people can double-click on to set the registry key. job000209: No package for Solaris There's no package for the Bugzilla integration on Solaris that's as convenient to install as the RPM for Linux. job000210: Bugzilla states/resolutions with spaces in them cause broken jobspec If the Bugzilla database schema is altered so that states and resolutions have spaces in them (this is legal in MySQL) then the replicator generates a broken jobspec. job000211: Mail can get lost If the replicator tries to send e-mail and fails (typically because of a problem with the SMTP servr), then it just drops the mail. The contents is therefore lost. job000213: Performance of Bugzilla is reduced The replicator takes locks on a number of tables each time it polls, even if there is nothing to do. This reduces the Bugzilla performance. NICE job000061: Fixes are invisible in TeamTrack by default The fixes table only shows up in a case in TeamTrack if the user has checked the "version control" option in their profile. By default, it's unchecked. job000152: "commenton" parameters not enforced by Bugzilla integration Bugzilla has a set of parameters called "commenton" which forces Bugzilla users to enter comments (in the long description) when they do certain things to the bug record. We don't enforce these parameters through the P4DTI. job000153: "despot" not supported by Bugzilla integration Bugzilla has a feature called "despot" which we don't support. [Need a description of what it is here. RB 2000-12-13] job000154: "shadowdb" is not supported by the Bugzilla integration Bugzilla has a feature called "shadowdb" which creates a read only copy of the database for faster public access. The integration doesn't support this feature. job000161: Replicator appears to hang When the replicator starts up for the first time, it prints some log messages and then stops. This is disconcerting to the user, as you can't tell if it's hanging or waiting. job000171: AG should note broken MySQL versions According to Shiv Sikand, MySQL 3.23.29-gamma and 3.23.29a-gamma are broken, in that they stop Bugzilla logins from working. We should note these in the Prerequisites section of the AG. job000175: System fields like P4DTI_RID are editable by users in Perforce Some fields in the Perforce jobspec should not be edited by the user, for example P4DTI_RID and P4DTI_ISSUE_ID. But these are easily editable in the Perforce command line and GUI. We have to depend on people's carefulness. job000179: Teamtrack.dll can get lost An inexperienced Windows user who copies the P4DTI files from the unpack directory to another one might end up without teamtrack.dll (since DLLs are hidden by Windows tools like Windows Explorer by default). job000203: Users can masquerade as other users Because users are matched by e-mail address between the defect tracker and Perforce, you can fool the replicator by running "p4 user" and editing your email address. job000206: Administrator Guide needs some improvements Here are some miscellaneous improvements that should appear in the AG: 1. Advice needed on setting up p4web (e.g., use browse mode). 2. Have a table of auxiliary scripts with purpose. 3. Some metrics about how much time the replicator takes and what you can do to improve the performance. 4. Tell people not to switch TeamTrack databases under the feet of the replicator. 6. GETTING SUPPORT For problems relating to Perforce or the P4DTI in general, contact Perforce Support by writing to or see the technical support page at for contact information. For problems relating to TeamShare, contact TeamShare Technical Support by writing to or see the support page at for contact information. Bugzilla is not supported by any one person or organization. Consult the documentation that came with Bugzilla, or visit . 7. PROJECT CONTACTS You may want to join the p4dti-discussion mailing list. The goals of the list are: 1. to provide feedback to the project on requirements, design, implementation, etc.; 2. to allow people to exchange information and experience with using and adapting the project; 3. to keep people informed about project progress. To join, send a message with the word "subscribe" in the _body_ to or send the word "help" for general information. Please note that the mailing list will be archived and the archive may be published. A. REFERENCES None. B. DOCUMENT HISTORY 2000-10-19 RB Created using FreeBSD release documents as a model. 2000-11-08 RB Updated to explain purpose of release 0.3.3. Added references to reports. Improved instructions for obtaining special client and builds. Added a summary of changes. Added readership and confidentiality. 2000-11-20 RB Made it clearer where the Administrator's Guide and the installer file come from. 2000-11-30 RB Partially updated for version 0.4. 2000-12-01 RB Updated references to the "SAG" to "AG" since it's now called the Administrator's Guide. 2000-12-05 RB Added list of known issues generated from Ravenbrook jobs. Updated support information. 2000-12-05 GDR Section 4 says "first public release" only. 2000-12-07 GDR Updated for release 0.4.1. 2000-12-08 RB Updated for release 0.4.2. 2000-12-13 RB Updated for version 0.5. Updated for release 0.5.0. 2000-12-15 NB Added pointer to AG from Linux instructions. 2000-12-15 NB Clarified pointer to defect reports. 2000-12-15 RB Changed Bugzilla install directory to p4dti-bugzilla-RELEASE for consistency with many other Unix packages. 2000-12-15 NB Conditionalize TeamTrack text. 2000-12-15 NB Updated for release 0.5.1. 2000-12-18 NB Merged back into masters. 2001-02-15 RB Updated for version 1.0, pointing people at Perforce and TeamShare rather than Ravenbrook for information and support. 2001-02-19 GDR Added fixed and known issues for release 1.0.0. --- Copyright 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute copies and derivative works of this document provided that (1) you do not charge a fee for this document or for its distribution, and (2) you retain as they appear all copyright and licence notices and document history entries, and (3) you append descriptions of your modifications to the document history. $Id: //info.ravenbrook.com/project/p4dti/release/1.0.0/readme.txt#1 $