PERFORCE DEFECT TRACKING INTEGRATION RELEASE NOTES FOR RELEASE 1.5.3 Nick Barnes, Ravenbrook Limited $Date: 2002/12/13 $ CONTENTS 1. Introduction 2. Supported configurations 3. Getting support 4. Project contacts 5. What's new 5.1. What's new in release 1.5.3 5.2. What was new in release 1.5.2 5.3. What was new in release 1.5.1 5.4. What was new in release 1.5.0 5.5. What was new in release 1.4.2 5.6. What was new in release 1.4.1 5.7. What was new in release 1.4.0 5.8. What was new in release 1.3.2 5.9. What was new in release 1.3.1 5.10. What was new in releases up to 1.3.0 A. References B. Document history C. Copyright and licence 1. INTRODUCTION These are the release notes for release 1.5.3 of the Perforce Defect Tracking Integration (P4DTI). The P4DTI connects your defect tracking system to Perforce, so that you don't have to switch between them and enter duplicate information about your work. It also links changes made in Perforce with defect tracker issues, making it easy to find out why a change was made, find the work that was done to resolve an issue, or generate reports relating issues to files or codelines. For instructions on installing the P4DTI, see the product readme (readme.txt). For up-to-date information about releases of the P4DTI, see the product information page . From there you will find links to the latest releases, including reports of defects found. If you want to adapt or extend the P4DTI, please go to the product information page and download the Integration Kit. It contains full source code and documentation to help you. The readership of this document is anyone who wants to download and use the Perforce Defect Tracking Integration. This document is not confidential. 2. SUPPORTED CONFIGURATIONS Perforce versions: - Perforce 2000.2, 2001.1, 2002.1, or 2002.2, on any platform; The TeamTrack integration supports: - TeamTrack 5.5 on Windows NT 4 or Windows 2000; - Python 2.0 on Windows NT 4 or Windows 2000; - Running the P4DTI replicator on Windows NT 4 or Windows 2000. Use of TeamTrack 4.5 or 5.0 on Windows NT 4 or Windows 2000 is also supported, but is deprecated: support may be dropped in a future version of the P4DTI. The P4DTI does not support a TeamTrack server running on a secure web server, nor does it support the "TeamTrack Online" hosted defect tracking service from TeamShare. The Bugzilla for Unix integration supports: - Bugzilla 2.14.4 or 2.16.1 on Red Hat Linux 7.3 or Solaris; - MySQL 3.22.19 or later; - Python 1.5.2, 2.0, or later; - Running the P4DTI replicator on any Unix. Use of Bugzilla 2.10, 2.12, 2.14, and 2.14.1 on Red Hat Linux 7.3 or Solaris is also supported, but is deprecated: support may be dropped in a future version of the P4DTI. Support for Python 1.5.2 is deprecated, and may be dropped in a future version of the P4DTI. Note that although the Bugzilla integration runs on a variety of Unix operating systems, we've only tested it on Red Hat Linux 7.3. Also note that the P4DTI will not be able to run Bugzilla's "processmail" script unless it is running on the same machine as Bugzilla itself. The Bugzilla for Windows integration supports: - Bugzilla 2.14.4 on Windows NT4 or Windows 2000; - Python 2.0 or later with Win32 plug-ins; - Running the P4DTI replicator on Windows NT 4 or Windows 2000. No earlier or later version of Bugzilla is supported on Windows at this time. 3. GETTING SUPPORT For problems relating to Perforce or the P4DTI in general, contact Perforce Support by writing to or see the technical support page at for contact information. For problems relating to TeamShare, contact TeamShare Technical Support by writing to or see the support page at for contact information. Bugzilla is not supported by any one person or organization. Consult the documentation that came with Bugzilla, or visit . 4. PROJECT CONTACTS You may want to join the p4dti-discussion mailing list. The goals of the list are: 1. to provide feedback to the project on requirements, design, implementation, etc.; 2. to allow people to exchange information and experience with using and adapting the project; 3. to keep people informed about project progress. To join, send a message with the word "subscribe" in the _body_ to or send the word "help" for general information. Please note that the mailing list will be archived and the archive may be published. 5. WHAT'S NEW This section lists defects that have been fixed. 5.1. WHAT'S NEW IN RELEASE 1.5.3 ESSENTIAL job000642: AG "patching Bugzilla" section needs to explain "patch" for Windows The P4DTI for Bugzilla needs to apply a patch to Bugzilla using the program "patch". This program is provided as standard on Unix and Linux, but not on Windows. In the AG, section 3.4.1 "Software Prerequisites", we say that "patch" is required and provide a link to an implementation of "patch" for Windows. However, this information is not included in section 5.4.1, "Patching Bugzilla", which a user installing the P4DTI will be reading when "patch" is actually required. job000643: Perforce 2002.2 not supported The Perforce 2002.2 release is imminent. The P4DTI works with this release but does not say so in the documentation (in particular in the release notes). 5.2. WHAT WAS NEW IN RELEASE 1.5.2 ESSENTIAL job000633: Readme.txt has an installer filename wrong The P4DTI readme file, readme.txt, says 'On the Bugzilla Windows machine, run the "p4dti-bugzillawin-1.5.1.exe" installer program that came with this document.' The installer program is actually called "p4dti-bugzilla-1.5.1.exe". job000635: AG doesn't offer "patch" utility for Windows Installing the P4DTI for Bugzilla requires the use of the utility "patch". This is not standard on Windows, and the P4DTI for Bugzilla is now supported on Windows. The AG should explain that "patch" has to be installed, and offer a source of it for Windows. Our test machine has Cygwin installed, which comes with "patch". job000636: Applying Bugzilla patch can put file in wrong place The Bugzilla 2.16.1 patch provided with the P4DTI includes the entirety of a a new file, template/en/custom/bug/edit.html.tmpl. If the patch is applied simply with "patch < bugzilla-2.16.1.patch", this new file is created in the main bugzilla directory. job000637: MySQLdb RPM depends on mx Python module which may not be present The Bugzilla P4DTI requires the Python "MySQLdb" package, to connect to the MySQL database containing the Bugzilla data. This package is available in various forms. The AG links to an RPM for MySQLdb 0.9.1 for Red Hat Linux 7.3. This RPM requires another package, called "mx", to be present in order to install successfully. The AG should either link to an RPM which doesn't require mx or should mention that this mx package is necessary and provide a link to it. OPTIONAL job000632: AG says "doeditparams.pl" not "doeditparams.cgi" In describing the Bugzilla patch, the AG states that it modifies the file "doeditparams.pl". The actual name of this file is "doeditparams.cgi". 5.3. WHAT WAS NEW IN RELEASE 1.5.1 CRITICAL job000628: Bugzilla 2.16 and 2.16.1 patches are broken The patches provided with P4DTI 1.5.0 for Bugzilla 2.16 and 2.16.1 are broken. Using the Bugzilla bug form with these patches results in a "Software error" message in the Bugzilla user interface, complaining about global symbols "$rid" and "$sid": 5.4. WHAT WAS NEW IN RELEASE 1.5.0 ESSENTIAL job000389: Bugzilla integration not supported on Windows Bugzilla can be made to work on Windows but the P4DTI is not supported on Windows. job000533: p4_user with wrong email address confuses Bugzilla user translator In the Bugzilla integration, if the P4DTI's perforce user (configuration item p4_user) does not have an email address matching the P4DTI's Bugzilla user (configuration item replicator_address) then the user translator will translate that Perforce user to the P4DTI's Bugzilla user, even if there is a separate Bugzilla user with email address matching that Perforce user. In other words, the translations in the two directions do not match each other. job000551: P4DTI kit for Unix has DOS line endings Since the release build procedure was automated we've been creating integration kits for Unix with DOS line endings. job000552: Bugzilla for Windows integration is in separate source tree A port of P4DTI to Bugzilla for Windows was done by Parrus Technologies and released by at Perforce. Perforce have asked us to integrate it with the master sources. job000608: Bugzilla 2.14 security patch releases not supported. Bugzilla 2.14.2, 2.14.3, and 2.14.4 have been released as security patch releases to Bugzilla. None of them are currently supported by the P4DTI. We should at least support the most recent patch release (currently 2.14.4). job000609: Test suite for Bugzilla on Windows doesn't work The test suite doesn't support testing for Bugzilla on Windows. To properly support the P4DTI for Bugzilla on Windows (job000389) we have to make the test suite work there. job000610: Bugzilla/Windows integration doesn't support processmail The integration for Bugzilla on Windows doesn't support the use of processmail. The processmail scripts are deferred correctly but then can't be executed because of a limitation of the use of CMD on Windows (can't change directory and run a command with one invocation of os.system()). job000611: TeamTrack test database licenses expired The Teamtrack test databases have licenses which have expired. We need new licenses to enable us to test TeamTrack support. job000613: TeamTrack interface MSVC workspace is confused code/p4dti.dsw and related files control the build of the Python Teamtrack interface on Windows. Unfortunately they have become a little confused: some build (in debug mode) for Python 1.5.2, there is a redundant file, there is a non-existent project ("p4dti"), they have acquired some relative links, and so on. job000614: TeamTrack tests run when configured for Bugzilla on Windows The automated tests for the Python TeamTrack interface run when running the complete automated test suite for the integration with Bugzilla on Windows. job000615: Bugzilla shows extra changelists with multiple Perforce servers When integrating a Bugzilla with more than one Perforce server, the fixes table in a Bugzilla bug with fixes from one Perforce server will show the changelists from all Perforce servers with matching changelist numbers. OPTIONAL job000612: Bugzilla integration doesn't check use_windows_event_log config parameter The Bugzilla integration can now run on Windows. On that platform, the config file may contain a use_windows_event_log parameter. The value of this parameter must be checked (to be 0 or 1). job000616: Linux/Unix system log usage not configurable Under Linux and Unix the P4DTI replicator will send log messages to the system log. The corresponding functionality under Windows is controlled by a configuration parameter "use_windows_event_log". The Linux/Unix system log can theoretically filter out P4DTI messages (with syslog.conf) but it is a good idea to provide a configuration switch as well. job000617: Test suite not very useful under Python 2.2 Changes made to the PyUnit unit test framework between Python 2.1 and Python 2.2 have reduced the utility of the P4DTI test suite. In particular, running a set of tests only produces a list of single-character results ("." for a pass, "F" for a failure, "E" for an error) while the tests are running, preventing any sort of analysis until the run completes. 5.5. WHAT WAS NEW IN RELEASE 1.4.2 ESSENTIAL job000512: Bugzilla P4DTI doesn't work with Python 2.2 The Bugzilla P4DTI inadvertently uses a feature of Python that doesn't work in Python 2.2. Running the P4DTI with Python 2.2 gives an error in the use of has_key at line 412 of bugzilla.py. 5.6. WHAT WAS NEW IN RELEASE 1.4.1 CRITICAL job000461: Bugzilla integration doesn't support multiple Perforce servers The Bugzilla integration doesn't work with multiple Perforce servers. The second replicator instance will fail. ESSENTIAL job000437: Jobspec not suitable for creating new jobs The P4DTI creates a Perforce jobspec that's fine for jobs which are replicated from the defect tracker, but not so good for jobs that are created in Perforce. job000478: TeamTrack 5.02 is supported The P4DTI currently supports TeamTrack 4.5, 5.0, 5.02, and 5.5 (see also job000409 and job000433). This adds to the support burden at Perforce and TeamTrack. Perforce have asked that the set of supported platforms be changed. Specifically, TeamTrack 5.01 and 5.02 should _not_ be supported, and the documentation should recommend that customers upgrade to TeamTrack 5.5. job000483: refresh.py comments are out-of-date The comments in refresh.py indicate that the script will "delete all the jobs on the Perforce server" and refresh them from the defect tracker. This is incorrect; refresh.py no longer deletes any jobs. job000491: jobspec has bad defaults for optional user fields In the Bugzilla integration, if the administrator has added a user field (e.g. 'reporter', 'qa_contact') to replicated_fields, creating jobs in Perforce with the default value of this field will fail. job000494: Bugzilla integration fails if qa_contact is in replicated_fields There is a Bugzilla field "qa_contact", which holds the User name of the "QA contact" assigned to a bug. This may be zero, if the Bugzilla parameter "useqacontact" is turned off, or if there is no QA contact assigned (for instance, if there is no initial QA contact for the component against which this bug is reported). If this field is zero, and the P4DTI is configured to replicate this field to Perforce (i.e. if 'qa_contact' is in replicated_fields in config.py) then the P4DTI will fail. job000501: new jobs break TeamTrack integration if LASTMODIFIEDDATE or SUBMITDATE are replicated In the TeamTrack integration, if either of the fields LASTMODIFIEDDATE and SUBMITDATE are replicated (i.e. are in the replicated_fields configuration parameter) and the P4DTI is set up to replicate new jobs (i.e. replicate_job_p is defined to return 1), and a new job is created without filling in these fields, the P4DTI fails. OPTIONAL job000408: Table of Bugzilla fields in AG is incomplete for Bugzilla 2.14 The Administrator's Guide has a table in section 5.1.5 listing the fields in a Bugzilla bug. The set of fields has changed in Bugzilla 2.14, with the addition of four new fields (reporter_accessible, qacontact_accessible, cclist_accessible, assignee_accessible). These fields are not really suitable for replication, but they should be added to the table in any case. job000492: Test suite doesn't cope with 'reporter' in replicated_fields If you add 'reporter' to replicated_fields in test/config_swan.py, the test suite fails in test_p4dti.py:migrate, saying there's no Bugzilla user corresponding to Perforce user ''. This is because the translate_jobspec function provided by that test doesn't provide a 'reporter' field (required by the new jobspec, but not present in the migrated jobs). job000493: Bugzilla migration fails if delta_ts in replicated_fields If the delta_ts field (the timestamp of the most recent change to a Bugzilla bug) is included in the replicated_fields configuration item, and the user's translate_jobspec function doesn't fill it in, migration will fail. job000495: Test suite doesn't cope with 'qa_contact' in replicated_fields If you add 'qa_contact' to replicated_fields in test/config_swan.py, the test suite fails in test_p4dti.py:migrate, saying there's no Bugzilla user corresponding to Perforce user ''. This is because the translate_jobspec function provided by that test doesn't provide a 'QA_Contact' field (required by the new jobspec, but not present in the migrated jobs). job000496: Bugzilla integration breaks migration if numeric fields are replicated Bugzilla bugs include a small number of numeric fields. One is actually a bitset ('groupset'), one is a genuine number ('votes'), and the rest are actually booleans: 'everconfirmed', 'qacontact_accessible', 'cclist_accessible', 'assignee_accessible', 'reporter_accessible'. If one of these fields is in 'replicated_fields', then migrating jobs without a translate_jobspec function to fill in a numeric value for this field will cause the migration to fail. 5.7. WHAT WAS NEW IN RELEASE 1.4.0 CRITICAL job000433: TeamTrack 5.02 not supported The P4DTI is not known to work with TeamTrack 5.02 (build 50207). job000447: Bugzilla 2.14.1 not supported Bugzilla 2.14.1 was released on 2002-01-05. This is a security patch release, fixing various holes in 2.14 (see the release notes -- link below). job000453: Perforce 2002.1 not supported P4DTI does not support Perforce 2002.1. job000459: TeamTrack 5.5 not supported The P4DTI is not known to work with TeamTrack 5.5 (build 55012). job000463: Bugzilla integration fails if mx.DateTime module is installed The Bugzilla integration breaks if the Python mx.DateTime module is installed on the P4DTI server system. ESSENTIAL job000016: Double replication causes many conflicts There is no reliable way for the replicator to distinguish its changes to jobs from changes made by other Perforce users. This means that everything replicated from the DT to Perforce is then replicated back again, and conflicts are very likely when making edits in the DT in quick succession. job000042: Rapid changes in the DT cause conflicts The replicator thinks there's a conflict if a DT issue is rapidly changed more than once. It replicates the issue to the job, then wakes up and replicates it back (see job job000033), but by now it's changed again in the DT, so the replicator believes there's a conflict. job000244: TeamTrack names the wrong user in an error message I was logged into Perforce as user "lmb" and set the state of a job to "assigned"; the corresponding user in TeamTrack didn't have the privilege to make this transition. I was expecting to get the error message "User lmb does not have privilege for transition Assign." but instead I got "User P4DTI-replicator0 does not have privilege for transition Assign." job000245: Unhelpful error from TeamTrack if you can't edit an issue If you edit an issue without permission, you get an error from TeamTrack like this: "User P4DTI-replicator0 does not have privilege for transition ." This isn't very helpful. job000309: Underscores don't work in Perforce jobviews If you have an underscore in your jobview then it doesn't work. For example, the jobview "state=ready_for_test" doesn't select a job even if the State field for that job has the value "ready_for_test". job000343: When using multiple Perforce servers, TeamTrack uses wrong changelist link When the P4DTI is running with multiple Perforce servers, TeamTrack can link to the wrong changelist URL in the fixes section of the issue. That is, it is only picking one changelist URL and using it for all issues, regardless of which Perforce server the issue is replicated to. job000352: Bugzilla emailsuffix parameter not supported Bugzilla has a parameter "emailsuffix" which is appended to all email addresses. P4DTI has no support for this, and will fail to translate users correctly if it is set. job000409: Not clear if TeamTrack 5.01 is supported or not It's not clear from the release notes and documentation whether the P4DTI supports TeamTrack 5.01. job000413: Bugzilla integration does not work with MySQLdb 0.9.0 or later The P4DTI Bugzilla integration uses a Python database library called MySQLdb to access the Bugzilla database. The P4DTI supports release 0.2.2 and 0.3.0 or this library. There are several more recent releases. The most recent releases do not work with the P4DTI. job000426: Migration to TeamTrack fails unless issue ids are zero-filled to 5 digits If you change the number of digits to which issue ids are zero-filled, migration and replication of new issues doesn't work. job000429: Error message in conflict e-mail might be wrong When the P4DTI sends out an e-mail explaining that a job and issue have been edited simultaneously and that it's overwritten the job with the issue, it might include an incorrect error message. job000436: Migration instructions don't help with deleting and restoring databases When you migrate, you'll probably have to delete and restore your defect tracker database several times. The instructions for migration in the Advanced Administrators Guide should explain how to do this. job000442: Can't replicate 'line' fields with hashes in them to Perforce If you have a field with a # in it in the defect tracker, and if that field is replicated to a 'line' field in Perforce, the contents don't get replicated correctly to Perforce because the # is interpreted as the start of a comment. job000445: Bugzilla replicator doesn't like spaces in enum fields The Bugzilla replicator doesn't translate an "enum" field correctly from Bugzilla to Perforce if the values in the field include spaces or other non-word characters. Two enum fields ("bug_status" and "resolution") are handled specially. For other enum fields (i.e. "priority" and "op_sys" in the default Bugzilla schema) there is no special handling and the replicator will fail if given a value with a non-word character such as a space. OPTIONAL job000086: Users can "fix" issues that they don't have permission to change A user can change the status for a job, and the corresponding defect tracker issue will change in status, even if the user doesn't own or have permission to change it. The user has to sneak in an illegal "p4 fix" after a legitimate user has edited the job and then delete the fix. job000276: Can't fix (and then delete the fix) a job owned by user (None) in the TeamTrack integration If an issue has no owner in TeamTrack (and therefore the owner of the corresponding job is (None) in Perforce) then you can't transition the job by fixing and then immediately delete the fix. The replicator is unable to replicate the change to the job and it overwrites the job with the issue. job000289: Users with spaces produce confusing errors A Bugzilla or TeamTrack user name containing spaces causes confusing errors to be produced by the P4DTI, especially when it doesn't correspond to a Perforce user, and perhaps even when it does. job000333: Poor error message from TeamTrack 5.0 when you specify wrong user or password If you specify the wrong user id or password in my P4DTI configuration, you get a misleading error message from TeamTrack. job000427: Can't delete associated filespec in Bugzilla integration In the Bugzilla integration, if you create an associated filespec record in Perforce (by adding a line to the P4DTI-filespecs field) and then delete it, then the deletion can't be replicated to Bugzilla. 5.8. WHAT WAS NEW IN RELEASE 1.3.2 CRITICAL job000422: Refreshing fails after migration if new workflow doesn't match old workflow If you have fixes in Perforce with statuses other than "closed", and if you migrate your jobs from Perforce to the defect tracker and in doing so change the workflow so that some fix statuses disappear, then the P4DTI can't refresh the fixes after the migration. ESSENTIAL job000249: Refresh script fails if you change the closed_state Suppose you start out with closed_state=None and people make fixes in Perforce to the state "resolved". Then you set closed_state="resolved" and run the refresh script. This fails because it tries to make fixes to the state "resolved", which Perforce refuses because there is no Perforce state "resolved" (Perforce now uses "closed" instead). job000421: Migration of jobs to Bugzilla fails if a job is owned by a non-existent user If you have jobs in Perforce that are owned by a non-existent user, or a user who's been deleted, then the P4DTI can't migrate those jobs to Bugzilla. 5.9. WHAT WAS NEW IN RELEASE 1.3.1 CRITICAL job000418: Migration of users to Bugzilla fails for lack of migrated_user_groups If you attempt to migrate users from Perforce to Bugzilla following the instructions in the Advanced Administrator's Guide, section 4.4, the operation fails because there is no value for the configuration parameter migrated_user_groups. job000419: Can't migrate users to Bugzilla 2.14 Migrating users from Perforce to Bugzilla works for Bugzilla 2.10 and 2.12, but in Bugzilla 2.14 it fails with a complaint from MySQL that the field 'password' is unknown. ESSENTIAL job000420: Too time-consuming to find all problems with migration With our current process, migrating jobs involves repeatedly running migrate.py, finding a problem with a particular job not being translated for some reason, adjusting the translate_jobspec and prepare_issue functions to take account of the exceptional case, and running migrate.py again. Since migration starts with the first job each time, this procedure is potentially O(n^2) in the number of jobs. 5.10. WHAT WAS NEW IN RELEASES UP TO 1.3.0 For information about what was new in releases up to and including 1.3.0, see the release notes for that release: A. REFERENCES None. B. DOCUMENT HISTORY 2002-02-01 GDR Noted support for TeamTrack version 5.5. 2002-02-04 GDR Updated for release 1.4.0. 2002-03-14 NB Removed support for TeamTrack 5.02. 2002-04-09 NB Updated for release 1.4.1. 2002-04-19 NB Updated for release 1.4.2. 2002-06-04 GT Updated for Bugzilla on Windows 2000 port for release 1.4.2. 2002-06-26 RB Merged Bugzilla for Windows 2000 port into Ravenbrook sources. 2002-10-30 NB Tidied up in preparation for 1.5 releases. 2002-11-01 NB Updated for release 1.5.0. 2002-11-20 NB Updated for release 1.5.1. C. COPYRIGHT AND LICENCE This document is copyright (C) 2002 Perforce Software, Inc. All rights reserved. Redistribution and use of this document in any form, with or without modification, is permitted provided that redistributions of this document retain the above copyright notice, this condition and the following disclaimer. THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. $Id: //info.ravenbrook.com/project/p4dti/release/1.5.3/release-notes.txt#1 $