PERFORCE DEFECT TRACKING INTEGRATION RELEASE NOTES FOR RELEASE 1.0.1 Gareth Rees, Ravenbrook Limited $Date: 2001/02/23 $ CONTENTS 1. Introduction 2. Supported configurations 3. Getting support 4. Project contacts 5. Known defects 6. What's new 6.1. What's new in release 1.0.1 6.2. What was new in release 1.0.0 A. References B. Document history 1. INTRODUCTION This is the release notes for release 1.0.1 of the Perforce Defect Tracking Integration (P4DTI). The P4DTI connects your defect tracking system to Perforce, so that you don't have to switch between them and enter duplicate information about your work. It also links changes made in Perforce with defect tracker issues, making it easy to find out why a change was made, find the work that was done to resolve an issue, or generate reports relating issues to files or codelines. For instructions on installing the P4DTI, see the product readme (readme.txt). For up-to-date information about releases of the P4DTI, see the product information page . From there you will find links to the latest releases, including reports of defects found. If you want to adapt or extend the P4DTI, please go to the product information page and download the Integration Kit. It contains full source code and documentation to help you. The readership of this document is anyone who wants to download and use the Perforce Defect Tracking Integration. This document is not confidential. 2. SUPPORTED CONFIGURATIONS This release supports: - Perforce 2000.2 on any platform; - TeamTrack 4.5 running on Windows NT 4 or Windows 2000; - Bugzilla 2.10 on Red Hat Linux 6.2 or Solaris using MySQL The Bugzilla integration may work on other operating systems, but has not been tested. 3. GETTING SUPPORT For problems relating to Perforce or the P4DTI in general, contact Perforce Support by writing to or see the technical support page at for contact information. For problems relating to TeamShare, contact TeamShare Technical Support by writing to or see the support page at for contact information. Bugzilla is not supported by any one person or organization. Consult the documentation that came with Bugzilla, or visit . 4. PROJECT CONTACTS You may want to join the p4dti-discussion mailing list. The goals of the list are: 1. to provide feedback to the project on requirements, design, implementation, etc.; 2. to allow people to exchange information and experience with using and adapting the project; 3. to keep people informed about project progress. To join, send a message with the word "subscribe" in the _body_ to or send the word "help" for general information. Please note that the mailing list will be archived and the archive may be published. 5. KNOWN DEFECTS IMPORTANT: Read the defect reports available from the product information page . You will find information about defects found _after_ release. There could be vital information there which we couldn't know when we wrote this document. If you don't read the document you might meet the same problems, waste a lot of time, and possibly DESTROY ALL YOUR DATA. This is a list of major defects which were known at the time of release. CRITICAL job000200: No supported TeamTrack release works with integration There's no officially supported TeamTrack release that works with the integration. job000205: Configuration is still too difficult The Administrator's Guide is too hard to use. Administrators are busy people and don't read documentation, not even the very nice documentation that we've put so much effort into. ESSENTIAL job000016: Double replication causes many conflicts There is no reliable way for the replicator to distinguish its changes to jobs from changes made by other Perforce users. This means that everything replicated from the DT to Perforce is then replicated back again, and conflicts are very likely when making edits in the DT in quick succession. job000022: No migration from Perforce jobs The Administrator's Guide talks about migrating from using Perforce jobs to using the integration with a DT. We don't provide any method of doing this. The software assumes complete control of the jobs subsystem of Perforce, and will damage any existing jobs data. job000042: Rapid changes in the DT cause conflicts The replicator thinks there's a conflict if a DT issue is rapidly changed more than once. It replicates the issue to the job, then wakes up and replicates it back (see job job000033), but by now it's changed again in the DT, so the replicator believes there's a conflict. In version 0.4 there will be a "defect tracker wins" policy by default, so this will not stop replication, but it will send annoying e-mails reporting conflicts where there are none, and thus increase the burden of the system on its users. job000054: There are few reports There are no reports to help with analysis of the information gathered by the integration. This is a big reason for having the integration in the first place. (See requirement 5, requirement 60.) job000064: There's no product information page or sheet There's no product information sheet to tell people (esp. managers) at a high level what the integration does. In particular, we don't tell people that the integration enforces DT workflow and security in Perforce. job000065: Not enough logging control There's only one level of logging at the moment. We have to edit the code to get more debugging output in the log. This won't be possible or easy when beta testing or supporting the integration. job000066: We don't explain which jobspecs won't work "p4 -G job -o" applies much more stringent tests than "p4 job -o". It won't even give you a form unless all the values are correct. This means that tricks like ones in Perforce's jobspec, where an illegal value like "setme" is specified as the default value for a select field, can't be used with the integration. job000069: No resolver documentation We must write documentation for the resolver. For example, there are two ways of resolving conflicts: either set to "discard" or fix up the problem and set to "keep". Need to document both of these. job000076: Deleting records gives the replicator a headache The replicator doesn't cope well when a defect tracker record is deleted. When you delete an issue, then the next time (and every subsequent time) you edit the corresponding job it will fail to fetch the issue from the DT and send e-mail to the administrator. job000077: UG contains no version or release related documentation There's no documentation corresponding to use cases 6.12 and 6.15 dealing with finding issues related to versions or releases and vice verse. job000078: Replication failure can cause virtually irrecoverable database records If the initial replication goes wrong (e.g. fails to create a Perforce job because the server is down) then the DT issue is marked with P4DTI action "wait" but there's no corresponding job to set to "keep" in order to overwrite it. You can't fix the problem without going into the database directly using SQL or our own Python interface. job000082: AG has no training and documentation section The AG has no guidance on training and documentation for users. job000086: Users can "fix" issues that they don't have permission to change A user can create a fix for any job, and the corresponding defect tracker issue will change in status, even if the user doesn't own or have permission to change it. job000124: Changes by disabled Bugzilla users may be replicated Bugzilla has a mechanism for disabling users. We don't check this as part of checking user permissions. We should. job000128: Can't replicate fix to deleted changelist The replicator can't replicate a fix to a deleted changelist, and this gets it into a right tangle: it can't continue replicating. Luckily this is a rare occurrence: normally the deletion of a changelist is replicated to the defect tracker, so the situation doesn't arise. It's only if you delete a changelist and simultaneously cause the replicator to replicate a fix for that changelist back from the defect tracker that this can happen. job000134: The replicator log grows without limit The replicator log just keeps on growing. job000137: You can fix a job that you haven't been assigned A user can add a fix for a job that they don't have permission to transition or update. job000148: Too long to replicate because of reading USERS, STATES, etc. many times The replicator may read auxiliary tables in TeamTrack (like the USERS table) many times in the course of replicating a single job or issue. This slows replication down and increases the possibility of conflicts. job000149: We don't use system logging facilities. We don't use system logging facilities (syslog/Windows Event Log); instead we log to a file. We should use system logging. Managing applications which don't is irritating. job000159: Fixes which the replicator rejects still get replicated If a Perforce user makes a fix which the defect tracker disallows for some reason (e.g. it's an illegal transition in the defect tracker, the user doesn't have the necessary permissions to make the transition, etc), then the replicator will correctly refuse to replicate the change to the issue's status and will undo the change to the job's status, in accordance with the "defect tracker wins" policy. However, the fix record itself will be replicated, and will show up in the defect tracker's interface. This is misleading. Better behaviour would be for the replicator to remove the fix (and not replicate it). job000183: Mysterious problem with elapsed time fields in Perforce A user reported this error: Perforce error: Error in job specification. Error detected at line 18. Invalid date '1970/01/01 00:30:00'. This is mysterious because this date is valid in Perforce. job000192: tTrack triggers not run A beta tester found that a case transition caused by the P4DTI did not run the tTrack triggers associated with the transition. job000228: UG text is somewhat specific to tTrack The text of the UG is somewhat specific to the tTrack integration. For instance, section 4 suggests filtering jobs on the "owner" field, which is not present with a Bugzilla integration (it's called "Assigned_To" instead). OPTIONAL job000020: If the replicator detects a conflict and there's no job then it fails If the replicator detects a conflict between a DT issue and a job, and the job doesn't exist, it fails mysteriously. This can happen if there's a bug in the replicator or the Perforce jobspec that means that the initial replication from the DT fails to make the job. It's hard for a program to tell if a job exists in Perforce, so this is a problem of robustness. Most likely to result from a problem with the configuration, or perhaps if the Perforce server is down. job000030: Users can't get help based on messages The P4DTI produces a lot of messages. It would be good to allow users to follow some sort of link from these to more information and experience. job000036: New jobs in Perforce don't get replicated to TeamTrack. New jobs in Perforce don't get replicated to TeamTrack. job000040: You can't fix the default changelist There's no way to associate a job fix with the default changelist, so that when it is submitted, the job is fixed. Instead, you have to make a numbered changelist first. job000044: There's no way to prevent a transition without a fix Users would like to prevent issues transitioning [sic] to certain states unless there's a fix that links the issue to a change. job000046: The replicator process is hard to manage The replicator process is hard to manage, especially on Windows NT, where one has to launch it using a non-trivial command line at a prompt. job000062: Fixes not replicated until another change is made When we added a fix to a job, the replicator doesn't actually manage to replicate the fix until or unless you change a field in the job. We've not been able to reproduce this problem since alpha testing, and it only occurred at one site. job000084: No glossary in the UG There's no glossary in the user's guide. job000098: Perforce UIs don't link to defect tracker UIs p4win and/or p4web should link jobs to defect tracker issues. We could help with this by putting URLs into the job (e.g. in the form recommended by the appendix to RFC 1738). Then it would be up to p4win and/or p4web to turn those into live links. job000099: The p4 module has a security hole Any user could cause the replicator to run arbitrary shell commands by putting appropriate shell meta-characters in a jobname or Perforce user name. job000110: Replicator falls over if you copy an issue in TeamTrack If you do a copy transition in TeamTrack, then the replicator can't cope with the new issue. job000114: There's no description of the limitations of the integration People thinking of installing the integration need to know what its limitations are. There needs to be a "will this work for your organization?" document. job000125: Bugzilla version not checked We don't check the Bugzilla version (which should be 2.10 for the integration to work). job000139: Trigger scripts duplicate replicator code The example trigger script we provide has its own interface to Perforce, duplicating the "p4" class in the replicator. It would be better to share the code, for maintainability and robustness. job000145: Deleted TeamTrack states appear in Perforce jobspec If you delete an issue state in TeamTrack, it still appears among the values for the State field in the Perforce jobspec. job000151: Source code documentation is inadequate The documentation in comments in the source code is generally inadequate. Each source code file should conform to the general rules. There needs to be more cross-referencing to design documents. The Bugzilla patches need to be cross-referenced to the Python sources. RB 2000-12-13 job000162: Replicator needs restarting when you add new users to Perforce or the defect tracker When you add a new user to Perforce or the defect tracker or change a user's e-mail address, you have to restart the replicator. job000196: Illegal transitions should be prevented by Perforce clients. Perforce users can make job transitions which are prohibited by the DT workflow. The P4DTI reports and undoes these illegal transitions, but it would be better if the P4 clients did not allow them in the first place. job000197: P4DTI doesn't do ESMTP authentication Some ESMTP servers (e.g. Microsoft Exchange) may be configured to require authentication (RFC 2554). The P4DTI doesn't do this, so can not send email via these servers. Currently the P4DTI will not work if it can't send email. job000201: Empty assigned_to field can't be replicated If the "Assigned to" field in Bugzilla is empty, then it can't be replicated. You get the error "Error in job specification. Missing required field 'Assigned_To'." job000207: Deleted jobs aren't restored immediately If you delete a job in Perforce, then it gets restored when the issue is next updated in the defect tracker. It would be better if it were restored immediately. job000209: No package for Solaris There's no package for the Bugzilla integration on Solaris that's as convenient to install as the RPM for Linux. job000211: Mail can get lost If the replicator tries to send e-mail and fails (typically because of a problem with the SMTP servr), then it just drops the mail. The contents is therefore lost. job000213: Performance of Bugzilla is reduced The replicator takes locks on a number of tables each time it polls, even if there is nothing to do. This reduces the Bugzilla performance. job000216: readme.txt mentions beta sites The readme.txt mentions particular beta sites (in the known issues section). Probably we should avoid this. job000220: Perforce superuser password is in clear in config.py The replicator's Perforce user's password is in clear in config.py (and so is the replicator's TeamTrack/Bugzilla user's password). This is a security risk. job000227: AG doesn't have Bugzilla error messages Section 11.2 of the AG lists and explains P4DTI error messages. Most of the error messages produced by the Bugzilla integration are not mentioned in this section. job000231: Perforce state field contains bogus states if replicating a subset of projects from TeamTrack If you're using the replicate_p configuration parameter to limit the replicated issues to a subset of projects in TeamTrack, then it's annoying that the legal values for the Perforce state field include states from projects and workflows that aren't being replicated. NICE job000061: Fixes are invisible in TeamTrack by default The fixes table only shows up in a case in TeamTrack if the user has checked the "version control" option in their profile. By default, it's unchecked. job000152: "commenton" parameters not enforced by Bugzilla integration Bugzilla has a set of parameters called "commenton" which forces Bugzilla users to enter comments (in the long description) when they do certain things to the bug record. We don't enforce these parameters through the P4DTI. job000153: "despot" not supported by Bugzilla integration Bugzilla has a feature called "despot" which we don't support. [Need a description of what it is here. RB 2000-12-13] job000154: "shadowdb" is not supported by the Bugzilla integration Bugzilla has a feature called "shadowdb" which creates a read only copy of the database for faster public access. The integration doesn't support this feature. job000175: System fields like P4DTI_RID are editable by users in Perforce Some fields in the Perforce jobspec should not be edited by the user, for example P4DTI_RID and P4DTI_ISSUE_ID. But these are easily editable in the Perforce command line and GUI. We have to depend on people's carefulness. job000179: Teamtrack.dll can get lost An inexperienced Windows user who copies the P4DTI files from the unpack directory to another one might end up without teamtrack.dll (since DLLs are hidden by Windows tools like Windows Explorer by default). job000203: Users can masquerade as other users Because users are matched by e-mail address between the defect tracker and Perforce, you can fool the replicator by running "p4 user" and editing your email address. job000206: Administrator Guide needs some improvements Here are some miscellaneous improvements that should appear in the AG: 1. Advice needed on setting up p4web (e.g., use browse mode). 2. Have a table of auxiliary scripts with purpose. 3. Some metrics about how much time the replicator takes and what you can do to improve the performance. 4. Tell people not to switch TeamTrack databases under the feet of the replicator. 6. WHAT'S NEW This sections lists defects that have been fixed. 6.1. WHAT'S NEW IN RELEASE 1.0.1 The following defects were fixed between release 1.0.0 and release 1.0.1. CRITICAL job000027: There are outstanding defects that were found in release 0.3.0. job000221: Refreshing Perforce jobs fails in Bugzilla integration A user of 0.5.1 attempted to refresh Perforce jobs, but that failed with the following error: File "bugzilla.py", line 192, in sqlquote raise error, ("sqlquote given non-string %s." % str(value)) P4DTI Bugzilla interface error: sqlquote given non-string None. ESSENTIAL job000135: Replicator makes no more progress if replication to Perforce fails If replication from the defect tracker to Perforce fails for whatever reason, the replicator gets into a loop where it keeps e-mailing the administrator. job000166: May fail with later MySQLdb versions We may fail with later MySQLdb versions, in particular 0.3.0 which I believe is gaining popularity. We haven't tested this. We should test and make any appropriate fixes. If there are serious incompatibilities, we should at least test the version (MySQLdb.__version__). It may not be appropriate to require 0.2.2, as the MySQLdb version is a system-wide property and users may have other requirements forcing them to use 0.3.0. Also, operating systems and Python or MySQL packages are likely to be distributed with 0.3.0 (or above) in future. job000174: The AG doesn't tell you how to upgrade from the beta A small number of sites will be running a beta release like 0.4.2 or 0.5.1. When release 1.0.x comes out they will naturally want to upgrade. But the AG doesn't have any instructions telling them how to do this. job000188: tTrack integration fails with non-ASCII characters The tTrack integration does not cope with non-ASCII characters. For instance, "é" (character 0xe9). If this character is placed in a tTrack field, it gets into Perforce as "˙ffffffffffe9". Note that the first character is 0xff. Note also that the number of f's seems to depend on the exact platform: it's four on sandpiper (suggesting a 24-bit encoding) and ten on the reporting user's machine (suggesting a 48-bit encoding ?!). This encoding also appears in the log output; I don't think the P4DTI is doing it. Going the other way, putting "é" into a Perforce job causes the P4DTI to fail when trying to update tTrack (no message from the TeamShare API). job000191: Admin can't easily find release when contacting support In the AG we tell the administrator to report which release of P4DTI they are using. But how do they find this out? The documentation isn't clear and the program doesn't report it. job000194: tTrack states include tSupport states tSupport and tTrack both have states. When replicating from tTrack, the jobspec status field includes tSupport states, which are not possible for tTrack cases. This is not ideal. job000195: keyword translation is too conservative We translate keywords (items for Perforce job fields of type 'word') by using a hexadecimal escape %xx for characters other than [a-zA-Z0-9(),.?!-]. This is very conservative, and rules out a number of characters commonly used in defect trackers. We also translate the space character ' ' into the underscore character '_'. This has the problem that we do not escape underscores, so underscores in a defect tracker become underscores in Perforce and then translate back to spaces. We need a translator which is (a) bijective, (b) less conservative, and hopefully (c) less obfuscating. job000215: The replicator can send out mail bombs The replicator can get stuck in a loop where it sends e-mail to the administrator every 10 seconds (or whatever the poll_period is). This is very unfriendly. Not only that, the e-mail message can double in size each time because each message includes the formatted_traceback from the previous message. job000219: Existing jobs in Perforce may become unusable when the jobspec is changed When the P4DTI starts up for the first time, it changes the jobspec without checking to see whether any jobs are present. These jobs may become unusable when the jobspec is changed. job000222: Deleting fix causes replicator to crash A user tried out "p4 fix -d ...". The Bugzilla integration crashed with the following error: File "bugzilla.py", line 482, in delete_fix ('bug_id = %d and changelist = %d ' TypeError: not enough arguments; expected 4, got 3 job000225: If you "p4 fix" when there's no closed state, the replicator can't replicate When you do "p4 fix -c CHANGE JOB" then Perforce sets the status of the job to "closed". If "closed" is not a legal status in Perforce then the replicator can't get at the job any more, so it fails to poll. job000229: The readme is too hard to use The readme.txt is too hard to use. The lists of fixed and open issues are too long; they should be in a file of their own (release-notes.txt). job000230: Messages from the replicator refer to non-existent sections of the manuals Messages from the replicator refer the reader to sections of the UG and AG. But sometimes these sections don't exist. 1. When a job is overwritten, the replicator refers you to "section 2.5 of the P4DTI User Guide". 2. Tooltips for P4DTI-filespecs and P4DTI-action refer you to section ? of the Administrator's Guide. OPTIONAL job000052: Likely problems not in troubleshooting section Likely errors: 1. can't login: did you set the user name in TeamTrack to match the replicator id? 2. run out of users in Perforce? have you changed the replicator id recently? If so, make sure you delete the old Perforce user for the replicator. job000167: Released manuals should be on the website The released manuals are currently not available on the website (except by downloading and installing a release). This has several undesirable consequences. For instance, users browse the master or version manuals, which do not reflect the release which they have downloaded. Also it makes it impossible to give a URL into a released manual. job000208: TeamTrack integration doesn't provide a .reg file It's error prone to tell people to use regedit to set the "VC Integration" registry key. It would be better to provide a .reg file which people can double-click on to set the registry key. job000210: Bugzilla states/resolutions with spaces in them cause broken jobspec If the Bugzilla database schema is altered so that states and resolutions have spaces in them (this is legal in MySQL) then the replicator generates a broken jobspec. job000223: Quote in change comment terminates display in TeamTrack A quote character (single or double) in a change comment terminates the display of that comment in the "Perforce fixes" table in TeamTrack. job000226: Newlines don't show up in change descriptions in TeamTrack Newlines in change descriptions don't show up in the "Perforce fixes" table. Instead, lines are placed adjacent to each other. job000232: Log and e-mail messages are confusing if jobname is different from the issue name Log messages and e-mail messages describe an issue using the name of the corresponding job. This is OK when they are the same (which is the usual case), but misleading when they are not (which could happen if you've migrated your jobs and kept the jobname). NICE job000171: AG should note broken MySQL versions MySQL 3.23.29-gamma and 3.23.29a-gamma are broken, in that they stop Bugzilla logins from working. We should note these in the Prerequisites section of the AG. 6.2. WHAT WAS NEW IN RELEASE 1.0.0 The following defects were fixed between release 0.5.1 and release 1.0.0. CRITICAL job000181: Assertion failure in translate_1_to_0 If an optional field in Perforce has no value, then a field translator may fail with an assertion error. job000190: Connection to TeamTrack hangs for several minutes When release 0.4.2 attempts to connect to TeamTrack 4407, it hangs for several minutes before connecting. job000217: readme.txt has too much detail of fixed bugs It's good to identify fixed bugs in the readme.txt, but if we do so verbosely then even mildly impatient readers will never get to the 'known bugs' section, which is arguably much more important. job000218: readme.txt has a confusing date at the top The readme.txt file says "Richard Brooksby, Ravenbrook Limited, 2000-10-30", right at the top, because it conforms to our standard document style. This will be very confusing to users, and may well put people off in time (e.g. when it still says this in late 2002, and users download it and say "eeek, this hasn't been maintained for two years"). ESSENTIAL job000006: TeamShare API error reporting is inadequate. When TeamTrack refuses to update or transition an issue because a user doesn't have the privilege to do so, it raises an error but provides only an empty error message. The replicator can't provide any information to the user to explain why the operation failed. job000033: Incompatible with other TeamShare API programs The P4DTI is incompatible with other TeamShare API programs because changes made by other TeamShare API programs are not replicated. The replicator has no reliable way of distinguishing its changes to TeamTrack from other users' changes in TeamTrack; this is because all API users show up as user 0 in the CHANGES table. So changes made in TeamTrack by other API users (e.g. other integrations or scripts) will not be replicated accurately. job000048: Consistency checking of the configuration is inadequate Consistency checking of the configuration needs to be much stronger, to catch problems so that they don't affect the replicator when it's in use. job000075: No automatic check of configuration How do you tell if the configuration is correct? There should be automatic checking of the configuration. See e-mail "Checking the configuration automatically" for some ideas. job000092: Long descriptions aren't replicated by Bugzilla integration In Bugzilla, bugs have short descriptions (called "Summary" in the web interface) and long descriptions. Each bug may have many long descriptions; they carry a timestamp and are displayed in date order. They also carry user names. Currently these long descriptions are not replicated. Perhaps they should be; the short description can be too short and/or misleading. Note that it is not possible to edit or delete long descriptions from the Bugzilla web interface; only to submit new descriptions to be appended. job000093: Can't replicate other Bugzilla fields Bugzilla bugs have loads of fields. We only replicate a small number of them. We don't support the "replicated_fields" parameter in the Bugzilla configuration. job000103: Can't easily add to replicated_fields list The configurator doesn't cope if you change the replicated_fields. The P4DTI administrator can't add a new field to the replicated_fields list (except at the end of that list) because the new field will get a Perforce field number that was previously used by another field. So the data that was previously in that other field gets captured by the new field. If you have used the automatic configuration generator and you edit the replicated_fields list, fields probably get the wrong values in Perforce, because the generated jobspec's field numbers probably don't match the old values. job000109: Can't get at Perforce information from TeamTrack TeamTrack lists associated changes for each issue, but you can't easily get at any more details without going to the Perforce interface. It's inconvenient, and not every TeamTrack user will even be licensed to go to the Perforce interface. job000116: Bugzilla integration doesn't do enough checking The Bugzilla integration does hardly any checking, either of the configuration or of data encountered during replication. This is in contrast to the TeamTrack integration, which now does quite a lot of checking. job000117: Jobview and job filter confused in the UG The UG says that the list of jobs displayed in the GUI dialogs is affected by the users jobview. This isn't true. It's the job filter that's used. The jobview only affects the command line interface. job000122: Server failures aren't handled gracefully If the Perforce server or the defect tracker is down, the replicator produces unhelpful error messages and e-mails to the administrator. job000138: "Add Job Fix" always sends jobs to "closed" The "Add Job Fix" command in the Perforce Windows GUI always sends a job to status "closed". Users will want to link changes using other keywords. job000141: Can't add to replicated_fields list The P4DTI administrator can't add a new field to the replicated_fields list (except at the end of that list) because the new field will get a Perforce field number that was previously used by another field. So the data that was previously in that other field gets captured by the new field. job000155: Bugzilla integration doesn't send mail Bugzilla sends lots of email, when bugs change. We don't do that. Bugzilla has a Perl script called "processmail", which it runs whenever a bug changes. It analyses the bug activity and may send mail to one or more Bugzilla users describing that activity. We don't do this, or emulate it, so Bugzilla bugs will be changed by the replicator without email being sent out. job000156: Pending changelists aren't clearly indicated as such in Bugzilla You can't tell the difference in the P4DTI section of the Bugzilla bug form between a pending changelist and a submitted one.Ę This means that you might mistakenly think that work has been done when in fact it is only pending and may later be reverted. job000157: Improper installation on Linux The integration is distributed as a simple tarball on Linux and doesn't install anywhere in particular. The AG says to put it in the administrator's home directory for the moment, which isn't good enough. job000172: Replicating some fields can break Bugzilla Some fields can not be changed using the Bugzilla interface, or only changed in highly restricted ways. Now that these fields can be replicated into Perforce, we should restrict changing these fields in Perforce. For instance, the 'votes' field can only be changed by voting, which involves checking and updating other fields in other tables and may have other side-effects. For another instance, the 'product', 'version', and 'component' fields must be keys into other tables. For a third instance, the created_ts field is never updated. job000173: Wrong Perforce server version causes installation to fail mysteriously If you have the wrong Perforce version (2000.1 or earlier) then installation will fail, with an error like this: "Perforce error: Error in job spec specification. Missing required field 'Values-State'." It should instead say something like "Perforce release not supported; changelevel NNNNN required." job000178: AG doesn't give advice on making problem reports The AG doesn't give advice on how to get effective support for the P4DTI. Typically someone e-mails a section of their P4DTI log -- for example, the error message at the end. This is rarely enough information to track down the problem. job000180: bugzilla_user is confusing. Many people find the bugzilla_user configuration parameter confusing. We should get rid of it and use the replicator_address parameter in its place. job000182: Elapsed time fields aren't replicated properly Date fields in TeamTrack have a "display type" which can be "Date only", "Date and time", "Time only" and "Elapsed time". The repicator copes well with the first two, but poorly with the second two (an elasped time of 10 hours in TeamTrack becomes a date of 1971/01/01 10:00:00 in Perforce). job000184: Bugzilla integration doesn't work if your database is not called 'bugs' If your Bugzilla database is not called 'bugs', then P4DTI tries to connect to 'bugs' anyway. So it doesn't work! job000185: python RPMs no good The Python 1.5.2 RPMs are no good for building the MySQLdb interface. In the AG we offer the Python 1.5.2 RPM. This is insufficient on its own to build the interface (because it doesn't have the header files). There is a python-devel RPM which does include header files, but for some reason the MySQLdb 0.2.2 and 0.3.0 interfaces don't compile out of the box with it. Building python 1.5.2 from the tarball distribution works OK, as long as you include the optional syslog module (by uncommenting the syslog line from Modules/Setup before running make). job000186: AG suggests python RPM which is insufficient MySQLdb can't be built using just the python RPM linked to from the AG. Our users need to build and install MySQLdb. To build it they need the python 1.5.2 header files. The python RPM linked to from the AG doesn't have any header files. They are contained in a python-devel RPM. However, the python-devel-1.5-2 RPM which we installed on swan (which may have downloaded from ftp.python.org; www.python.org does not now link to any RPMs for python 1.5.2) provides header files which are not quite right for building MySQLdb. I haven't dug very deeply here. I managed to build MySQLdb 0.2.2 by editing its source code. This is not very satisfactory. For MySQLdb 0.3.0 I also had problems. I resolved these by uninstalling the python and python-devel RPMs and building python from the sources (python-1.5.2.tar.gz). The optional syslog module is required by the P4DTI, so I had to edit Modules/Setup before running make. We need to fix the prerequisites section of the AG so that users get a python environment on which they can build MySQLdb and run the P4DTI. job000187: Creating Bugzilla tables twice We call bugzilla.create_p4dti_tables() twice on startup; once when making a bugzilla.bugzilla in configure_bugzilla and once when making a dt_bugzilla.dt_bugzilla in init.py. We can remove the call from inside dt_bugzilla.__init__. job000189: replicated_fields can't be changed without replicating all jobs In section 10 of the prepared manuals we tell people to run the refresh script if they change the replicated_fields parameter, but this forces all issues to be replicated, which they may not want. job000198: We don't check that bugzilla_directory works The configuration parameter bugzilla_directory names a directory in which the P4DTI will find a script called 'processmail'. The P4DTI runs this script after it changes a Bugzilla bug, and it sends email to Bugzilla users just as if the bug was changed through the Bugzilla web interface. We do not currently check that this parameter names a directory, or that the directory it names contains a processmail script. If either of these things is false, the P4DTI will fail to invoke the processmail script. job000199: Auxiliary scripts send e-mail The auxiliary scripts, check.py and refresh_perforce.py, send e-mail to the administrator saying that the replicator is starting up. They shouldn't do this, because it's unnecessary and the message is not true. job000202: Errors from Perforce not reported well Some Perforce errors are reported only as "The Perforce client exited with error code %d. Is the server down or the server address incorrect?" This provides little help in tracking down the problem. job000204: Issue owned by non-existent Perforce user causes crash If the Perforce server is running without any spare licences, then when the replicator tries to overwrite a job whose owner is a user that doesn't exist in Perforce, then you get the error message "Can't create a new user - over licence quota". job000212: TeamTrack 4.5 not supported The meaning of the arguments to TSServer::AddField() have changed in TeamTrack 4.5, which means that the new fields aren't be properly added to the CASES table. job000214: No way of controlling the poll period There's no easy way for the administrator to control the poll period. If 10 seconds gives performance problems then there should be a way of increasing the delay. OPTIONAL job000024: "\012" in fixes table in TeamTrack Change descriptions in the "Perforce fixes" section of a TeamTrack case description display the octal escape "\012" as those four characters, rather than as a newline (ASCII 10). job000032: Deletion of fixes and filespecs in TeamTrack may cease to work in future releases. Deletion of fixes and filespecs from the TeamTrack interface is correctly replicated to Perforce at present. But this depends on an undocumented feature of TeamTrack and so may cease to work when new releases of TeamTrack appear. job000101: Different transitions for different issue types may confuse the replicator In TeamTrack, different issue types (bugs, enhancements, etc) may have different workflows: that is, certain transitions may be enabled for some issue types but not for others. The replicator doesn't take this into account and so may pick the wrong transition, which means that the transition may fail. It may happen that the replicator can't transition any issues for this reason. job000160: 'closed' doesn't map to 'RESOLVED' in the Bugzilla integration The 'closed' status for Perforce jobs is special: p4 fix without a -s flag will cause a transition to 'closed'. The natural status for this to translate to in Bugzilla is 'RESOLVED': engineers should make bugs RESOLVED when they resolve them. But currently 'closed' in Perforce translates to 'CLOSED' in Bugzilla: to move a bug to 'RESOLVED', Perforce users must do 'p4 fix -s resolved'. Perforce users will want to move jobs to 'RESOLVED' often but to 'CLOSED' rarely. Using 'p4 fix' without '-s' will usually cause the replicator to reject the update because there are few legal transitions to 'closed'. job000163: jobspec fields too small for Bugzilla. The replicator makes a new Perforce jobspec. The "assigned_to" field has size 32. This must in fact be large enough to hold a Bugzilla login_name (from the "profiles" table in Bugzilla), which is varchar(255). The same may apply to other jobspec fields. job000165: Bugzilla configuration parameters are not checked. We don't check the Bugzilla configuration parameters. For instance, we don't check that dbms_port is a number, or dbms_host is a string, or that bugzilla_user is an email address. The result is that if a user sets a configuration parameter incorrectly, or leaves it unset, they may get a very obscure internal error from, for instance, MySQLdb. job000169: Change numbers are links in TeamTrack even when no changelist URL has been specified If changelist_url is None in the configuration, then changelist numbers are still links in the table of fixes in TeamTrack, but they don't link to anywhere sensible. job000170: Replicator may be unable to send e-mail if the default replicator_address is unchanged The file config_bugzilla.py says replicator_address = 'p4dti-%s' % rid. Ravenbrook's SMTP server allows the replicator to send e-mail from this address, but not all SMTP servers allow this. job000193: Bugzilla integration fails if DateTime module is installed The Bugzilla integration breaks if the Python DateTime module is installed on the P4DTI server system. The Python DateTime module is a third-party module for creating and manipulating dates and times. If it is present, MySQLdb uses DateTime types for time fields (date, time, datetime, and timestamp). However, this MySQLdb/DateTime combination breaks when a MySQL field contains a zero value (e.g. "0000-00-00 00:00:00"). This happens in Bugzilla (e.g. the 'lastdiffed' field of a new bug). We have not tested the P4DTI with the DateTime module at all; it is possible that using it causes other problems even if zero datetimes are not present. I have reported this problem to the author of MySQLdb, Andy Dustman . NICE job000161: Replicator appears to hang When the replicator starts up for the first time, it prints some log messages and then stops. This is disconcerting to the user, as you can't tell if it's hanging or waiting. job000168: Easy to set dbms_port to a string. It's easy to set the configuration parameter dbms_port to a string (e.g. '3306', 'localhost:3306', etc) rather than an int. Especially easy given that p4_port has to be a string. In release 0.5.1 it was especially unclear that this had to be an integer (config_bugzilla.py has it set to ????). config.py now has a sensible value here (i.e. 3306), but this requirement should be documented in the comment and in the AG, and tested in configure_bugzilla.py (see job000165). A. REFERENCES None. B. DOCUMENT HISTORY 2001-02-22 GDR Created using Perforce release notes as a model. 2001-02-23 GDR Updated for release 1.0.1.