PERFORCE DEFECT TRACKING INTEGRATION README FOR RELEASE 0.4.2 Richard Brooksby, Ravenbrook Limited, 2000-10-30 CONTENTS 1. Introduction 2. Supported configurations 3. Installation 3.1. Installing the TeamTrack integration 3.2. Installing the Bugzilla integration 4. Release notes 4.1. What's new 5. Known defects 6. Getting support 7. Project contacts A. References B. Document history 1. INTRODUCTION This is a beta release of the Perforce Defect Tracking Integration (P4DTI). The primary purpose of this release is to test the software in realistic conditions at real customer sites, to find defects and fix them before the general release in 2001-02. The Perforce Defect Tracking Integration version 0.4 is beta test software, only intended for use during the beta program of the project. The software _will_ be defective. We recommend that you do not rely on the integration in your organization, but we would like you to try it out and let us know about any problems. We are very interested in your feedback on the beta. Please write to . For up-to-date information about releases of the P4DTI, see the releases index . Reports of defects found in releases will be posted there. For general information about the project, see the project front page . The readership of this document is anyone who wants to download and use the Perforce Defect Tracking Integration. This document is not confidential. 2. SUPPORTED CONFIGURATIONS This release supports: - Perforce 2000.2 on any platform; - TeamTrack internal build 4402 (available from Ravenbrook) running on Windows NT 4. Instructions for obtaining the internal TeamTrack build are included in the Administrator's Guide that is installed when you follow the instructions in the "Installation" section below. 3. INSTALLATION 3.1. Installing the TeamTrack integration On the TeamTrack server machine, run the "p4dti-teamtrack-0.4.2.exe" installer program that came with this document. This will unpack the integration. We recommend installing the integration in "C:\Program Files\P4DTI\" but you can ask the installer to put it somewhere else. Then consult the Administrator's Guide (installed by default in "C:\Program Files\P4DTI\ag\index.html") for further instructions. 3.2. Installing the Bugzilla integration [Bugzilla is not supported by this release. It will be supported in version 0.5. RB 2000-12-05] 4. RELEASE NOTES 4.1. What's New This is the first public release of P4DTI. 5. KNOWN DEFECTS IMPORTANT: Read the defect reports in the releases index . You will find information about defects found _after_ release. There could be vital information there which we couldn't know when we wrote this document. If you don't read the document you might meet the same problems, waste a lot of time, and possibly DESTROY ALL YOUR DATA. This is a list of major defects which were known at the time of release. job000006: TeamShare API error reporting is inadequate. When TeamTrack refuses to update or transition an issue because a user doesn't have the privilege to do so, it raises an error but provides only an empty error message. The replicator can't provide any information to the user to explain why the operation failed. job000016: Double replication causes many conflicts There is no reliable way for the replicator to distinguish its changes to jobs from changes made by other Perforce users. This means that everything replicated from the DT to Perforce is then replicated back again, and conflicts are very likely when making edits in the DT in quick succession. We observed this at Quokka Sports. RB 2000-11-28 job000022: No migration from Perforce jobs The Administrator's Guide talks about migrating from using Perforce jobs to using the integration with a DT. We don't provide any method of doing this. The software assumes complete control of the jobs subsystem of Perforce, and will damage any existing jobs data. job000024: "\012" in fixes table in TeamTrack Change descriptions in the "Perforce fixes" section of a TeamTrack case description display the octal escape "\012" as those four characters, rather than as a newline (ASCII 10). job000033: Incompatible with other TeamShare API programs The P4DTI is incompatible with other TeamShare API programs because changes made by other TeamShare API programs are not replicated. The replicator has no reliable way of distinguishing its changes to TeamTrack from other users' changes in TeamTrack; this is because all API users show up as user 0 in the CHANGES table. So changes made in TeamTrack by other API users (e.g. other integrations or scripts) will not be replicated accurately. job000037: Consistency checker script is inadequate The consistency checker described in section 7.1 of the Administrator's Guide doesn't check configuration parameters, that the P4DTI_* fields are present in the defect tracker and Perforce, or changelists. job000039: P4DTI defects aren't visible to the world Need to make our own defect tracking system appear in to the world, in or alongside the project web site, so that people can see them from outside, and so that we "eat our own dogfood" and use the P4DTI ourselves, and be seen to do so, and provide a demonstration of the P4DTI to the world, etc. etc. job000042: Rapid changes in the DT cause conflicts The replicator thinks there's a conflict if a DT issue is rapidly changed more than once. It replicates the issue to the job, then wakes up and replicates it back (see job job000033), but by now it's changed again in the DT, so the replicator believes there's a conflict. In version 0.4 there will be a "defect tracker wins" policy by default, so this will not stop replication, but it will send annoying e-mails reporting conflicts where there are none, and thus increase the burden of the system on its users. job000047: Historical bugs are replicated but uninteresting At startup, the replicator replicates all issues from the DT starting at the beginning of time. This takes a long time, uses up space, clutters up databases, and is usually unnecessary. RB 2000-11-21 This now only affects the Bugzilla integration. RB 2000-12-04 job000048: Consistency checking of the configuration is inadequate Consistency checking of the configuration needs to be much stronger, to catch problems so that they don't affect the replicator when it's in use. job000054: There are few reports There are no reports to help with analysis of the information gathered by the integration. This is a big reason for having the integration in the first place. (See requirement 5, requirement 60.) job000060: Assertions and run-time errors conflated Assertions should be things which the programmer believes will always be true. Other conditions are run-time errors and should have recovery code, and try not to be fatal errors in the replicator, as it has to be a reliable daemon or service. Assertions probably should be fatal, and provide plenty of backtrace information for debugging. job000061: Fixes are invisible in TeamTrack by default The fixes table only shows up in a case in TeamTrack if the user has checked the "version control" option in their profile. By default, it's unchecked. job000064: There's no product information page or sheet There's no product information sheet to tell people (esp. managers) at a high level what the integration does. In particular, we don't tell people that the integration enforces DT workflow and security in Perforce. job000065: Not enough logging control There's only one level of logging at the moment. We have to edit the code to get more debugging output in the log. This won't be possible or easy when beta testing or supporting the integration. job000068: Single-step every line of code you write You should always step through every line of code you write. It finds many defects early. You need to learn how to use the Python debugger to do this. It's also necessary for debugging in the field. job000070: No coding practice documentation There's no documentation for coding practices, especially on version branches, which could make merging very hard, especially when other people are extending the system. job000075: No automatic check of configuration How do you tell if the configuration is correct? There should be automatic checking of the configuration. See e-mail "Checking the configuration automatically" for some ideas. job000076: Deleting records gives the replicator a headache The replicator doesn't cope well when a defect tracker record is deleted. When you delete an issue, then the next time (and every subsequent time) you edit the corresponding job it will fail to fetch the issue from the DT and send e-mail to the administrator. job000077: UG contains no version or release related documentation There's no documentation corresponding to use cases 6.12 and 6.15 dealing with finding issues related to versions or releases and vice verse. job000078: Replication failure can cause virtually irrecoverable database records If the initial replication goes wrong (e.g. fails to create a Perforce job because the server is down) then the DT issue is marked with P4DTI action "wait" but there's no corresponding job to set to "keep" in order to overwrite it. You can't fix the problem without going into the database directly using SQL or our own Python interface. job000082: AG has no training and documentation section The AG has no guidance on training and documentation for users. job000085: The SAG does not tell me how to set up a Bugzilla integration The Bugzilla integration is ready for some testing. I'm trying to set up a test server on swan, by following the instructions in the SAG. They are rather specific to tTrack and/or Windows (e.g. "4.3. Updating the Windows Registry"), so I can't use them. job000086: Users can "fix" issues that they don't have permission to change A user can create a fix for any job, and the corresponding defect tracker issue will change in status, even if the user doesn't own or have permission to change it. This hasn't been confirmed by testing as of 2000-11-28. job000092: Long descriptions aren't replicated by Bugzilla integration In Bugzilla, bugs have short descriptions (called "Summary" in the web interface) and long descriptions. Each bug may have many long descriptions; they carry a timestamp and are displayed in date order. They also carry user names. Currently these long descriptions are not replicated. Perhaps they should be; the short description can be too short and/or misleading. Note that it is not possible to edit or delete long descriptions from the Bugzilla web interface; only to submit new descriptions to be appended. job000093: Can't replicate other Bugzilla fields Bugzilla bugs have loads of fields. We only replicate a small number of them. We don't support the "replicated_fields" parameter in the Bugzilla configuration. job000094: Bugzilla integration design not documented. There is a lot of design in the Bugzilla integration; none of it is in the design documentation. job000095: Bugzilla integration is verbose to stdout but doesn't log The Bugzilla integration reports all SQL statements and results to stdout, but doesn't use the logging mechanism at all. job000097: Replicated fixes and filespecs not visible from Bugzilla We haven't changed the Bugzilla code at all, so it can't see or interact with replicated information. In particular, the replicated fixes and filespecs don't appear in the Bugzilla web interface. job000103: Can't easily add to replicated_fields list The configurator doesn't cope if you change the replicated_fields. The P4DTI administrator can't add a new field to the replicated_fields list (except at the end of that list) because the new field will get a Perforce field number that was previously used by another field. So the data that was previously in that other field gets captured by the new field. If you have used the automatic configuration generator and you edit the replicated_fields list, fields probably get the wrong values in Perforce, because the generated jobspec's field numbers probably don't match the old values. job000109: Can't get at Perforce information from TeamTrack TeamTrack lists associated changes for each issue, but you can't easily get at any more details without going to the Perforce interface. It's inconvenient, and not every TeamTrack user will even be licensed to go to the Perforce interface. job000112: Can't easily replicate by project It's hard to set the TeamTrack integration to replicate only those issues in a particular project or group of projects. job000114: There's no description of the limitations of the integration People thinking of installing the integration need to know what its limitations are. There needs to be a "will this work for your organization?" document. job000116: Bugzilla integration doesn't do enough checking The Bugzilla integration does hardly any checking, either of the configuration or of data encountered during replication. This is in contrast to the TeamTrack integration, which now does quite a lot of checking. job000120: Bugzilla integration doesn't enforce workflow restrictions. Bugzilla has some workflow restrictions (e.g. only some transitions are legal, only some users may make some transitions). We don't enforce these. job000122: Server failures aren't handled gracefully If the Perforce server or the defect tracker is down, the replicator produces unhelpful error messages and e-mails to the administrator. job000123: Can't get at Perforce information from Bugzilla interface Bugzilla doesn't provide any interface to Perforce data associated with a bug. You can't easily get at any more details without going to the Perforce interface. It's inconvenient, and not every Bugzilla user will be licensed to go to the Perforce interface. job000124: Changes by disabled Bugzilla users may be replicated Bugzilla has a mechanism for disabling users. We don't check this as part of checking user permissions. We should. job000128: Can't replicate fix to deleted changelist The replicator can't replicate a fix to a deleted changelist, and this gets it into a right tangle: it can't continue replicating. Luckily this is a rare occurrence: normally the deletion of a changelist is replicated to the defect tracker, so the situation doesn't arise. It's only if you delete a changelist and simultaneously cause the replicator to replicate a fix for that changelist back from the defect tracker that this can happen. job000129: Pending changelists aren't clearly indicated as such in TeamTrack You can't tell the difference in TeamTrack's Version Control History section between a pending changelist and a submitted one. This means that you might mistakenly think that work has been done when in fact it is only pending and may later be reverted. job000134: The replicator log grows without limit The replicator log just keeps on growing. job000135: Replicator makes no more progress if replication to Perforce fails If replication from the defect tracker to Perforce fails for whatever reason, the replicator gets into a loop where it keeps e-mailing the administrator. job000137: You can fix a job that you haven't been assigned A user can add a fix for job that they don't have permission to transition or update. job000138: "Add Job Fix" always sends jobs to "closed" The "Add Job Fix" command in the Perforce Windows GUI always sends a job to status "closed". Users will want to link changes using other keywords. job000142: (None) appears in TeamTrack instead of user who made the change The user who made the change to an issue is given as (None) in the state transition list and the change history, rather than the actual user who made the change. 6. GETTING SUPPORT For support during the beta programme, please send e-mail to . We can set up telephone meetings, but please send e-mail first. Please do not contact us for support for this release after 2001-02 (February). This beta release isn't supported by Perforce Software or TeamShare, Inc. We plan to roll out full support during the beta programme so that it is ready in time for the general release. Support arrangements will be announced on the project web site and in the documentation of a later release. We _are_ very interested in your experiences using this software. Please write to the addresses listed in the "Project Contacts" below. 7. PROJECT CONTACTS If you have feedback on this beta software please write to . If you have questions about the project please write to . If you have a comment on the project, you can write to . You can also join the p4dti-discussion mailing list. The goals of the list are: 1. to provide feedback to the project on requirements, design, implementation, etc.; 2. to allow people to exchange information and experience with using and adapting the project; 3. to keep people informed about project progress. To join, send a message with the word "subscribe" in the _body_ to or send the word "help" for general information. Please note that the mailing list will be archived and the archive may be published. A. REFERENCES [RB 2000-11-04] "Perforce Defect Tracking Integration Alpha Programme Report"; Richard Brooksby; Ravenbrook Limited; 2000-11-04; . B. DOCUMENT HISTORY 2000-10-19 RB Created using FreeBSD release documents as a model. 2000-11-08 RB Updated to explain purpose of release 0.3.3. Added references to reports. Improved instructions for obtaining special client and builds. Added a summary of changes. Added readership and confidentiality. 2000-11-20 RB Made it clearer where the Administrator's Guide and the installer file come from. 2000-11-30 RB Partially updated for version 0.4. 2000-12-01 RB Updated references to the "SAG" to "AG" since it's now called the Administrator's Guide. 2000-12-05 RB Added list of known issues generated from Ravenbrook jobs. Updated support information. 2000-12-05 GDR Section 4 says "first public release" only. 2000-12-07 GDR Updated for release 0.4.1. 2000-12-08 RB Updated for release 0.4.2. --- Copyright 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute copies and derivative works of this document provided that (1) you do not charge a fee for this document or for its distribution, and (2) you retain as they appear all copyright and licence notices and document history entries, and (3) you append descriptions of your modifications to the document history. $Id: //info.ravenbrook.com/project/p4dti/version/0.4/readme.txt#5 $