P4DTI VERSION 2 REQUIREMENTS ANALYSIS Richard Brooksby, Ravenbrook Limited, 2002-08-30 1. INTRODUCTION This document analyses the requirements put forward for P4DTI version 2 by Perforce Software, and presents some solutions with estimates of effort required by Ravenbrook. The intended readership of this document is Perforce and Ravenbrook P4DTI staff. This document is not confidential. 2. REQUIREMENTS This is a summary of Gerry Thompson's "Feature requests for P4DTI 2.0" [Thompson 2002-07-30]. 1. Bugzilla integration enhancements 1.1. Bugzilla 2.16 1.2. Bugzilla 2.18 1.3. Custom fields in Bugzilla 2.18, supporting all Perforce field types 1.3.1. Enhance Bugzilla 2.18 to support requirement 1.3. 2. Easier installation 3. Solaris 3.1. job000209 (No package for Solaris) 3.2. job000320 (P4DTI is not tested on Solaris) 3.3. job000324 (Bugzilla patch doesn't work well on Solaris 8) 4. Easier administration and debugging 4.1. job000524 4.2. job000525 (Perforce job008041) 4.3. Explanation of P4DTI database schema 4.4. Diagnose and modify bad data so that P4DTI can continue 4.5. Support tool for quering and fixing MySQL database entities in Bugzilla 5. Share p4 logger with other daemons 6. Perforce as a defect tracker (Perforce job008768) 6.1. Don't touch jobspec except to add fields 6.2. Share p4 logger with other daemons (same as requirement 5) 7. Provide management information 7.1. Using p4report 7.2. Query: How fast is codeline changing? 7.3. Query: How fast are serious bugs getting fixed? 7.4. Query: Will we meet our targets? 8. Configurable jobspec (Perforce job0008678) 3. ANALYSIS 3.1. Bugzilla Bugzilla 2.16 support (requirement 1.1) is already planned for P4DTI version 1.5, and so is not included in this analysis [Thompson 2002-09-27]. Bugzilla 2.18 support, including custom fields, is hard to estimate. Bugzilla 2.18 includes many changes, and it's quite unclear how some of them will work. However, we can esimate the effort required from our previous experience with Bugzilla version changes. There's aready a patch for custom field support (see ) so it's unlikely that Ravenbrook would need to develop custom fields from scratch. At the very least we could adapt this patch to support all Perforce field types and roll it into the P4DTI Bugzilla patch. Estimate: Analysis: 1w 5d 30h Development: 3w 15d 90h Documentation: 1w 5d 30h Testing: 4w 20d 120h Total: 9w 45d 270h 3.2. Easier installation The actual "installation" step is very simple. What takes time is assembling and installing the prerequisite packages, and then configuring the P4DTI and debugging the configuration. So, there are two parts to the solution: 1. bundling and automatic installation of as many prerequisites as possible; 2. improved configuration and debugging. I believe there is a significant overlap between this second part and requirement 4, "easier administration and debugging" (see section 3.4 for analysis). In addition to the solution proposed in that section, I propose a step-by-step guided configuration procedure using web-based forms, with documentation included at each stage, and cross-referenced to the manual. This will reduce the incidence of people trying to set up the P4DTI without understanding the configuration parameters. As part of the analysis of this requirement, I would like to talk to customers who have installed the P4DTI about their experiences, so that we can focus our efforts on the most difficult and time-consuming stages. We already provide links to all the prerequisites and redistribute them. Bundling them with the P4DTI download may help a little. It's questionable whether we want to install them automatically. In many cases, the necessary prerequisites will already be installed on the system. For example, we wouldn't want to install Bugzilla, as it's probably already in use, and therefore so is MySQL. However, we may be able to bundle and install any necessary Python packages, using the Python package system. Estimate: Analysis: 1w 5d 30h Development: 3w 15d 90h Documentation: 2w 10d 60h Testing: 4w 20d 120h Total: 10w 50d 300h 3.3. Solaris We would need to arrange access to a Solaris machine for development and testing. In the past we have arranged access to a machine at Perforce for this purpose. If this is not possible, the cost of a suitable Solaris machine must be added to the estimate below. One of the main difficulties on Solaris is the lack of a suitable "patch" program for patching Bugzilla. We either need to write a Python program which will do the patching, or add GNU patch as a prerequisite. A Python patcher might be a good idea across all platforms, and may be necessary to intelligently patch customized Bugzilla installations in future. The other main part of this work is investigating the Solaris packaging system and developing a package containing the P4DTI. In addition, the test suite will need to be tested on Solaris, and the test procedure will have to include Solaris in future. This will increase the cost of each new release of the P4DTI. Estimate Analysis 3d 18h Development 1w 5d 30h Documentation 2d 12h Testing 2d 12h Total 12d 72hh 3.4. Easier administration and debugging 3.4.1. A web interface P4DTI configuration, administration, and debugging is currently done through a command line and text file interface. P4DTI administrators are required to edit some Python code. They must do this while working their way through a fairly long manual. There is no immeditate feedback or assistance with configuration. Monitoring P4DTI activity means reading a log file or dealing with e-mail messages from the replicator. I propose changing the nature of the P4DTI from a command line tool to a web-based tool. After installation, all configuration, control, and monitoring will be done through a CGI front-end, in a similar manner to Bugzilla. In addition, the web interface will be linked to on-line documentation. Configuration, with immediate validation Activity monitoring and statistics Error conditions Consistency checking Display of internal P4DTI data On-line documentation Advanced configuration A web-based interface can be linked to and from both Bugzilla and P4Web, to further integrate Perforce and the defect tracker. An interactive interface may make it possible to display and edit the relationship between defect tracker fields and the Perforce jobspec, giving greater flexibility. The front-end would also include a tool to force replication of a fix record or changelist that hasn't been replicated, or re-replicate jobs, etc. This would be like the current "refresh" script, but more specific and interactive. Estimate Analysis 1w 5d 30h Design 2w 10d 60h Development 3w 15d 90h Documentation 2w 10d 60h Testing 5w 25d 150h Total 13w 65d 390h 3.5. Share the p4 logger It's easy to change the P4DTI replicator not to clear the logger, but this will lead to the logger record growing indefinitely. This may require a change to the Perforce server, because _someone_ has to clear the logger at some point, and this means some kind of liaison between the various daemons using it. Some research will need to be done to work out a solution. Estimate Analysis 1w Development 1w Testing 1w Total 3w 3.6. Perforce as a defect tracker This means developing a "dt_p4.py" module to treat a Perforce server like a defect tracker. Assuming this is straightforward, here's the estimate. Estimate Analysis 1w Development 2w Testing 2w Total 5w 3.7. Provide management information It's not clear that P4Report is the best means to achieve this, as we need to combine information from both the defect tracker database and the Perforce database. Do Perforce want us to use P4Report for some other reason (e.g. marketing, or to help develop P4Report)? We need to investigate what P4Report can do. We can certainly adapt and generalize our own issue reporting script to work with the P4DTI through the proposed web front-end. The main challenge is generalizing it to cope with different branching strategies, or even ad-hoc branching. Estimate Analysis 1w Development 2w Testing 2w Total 5w 3.8. Configurable jobspec We could read the jobspec, the DT config, and the P4DTI config, and provide a web-based front-end for connecting the two, effectively doing "advanced configuration". Gerry stated that no estimate was required for this at present. A. REFERENCES [Thompson 2002-07-30] "Feature requests for P4DTI 2.0" (e-mail message); Gerry Thompson; Perforce Software; 2002-07-30. [Thompson 2002-09-27] "defect tracker version support in P4DTI 1.5.x" (e-mail message); Gerry Thompson; Perforce Software; 2002-09-27. B. DOCUMENT HISTORY 2002-09-26 RB Provided estimates for some parts. Added solutions section. 2002-09-28 RB Wrote up solutions section. 2002-09-29 RB Analysed more requirements. 2002-09-30 RB Completed all sections, ready for review. 2003-01-15 NB Removed dollar values per request from Perforce. $Id: //info.ravenbrook.com/project/p4dti/doc/2002-08-30/p4dti-2-requirements-analysis/index.txt#3 $