Requirements and use cases for Perforce/defect tracking integration Gareth Rees, Ravenbrook Limited, 2000-05-03 1. Introduction This document records the main user requirements for integration between Perforce's software configuration management system and defect tracking systems. I intend that -- users of Perforce should be able to check that their requirements are included here; and -- architecture proposals and designs for Perforce/defect tracking integration should be checked against the requirements described here. [if these are the things that users want to do they are also the things that should be documented in two places: the Perforce user guide (jobs section) and the user guide for integrating.] The intended readership is therefore -- users of Perforce; and -- anyone developing architecture proposals and designs for Perforce/defect tracking integration. This document is not confidential. I intend it to be released to Perforce users with permission and with sources removed. 2. Sources The requirements and use cases in this document are derived from the following sources: -- Responses to a survey of Perforce users' defect tracking requirements. -- Ravenbrook's change management handbook [RB 1998-02-17]. -- The "Jobs" sections of the Perforce user manual [Perforce 1999-12-28]. -- Kelly Shaw's description of TeamShare's requirements for integration of TeamTrack with software configuration management systems [Shaw 2000-04-18]. -- Meeting with James Zhou of TechExcel, developers of DevTrack [RB 2000-05-03]. 3. Actors Analyst Someone who uses documents without changing them (in particular, to analyze the cause and scope of an issue). Developer Someone who makes changes to documents. Manager Someone who schedules and assigns work. Stakeholder Someone with a stake in the outcome of a project but not involved in its development. User Any user of the defect tracking system or Perforce. Any of the above roles. Perforce Software Configuration Management System. DTS Defect tracking system. 4. Terminology Actor An entity external to the defect tracking integration software whose actions have an effect on it. A user of the integration or a software system that interacts with it. Document A file stored in Perforce. Issue A discrepancy between a product and its requirements. Tracked by the DTS. Task A piece of work (the smallest unit of work tracked by the DTS). Note: not all DTSs will have separate entities for issues and tasks. In fact, most DTSs track a single entity, which may be used to represent issues or tasks, or some combination. Job The representation of an issue or task in Perforce. This is very likely to correspond directly to the existing Perforce jobs, but other possible implementations should not be ruled out. 5. General requirements 5.1 Use either tool Requirement Any routine development activity that affects both defect tracking and software configuration management should be able to be carried out from either the DTS or Perforce. Justification Some users are used to working in one system and do not want to learn to use another. Even users familiar with both systems want to be able to do work in whichever system is most convenient at a given time. They do not want to have to switch back and forth. Note For this reason, most of the use cases below say something "Developer uses check out command" without specifying which system the command is in. Requirements addressed Ease of use (3.2) Sources [Currier 2000-03-23] > The user sees no 'knowledge-base' separation induced details. > That is, the development/QA/release 'process' contains no > perturbations or details stemming from the fact that there are > two databases: a SCM/revision database and a issue/bug > tracking database. Additionally, queries that involve both > databases do not have to be split into two queries by the > user. Russell Jackson asks for both in the same message, apparently without realizing it [Jackson 2000-13-13]: > A tight integration means that you can use the bug tracking > system as an interface to Perforce [...] With a system that > notified the developers of all of their new jobs, and was > setup so that when a job was closed, the corresponding defect > would be closed, the developers wouldn't have to go into the > bug tracking system at all except to create new bugs. [Parrish 2000-04-10] > At a high level, my expectation is "simply" that engineers can > spend the majority of their time in Perforce, and not have to > switch applications to complete a change request, create new > change requests, choose which change request to work on next, > and so on. [...] > > The change request status would be visible via both Perforce > and the change request system. 5.2 Triggers for every event Requirement It should be possible to do something whenever an event takes place in the Perforce world. Justification We need to make an integration framwork flexible enough to integrate with whatever tools the customers want to integrate with (not just defect tracking systems). We cannot anticipate every customer requirement. Pre- and post-event triggers are a flexible and general mechanism. Requirements addressed Openness (3.4) Sources [Goffin 2000-03-13] > Command line ineroperation - e.g. pre/post submit triggers, > etc. could also help. [Jackson 2000-13-13] > Triggers on the job commands. These could be used in many ways > to connect the two systems together. [Pye 2000-03-17] > All Perforce commands should have pre and post triggers > (hooks) to allow general integration. [Anon 2000-03-17] > We use a lot of scripts that call P4, and parse the results. > Many of these run every few minutes, as there is no way to > trigger them. If 'post submit' triggers were available as > well, and more info was available in a format similar to 'p4 > fstat' then we could probably script our way out of most > things. 5.3 Don't close a job when you submit an associated changelist Requirement When a developer submits a changelist, Perforce should close associated jobs only if the developers says so. (In other words, the default action should be to leave the job in its current state.) Justification Tasks often take several submissions to complete. It is less harmful for the task to be recorded as incomplete and for a devloper to have to go back and record the task as completed that it is for the task to be recorded as completed when it is not. In some development methodologies it is important to encourage developers to submit changes as often as possible: this improves communication between developers and recovery from development mistakes or system failure. If jobs are closed when a changelist is submitted this will make developers reluctant to submit frequently. Sources [Tavola 2000-03-16] > I hate the way Perforce automatically marks jobs as closed > whenever you submit a changelist the "fixes" that job. Many > changes only partially address a job; they do not actually fix > the job. [Anon 2000-03-17] > closing a job when a change is submitted is jumping the gun - > it might fail in test and come back 6. Use cases 6.1 Create task Scenario Manager is scheduling and assigning work to address one or more issues. Actors Manager, Developer, DTS, Perforce Pre-conditions (Issues present in DTS.) Manager has permission to create task. Events (Manager selects issues which task will address.) Manager creates task. Manager assigns task to developer. Start transaction. DTS creates task record. Perforce creates job record. End transaction. (Developer is notified of the assignment.) Post-conditions Developer is assigned a task to work on. The task record in the DTS and the job record in Perforce are consistent with the state of the task and with each other. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Sources [Perforce 1998-12-28 chapter 10] 6.2 Associate revisions of documents with task Scenario Analyst is working on a task and discovers that the task affects certain revisions of certain documents. Actors Analyst, DTS, Perforce Pre-conditions Task record in DTS. Job record in Perforce. Events Analyst selects revisions of documents. Analyst associates revisions of documents with task. Start transaction. DTS records association of revisions of documents with task. Perforce records association of revisions of documents with job. End transaction. Post-conditions revisions of documents are associated with the task. The task record in the DTS and the job record in Perforce are consistent with the state of the task and with each other. Requirements addressed Consistency [RB 2000-04-11, 3.1] Documentations [RB 2000-04-11, 3.3] Sources [RB 2000-05-02] 6.3 Check out copies of revisions of documents associated with task Scenario Developer selects a task that he has been assigned and checks out copies of revisions of documents associated with the task. Actors Developer, DTS, Perforce. Pre-conditions Task has been associated with a set of revisions of documents. Events Developer selects a task. Developer selects revisions of documents from those associated with the task. Developer chooses a client workspace. Developer uses check out command. Start transaction. Perforce checks out revisions of documents associated with the task record into the client workspace. Perforce associates the workspace with the job. DTS updates its task record to reflect the state of the task. Perforce updates its job record to reflect the state of the task. End transaction. Post-conditions Developer has access to checked out copies of the revisions of the documents associated with the chosen task. The task record in the DTS and the job record in Perforce are consistent with the state of the task and with each other. The workspace is associated with the job. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Sources [Shaw 2000-04-18] > Check out code to work on a bug. Somehow, the document, > including version, branch, etc., must be associated with the > bug. 6.4 Make a branch to work on a task Scenario Developer needs to do experimental or prototype development in order to find a good solution to an issue. He decides that this work is extensive enough to be worth doing on a branch. Actors Developer, DTS, Perforce. Pre-conditions Task has been associated with a set of revisions of documents. Events Developer selects a task. Developer selects revisions of documents from those associated with the task. (Developer chooses a name for the branch.) (Or a name is automatically chosen for the branch based on the task.) Start transaction. Perforce creates a branch with the given name and documents. DTS associates new revisions of documents with the task. Perforce associates new revisions of documents with the job. End transaction. Post-conditions Branch has been created. The task record in the DTS and the job record in Perforce are consistent with the state of the task and with each other. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Sources [RB 1998-02-17] 6.5 Associate changelists with tasks Scenario Developer realises that the changes he made yesterday will also complete the tasks he is assigned to work on today. He needs to record this in the defect tracking system to that SQA can check that he is right. Actors Developer, DTS, Perforce Pre-conditions Changelists have been submitted. Developer is assigned to tasks. Events Developer selects changelists. Developer selects tasks. Developer issues associate command. Start transactions. DTS associates changelists with tasks. Perforce associates changelists with corresponding jobs. End transaction. Post-conditions The changelists are associated with the appropriate job records in Perforce and the appropriate task records in the DTS. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Documentation [RB 2000-04-11, 3.3] Sources [Bachmann 2000-03-14] > We want traceability from development activities back to why > they were done in the first place. When code is checked back > into Perforce, the developer should be able to easily trace > back to requirements, functionality and QA/defects what is the > scope of the new code. With disconnected systems, to acheive > the same goal requires a very strong people oriented process, > and still much is missed. [Mack 2000-03-18] > Associate tasks with changelists (a field in a job?) [...] > relating changelist numbers and defect-IDs is important to us, > for tracking product changes [Parrish 2000-04-10] > There would be a way of associating perforce changes with the > change request, and that information would be visible directly > through the change request system. [Perforce 1998-12-28 chapter 10] 6.6 Submit a changelist Scenario Developer has done some work on some tasks he was assigned to work on and decides to submit the changes so far. Actors Developer, DTS, Perforce. Pre-conditions Developer is working on one or more tasks and has checked-out revisions of documents associated with the tasks in a workspace. Events Developer submits changes. Start transaction. Perforce submits changelist. Perforce associates the changelist with each job associated with the workspace. DTS associates the changelist with each corresponding task. End transaction. Post-conditions The changes to the documents are recorded in Perforce. The changelists are associated with the appropriate job records in Perforce and the appropriate task records in the DTS. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Documentation [RB 2000-04-11, 3.3] Sources [Shaw 2000-04-18] > Checking in code to fix a bug. The changes made to code to > support a bug fix must be associated with the bug. There must > also be some way to merge changes when multiple users have the > document checked-out. [Anon 2000-03-17] > In an integrated SCM and defect tracking system, both systems > would automatically link changes to defects and vice versa. 6.7 Complete a task Scenario Developer has completed the work for a task and submitted all the changes for that task. He now wants to record the task as being completed. Actor Developer, DTS, Perforce. Pre-conditions Developer is working on one or more tasks. No outstanding changes for any document in a workspace associated with the completed tasks. Events Developer selects tasks. Developer indicates that the tasks are completed. Start transaction. DTS updates task record for new state of each task. Perforce records job record for new state of each task. End transaction. (Workflow automation takes place; next person who has to deal with each task is notified.) Post-conditions The task records in the DTS and the job records in Perforce are consistent with the state of the tasks and with each other. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Sources [Mack 2000-03-18] > Close my open tasks [Perforce 1998-12-28 chapter 10] 6.8 Simultaneously submit changes and complete task Scenario Developer has completed the work for some tasks and now wishes to submit the final changes and record the completion of the tasks in one step. Actors Developer, DTS, Perforce. Pre-conditions Developer is working on one or more tasks and has checked-out revisions of documents associated with the tasks in a workspace. Events Developer selects tasks. Developer indicates that the tasks are completed. Developer submits changes. Start transaction. Perforce submits changelist. Perforce associates the changelist with each job associated with the workspace. DTS associates the changelist with each corresponding task. DTS updates task record for new state of each task. Perforce records job record for new state of each task. End transaction. (Workflow automation takes place; next person who has to deal with each task is notified.) Post-conditions The changes to the documents are recorded in Perforce. The changelists are associated with the appropriate job records in Perforce and the appropriate task records in the DTS. The task records in the DTS and the job records in Perforce are consistent with the state of the tasks and with each other. Requirements addressed Consistency [RB 2000-04-11, 3.1] Ease of use [RB 2000-04-11, 3.2] Documentation [RB 2000-04-11, 3.3] Sources [Chen 2000-03-13] > Integrated means that when I do a p4 submit, I can also > simultaneously update the issue tracking system with the > appropriate updates. (issue closed, what change number, what > files, etc.) [Geiger 1999-03-13] > For example, when a developer submits changes that affect > defects (whether comprising a completed fix, a part of a fix, > or the removal of a fix), they should be able to record these > effects as an integral part of the information provided at > submit time. [Heinz 2000-03-17] > Integrated to "me" means being able to close a defect at the > perforce interface of change submission. [Humphrey 2000-03-20] > It would be nice if the submission automatically did the > right thing with the affected defects. The right thing > includes: changing the defect state, supplying descriptive > text about the change (the same as the high level > description), and providing classification information. > Classification information includes: stage of development when > defect was fixed, root cause of defect, stage of development > when defect was introduced etc. > > It would be nice if the developer could provide all of the > information only once - at sumbit time. That is, it would be > nice if the developer could submit his changes and then go to > the next thing, *without* having to also remember to go to the > defect tracking system and update data there as well. [Perforce 1998-12-28 chapter 10] [Tavola 2000-03-16] > When you check in a changelist in Perforce, the Perforce > Review Daemon automatically adds information to the jobs in > JobTracker about which jobs were addressed by which source > files. 6.9 Find tasks being worked on in workspace Scenario Developer wants to know why he checked out a set of revisions of documents, and what to do next. He needs to look at the task records for the tasks that he is working on. Actors Developer, DTS/Perforce. Pre-conditions Tasks have been associated with sets of revisions of documents. Developer is working on tasks. Developer has revisions of documents associated with tasks checked out in a workspace. Events Developer selects workspace. Developer requests the tasks the documents were checked out for. Perforce or DTS presents the corresponding task records. Post-conditions Developer knows what tasks he is working on in that workspace. Requirements addressed Ease of use [RB 2000-04-11, 3.2] Sources 6.10 Find workspace in which task is being worked on Scenario Developer wants to continue working on a task he has already done some work on. He wants to know where the work in progress is to be found. Actors Developer, DTS/Perforce. Pre-conditions Developer has revisions of documents associated with a task checked out in a workspace. Events Developer selects task Developer requests the workspace in which the task is being worked on. Perfrorce or DTS presents the corresponding workspace. Post-conditions Developer knows where he was working on the task. Requirements addressed Ease of use [RB 2000-04-11, 3.2] Sources 6.11 Find issues associated with documents Scenario User wants to know the history of work on some documents. He wants to know why the documents are the way they are and believes that the issues associated with the document may provide an explanation. Actors User, DTS. Pre-conditions Events User selects documents. User asks for associated issues. DTS presents associated issues. Post-conditions User knows which issues are associated with the document. Requirements addressed Documentation [RB 2000-04-11, 3.3] Analysis [RB 2000-04-11, 3.5] Sources [Jackson 2000-13-13] > Methods to identify all of the jobs that a source file was > included in. [Perforce 1998-12-28 chapter 10] 6.12 Find issues associated with version or release Scenario Manager is assessing the quality of a release candidate and wants to know which defects are present in the candidate and which have been fixed. Actors Manager, DTS. Pre-conditions Events Manager selects version of documents. Manager asks for associated issues. DTS presents associated issues. Post-conditions Manager knows which issues are associated with the version of the documents. Requirements addressed Documentation [RB 2000-04-11, 3.3] Analysis [RB 2000-04-11, 3.5] Sources [Currier 2000-03-23] > 1) what 'bugs/issues' have been fixed in this release? > 2) what are the differences in defect repairs lists for two > arbitrary releases? > 3) what bugs/issues are fixed in this codeline? Between 2 > codelines? > 4) what changes are present in this codeline? Between 2 > codelines? > > Other queries: > > 5) which releases have this bug fix? > 6) which codelines are missing this bugfix? > 7) have we dropped any repairs since the previous release? [Geiger 1999-12-08] > Many (most?) defect tracking systems I am familiar with lack > any such systemic glue, relying on users to "fill in" > information about what bugs are present, or fixed, in which > releases. To me this seems hopelessly simple minded and > destined to fail, in the face of the actual complexities." [Heinz 2000-03-17] > Customers want a report of what's fixed in a release, what > progress has been made, in order to decide whether to upgrade, > to make judgements about the company's ability to provide, to > evaluate the product and the company, etc. [Perforce 1998-12-28 chapter 10] 6.13 Find issues associated with changelist Scenario Manager is reviewing a changelist and wants to know why the changes were made. Actors Manager, DTS. Pre-conditions Events Manager selects changelist. Manager asks for associated issues. DTS presents associated issues. Post-conditions Manager knows which issues are associated with the changelist. Requirements addressed Documentation [RB 2000-04-11, 3.3] Analysis [RB 2000-04-11, 3.5] Sources [Perforce 1998-12-28 chapter 10] [Anon 2000-03-17] > Both systems would automatically link changes to defects and > vice versa. The changes that are made to achieve a particular > Change Product request (CPR) should be associated with it, and > conversely, a change should have a CPR associated to justify > its existence. > > Test can check where development is up to by seeing if changes > have gone in against CPRs. Development can focus on the right > areas (high priority bugs). Management can figure out the > effort being applied - in terms of changes at least - into > fixes. 6.14 Find revisions of documents associated with an issue Scenario Tester is developing a regression test case for an issue. They want to read the documents associated with the issue. Actors User, DTS, Perforce. Pre-conditions Events User selects issue. User requests associated documents. Perforce gets copies of revisions of documents. Documents are presented to user. Post-conditions User can read revisions of documents associated with issue. Requirements addressed Documentation [RB 2000-04-11, 3.3] Analysis [RB 2000-04-11, 3.5] Sources 6.15 Find version or release associated with an issue Scenario Customer wants to know if a defect they are concerned about is fixed in the new release. Manager must prepare a response. Actors Manager, DTS. Pre-conditions Events Manager selects issue. Manager selects version of release. DTS presents Note This is not an integration use case, but it appeared in the user survey. Sources [Geiger 1999-12-08] [Geiger 2000-03-13] 6.16 Find changes resulting from an issue Scenario Stakeholder wants to see evidence that an issue has been dealt with. Manager needs to provide a list of changes to convince them that all the necessary steps have been taken (and no unnecessary steps!). SQA want to verify that an issue has been dealt with correctly. They need to see all the changes resulting from the issue so that they can check that each change is properly justified. Tester wants to write a white box regression test for an issue. She needs to look at the changes resulting from the issue so she can direct the test at the areas most likley to have been broken by the change. Actors Manager, DTS, Perforce Pre-conditions Events Manager selects issue. Perforce presents changes resulting from the issue. Post-conditions Manager knows which changes resulted from the issue. Requirements addressed Documentation [RB 2000-04-11, 3.3] Analysis [RB 2000-04-11, 3.5] Sources [Perforce 1998-12-28 chapter 10] [Goffin 2000-03-13] > We need to prove to external auditors that defects found in > the development (or test) process have been resolved. Where > this resolution includes changes to product code (or in our > case documentation as we keep user manuals, etc in Perforce > too.) then there must be traceability into the changed > modules. 6.17 Attempt to read document without permission Scenario Organization is working on a highly confidential project. Manager wants to prevent developers from reading source code or documentation unless the developer is assigned to work on it. Sources [RB 2000-05-03] 6.18 Attempt to edit document without permission Scenario Organization is working on a safety-critical project. Manager wants to prevent developers from making changes to source code or documentation unless the developer is assigned to work on it. Sources [RB 2000-05-03] 6.19 View list of assigned tasks Scenario Developer comes into work on a Monday morning and thinks, "I wonder what my manager has in store for me today?" Sources [Mack 2000-03-18] > Get a list of my open tasks (Jobs?) [Parrish 2000-04-10] > The change request status would be visible via both Perforce > and the change request system. Each user would need to be able > to view all their change requests, and also be able to > transfer ownership. Note This use case forms part of many of the above use cases. In order to select an assigned task the user must be able to see their assigned tasks. A. References [Bachmann 2000-03-14] Glenn Bachmann Re: Requirements for Perforce and Defect Tracking 2000-03-14T12:27:03Z [RB 1998-02-17] Richard Brooksby Change management handbook Ravenbrook Limited, 1998-02-17 [RB 2000-04-11] Richard Brooksby User requirements for defect tracking integration Ravenbrook Limited, 2000-04-11 [RB 2000-05-02] Richard Brooksby Meeting with James Zhou, TechExcel Ravenbrook Limited, 2000-05-02 [RB 2000-05-03] Richard Brooksby Draft of Perforce/DevTrack Integration Meeting, 2000-05-02 2000-05-03T21:49:45Z [Chen 2000-03-13] Boris Chen Re: Requirements for Perforce and Defect Tracking 2000-03-13T18:38:53Z [Currier 2000-03-23] sandy Currier Re: [Fwd: Requirements for Perforce and Defect Tracking] 2000-03-23T22:12:03Z [GDR 2000-05-11] "Defect Tracking Project Meeting, 2000-05-11"; Gareth Rees; Ravenbrook Limited; 2000-05-11. [Geiger 1999-12-08] Richard Geiger extended fixes database 1999-12-08T22:39:19Z [Geiger 2000-03-13] Richard Geiger Re: Requirements for Perforce and Defect Tracking 2000-03-13T19:16:12Z [Goffin 2000-03-13] Paul Goffin RE: Requirements for Perforce and Defect Tracking 2000-03-13T16:18:53Z [Heinz 2000-03-17] Ines Heinz Re: Requirements for Perforce and Defect Tracking 2000-03-17T23:03:21Z [Humphrey 2000-03-20] Alan Humphrey RE: Requirements for Perforce and Defect Tracking 2000-03-20T18:05:40Z [Jackson 2000-13-13] Russell Jackson Re: Requirements for Perforce and Defect Tracking 2000-03-13T18:21:46Z [Mack 2000-03-18] Ed Mack RE: Requirements for Perforce and Defect Tracking 2000-03-18T00:24:33Z [Parrish 2000-04-10] Jeff Parrish Re: Requirements for Perforce and Defect Tracking 2000-04-10T16:49:24Z [Perforce 1999-12-28] Perforce 99.2 p4 command line user's guide Perforce, 1999-12-28 [Pye 2000-03-17] Michael Pye Re: [p4] Perforce integration with defect tracking systems 2000-03-17T01:29:25Z [Shaw 2000-04-18] Kelly Shaw Re: Perforce Integration 2000-04-18T15:58:00Z [Tavola 2000-03-16] Geoff Talvola Re: [p4] Perforce integration with defect tracking systems 2000-03-16T16:53:36Z [Anon 2000-03-17] A Perforce customer who wishes to remain anonymous RE: Requirements for Perforce and Defect Tracking 2000-03-17T15:31:53Z B. Document history 2000-05-03 GDR Based on Perforce user survey results. 2000-05-05 GDR Added material from Change Management Handbook. 2000-05-08 GDR Added material from discussion with RB. 2000-05-25 RB Marked as "not confidential" as agreed with Perforce [GDR 2000-05-11, 5]. 2000-05-30 RB Anonymized contributor of [Anon 2000-03-17]. Copyright (C) 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution. $Id: //info.ravenbrook.com/project/p4dti/doc/2000-05-03/reqs-and-use-cases/index.txt#3 $