P4DTI issue job000950

TitleBugzilla integration too slow
Assigned userNick Barnes
DescriptionIntegrating with Bugzilla 2.17.x or 2.18 involves more database access than earlier integrations (e.g. groups tables, product and components tables). When running some stress tests the integration appeared very slow.
AnalysisWe don't cache any data at the moment. We should cache a lot of results (e.g. "what is the name of product 17?", "in what groups is user 37?", "tell me all the user IDs and email addresses"). We can flush the cache at the end of each poll; it can't get stale during a poll unless we invalidate it ourselves, because we have the tables locked.
Add a slot to the bugzilla class called "cache". This should be a dictionary from an item describing a data item (e.g. the pair ("components", product_name)) to the processed result of a database query (in this example, a table mapping component name to component records, for the given product). Before making a query, check the cache. At the end of each poll, reset the cache to an empty dictionary.
How foundautomated_test
EvidenceI just know.
Observed in2.1.2
Introduced in2.1.2
Created byNick Barnes
Created on2004-06-24 12:23:09
Last modified byNick Barnes
Last modified on2004-07-16 12:00:53
History2004-06-24 NB Created.


Change Effect Date User Description
97171 closed 2004-05-28 16:43:43 Nick Barnes Bugzilla 2.17.7 support:
  bugmail.pl replacement for processmail.
  Bugzilla 2.17.5 database schema changes (especially new groups system, new product and component IDs, group_control_map controls);
  Cache results of some database queries to speed operation with new or old schemas;
  Changed bug change permission rules;
  Clean up better when deleting a bug;
  Correct values in bugs_activity entries when changing user columns.