MPS issue job003971

TitleNo automatic feedback about overheads of "hot" versus "rash" MPS
Assigned userRichard Brooksby
DescriptionThe "hot" variety of the MPS is intended for production, but still includes most assertions, as long as they are not on critical paths. The intention is that the "hot" variety perform almost as well as a "rash" variety, which has all all assertions disabled. We should have automatic feedback to the development team about whether this requirement is being met.
AnalysisContinuous integration should run regular comparisons of hot and rash builds, at the very least using our own gcbench and djbench programs, but ideally with real clients' benchmarks (see also job003834).
If possible, we should collect, tabulate, and plot this data to see the impact of our changes.
How foundunknown
EvidenceAt changelist 189674 on Silverbird (OS X):

    make -f lii6ll.gmk VARIETY="hot" gcbench djbench
    make -f lii6ll.gmk VARIETY="rash" gcbench djbench
    xci6ll/hot/gcbench -x 1564912146 amc -> 14.1s
    xci6ll/rash/gcbench -x 1564912146 amc -> 13.4s
    xci6ll/hot/djbench -x 861801839 mvff -> 23.9s
    xci6ll/rash/djbench -x 861801839 mvff -> 16.2s

This quick test shows we're doing OK for GC, but not so great for the manual MVFF pool.
Created byRichard Brooksby
Created on2016-03-07 10:50:29
Last modified byGareth Rees
Last modified on2016-04-20 13:16:17
History2016-03-07 RB Created.


Change Effect Date User Description
191339 closed 2016-04-20 13:15:18 Gareth Rees Add a test checking that the hot variety is not too much slower than the rash variety. Turn this on for continuous integration testing via "make test".