.. mode: -*- rst -*-

Tests
=====

:Tag: design.mps.tests
:Author: Richard Kistruck
:Date: 2008-12-04
:Status: incomplete design
:Revision: $Id: //info.ravenbrook.com/project/mps/version/1.116/design/tests.txt#1 $
:Copyright: See `Copyright and License`_.
:Index terms: pair: tests; design


Introduction
------------

_`.intro`: This document contains a guide to the Memory Pool System
tests.

_`.readership`: This document is intended for any MPS developer.


Smoke tests
-----------

_`.smoke`: The "smoke tests" provide quick checks that the MPS is
working. They run quickly enough for it to be practical to run them
every time the MPS is built.

_`.randomize`: Each time a test case is run, it randomly chooses some
of its parameters (for example, the sizes of objects, or how many
links to create in a graph of references). This allows a fast test
to cover many cases over time.

_`.randomize.repeatable`: The random numbers are chosen
pseudo-randomly based on a seed initialized from environmental data
(the time and the processor cycle count). The seed is reported at test
startup. Each test can be run with a specified seed. This ensures
that the deterministic tests are repeatable.


Smoke test list
...............

_`.test.finalcv`: Registers objects for finalization, makes them
unreachable, deregisters them, etc. Churns to provoke minor (nursery)
collection.

_`.test.finaltest`: Creates a large binary tree, and registers every
node. Drops the top reference, requests collection, and counts the
finalization messages.

_`.test.zcoll`: Collection scheduling, and collection feedback.

_`.test.zmess`: Message lifecycle and finalization messages.


Performance test
----------------

_`.test.ratio`: The ``testratio`` target checks that the hot variety
is not too much slower than the rash variety. A failure of this test
usually is expected to indicate that there are assertions on the
critical path using ``AVER`` instead of ``AVER_CRITICAL`` (and so on).
This works by running gcbench for the AMC pool class and djbench for
the MVFF pool class, in the hot variety and the rash variety,
computing the ratio of CPU time taken in the two varieties, and
testing that this falls under an acceptable limit.

Note that we don't use the elapsed time (as reported by the benchmark)
because we want to be able to run this test on continuous integration
machines that might be heavily loaded.

This target is currently supported only on Unix platforms using GNU
Makefiles.


Document History
----------------

- 2008-12-04 Richard Kistruck. Create. Describe finalization tests.

- 2010-03-03 Richard Kistruck. Correction: it's fin1658a.c and
  job001658, not 1638.

- 2010-03-03 Richard Kistruck. Add zmess.c, zcoll.c. zmess.c subsumes
  and replaces fin1658a.c.

- 2013-05-23 GDR_ Converted to reStructuredText.

.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/


Copyright and License
---------------------

Copyright © 2013-2016 Ravenbrook Limited <http://www.ravenbrook.com/>.
All rights reserved. This is an open source license. Contact
Ravenbrook for commercial licensing options.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

#. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.

#. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

#. Redistributions in any form must be accompanied by information on how
   to obtain complete source code for this software and any
   accompanying software that uses this software.  The source code must
   either be included in the distribution or be available for no more than
   the cost of distribution plus a nominal fee, and must be freely
   redistributable under reasonable conditions.  For an executable file,
   complete source code means the source code for all modules it contains.
   It does not include source code for modules or files that typically
   accompany the major components of the operating system on which the
   executable file runs.

**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed.  In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**