MPS issue job003659

TitleToo hard to maintain the test suite
Statusclosed
Priorityoptional
Assigned userGareth Rees
OrganizationRavenbrook
DescriptionMultiple interacting problems:
* It's too hard to add new test cases and maintain the existing test cases.
* The easiest way to add a new test is to copy and paste an existing test case, but as a result there's a lot of duplicated code in the test suite.
* Too much book-keeping to add a new test case to the automated test suite (have to edit comm.gmk, commpre.nmk and fiddle with the Xcode project).
* The test script is embedded in the Xcode project as just one line, so merges are difficult.
AnalysisSolve the book-keeping problems by:
* Putting list of tests in one place
* Moving Xcode test script into tool subdirectory
How foundinspection
EvidenceNone as yet.
Created byGareth Rees
Created on2014-01-09 14:51:37
Last modified byGareth Rees
Last modified on2014-03-27 17:21:39
History2014-01-09 GDR Created.

Fixes

Change Effect Date User Description
185050 closed 2014-03-27 15:19:42 Gareth Rees Test case database now notes which test cases use threads.
New test suite "testansi" consists of test cases that run on the generic ("ANSI") platform.
New target "ansi" builds the MPS with the CONFIG_PF_ANSI CONFIG_THREAD_SINGLE and CONFIG_PROTECTION_NONE settings.
Build and test the "ansi" target as part of "make test" for the benefit of the buildbots (just Linux and FreeBSD for the moment).
184971 open 2014-03-24 12:12:24 Gareth Rees Don't list the test cases to be run in commpre.nmk: list them instead in testrun.bat for consistency with other platforms.
Format lists of files and test cases with one per line so that merges are easier.
Improve testrun.bat: better success/failure output; fewer gotos.
184965 open 2014-03-24 11:41:01 Gareth Rees Ensure that all test cases call testlib_init so that the testlib assertion handler is installed: this ensures that on Windows you can set MPS_TESTLIB_NOABORT to avoid the dialog box from abort().
184960 open 2014-03-22 18:25:33 Gareth Rees Better to use /bin/sh and not have such fancy syntax.
184959 open 2014-03-22 18:15:06 Gareth Rees testrun.sh needs to be run by bash in order to use a list.
184958 open 2014-03-22 18:04:56 Gareth Rees Don't list the test cases to be run in the Xcode project: list them in testrun.sh instead so that this is shared with the FreeBSD and Linux platforms.
Format the list with one test case per line so that merges are easier.