| Title | Arena extensions are not adaptive |
| Status | open |
| Priority | nice |
| Assigned user | Gareth Rees |
| Organization | Ravenbrook |
| Description | This table (taken from [1]) shows the effect of varying the initial allocation of address space to the arena when running the test case "scheme-advanced test-leaf.scm": Space Extns Colls Time 2 32 371 52.0 4 21 370 47.0 8 0 * * 14 0 * * 16 0 2436 160.5 18 0 1135 89.1 20 0 673 60.6 22 0 484 48.7 24 0 400 43.1 32 0 368 41.2 You'll note that when the original arena size is small the number of extensions is very large, and yet the working set size is probably somewhere around 16 MB. Why so many extensions? |
| Analysis | The code for computing the chunk size is in vmArenaChunkSize() in arenavm.c. In normal use this always allocates the same amount, namely vmarena->extendBy (which is just the original arena size) There are two problems here: 1. There seem to be a lot more extensions here than are justified by the working set size. With 2 MB initial allocation, the arena must have been using about 66 MB to store a working set of about 16 MB. This must be due to running out of zones in arenaAllocPolicy. This is covered by job003384. 2. The extension size does not adapt. It would be sensible to extend with a value that is a multiple of the current arena size, so that the arena grows exponentially. Need to think how this interacts with mps_arena_vm_growth. It ought to be possible to deprecate this function (or even remove it, as it was never documented). See job003795. But check the background and the customer requirement first [2]. DL notes that exponential growth is a bad idea when you are short of address space (for example, 32-bit process using many libraries each of which needs its own chunk of address space). In this case you'd like your address space to be a bit bigger than your working set, but the exponential growth will ensure that a fixed fraction of it is wasted. So maybe this is a non-starter. But it would still be nice to avoid mps_arena_vm_growth, maybe via some other option. |
| How found | manual_test |
| Evidence | [1] <https://info.ravenbrook.com/project/mp.../user-guide/manual/html/guide/perf.html>[2] < https://info.ravenbrook.com/mail/2007/12/19/12-17-36/0.txt> |
| Created by | Gareth Rees |
| Created on | 2014-05-14 16:46:27 |
| Last modified by | Gareth Rees |
| Last modified on | 2014-05-16 10:17:49 |
| History | 2014-05-14 GDR Created. |