MPS issue job001811

TitleMPS AMC small nailed objects risk big retention by placement after big objects
Assigned userRichard Kistruck
DescriptionMPS AMC small nailed objects risk big retention by placement after big objects

Related jobs:
  - job001809 "MPS ambiguous internal pointer makes AMC pool retain memory as pads" (fixes an earlier issue where a free-able pad was not being freed)

RHSK 2009-04-03
After allocating a big object (which always starts its own segment) there may be some space left at the end of the MPS segment. If a small object is allocated there, and is then preserved by a legitimate, persistent, ambiguous reference, the MPS is unable to move it. The big object may either be preserved (by copy) or collected: either results in a large pad object. But the MPS cannot free the segment it is on, because of the small object. So that large useless pad persists, taking up memory and contributing nothing.
AnalysisRHSK 2008-05-01
BufferFill for very big object is rounded up, and this excess might then be
used for a few small objects. This is bad strategy: if one of these
small objects is nailed, it could retain the whole enormous segment.
Very big objects should live on a segment by themselves.

RHSK 2009-04-03
A solution would be to set apStruct.limit to just big enough for the requested big allocation. Any further allocation after that will trip, pad-out the rest, and BufferFill with a new segment.

RHSK 2009-10-01
Fixed by treating any segment of at least AMCLargeSegPAGES as 'large', setting the allocation-point limit to precisely the mps_reserve() amount requested, and padding any remaining space in the last page of the segemnt -- this is LSP = Large Segment Padding, and the pad is an LSP pad.

Gather statistics and emit diagnostics to show how well this policy is performing.
How foundinspection
Evidencethinking about the code
See also [pads on heap] <>
and the discussion that followed.
Observed in1.108.1
Introduced in1.100.0
Created byRichard Kistruck
Created on2008-05-01 17:10:22
Last modified byRichard Kistruck
Last modified on2009-10-01 17:25:46
History2008-05-01 RHSK Created.
2009-04-03 RHSK Propose solution.
2009-07-22 RHSK Link to job001809
2009-10-01 RHSK Write-up fix.


Change Effect Date User Description
168704 closed 2009-09-18 16:15:50 Richard Kistruck MPS br/padding AMC LSP: rename PagesPerSegMediumLIMIT as AMCLargeSegPAGES, and make it a define in config.h. Corrections in des/poolamc.
168688 closed 2009-09-17 16:59:39 Richard Kistruck MPS br/padding:
poolamc.c tidy up:
  neater implementation of obj1pip (amcReclaimNailed)
  neater implementation of amcResetTraceIdStats -- no need for a function any more
  delete lots of obsolete temporary diagnostic (superseded by AMCTraceEnd_pageret)
  a few more avers (especially on buffer empty)
also revert temporary diagnostic changes in arena.c, config.h, diag.c, global.c
diag.c: fix diag-buffer at 100 screenfuls (200000 chars).
zcoll.c: reinstate Make 50000 with occasional big objs.
168661 closed 2009-09-14 17:12:43 Richard Kistruck MPS br/padding des/poolamc: complete documentaiton of AMC pads and retained pages.
168478 closed 2009-08-20 17:44:16 Richard Kistruck MPS br/padding: new PoolTraceEndMethod: do end-of-trace work
 Tracer calls PoolTraceEnd() when the trace is TraceFINISHED.
 AbstractPoolClass uses PoolTrivTraceEnd -- a NOOP. [mpm.h, mpmst.h, mpmtypes.h, pool.c, poolabs.c]
 AMC overrides with AMCTraceEnd, to emit diagnostic on how well the trace went! [poolamc.c]
168451 closed 2009-08-18 17:14:19 Richard Kistruck MPS br/padding zcoll.c: Change BigSmall() test to BigdropSmallambig() --
creates a big object and drops the ref to it, followed by a small object and keeps an ambig ref to it (to nail the AMCSeg it is in).
Loops 100 times. This is for testing job001811 changes.
167759 open 2009-04-08 16:04:09 Richard Kistruck MPS br/padding poolamc.c: change AMCBufferFill, so that for large requests (> 8 ArenaAligns) it gives only requested size; pad the rest. See job001811. The number 8 is provisional...