38. Thread manager

38.1. Introduction

.intro: This is the design of the thread manager module.

.readership: Any MPS developer; anyone porting the MPS to a new platform.

.overview: The thread manager implements two features that allow the MPS to work in a multi-threaded environment: exclusive access to memory, and scanning of roots in a thread’s registers and control stack.

38.2. Requirements

.req.exclusive: The thread manager must provide the MPS with exclusive access to the memory it manages in critical sections of the code. (This is necessary to avoid for the MPS to be able to flip atomically from the point of view of the mutator.)

.req.scan: The thread manager must be able to locate references in the registers and control stack of the current thread, or of a suspended thread. (This is necessary in order to implement conservative collection, in environments where the registers and control stack contain ambiguous roots. Scanning of roots is carried out during the flip, hence while other threads are suspended.)

.req.register.multi: It must be possible to register the same thread multiple times. (This is needed to support the situation where a program that does not use the MPS is calling into MPS-using code from multiple threads. On entry to the MPS-using code, the thread can be registered, but it may not be possible to ensure that the thread is deregistered on exit, because control may be transferred by some non-local mechanism such as an exception or longjmp(). We don’t want to insist that the client program keep a table of threads it has registered, because maintaining the table might require allocation, which might provoke a collection. See request.dylan.160252.)

.req.thread.die: It would be nice if the MPS coped with threads that die while registered. (This makes it easier for a client program to interface with foreign code that terminates threads without the client program being given an opportunity to deregister them. See request.dylan.160022 and request.mps.160093.)

.req.thread.intr: It would be nice if on POSIX systems the MPS does not cause system calls in the mutator to fail with EINTR due to the MPS thread-management signals being delivered while the mutator is blocked in a system call. (See GitHub issue #9.)

.req.thread.errno: It would be nice if on POSIX systems the MPS does not cause system calls in the mutator to update errno due to the MPS thread-management signals being delivered while the mutator is blocked in a system call, and the MPS signal handlers updating errno. (See GitHub issue #10.)

.req.thread.lasterror: It would be nice if on Windows systems the MPS does not cause system calls in the mutator to update the value returned from GetLastError() when the exception handler is called due to a fault. This may cause the MPS to destroy the previous value there. (See GitHub issue #61.)

38.3. Design

.sol.exclusive: In order to meet .req.exclusive, the arena maintains a ring of threads (in arena->threadRing) that have been registered by the client program. When the MPS needs exclusive access to memory, it suspends all the threads in the ring except for the currently running thread. When the MPS no longer needs exclusive access to memory, it resumes all threads in the ring.

.sol.exclusive.assumption: This relies on the assumption that any thread that might refer to, read from, or write to memory in automatically managed pool classes is registered with the MPS. This is documented in the manual under mps_thread_reg().

.sol.thread.term: The thread manager cannot reliably detect that a thread has terminated. The reason is that threading systems do not guarantee behaviour in this case. For example, POSIX says, “A conforming implementation is free to reuse a thread ID after its lifetime has ended. If an application attempts to use a thread ID whose lifetime has ended, the behavior is undefined.” For this reason, the documentation for mps_thread_reg() specifies that it is an error if a thread dies while registered.

.sol.thread.term.attempt: Nonetheless, the thread manager makes a “best effort” to continue running after detecting a terminated thread, by moving the thread to a ring of dead threads, and avoiding scanning it. This might allow a malfunctioning client program to limp along.

.sol.thread.intr: The POSIX specification for sigaction says that if the SA_RESTART flag is set, and if “a function specified as interruptible is interrupted by this signal, the function shall restart and shall not fail with EINTR unless otherwise specified.”

.sol.thread.intr.linux: Linux does not fully implement the POSIX specification, so that some system calls are “never restarted after being interrupted by a signal handler, regardless of the use of SA_RESTART; they always fail with the error EINTR when interrupted by a signal handler”. The exceptional calls are listed in the signal(7) manual. There is nothing that the MPS can do about this except to warn users in the reference manual.

.sol.thread.errno: The POSIX specification for sigaction says, “Note in particular that even the “safe” functions may modify errno; the signal-catching function, if not executing as an independent thread, should save and restore its value.” All MPS signals handlers therefore save and restore errno using the macros ERRNO_SAVE and ERRNO_RESTORE.

.sol.thread.lasterror: The documentation for AddVectoredExceptionHandler does not mention GetLastError() at all, but testing the behaviour reveals that any value in GetLastError() is not preserved. Therefore, this value is saved using LAST_ERROR_SAVE and LAST_ERROR_RESTORE.

38.4. Interface

typedef struct mps_thr_s *Thread

.if.thread: The type of threads. It is a pointer to an opaque structure, which must be defined by the implementation.

Bool ThreadCheck(Thread thread)

.if.check: The check function for threads. See design.mps.check.

Bool ThreadCheckSimple(Thread thread)

.if.check.simple: A thread-safe check function for threads, for use by mps_thread_dereg(). It can’t use AVER(TESTT(Thread, thread)), as recommended by design.mps.sig.check.arg.unlocked, since Thread is an opaque type.

Arena ThreadArena(Thread thread)

.if.arena: Return the arena that the thread is registered with. Must be thread-safe as it needs to be called by mps_thread_dereg() before taking the arena lock.

Res ThreadRegister(Thread *threadReturn, Arena arena)

.if.register: Register the current thread with the arena, allocating a new Thread object. If successful, update *threadReturn to point to the new thread and return ResOK. Otherwise, return a result code indicating the cause of the error.

void ThreadDeregister(Thread thread, Arena arena)

.if.deregister: Remove thread from the list of threads managed by the arena and free it.

void ThreadRingSuspend(Ring threadRing, Ring deadRing)

.if.ring.suspend: Suspend all the threads on threadRing, except for the current thread. If any threads are discovered to have terminated, move them to deadRing.

void ThreadRingResume(Ring threadRing, Ring deadRing)

.if.ring.resume: Resume all the threads on threadRing. If any threads are discovered to have terminated, move them to deadRing.

Thread ThreadRingThread(Ring threadRing)

.if.ring.thread: Return the thread that owns the given element of the thread ring.

Res ThreadScan(ScanState ss, Thread thread, Word *stackCold, mps_area_scan_t scan_area, void *closure)

.if.scan: Scan the stacks and root registers of thread, using ss and scan_area. stackCold points to the cold end of the thread’s stack—this is the value that was supplied by the client program when it called mps_root_create_thread(). In the common case, where the stack grows downwards, stackCold is the highest stack address. Return ResOK if successful, another result code otherwise.

38.5. Implementations

38.5.1. Generic implementation

.impl.an: In than.c.

.impl.an.single: Supports a single thread. (This cannot be enforced because of .req.register.multi.)

.impl.an.register.multi: There is no need for any special treatment of multiple threads, because ThreadRingSuspend() and ThreadRingResume() do nothing.

.impl.an.suspend: ThreadRingSuspend() does nothing because there are no other threads.

.impl.an.resume: ThreadRingResume() does nothing because no threads are ever suspended.

.impl.an.scan: Just calls StackScan() since there are no suspended threads.

38.5.2. POSIX threads implementation

.impl.ix: In thix.c and pthrdext.c. See design.mps.pthreadext.

.impl.ix.multi: Supports multiple threads.

.impl.ix.register: ThreadRegister() records the thread id the current thread by calling pthread_self().

.impl.ix.register.multi: Multiply-registered threads are handled specially by the POSIX thread extensions. See design.mps.pthreadext.req.suspend.multiple and design.mps.pthreadext.req.resume.multiple.

.impl.ix.suspend: ThreadRingSuspend() calls PThreadextSuspend(). See design.mps.pthreadext.if.suspend.

.impl.ix.resume: ThreadRingResume() calls PThreadextResume(). See design.mps.pthreadext.if.resume.

.impl.ix.scan.current: ThreadScan() calls StackScan() if the thread is current.

.impl.ix.scan.suspended: PThreadextSuspend() records the context of each suspended thread, and ThreadRingSuspend() stores this in the Thread structure, so that is available by the time ThreadScan() is called.

38.5.3. Windows implementation

.impl.w3: In thw3.c.

.impl.w3.multi: Supports multiple threads.

.impl.w3.register: ThreadRegister() records the following information for the current thread:

A HANDLE to the process, with access flags THREAD_SUSPEND_RESUME and THREAD_GET_CONTEXT. This handle is needed as parameter to SuspendThread() and ResumeThread().

The result of GetCurrentThreadId(), so that the current thread may be identified in the ring of threads.

.impl.w3.register.multi: There is no need for any special treatment of multiple threads, because Windows maintains a suspend count that is incremented on SuspendThread() and decremented on ResumeThread().

.impl.w3.suspend: ThreadRingSuspend() calls SuspendThread().

.impl.w3.resume: ThreadRingResume() calls ResumeThread().

.impl.w3.scan.current: ThreadScan() calls StackScan() if the thread is current. This is because GetThreadContext() doesn’t work on the current thread: the context would not necessarily have the values which were in the saved registers on entry to the MPS.

.impl.w3.scan.suspended: Otherwise, ThreadScan() calls GetThreadContext() to get the root registers and the stack pointer.

38.5.4. macOS implementation

.impl.xc: In thxc.c.

.impl.xc.multi: Supports multiple threads.

.impl.xc.register: ThreadRegister() records the Mach port of the current thread by calling mach_thread_self().

.impl.xc.register.multi: There is no need for any special treatment of multiple threads, because Mach maintains a suspend count that is incremented on thread_suspend() and decremented on thread_resume().

.impl.xc.suspend: ThreadRingSuspend() calls thread_suspend().

.impl.xc.resume: ThreadRingResume() calls thread_resume().

.impl.xc.scan.current: ThreadScan() calls StackScan() if the thread is current.

.impl.xc.scan.suspended: Otherwise, ThreadScan() calls thread_get_state() to get the root registers and the stack pointer.