$Id: //info.ravenbrook.com/project/mps/doc/2006-05-11/telemetry-log-events/1150telem.txt#2 $ (Do I need a wiki to put stuff like this in? This needs to go into "The MPS Book".) (Work in progress...) (rhsk) ---- This is aiming at being a HOWTO document, or system-wide overview. Not at first draft yet. RHSK 2005-12-15. Sources: .ref.source: .ref.metrics: "design.mps.metrics" = .ref.design: .ref.guide: "doc.mps.guide.telemetry" = .obsolete.ref.guide.telemetry: Do NOT use "guide.telemetry", it is OBSOLETE XX __METERS__ Meter ::= accumulator (takes series X0, X1,..XN, keeps running tally of N, sumX, sumX**2, minX, maxX) meter methods: MeterInit(namestring, pOwner) MeterAccumulate(X) MeterWrite(mps_lib_FILE) MeterEmit() -- emits meter values in an event of type "MeterValues" So far so good, but Meters are never actually used in the MPS. :-( Note: The following keywords are related to meters: meter metric statistic diagnostic debug __EVENT LIFECYCLE__ Event ::= tuple of ( , , , ..., ) Events are Emitted by Event-annotations, which are scattered throughout MPS code. Event annotations have the form EVENT_(, , , ..., ). The ('format') expresses the datatype of each of the event's , by a sequence of code letters, one letter per argument. For example, an event where arg1 is a Pointer and arg2 and arg3 are Words would be annotated like this: EVENT_PWW(TraceStatReclaim, trace, trace->reclaimCount, trace->reclaimSize); Each EVENT_() emits a compact, *binary* Event-record into the MPS's pre-allocated eventBuffer. An Event-record is a binary record containing the and other args from the EVENT_ call. When full, or on mps_arena_destroy(), the sequence of Event-records in the eventBuffer is flushed: EventFlush() calls mps_io_write(), which with the mpsioan.c plinth writes a binary stream to "mpsio.log". Note: The format of the binary Event-record is *undefined*. The data is written by memcopying directly from a C-struct; this is non-portable, but very fast. In practice this non-portability is not a problem because the corresponding analysis tools are typically also written in C and compiled with the same compiler (and in any case Event-records are simple structures). Speed of event emission is paramount. Note: Release varities do NOT emit events (to avoid the performance hit). EVENT_() compiles to NOOP. [Many of the identifiers that control conditional-compilation are ill-chosen, as noted herein. RHSK 2005-12-15]. The logic is: EVENT_() only generates code if "EVENT" [ill-chosen!] is defined, which is controlled by CONFIG_LOG [ill-chosen!], which is on for CONFIG_VAR_TI or CONFIG_VAR_II, which is .variety.ti and .variety.ii. Some events are also wrapped in STATISTIC_STAT() (.ref.metrics), which is controlled by DIAGNOSTICS [ill-chosen!] (.impl.mpm.h), which is controlled by CONFIG_DEBUG [ill-chosen!] (.impl.config.h). Note: We don't want the event-emitting varieties (.variety.ti) to run tediously slowly all the time either, so there is some coarse-grained run-time logic to control which Events are emitted. Each event is of one of the 7 'event kinds', which are approximately grouped by frequency, from occasional (EventKindArena, EventKindTrace) to extremely frequent (EventKindRef, EventKindObject). The 'telemetry control' bit-mask has one bit -- an On/Off switch -- for each kind. Default is all kinds Off. The example plinth (mpsliban.c + mpsioan.c) reads the environment to initialise the telemetry-control bit-mask, so you can turn on all EventKinds in a Unix shell (eg. /bin/bash) with "export MPS_TELEMETRY_CONTROL=255" before you run the MPS. This is only a coarse-control, letting you skip the extremely frequent events (and so avoid enormous log-files) if the events you are interested in are the rare ones. For fine-control, to select just the events you care about, you must use the off-line tools (see below). Use an external tool to analyse the events. These are often called "off-line tools", because you usually stream the events to a log file ("mpsio.log" if using the example mpsioan.c plinth), and then analyse them later. With a different plinth, it's quite possible to simultaneously run an external event-analysis tool with a 'live tether' to the MPS. Why do you need an external tool? Because the event stream is binary and unreadable (for speed and compactness in the MPS), and because the MPS does not offer fine-grained control over which events it emits. There is a 'simple' tool called dumper.c. It doesn't compile on .platform.xcppgc. It may be obsolete. There is another tool called "eventcnv" which is built on .platform.xcppgc. Instructions are in .ref.guide. For a weak example of use see . I don't know how to use it yet. There are also references to an event analyser called "lumberjack". It may be obsolete. And there's an event replayer. The MPS exports functions that let the client attach [is "attach" right? RHSK 2005-12-15] extra data (character string labels) onto events, to make them easier to analyse in the context of a client-domain investigation. The client can also use the coarse-grained event control, and flush the eventBuffer: (from branch 1998-01-06) mps_telemetry_control mps_telemetry_flush mps_telemetry_intern mps_telemetry_label mps_lib_telemetry_control -- initial values, fetched from eg. shell environment variables Note: The following keywords are related to events: telemetry event io logging __EVENTS__ what's the 'event clock' typedef Word EventType; #define EventEventTime ((EventType)0xEF213E99) /* TIME */ EventType is a sig. typedef struct { Word code; Word clock; } Event0Struct; Event->any.code is a Word. It holds the EventType sig. [eventdef.h] * - Code: The unique 16-bit code associated with this event type, * not currently used (see ); [eventcom.h] * These definitions will be unnecessary when the event codes are * changed to 16-bit. See . #define EventEventTime ((EventType)0xEF213E99) /* TIME */ [event.h] /* @@@@ As an interim measure, send the old event codes */ \ EventMould.any.code = Event##type; \ 1340 //info.ravenbrook.com/project/mps/branch/2001-08-13/trunk/src/event.h -c 18465 event.h#4 MMdevel_event_string trying to find what job this was done under, and why it was left incomplete //info.ravenbrook.com/project/mps/branch/2001-08-13/trunk/src/event.h#4 work was never finished. Okay: Let's use "Event-type" for the abstract concept. There's an Event-type-sig (eg. 0xEFB07141), with associated C-type "EventType". This is the 'old' way to identify an event-type, but it is still in use. Event-type-sigs are defined in eventcom.h. There's a 16-bit Event-code (eg. 0x0015), with associated C-type B. DOCUMENT HISTORY 2005-12-09 RHSK Created from rhsk/job001150 2006-05-22 RHSK Added more notes about Events.