State, replay and conformance

A Terse32 machine is deterministic: the same image with the same inputs gives the same state after every frame, on any host. This article defines that state byte for byte (the state serialization), its hash, how a run is replayed and compared, and the snapshot file that saves a machine and restores it. It is how two implementations are checked against each other: run the same image with the same inputs on both and compare hashes; the first frame that differs is where they diverge.

The state serialization

The serialization covers everything that can influence later frames or what is shown, and nothing else. It exists only between frames: after a frame ends and before the next begins (a machine just reset counts). Integers are little-endian, u32 unsigned, i32 two's complement; a flag is 00h or 01h. There is no padding. Sections follow in this order.

OffsetSizeField
013ASCII terse-state-1
1332Image ID (BLAKE3-256 of the image file)
454pc (after a fault, the faulting instruction)
49124x1 to x31, u32 each
1734Frame counter (what frame_count returns next frame)
1771Stopped: 0 running, 1 exited, 2 faulted, 3 halted by the host
1781Exit code; if halted, the slow device's slot; else 0
1791Fault kind: 1 illegal instruction, 2 breakpoint, 3 fetch, 4 load, 5 store, 6 misaligned; else 0
1804Illegal instruction's encoding (16-bit in the low half), else 0
1844Fault pc, else 0
1884Fault address, else 0
1924Budget carried into the next frame (i32, zero or negative)
1964L: RAM size, ram_pages x 8,192
200LAll of RAM from 0010_0000h
200 + LkGrants: for each device the image lists as optional, in header order, 1 if the machine was granted it, else 0

The image ID stands for the ROM, the RAM size and the device list, which never change; which optional devices the host granted at reset does not, so the grants follow RAM (k is 0 for an image without optional devices). Below, the machine has a device if its image requires it or it was granted. From here offsets are relative to the start of each section.

System, 37 + 4n bytes:

OffsetSizeField
04Unix time latched for the frame just run
41n: events queued, 0-64
54nThe events from the head: type, a, b, c
5 + 4n32Controllers 0-3, 8 bytes each, as controller returned them last frame (zero if not connected)

Terminal (device 2), 1 or 10,379 bytes:

OffsetSizeField
01Present: the machine has device 2 (if 0, the section ends)
19,600Cells 0-2,399, 4 bytes each
9,6016Cursor column, row, foreground, background, cursor shown, pending wrap
9,607768Palette 0-255: red, green, blue
10,3751k: bytes of an incomplete UTF-8 sequence given to write, 0-3
10,3763Those bytes, then zeros

Vector display, 1 or 8 + m bytes: present (the machine has device 3 or 9), style (0 clean, 1 phosphor), persistence (160 at reset), fill mode (0 fill, 1 outlines), m (u32, 0-16,384), then the list last accepted. Style and persistence are shared with device 9; an image without device 3 has fill mode 0 and m = 0.

Content (device 4), 1 or 425 bytes: present, then 8 request records of 53 bytes for handles 1-8:

OffsetSizeField
01op pending (01h-11h), or 0 for a free handle (all zero)
14Address of the request block
532id read at submit (a record's name for save, load, delete; a put's blob ID)
374offset read at submit (publish: the kind; set profile: the listed bit)
414length read at submit
454addr read at submit
494size read at submit (put, save; find: query length)

Fields an op does not read at submit are zero.

Page (device 7), 1 or 21 + d bytes: present, the rectangle x, y, width, height (u16 each, as clamped), the scroll value (u32, as stored), the background (u32, FF00_0000h at reset), d (u32: document length 1-65,536, or 0 for none) and the document.

Machines (device 8): absent (0 bytes) if the image does not list device 8. Otherwise a present flag, 1 only for the shell of a system; if it is 0 the section ends. Then focus (1), 8 handle records of 138 bytes, b (1) and b built-in image IDs (32 each), r (1) and r unread open requests (each the handle, then the 82 bytes request writes), in handle order, then the view (18 bytes, all zero for none: handle, direction, the rectangle as four u16, start progress u16, start frame u32, duration u16). Total 1,126 + 32b + 83r.

OffsetSizeHandle record
01State
11Run flag
28Rectangle x, y, w, h (u16, as clamped)
1047Bytes 1-47 of the info record
5781Identity given to start_program, or zeros

3D display (device 9): absent unless the machine has device 9; then m (u32, 0-16,384) and the list.

Sound (device 6): absent unless the machine has device 6; then the 16 voices of 38 bytes (source, wavetable, sample slot, phase, increment, volume, pan, attack, decay, sustain, release, stage, level, modulator, depth, feedback, ring, mix, noise register, ended, last two outputs), the 32 wavetables of 64 bytes, and the 4 sample slots, each its length (u32), its loop start (u32) and its bytes: 2,688 bytes plus the samples. The layout is in STATE.md section 1.12 and the voice in the sound device.

Argument: 32 bytes, the blob ID the shell gave the machine, only if it gave one; else nothing. There is no flag: a reader finds the argument exactly when 32 bytes remain after the sound section, none when 0 remain, and refuses any other remainder.

The sections that may be absent were added so that the hashes of older images did not change. The total is 45 + 128 + 23 + 4 + L + 37 + 4n plus the device sections.

What is left out

The hash

The state hash is the BLAKE3-256 hash (unkeyed) of the serialization, written as 64 lowercase hex digits. Test vectors from the reference implementation, with no input:

programs/hello/hello.terse at reset (serialization 18,812 bytes), after 1 frame and after 30 frames:

f82b7079daea85863f079eaba67e0ad2816b905e9a1a494e7dab587889a4c058
2c8994cf8408449f927b16377cf44750419682c7fcc744d84748459c7ec8f9cf
819432152b38626f8fc54b43f50d5d4c501dc99e6904315b61dc78b796c611e2

"No input" means no events, Unix time 0, no random bytes (so random gives zeros), controllers disconnected and no completions. That is what terse run supplies, apart from its --events script.

Replay

A machine's only inputs are, per frame: its events (in order), the latched Unix time, the random bytes, the controller states, the content completions (which handle, in which frame, status, size and bytes) and, for a machine a shell started, what open returns. Supply the same inputs in the same frames and the run repeats exactly, hash for hash. Page layout reaches the machine only as events, so a run replays even where layouts differ. terse run --events <file> replays a scripted input file (one line per event: frame, command, arguments); time stays 0 and there are no random bytes, and content requests complete at the start of the next frame.

Snapshots

A snapshot is the serialization plus the program's identity. A machine is restored from a serialization S and the image whose ID S names: it has the devices S's grants say (an optional device the host has and S lacks stays absent; one S has that the host cannot give fails the restore), and S is read section by section, with the sections those devices imply. It is refused if it is cut short or too long, names another image, has a present machines section (a shell), or does not serialize back to exactly S (so every field is in range: flags, stopped and fault fields consistent, budget not positive, L equal to the image's RAM, n at most 64, cursor on the grid, valid lists and lengths, a clamped page rectangle, pending requests with known ops, blocks in RAM and fields as a submit leaves them).

The restored machine has S's hash and is between frames; its next frame is numbered by S's counter. Host bookkeeping starts afresh: pending content requests fail in the next frame, the page is laid out as a new document, the afterglow is empty. Time and random bytes are what the host supplies from then on.

The snapshot file:

OffsetSizeField
04TSNP (54h 53h 4Eh 50h)
41Version 0
581Identity, as identity writes it; zeros for none
864Unix time latched for the machine's last frame
904N: length of S
94200S bytes 0-199 (up to and including L)
29416Page map: bit k (byte k/8, bit k mod 8) set if RAM page k is stored
3108,192pThe p stored pages, in order; the others are zero
...N - 200 - LThe rest of S
...32The state hash of S

A writer stores exactly the pages that are not all zero; a reader accepts any page stored. Bits past the last page must be clear. A reader also refuses N below 200 or above 1,423,091 (the longest serialization), and L that is 0, not a multiple of 8,192, above 1,048,576 or more than N - 200. Bytes after the hash, or a hash that does not match, are refused.

terse run <image> --save-state <file> writes the file after the last frame; --load-state <file> starts from one instead of reset (the image is the one given, else the one the snapshot names). --frames 0 --hash then prints the saved hash, and a continued run has the hashes of the uninterrupted one. Both are for a single machine. Shells take snapshots of their machines with the Machines device.

The conformance image

programs/conform-reading and programs/conform-program are the same source built for the reading and the program profile (terse run --profile reading or --profile program), each with its own events and expected files. programs/conform/conform.terse is a test image for whoever writes a host. It needs devices 2, 3, 4, 7 and 9 and 16 RAM pages, and checks the CPU (every instruction class, M-extension corner cases, compressed forms, code in RAM and self-modifying code), the budget and the charges, events, queue overflow and controllers, every system call, bad and empty buffers, the terminal, the vector display and its fill cap, the content device without a network, and the page. Frames 49-108 are a worst-case stress phase for measuring frame rate.

Run it headless for 200 frames with an empty store and no subscriptions, and compare with programs/conform/expected-hashes.txt (the hash every 10 frames). Create the two empty directories first:

    terse run programs/conform/conform.terse \
      --events programs/conform/conform.events \
      --store /tmp/conform/store \
      --config /tmp/conform/config \
      --frames 200 --hash-every 10

With --dump --dump-vectors instead of --hash-every 10 it prints the final screen (compare with expected-screen.txt): one PASS or FAIL line per section it can check itself, and the display list. The hashes are the authoritative check; they also cover what the program cannot see, such as the palette, the list, every register and every unit of budget. Esc exits with the number of failed sections.

See the machine for frames and the budget, and charges.