JNEXT
Skip to content

4.1 The test triplet

Every batch of changes on main has to clear three layers of testing before it is pushed. The project calls them the triplet, and running them takes four commands:

$ make clean && make gui-release
$ make unit-test
$ ./build/test/fuse_z80_test build/test/fuse
$ make regression

The triplet is the core of the batch gate, which adds make unit-test-sdl, make unit-test-win and make regression-win.

  • When it runs: on main's tip, once, after all of a work session's landings and before any push.
  • On a branch, nobody runs it. The author runs the suites and regression rows the change touches, plus make unit-test before each review. The reviewer runs the targeted rows again.
  • When it is red: the failure is traced to the landing that caused it, which is fixed forward or reverted, and the gate runs again. Nothing is pushed red.

There must be no FAIL and no SKIP in any layer. A SKIP is a row that was not tested, so every harness exits non-zero on one, in every mode (owner decision, 2026-10-06); the SKIP badge stays only to say what to install or fix.

Rebuild first — always

The build at the top is not a courtesy, and it is not left to your good habits either. make regression declares unit-test-build, gui-release and sdl-release as real prerequisites, because the suite runs those exact binaries. Most rows run build/gui-release/jnext; the four rows that exercise the SDL-only frontend need build/sdl-release/jnext instead, because SdlApp is instantiated only in a build configured with ENABLE_QT_UI=OFF; and the rewind-func row runs build/test/rewind_test, which make clean deletes. Building all of them from the target is what makes "the tests ran against this source" true by construction rather than by discipline. (make regression-rows declares gui-release-non-pgo instead of gui-release, so a branch runs no PGO build; ROWS_PGO=1 switches it back.)

The rule was written after a 14-hour-old binary produced two bogus FAILs immediately before a version bump. Staleness cuts the other way too, and that direction is worse: an old binary can equally fail to contain the bug a test would have caught, and then it reports a pass it never earned.

In a fresh agent worktree, run make worktree-bootstrap before anything else. roms/* is git-ignored, so a new checkout has no SD-card image and cannot run the suites at all.

Layer 1 — the unit suites

make unit-test does not go looking for test binaries. It runs exactly the suites declared in test/unit-tests.conf, in parallel, each one bounded by a 300 s timeout. That manifest currently declares 90 suites totalling 6610 rows: the suite count is pinned in the file's own # expect: 90 line, and the row total is the sum of the per-suite counts in that same file. The committed dashboard, test/SUBSYSTEM-TESTS-STATUS.md, reports the same 6610. 4.2 explains what those pins are for.

These suites link the emulator libraries directly and drive the classes through their public API, which makes them fast and hermetic — sd_rom_extractor_test is the only one that reads the SD image. This is also where VHDL compliance is asserted, row by row. What they cannot catch is anything that only exists once the whole machine is assembled: they render no frame, open no window, and drive neither frontend.

make unit-test also pulls in the cheap structural gates ahead of the build — the tautological-assertion lint, the Makefile help lint, the traceability accounting, self-test and duplicate-ID checks, the documentation checks, and the hermetic half of the packaging contract tests. They live there rather than only in CI for a simple reason: a gate the inner loop never reaches is not a gate.

Layer 2 — the FUSE Z80 opcode suite

$ ./build/test/fuse_z80_test build/test/fuse

All 1356 rows pass. This is the FUSE project's data-driven opcode corpus, which covers every instruction, including undocumented behaviour and per-instruction cycle counts. It is the only thing that proves the CPU core itself is right — and it proves nothing else. The Z80N extensions are covered by z80n_test (85 rows), and everything Next-specific lives elsewhere.

Note that fuse_z80_test is also a declared row of test/unit-tests.conf, pinned at 1356 with its data directory passed as an argument, so make unit-test has already run it. The standalone invocation above runs the same binary against the same data; it is a convenience for reading that one number, not a third independent execution.

Layer 3 — the screenshot and functional regression

$ make regression

This layer launches the real jnext binary headless and looks at what comes out of it. It reports 215 rows, made up of 6 preflight lints, 1 SD-image provisioning row, 66 screenshot rows and 142 functional rows. The last two numbers come from the # expect: 66 and # expect: 142 pins in regression_tests.conf and functional_tests.conf, and the arithmetic that ties them to the total is the harness's own end-of-run accounting assertion.

A screenshot row compares a whole frame, pixel for pixel, against a committed reference PNG. That is exactly as strong and as blunt as it sounds: it catches any rendering change anywhere in the pipeline, and when it fails it tells you the frame changed without telling you why. Functional rows cover the things a linked library cannot show at all — recording an MP4, an RZX round trip, audio underruns through a real device, the CLI's exit contracts, keypress delivery in both frontends.

The rows run in parallel — every CPU for the screenshots and the untagged functional rows, then the rows tagged quiet, which are bounded by real-time pacing (audio-underrun-func, screenshot-paused-func and a few more), a lane at a time on a host the first phase no longer loads (see The regression suite). JNEXT_TEST_JOBS caps the lanes; it is no longer needed for correctness, and CI passes none. Rows that merely spawn short-lived processes have failed under outside load too, which is why the suite itself flags a FAIL that happened on a loaded host.

A green make regression on a committed tree leaves a stamp keyed on the tree's non-doc content, and make regression-stamp-check finds it. A batch gate whose tree changed only in documentation since the last green run need not repeat it. Reviewers run targeted rows, never a full run.

What the triplet does not prove

It does not prove that the emulator matches real hardware anywhere no test row asserts that it does; making those gaps visible is the job of the traceability matrix (4.4). And it does not prove that the prose in the man page or in the guides is true — only that the generated outputs match their sources. 4.5 sets out exactly where that line sits.