The bench¶

The bench is one stock machine with the dev image, and the discipline around it. Almost every measured value on this site was obtained there, which is why the machine has a name and a definition: the bench reference. This page is the runbook: the bench machine, the clean-up rule, the bench diagnostics page, the bench tools, and the bench actuator.

How a result was obtained is recorded in the commit that carried it.

The bench machine¶

• The board. The bench reference is one Glowforge Basic built in late 2017; the control board is the same in the Basic, the Plus, and the Pro. The bench runs the dev image (forgefirm-image-dev) from an SD card. The dev image has a BusyBox userland, python3, gdb, and strace. Log in over SSH as root; the dev image permits a root login without a password. A serial console is available on ttymxc0 (serial access).
• Deploy a kernel. Write the new dev image to the SD card again (Build). This works because U-Boot (in the eMMC boot0 area) reads the saved environment at eMMC user-area offset 0x80000. That environment selects the boot device (on the bench, the SD card: mmcdev=0, mmcroot=/dev/mmcblk1p1). U-Boot then loads /boot/uEnv.txt and /boot/zImage from that rootfs partition. Thus the kernel always comes from the SD card that you wrote. The full eMMC map is on Boot and storage.
• Hot-swap a module. A change to the module alone can go on the board without a flash: copy glowforge.ko over /lib/modules/<kver>/extras/glowforge.ko, then rmmod glowforge && modprobe glowforge. A module reload turns the lid LED off (turn it on again through /sys/class/leds/lid_led*/target) and resets the analog configuration. The first liveness probe after a reload can read NO MOTION until the ladder probes again.
• Hot-swap against re-stamped kernels. The hot-swap loads only if the module was built against the patch state of the kernel that is on the board. An edit under the overlay of the kernel recipe (for example glowforge.dts) re-stamps CONFIG_LOCALVERSION_AUTO. The stamp does not come back when you revert the edit, because each do_patch makes a fresh git commit of the kernel tree. Thus, after an edit under the overlay, the module ships only with a full image flash. Kernel and BSP changes ride one image flash, in a batch. A .ko or overlay change is validated on the image that ships it, never hot-swapped onto a board that you are about to flash.
• Deploy a userspace program. The controller and the daemon are userspace programs. Replace the binary on the board. A flash is not necessary.
• Read the state through forgectrl or sysfs, never through the Grbl port. A second connection to TCP port 23 displaces the session that holds the port, whether a sender or a drill. Use the /status route of forgectrl, or the sysfs tree of the kernel module. They give the same state, without that effect.
• Power-cycle before a campaign. The acceptance tool takes a fresh-boot reference once for each boot. Take it after a power cycle, not after a warm reboot. The true fresh state of the machine is the powered-on one. That state is the lamp and sensor defaults of the PIC, with the start-up writes of forgectrl on top.

Bench hygiene¶

Remove everything that you put on the board for a test, in the same session. That includes scripts, helper binaries, sample logs, backups, staged copies, temporary configurations, and __pycache__.

• Stage test files in /tmp. It is lost at reboot.
• If a file must survive a reboot, put it in the one scratch directory /data/bench-scratch/. Delete that directory as a whole when the session ends.
• Nothing goes loose in /data. It holds only the factory state and the files of ForgeFIRM: forgefirm/, forgetest/, forgefirm.conf, ffboot, etc/, and log/forgefirm/.
• A bench tool that is worth a second use goes in forgefirm/scripts/bench/ in the repository. The dev image installs those tools under /usr/share/forgetest/bench/. Run them from there.
• Before the end of a bench session, list /data and remove what you added.

The bench diagnostics page¶

The acceptance daemon on the dev image also serves #bench (http://<machine>:8090/#bench). The page is the registry of the bench tools. It shows, for each tool, its safety class, its argument form, and its last run. The classes:

A tool runs as a subprocess with its output on the page. On the board, the machine is GF_HOST=127.0.0.1, the panel token is in GF_TOKEN, and the data files of the tool are under /data/forgetest/bench/ (FORGETEST_BENCH_DATA). The same scripts run from a LAN host with GF_HOST set. Each tool that can run on the board is registered. Two kinds of entry stay unported: the CI harnesses of the null-sink controller, and the decoder of factory .puls files. They do not run against the machine at all. They are listed so that the catalog of what exists is complete. Bench runs are recorded in /data/forgetest/bench.jsonl. They never enter a campaign.

The runner brackets each bench tool with a baseline pass, in the same way as each test. Before the run, it examines the machine against the fresh-boot idle state, and it restores what is off. After the run, on each exit path, it restores again. A takeover run also records the kernel attributes that the controller owns on entry, and it writes them back before forgectrl restarts.

The bench tools¶

The tools are in forgefirm/scripts/bench/. All of them run on the target board (dev image, python3 present) unless the table says otherwise. The tools that also run from a LAN host use gfbench.py:

• GF_HOST names the machine. sysfs goes through ssh (the ssh on PATH, or the client that GF_SSH names, for example GF_SSH='wsl -d <distro> -- ssh' from Windows). Grbl and forgectrl go over the LAN.
• With GF_HOST unset, the tools run on the board itself: sysfs directly, everything on 127.0.0.1. This is how the bench page runs them. Their data files are then under /data/forgetest/bench/ (FORGETEST_BENCH_DATA; next to the tool otherwise), and the panel token is in GF_TOKEN.

The tools that drive the thermal hardware directly (the flow characterization family) run with forgectrl, the owner of the thermal hardware, and the controller stopped. The takeover of the page does that. From a host, stop them first.

The live-fire drills¶

live_fire_drills.py <drill> [S] [F]:

Data files¶

These data files stay next to the tools:

• flow_matrix_results.json and flow_matrix_log.txt: the 60-run flow-detection matrix.
• flow_warm_results.json and flow_warm_log.txt: the warm-baseline validation.
• temp_calibration.json: the coolant sensor spot checks.
• lid_ir_ambient_baseline.csv: the lid-IR ambient level, lid closed, idle.

The build scripts use the Yocto cross toolchain and sysroot from the work directory of a target recipe in the build tree (FF_BUILD_TOP, default ../forgefirm/build). If that path is gone after a bitbake -c clean, you have three options:

• Build the named recipe again.
• Point TC at the current work directory of any target recipe.
• Build an SDK with bitbake meta-toolchain.

How the coolant-flow fire-gate threshold was derived¶

The cooling engine of forgectrl refuses to let the laser fire unless a heater-based flow check passes. With the pump commanded on, the loop heater runs at COOL_FLOW_HEATER_PCT (40 percent) for COOL_FLOW_CHECK_S (50 s). The rise of the downstream sensor over its settled baseline must stay below COOL_FLOW_RISE_C (14.4 °C). A stopped pump lets the output of the heater pool at the downstream sensor instead of a carry-away. The three constants are in forgectrl/src/cool.h. They are the compiled defaults behind the cool_flow_* settings.

The 14.4 °C threshold and the 40 percent / 50 s operating point come from flow_matrix.py. The matrix was 6 duties × 2 flow states × 5 interleaved repeats, that is 60 heating runs. Each run started from a common cooled baseline. flow_sampler.py sampled both sensors at 1 Hz. One heating trace was scored at each candidate check duration. The committed flow_matrix_results.json and flow_matrix_log.txt are that data set. The selection rule was the cheapest duty at which each observed no-flow rise was above each observed flow rise, with a comfortable d′. At 40 percent / 50 s:

• flow: 12.75 °C or less, over 17 observations;
• no flow: 16.04 °C or more, over 8 observations;
• d′ 8.4;
• about 0.8 °C of loop heating for each check.

The threshold is the balanced midpoint of the two bands (14.4 °C). flow_warm_validate.py then ran the real check again from heater-warmed baselines (flow_warm_results.json).

To reproduce this on another machine, boot the dev image. Then run the flow-matrix tool from the bench page (#bench). It is a takeover, and the full matrix takes about 1.6 h. The results land under /data/forgetest/bench/. Or run flow_matrix.py [duties] [repeats] from a host with GF_HOST (and GF_SSH if ssh needs a wrapper), with forgectrl and the controller stopped. The same derivation is also in forgectrl, as the flow-calibrate tool on the Diagnostics tab of the panel. It does 3 trials for each case at the operating point. It reports both bands and a recommended threshold. On the bench, its recommendation lands within a few tenths of a degree of 14.4. Apply a per-machine value through the cool_flow_rise setting. Do not edit the constant.

The bench actuator: forgefixture¶

The acceptance campaign asks a person for about eighty small things. Most of them are the same three: open the lid, pull the interlock, press the button. forgefixture (fixture/ in the forgefirm repository) is the box that does those on request. Thus the acceptance tool can run an operator test with nobody in the room. An ESP32-S3 DevKitC-1 on the bench network drives three relays that are wired into the connectors of the machine. The acceptance tool (forgetest, on the machine) asks it over HTTP. Then the tool watches the machine for the result, in the same way as it watches the hand of an operator.

Nothing here touches the safety chain of the laser. Two of the contacts are normally closed, and they sit in series with loops that the chain already reads. The third is normally open, across the button input, and it is only ever pulsed. With the fixture unpowered, unplugged, in a reboot, or crashed, the machine is a stock machine.

Hardware¶

The pins on the DevKit are plain GPIOs, with no strapping, USB, flash, or PSRAM role:

Connect the parts as follows:

• The relay coils (the VCC and GND of the modules) to the 3.3 V of the machine.
• The three IN pins to the GPIOs above.
• The GND of the DevKit to the GND of the modules.

The opto inputs of the modules share GND with their coils. Thus the ESP32 and the machine share a ground. Power the DevKit from a wall adapter, not from a PC, unless you want the ground of that PC on the machine.

Build and flash¶

The only input is fixture.env: the wifi network, the API key, and the hostname. Everything else is pinned (ESP-IDF v5.5, the mDNS component in dependencies.lock).

./fixture.sh env              # fixture.env from the example, with a fresh key; fill in the wifi
./fixture.sh build            # idf.py if installed, else the espressif/idf container (docker or podman)
./fixture.sh flash COM5       # or /dev/ttyUSB0; esptool from pip talks to the board directly
./fixture.sh monitor COM5     # the log
./fixture.sh test             # the host test of policy.c

Either USB port of the DevKit flashes. The port marked UART also shows the log. fixture.env is git-ignored and never leaves the bench. A build in the container works where docker or podman runs (Git Bash on Windows included). pip install esptool pyserial is the full host-side requirement for the flash and the monitor.

What it does on the network¶

The fixture joins the wifi as forgefixture (the HOSTNAME in fixture.env). It sends that name in its DHCP request. It announces forgefixture.local over mDNS, with a _forgefixture._tcp service. It connects again forever, and it never puts the radio to sleep. The API is HTTP on port 80, JSON, LAN only, with no OTA (the flash occurs over USB and nowhere else).

Each request carries the key in X-Fixture-Key. Without it, or with a wrong one, each path answers 401, and the attempt is logged with the address of the caller.

An action answers with the state as GET / shows it. The fixture does not read the switches of the machine. The tool verifies each action through the readings of the machine, which is the point.

curl -s -H "X-Fixture-Key: $KEY" http://forgefixture.local/
curl -s -H "X-Fixture-Key: $KEY" -d '{"state":"open"}' http://forgefixture.local/lid

The side of the tool¶

forgetest looks for /data/forgetest/fixture.json on the machine:

{"hostname": "forgefixture", "key": "<the API_KEY>", "ip": null,
 "channels": ["lid", "interlock", "button"], "arm_press": false}

ip overrides the mDNS lookup. The tool resolves <hostname>.local itself, because the image has no mDNS resolver. channels names what is wired. arm_press stays false unless the fixture is permitted to press the button to arm the laser for a live test. By default, that press belongs to a person. The file has mode 0600 and is bench-local. The tool probes the fixture before each run, and at most every 30 s otherwise. What holds:

• The machine still proves each action. The fixture does not read the switches. ctx.act asks the fixture, then waits for the reading of forgectrl, in the same way as it waits for the hand of an operator. If the box fails to do an action, the test falls back to the notice for the operator, and the record says so (evidence.actions[].by, fixture_error). In an unattended run there is nobody to fall back to, and the test ends ERROR with the refusal named. The tool spaces two button presses: the next one waits for the end of the last pulse, plus a 300 ms release. Thus the controller sees the release between them, and not one long press.
• An operator test that the fixture can run alone runs unattended. A test declares its actions and, with hands=(...), what else it asks of a person ("app" for a job in the Glowforge app). An operator test whose actions the fixture covers, and whose hands are empty, goes into the unattended queue and out of the attended one. Its Ready gates pass, because the fixture does the timed step. A prompt that the test raises anyway is a FAIL that names the undeclared step. It is never a wait for nobody. A live test never moves: the fire watch and the acknowledgment belong to a person.
• The button channel needs the jumper. With the enable jumper of the fixture out, the button is not covered, and the tests that press it stay attended. The arm press of a live test belongs to a person unless the bench configuration says arm_press: true. Then, and only with the jumper in, the fixture presses when the button lights, and the record says so.
• What the box still holds after a run is released and recorded (evidence.fixture.released), before the post pass of the baseline. Thus a lid that a failed test left open never reaches the next test.

What keeps it safe¶

• Each line is driven low first thing at boot, before the radio or the server exist, and after each reset.
• The button is a pulse whose end is armed before the line rises. A hung task trips the task watchdog, which panics and reboots to all-low.
• The button channel needs the jumper. A leaked LAN key cannot press the button on a fixture whose jumper is out.
• The contacts in the loops are NC. A fixture fault can only add an open. It can never mask a real lid open or a pulled interlock.
• No OTA, no configuration over the network, and nothing persisted other than the own calibration of the wifi driver.

Layout¶

main/         the firmware: main.c, wifi.c, api.c, relays.c, policy.c (the pure decisions)
test/         the host test of policy.c (./fixture.sh test)
fixture.env.example, sdkconfig.defaults, dependencies.lock