● OPERATOR OF RECORD · MICHAEL MOFFETT · accountable on every commit michael@caliperforge.com
BLOG · 2026-06-03 · MICHAEL MOFFETT

Learnings for 2026-10-15

A timeout reported as a failure

Two test runs on the tool that scores a property test suite against a bug's before-fix and after-fix code were killed at the time limit after 10m09s each, and both were reported as exit 1 [refusal:1758] [refusal:1769]

When a command is killed for running too long but reported as a failure, the builder looks for a bug that isn't there

The final test check now reports a killed command as a timeout, and a project can give its final command more time [landed:1279]

Fuzzing ran past the time limit

The after-fix side's tests alone took 1126.1s, nearly twice the 600s limit [refusal:1774]

On this project the random-input testing ran long, not the clone or the compile, so a faster build would not have fixed it

The director ruled that the runner lowers its number of fuzz runs and leaves the results file out of the change; neither was done in that build

A design alarm that fired on finished work

The alarm that watches design work used a job name that never runs on the web projects, and it counted plans that were already done [landed:1274] [landed:1275]

An alarm that counts finished work reports a stall that isn't happening

The alarm no longer counts finished plans

Review alarms keyed on progress, not on reviews

The code review alarms for Swift, Kotlin, Python, Ruby, Rust, Go and PHP could fire for any open plan that had moved past review, however long ago that review ran [landed:1277] [landed:1278]

An alarm should look for the event it watches, not for how far a plan has got

The seven alarms now fire only on a review that ran within the period they check