Replicator A sound-to-synth bench with an ear-judged golden set

A first attempt fitted a handful of short sound pieces by optimisation. It failed on a recording that was mostly flat broadband noise: the synth’s noise burst faded long before the sound did, and searching harder cannot fix a vocabulary that lacks the shape.

I rebuilt it around measurement. The recording passes through a bank of 16 log-spaced, roughly third-octave bands, from 300 Hz up to the recording’s own top frequency. The bank uses the same RBJ biquad coefficients as Web Audio’s bandpass filter, so the analysis matches the synth’s own filters. Each band’s envelope becomes a noise step, each confirmed resonance a sine, and each hard onset a 3 ms click.

Then the loop closes. The recipe is rendered through the real synth class into an OfflineAudioContext, read again through the same bank, and its gains and envelopes are corrected toward the recording; the best pass is kept. The copy button hands out exactly what was scored, to four significant figures and 10 µs.

Reading a tone through a narrow band gives a false picture. The filter delays and smears the onset (by about 10 ms at 400 Hz) and adds its own ringing to every decay, so a beep that switches on instantly reads as a 40 ms rise, and a resonance seems to ring longer than it does.

Rather than model that bias, the bench cancels it. It renders a solo probe tone, reads it through the identical band in the identical way, and, in two passes, moves the probe’s onset, offset, attack and level onto the recording’s reading. Decay times are corrected the same way. Whatever the filter does, it does to both.

The same idea decides what counts as a resonance: an impulse through that exact band gives the band’s own ring time, and a candidate must ring at least 1.5 times longer, and for at least 20 ms. Peaks are interpolated between FFT bins, because bin centres alone were up to ±11.7 Hz off, which is audibly out of tune at 500 Hz.

The floor is the same distance measured between two plays of the same recipe on different noise seeds: how far the reading moves when nothing changes but the noise. On the shutter, in the 12 Oct 2026 runs, noise alone moved the reading 1.76 dB and the recipe sat 1.71 dB from the recording: inside the floor.

The band score had a blind spot. At third-octave resolution, a held tone and a band of noise with the same envelope look alike, so I added a second, 1/12-octave “fine” score. For recordings with a low end it uses an 8,192-point transform below 300 Hz, because 2,048-point bins lumped a 32 Hz buzz in with its harmonics. The fine score reports and gates, but it never steers the calibration.

Two diagnostics show where a recipe differs: a per-band residual for the first 60 ms and for the rest, and an explain mode that mutes each piece in turn to rank which ones make a chosen window too loud. For the listening test itself there is a blind X/Y test: a coin flip hides which button plays the recording, and a tally counts correct guesses.

Beeps need sine tones and a shutter needs noise. The obvious design is a classifier that labels each spectral peak, and three were tried. The shutter’s noise passed all three.

So every file is now fitted in full several ways: plain noise bands; short tones from a fast tracker on a 5 ms hop; steady partials from an 8,192-point tracker at two prominence gates; and merges of these. The 8,192-point tracker can hold two tones 12 Hz apart as a beating pair. A tone set survives only if it improves the fine score by at least 0.25 dB and at least 15%, without costing more than 0.25 dB of band score; the closest survivor wins. A file that keeps tones mutes its clicks, then tries removing each noise band, quietest first, and keeps a removal only if the fine score does not rise and the band score stays within 0.25 dB of where the pass began.

In the 12 Oct 2026 runs, 7 of the 10 test recordings kept held tones and 3 did not, each by its own measurement. The shutter rejected all five tone sets: the best moved its fine score about 5%, short of the 15% bar. An interface beep kept six steady tones and finished in 8 steps.

Renders are deterministic: Math.random is swapped for a seeded generator around the synchronous play call, so rerunning unchanged code on an unchanged recording reproduces every score to within 10⁻⁶ dB.

The batch runner starts its own dev server and drives the bench’s own page in headless Chrome, so no second copy of the measurement can fall out of step. Each recipe is scored at four compaction rungs, the tolerances (0.5 to 4 dB) to which its envelopes are simplified. The run fails if any file fails, or if any accepted file is more than 0.05 dB worse on either score at any rung. A missing recording counts as a regression, so deleting a recording cannot remove its guard. Files join the set only by name, through a script that demands a written reason and, whenever it replaces a baseline, prints the old scores beside the new; there is no accept-all.

That is the shape of an evaluation harness for a generative system: a human-labelled golden set, a deterministic scorer with a stated noise reference, and a hard regression gate. It is also what makes the fitting code safe to hand to the AI coding agents I direct: a change that makes an accepted sound worse fails the run.