Calibration Agent

Frozen qubit first. Drag drift severity to 0 and press Run. Both controllers start from the same wrong pulse. The scheduled sweep fixes it in one shot at step 0 — one expensive burst of 6,400 measurements — and then does nothing for 30 steps. The agent climbs out over five or six steps and then jitters just under 100%, paying 128 shots every step. With nothing drifting, the sweep wins on fidelity and the agent wins on cost.

Now let it drift. Set severity to 1 and run again. Watch the teal line between sweeps: it sags as the qubit walks away from the last calibration, then snaps back the instant a sweep lands. The violet line does not sag, and in the lower panels you can see why — the agent’s drive frequency is following the dashed truth almost step for step, without ever being handed it.

Then move the slider while it runs. Severity is live. Push it to 1 mid-run and the scheduled controller has no idea until its next appointment. Set the sweep interval to 15 to make it competitive on fidelity again, and look at what happens to its shot count.

The agent holds three numbers — pulse duration, amplitude and drive frequency — and a fixed exploration width for each. Every step it draws one random nudge ε, plays the pulse at +ε and at −ε, and scores each candidate with two 32-shot experiments: three back-to-back pulses (which should return the qubit to |1⟩, and amplifies any rotation error threefold) and a Ramsey pair at half amplitude (which is steep in frequency error). The score is just the fraction of |1⟩ outcomes, averaged. Then it moves the pulse along ε by an amount proportional to the difference between the two scores.

That is policy gradient with symmetric sampling — the update is, in expectation, a step up the gradient of the expected reward, and the ± pairing cancels the baseline so that 32-shot rewards carry a usable signal. There is no model of the qubit inside the agent and no fit. It is a bandit that has learned which way is up.

One constraint keeps it honest: amplitude × duration is an exact degeneracy of the rotation angle, so a pulse twice as long at half the amplitude scores the same. Left alone the agent random-walks along that ridge into low amplitudes, where the same frequency error costs far more fidelity. The reward therefore carries a penalty for a pulse that no longer fits its 40 ns slot in the circuit schedule — a constraint any real scheduler imposes anyway.

Every K steps: sixteen Ramsey delays at 200 shots each, with the drive deliberately detuned by 6 MHz so the fringe frequency reports the sign of the error as well as its size; fit the fringe, move the drive. Then sixteen amplitudes at 200 shots each on the new frequency; fit the Rabi period, put the π pulse at half of it. 6,400 shots, then silence.

It is the better calibration when it runs — with no noise it lands within 0.1 MHz and 1% of amplitude, and the tests check that. Its problem is the silence. Between sweeps it neither learns nor spends, and the fidelity it delivers is set entirely by how far the qubit can drift in K steps. Shorten K and you buy fidelity with shots; the amortised cost at K = 30 is already 213 per step against the agent’s 128.

In July 2026 Google reported holding gate fidelities on a Willow processor with a reinforcement-learning agent that adjusted more than a thousand control parameters continuously, driven by measurement statistics rather than by scheduled characterisation. The claim that matters is the shape of the loop: counts in, parameter deltas out, no intermediate model of the device, and cheaper than the sweep it replaced.

This page is that loop on one qubit with three parameters, a drift model made of a random walk and a slow sinusoid, and an agent small enough to read in a few minutes. It reproduces the qualitative result — a measurement-driven agent that tracks drift at lower shot cost than periodic recalibration — on a system more than three hundred times smaller, with a reward the agent can evaluate in two experiments. On Willow the reward is a whole error-correction cycle’s logical error rate, the parameters interact through crosstalk, and the exploration budget is the real engineering problem. None of that is here.

It does not simulate decoherence, leakage to |2⟩, pulse shaping, DRAG, readout drift or crosstalk. The pulse is a rectangular rotation in the rotating-wave approximation and the only errors are a wrong angle and a wrong axis. Readout has a fixed 1% symmetric assignment error and nothing else.

It does not train a neural network. The agent is a Gaussian policy over three deltas with fixed exploration widths; there is no value function and no learned representation. It does not tune more than one qubit, does not learn the drift, and does not choose its own experiments. It cannot be exported to hardware — the pulse units are the simulation’s own.

What it does do is checked: lib/workbench/calibration.test.ts pins the gate fidelity to the average-fidelity formula and to the closed-form Rabi solution from the spin-precession lab, the sampled Rabi counts to the analytic sin² curve at every amplitude, the Ramsey fringe to (1 + cos Δτ)/2, the scheduled sweep to a known hidden offset, and both controllers to a 99% floor with the qubit frozen — from the same deliberately wrong start you see here.