This randomized benchmarking tutorial shows how to test whether a quantum backend is executing the gates your circuit depends on. You will build Clifford sequences, append their inverse, measure survival probability, fit decay, and compare standard, interleaved, and simultaneous randomized benchmarking on simulators and hardware. The workflow is designed to be rerun after calibration changes, software migrations, or hardware comparisons.
Overview
Randomized benchmarking (RB) estimates how errors accumulate as a quantum circuit becomes longer. Instead of testing one gate with a single fixed input, it samples many randomly chosen Clifford circuits and checks whether the final state returns to a known reference state. The resulting decay is less sensitive to state-preparation and measurement errors than a simple one-gate success test, although it does not eliminate those errors.
For a single-qubit experiment, the usual procedure is:
- Prepare a reference state, commonly
|0>. - Generate a sequence of
mrandom Clifford operations. - Calculate and append an inverse Clifford so the ideal total operation is the identity.
- Measure the probability of returning to the reference state.
- Repeat the experiment for several sequence lengths and fit the decay.
If the device were noiseless and the circuits were represented exactly, the survival probability would remain near one. On a noisy device, it generally decreases as m increases. The decay is a useful device-characterization signal, but it is not a complete description of every error mechanism. Leakage, drift, crosstalk, coherent over-rotations, compilation choices, and measurement errors can all affect the result.
RB is especially useful when comparing qubits, gate families, calibration periods, or backend configurations. It should be treated as a controlled experiment rather than a single permanent score. Record the circuit-generation settings, compiler behavior, qubits, shots, timestamp, and backend configuration so that a later run remains comparable.
What to track
1. The sequence design
Choose a set of sequence lengths that spans short circuits and the region where decay becomes visible. A practical starting design uses several lengths distributed across a roughly logarithmic or otherwise well-spread range. Avoid choosing only very short sequences: the change may be smaller than shot noise. Avoid choosing only very long sequences: the survival probability can approach its baseline and provide little additional information.
For each length, generate multiple independently randomized sequences. Keep the random seeds or sequence descriptions. Reusing the same circuits makes it harder to distinguish a device effect from an accidental property of one sequence. The number of sequences and shots per sequence should be increased when the decay is noisy, when comparing close results, or when the experiment is being used for a publication-quality estimate.
2. Clifford and native-gate details
A Clifford operation is usually compiled into the backend's native gates before execution. Therefore, standard RB primarily reports an average error associated with the compiled Clifford implementation, not the error of one native pulse in isolation. Track the Clifford-to-native decomposition, optimization settings, basis gates, qubit mapping, and whether adjacent operations were cancelled or merged.
The inverse must be calculated from the composed Clifford operation, not by reversing the list and replacing each Clifford with an informal inverse. In a framework-neutral workflow, maintain a representation of the net Clifford action, compute its mathematical inverse, and then append a circuit implementing that inverse. Verify the ideal circuit independently: with noise disabled, every generated sequence should return the reference state, subject only to simulator precision and measurement conventions.
3. The measured data
Save raw counts as well as calculated survival probabilities. For a reference state of |0>, survival probability is typically the fraction of shots measured as zero. For multiple qubits, define clearly whether survival means the all-zero bit string, a parity condition, or another projector. Do not silently change this definition between runs.
A common decay model is:
F(m) = A p^m + B
Here, m is the sequence length, p is the fitted decay parameter, and A and B absorb state-preparation, measurement, and long-sequence baseline effects. For a Hilbert-space dimension d, a commonly used estimate of the average error per Clifford is (d - 1)(1 - p) / d. For a single qubit, d = 2; for a two-qubit subsystem, d = 4. Treat this estimate as model-dependent, especially when errors are non-Markovian or strongly gate-dependent.
4. The comparison type
| Method | What changes | What it is useful for | Main caution |
|---|---|---|---|
| Standard RB | Random Clifford sequences with an inverse | Estimating average compiled-Clifford performance and comparing qubits | The result combines native gates used in the compilation |
| Interleaved RB | A target gate is inserted between random Cliffords | Estimating the performance of a particular gate relative to a reference RB experiment | Gate-dependent and coherent errors can make simple ratio formulas approximate |
| Simultaneous RB | RB sequences run on multiple qubits or subsystems at once | Detecting crosstalk or performance changes caused by parallel activity | Qubit selection, scheduling, and readout effects must be controlled |
For interleaved RB, compare the reference decay parameter with the interleaved decay parameter rather than interpreting the interleaved curve alone. The inferred target-gate error depends on the chosen model and on how the gate is compiled. Report the reference experiment, interleaved experiment, confidence interval or fit uncertainty, and the assumptions used in the conversion.
Cadence and checkpoints
Run a baseline RB experiment before making a hardware or software change. Then rerun it after changes that could affect gate execution, including a calibration update, a new qubit assignment, a compiler or transpiler migration, a modified optimization level, or a change in pulse or scheduling settings. A monthly or quarterly check can be useful for a stable development process, but the right cadence depends on how often the backend and workload change.
Use the same protocol for routine tracking:
- Keep qubit selection and connectivity fixed when measuring drift.
- Keep sequence lengths, randomization count, shots, and fitting bounds consistent.
- Record the backend identifier, software versions, execution date, calibration context, and queue or scheduling conditions where available.
- Include a small ideal-simulator check to catch inverse-circuit or measurement-definition bugs.
- Use a noisy simulator only as a model check, not as evidence of hardware quality.
In an ideal statevector simulator, the inverse construction should produce no decay. A noisy simulator should reproduce the qualitative behavior of its configured error model. Hardware results add sampling uncertainty and device variation, so compare distributions or fit uncertainty instead of treating one fitted number as exact.
How to interpret changes
A lower estimated error per Clifford is not automatically proof that every gate improved. First check whether the compiled circuits changed. A transpiler may use a different native-gate decomposition, remap the logical qubit, cancel operations, or alter the schedule. Those changes can improve or worsen RB without changing the underlying physical gate in isolation.
Also inspect the shape of the decay. A reasonably smooth exponential supports the basic RB model. Curvature, large sequence-to-sequence variation, or a poor fit can indicate drift, leakage, nonstationary noise, coherent errors, insufficient randomization, or an unsuitable length range. Do not hide these signals by forcing a visually convenient fit. Repeat the experiment, inspect residuals, and compare results across shorter time windows when the device may be changing during execution.
When comparing qubits, use matched protocols and avoid ranking devices from differences smaller than the experiment's uncertainty. When comparing native gate families, remember that standard RB measures the entire compiled Clifford. Interleaved RB can isolate a target operation more directly, but it should be paired with a reference run and interpreted with its assumptions stated.
Simultaneous RB is valuable when a circuit will operate gates in parallel. If individual RB looks stable but simultaneous RB degrades, investigate crosstalk, shared control resources, frequency collisions, scheduling, or readout interactions. This result is often more relevant to a multi-qubit workload than an isolated single-qubit score.
For an application such as a VQE workflow from simulator to hardware, use RB as one diagnostic alongside application-level baselines. RB can reveal that a backend changed, but it cannot by itself predict the accuracy of a deep variational circuit. Pair it with circuit-level tests, noise-aware simulation, and, where appropriate, the techniques discussed in this quantum error mitigation workflow.
When to revisit
Revisit this protocol on a scheduled monthly or quarterly basis when you are tracking a production-like backend, and immediately after a calibration, compiler, SDK, mapping, or pulse-schedule change. Update the implementation when your quantum software framework changes its Clifford-generation, transpilation, backend-execution, or result-format APIs. A practical migration checklist is available in the quantum SDK version compatibility guide.
Before each run, confirm five things: the inverse circuit returns the reference state in an ideal simulator; the selected qubits and gate set match the intended comparison; raw counts are stored; the fit model and uncertainty method are documented; and the result is compared with a previous run using the same definitions. If any of those conditions changes, label the measurement as a new series rather than silently appending it to the old one.
Finally, keep a small benchmark log containing the date, backend, qubits, protocol, sequence lengths, number of random sequences, shots, fitted parameters, uncertainty, compiler settings, and notes about calibration. That log turns randomized benchmarking from a one-time demonstration into a repeatable way to monitor quantum gate quality as simulators, hardware, and developer tools evolve.