Small enough to fit
The deployed network is 24 → 16 → 16 → 1 — 689 parameters in 2756 bytes of flash. A forward pass takes about 8 ms on the ATmega328, inside a 20.0 ms control tick.
A pendulum that teaches itself to balance
Print it, solder it, and train it at home for about £20. The controller is a
689-parameter network that learned to swing up and balance — trained in
simulation, fine-tuned on the real rig, then distilled small enough to run
standalone on an Arduino Nano.
Loading…
The five inputs the 689-parameter network reads, and the one action it writes — 50 times a second. Each trace is 1.5 s of history; the shaded sliver on the right is the 4-frame stack it is reading now (80 ms). Everything left of it the policy has already forgotten.
24 → 16 → 16 → 1 · 689 parameters · 2756 bytes of flash · 50 Hz — the same weights the Arduino is flashed with.
The panel above is not an animation. It loads the physics engine into your
browser and runs the exact weights that are flashed on the Nano — the same
policy_weights.h that arduino-cli compiles — through the same control law
the firmware executes. Drag the pendulum, or switch control off and watch it
fall: whatever state you leave it in, the policy has to recover from one nobody
scripted.
Why this is interesting
Section titled “Why this is interesting”A rotary inverted pendulum is the standard testbed for control theory: an unstable equilibrium you can only hold by moving the thing underneath it. Commercial teaching rigs cost thousands — a Quanser QUBE Servo 2 lists at around £4,500. This one is printed plastic, a NEMA17 stepper, a magnetic encoder and an Arduino Nano.
The interesting part is what runs on it. Not a hand-tuned PID, but a policy that was trained by reinforcement learning and then squeezed until it fit on an 8-bit microcontroller with 2 KB of RAM.
Trained where it is cheap
A large teacher learns in simulation in about 25 minutes on a laptop, across three stages of increasing domain randomisation.
Fine-tuned where it counts
Simulation alone does not transfer. Roughly 30–80 real-rig episodes close the gap, with the physical pendulum in the loop.
Distilled, not shrunk
Training a tiny network directly with RL failed twice on hardware. Imitating the big teacher works — the student’s slight underfit acts as gain reduction and buys robustness.
Start here
Section titled “Start here”Build the rigBill of materials, printing, assembly, wiring and first power-on.
Train and deploy a policyThe end-to-end pipeline, from measuring your rig to a pendulum that balances with no laptop attached.
How it worksThe transition contract, domain randomisation, and the transport-delay work that made deployment possible.
Watch the buildA video walkthrough of the mechanical build.