Name: Moonlight Sorting
Uploaded: 2023-09-09T00:00:00.000Z
Duration: 5 min 4 s
Description: Five minutes of a reinforcement learning agent training on a sorting task, built the way the Unity ML-Agents examples are built. The agent is not told the rule. It finds it by repetition.

Five minutes of a training run from September 2023. Machine learning, in the Unity ML-Agents mould: an environment, an agent inside it, a reward, and enough repetitions that a policy falls out the other end.

Nothing here is told what to do. The agent is not handed a sorting rule and asked to apply it. It is dropped into the task, scored on the outcome, and left running until the behavior that scores well is the behavior it reaches for first. What you are watching is that gap closing.

Small piece of work, and it earns its place for one reason. Reinforcement learning is the clearest demonstration there is that a system can arrive at a competent policy nobody wrote down. That is the property that makes an agent useful, and it is the same property that makes one worth instrumenting. Every behavioral monitor I have built since is downstream of watching runs like this one.