Create / Understand / 2024
Teaching noisy microrobots to move cargo together.
THE CENTRAL QUESTION
When a team of robots moves one object, success belongs to the group. That creates a learning problem: a robot can receive the same reward whether it pushed in the right direction, obstructed its neighbours or did nothing. At the microscale, random motion and collisions make the connection between one action and the final result even harder to see. The project introduces a reward that estimates each robot's marginal contribution.
Imagine several people pushing a table while everyone receives one score for the final motion. The score does not tell an individual whether a particular push helped. In reinforcement learning this is called credit assignment, and it becomes more severe as the group grows or the physical system becomes noisy.
After an action, a short simulation estimates how the group would have performed if one robot had been removed. The difference between the real outcome and this imagined alternative becomes that robot's reward. The method does not prescribe a pushing pattern; it supplies a more informative signal from which coordinated behaviour can emerge.
The experiments use light-controlled Janus particles about six micrometres wide to manipulate a rod roughly 100 micrometres long. Each particle sees only compact local information and chooses among a few motion commands. Rotation was learned directly in the experiment, while the more demanding target-transport policy was trained in simulation and transferred to the physical system.
Counterfactual rewards produced faster learning and better final performance than a single team reward or a hand-designed torque reward. For target transport, the policy discovered a staged strategy: translate the rod efficiently, then rotate it near the destination. It exceeded 90 percent success within the reported action budget and remained useful when some commands were randomized.
The work shows that better credit assignment can produce robust collective strategies without scripting every robot. Tests also explored larger swarms, including rotation with up to 200 agents. The platform still relies on external imaging and laser control in two dimensions, so it is a model for collective control rather than an immediately deployable in vivo swarm.