Action Space Efficiency in Robot Learning

Summer 2026

One narrow research question

This independent summer research project asked one question: under one MuJoCo pickup task, one controller integration, and one fixed behavioral-cloning family, which action representation learned more efficiently from demonstrations?

joint_delta commands small changes at each robot joint. ee_tool_delta commands small changes in the gripper tool position. The robot, task, controller, training method, and evaluation rules stayed fixed.

The project tested a focused robot-learning question. It did not test a complete vision-language-action system.

Primary result

Joint actions used the data more efficiently

Joint-delta produced a mean normalized success AUC of 0.624. End-effector tool-delta produced 0.370. The paired difference was 0.254, with a crossed-factor 95% interval from 0.149 to 0.353.

• Direction All 15 ladder-by-seed AUC differences favored joint-delta.
• Integrity The full 150-cell and 3,600-rollout gate passed.
• Check An independent calculation matched the primary values within 5.55e-17.
• Scope This is evidence for the tested design, not a universal ranking of robot action spaces.

Why controller first

Separate the system before judging the policy

A failed robot rollout does not automatically mean that the learned policy failed. The simulator, controller, action labels, gripper logic, or evaluator can also be wrong.

I first built and checked the SO-101 scene, a damped-least-squares Cartesian controller, both action adapters, scripted pickup behavior, strict event-order gates, and physical-sanity checks. I then replayed the recorded policy labels before I trained a model.

This order found simulator contact defects and invalid raw transition labels before they could be mistaken for learning results.

Evidence ladder

Each test answered a different question

Scripted success proved that the task was executable. Label replay proved that the recorded commands were executable. Neither result counted as learned-policy success.

Registered method

150 policy fits across five data budgets

The study used five demonstration budgets: 6, 12, 18, 24, and 30. It crossed those budgets with three immutable nested data ladders, five model seeds, and two action spaces.

• Cells 150 trained policy conditions.
• Trials 24 development rollouts per cell, for 3,600 total.
• Policy A frozen hybrid nearest-neighbor gripper and MLP arm policy.
• Endpoint Normalized success AUC against distinct demonstration count.
• Uncertainty A 10,000-replicate crossed-factor paired bootstrap.

The analysis treated ladders and model seeds as separate correlated factors. It did not treat the 15 pairs as 15 independent samples.

Failure analysis

The difference was mainly in event order

Joint-delta succeeded on 1,042 of 1,800 rollouts. End-effector tool-delta succeeded on 626 of 1,800. The end-effector policy had 845 event-order failures, compared with 549 for joint-delta. It also had more contact-dynamics failures.

Both policies were weak at six demonstrations. The curves were not monotonic. Joint success fell from 24 to 30 demonstrations, and end-effector success also varied. The AUC result integrates the full observed curve. It does not assume that more data always improved a policy.

The lower end-effector infeasible-step rate did not produce higher success. Controller constraint exposure alone did not explain the result.

Engineering and verification

A research system with fail-closed checks

The completed prototype includes the MuJoCo SO-101 arm, pickup and pick-place tasks, two action adapters, scripted demonstrations, action-label replay, state-based behavioral cloning, fixed-camera RGB data plumbing, and run provenance records.

A deterministic finalizer checked the protocol, cell manifests, trial rows, file hashes, and result shape before it produced CSV, Markdown, JSON, and SVG artifacts. The focused finalizer and protocol suite passed 37 tests. The full repository suite passed 205 tests.

The registered run exited cleanly with 150 cell manifests and 3,600 trial rows. A second implementation recomputed the headline statistics from the cell summaries.

Research judgment

Negative results stopped unproductive tuning

Raw observed end-effector transitions were about 30 times smaller than executable command labels and failed replay at 0/18. Lower end-effector action gains reduced performance. A privileged gripper state machine also failed to rescue the learned end-effector arm.

Global gripper weighting and a hybrid nearest-neighbor gripper improved both policies, but they did not remove the joint advantage. These results closed the pickup rescue program. More gain, loss, label, match, cap, or gripper tuning was not justified.

The stop decision was part of the result. It kept the project from turning into an open-ended search for one favorable run.

Limits and closeout

A completed prototype, not a universal claim

The result covers one simulated pickup task, one SO-101 model, one damped-least-squares controller integration, three data ladders, five model seeds, and one frozen state-based policy family.

It does not include hardware validation, learned pick-place, controller replication, a vision-conditioned policy, language input, or a complete VLA policy. Locked evaluation and the earlier protocol final holdout stayed closed.

The project is complete because it answered its registered question with verified evidence. Future controller replication, vision learning, or VLA work must start as separate research projects with new claims and new evaluation contracts.