If we want one model to co-design
any
robot, the network needs a shared vocabulary that spans
every embodiment. RoboTokens are that vocabulary: typed tokens for every link, joint, motor, state, and
action — organized so a single sequence describes both a robot’s
embodiment
(time-invariant) and
its
dynamics
(time-varying).
RoboTokens has the following key design goals:
Complete.
A RoboToken sequence captures everything about a rigid articulated robot: 5
embodiment token types (links, fixed joints, rotating/sliding joints, ball joints, motors) plus state
and action tokens.
Flexible.
Tokens reference each other by ID — a joint token knows the two link tokens
it connects — so the same format handles a 6-DoF arm or a 35-DoF biped.
Consistent.
The tokenizer canonicalizes redundant spatial offsets and stores inertial /
transform data in a single frame, reducing the variance the model has to learn through.
Extensible.
New token types slot in without changing the architecture — we add
target end-effector pose tokens for trajectory tracking by pointing them at the relevant end-effector
and timestep IDs, no retokenization of the rest of the robot required.
Optimizable.
Unlike autoregressive text formats like MJCF, diffusing continuous-valued
RoboTokens gives both
global controllability
(later tokens influence earlier ones across
denoising steps) and
differentiability
— the two ingredients needed for gradient-based
reward optimization at inference time.
