Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learning

OmniReset

Source

• Raw Markdown: paper_omnireset-2026.md
• PDF: paper_omnireset-2026.pdf
• Preprint: arXiv 2603.15789v3
• Venue record: OpenReview forum, ICLR 2026
• Official project page: OmniReset
• Official code and documentation: UW-Lab/UWLab
• Official checkpoints: UW-Lab assets on Hugging Face
• Microsoft Research publication page: Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learning
• Social source that linked this paper: Curriculum Mining

Status And Credibility

OmniReset is a 2026 paper published at ICLR 2026, a tier-1 machine-learning venue. The rendered v3 paper is dated April 2, 2026 and carries the ICLR 2026 conference-paper banner. The team spans the University of Washington, NVIDIA, and Microsoft Research. The official project page, BSD-3-Clause UWLab code repository, documentation, and released state-policy checkpoints provide a substantial artifact surface.

This is strong, current evidence for diverse simulator resets in dexterous-manipulation policy training. It is not direct evidence for JEPA, learned world models, or multivariate time-series pretraining, and the code/checkpoints were not executed during this ingest.

Core Claim

OmniReset argues that long-horizon robotic reinforcement learning is often bottlenecked less by PPO itself than by narrow state-action coverage. Instead of hand-authoring a reverse curriculum or supplying demonstrations, it programmatically constructs a broad initial-state distribution over recurring manipulation interactions, then uses large-batch on-policy RL to discover task-specific behavior.

flowchart LR
  Goal["target object + goal configurations + workspace"]
  Resets["near-goal, grasped, near-object, reaching resets"]
  PPO["large-batch PPO over state observations"]
  Expert["state-based manipulation policy"]
  Data["80k simulated expert trajectories"]
  Distill["RGB policy distillation + visual/dynamics randomization"]
  Robot["zero-shot real-robot deployment"]

  Goal --> Resets --> PPO --> Expert --> Data --> Distill --> Robot

The important curriculum distinction is subtle. OmniReset says it needs no explicit ordered curriculum for simulation-only behavior generation, but its broad reset distribution induces an emergent competence order: PPO first solves near-goal states, then propagates value toward grasped, near-object, and reaching states.

Reset Construction

The user specifies three pieces of task structure:

1. the target object to manipulate;
2. a set of successful goal configurations;
3. the robot workspace.

OmniReset then generates candidate initial states across recurring interaction modes:

• Near goal: target object close to successful configurations.
• Grasped: robot initialized with stable grasps on the target object.
• Near object: end effector close to the object and potential contact points.
• Reaching: robot positioned farther from the object so the policy must solve the full approach-and-manipulate sequence.

The paper precomputes 1,000 feasible grasp points and generates near-goal offsets by perturbing the object away from successful configurations. This is task-agnostic in the sense that the same reset recipe is reused, but it is not structure-free: it needs an identified target object, valid goal configurations, a workspace, grasp generation, a physics simulator, and task/environment implementation.

Training And Evaluation Interface

The state-based actor receives five-step histories of robot state, object poses, and previous actions. The critic also receives privileged simulator parameters. PPO uses generalized state-dependent exploration noise, and the number of parallel environments is scaled together with the batch size. The project page summarizes this as 64K+ environments; the paper’s ablation says large parallel batches are essential for propagating competence from near-goal to full reaching states.

The paper evaluates easy and hard versions of peg insertion, leg twisting, drawer insertion, cube stacking, wall-slide/block-reorientation, and cupcake placement. The hard variants cover a much wider target-object and goal distribution. A four-leg table demo sequences four separately trained policies through a scripted high-level controller, so it is not one end-to-end policy solving the whole assembly.

The main simulation baselines receive 100 successful demonstrations:

• BC-PPO;
• DeepMimic-style reward augmentation;
• demonstration-based success-weighted reset curriculum.

OmniReset does not receive those demonstrations for state-policy training. The result figures report high success on hard tasks where the baselines make limited progress, plus better perturbation robustness and broader successful initial-state coverage. The paper’s main text does not provide one compact numerical table for all simulation curves, so the wiki should not invent aggregate margins from the plots.

Real-World Transfer

The learned state policies generate 80,000 simulated expert trajectories for each distilled visuomotor policy. The student uses three $224\times 224$ RGB views, a pretrained ResNet-18 encoder, a Gaussian MLP action head, and five-observation history. Transfer also depends on calibrated kinematics, controller matching, system identification, visual randomization, dynamics randomization, and action-space finetuning.

The paper reports:

Metric	Peg	Leg	Drawer
State RL success in simulation	94.97%	93.94%	89.60%
Distilled image-policy success in simulation	55.45%	43.69%	84.00%
Distilled image-policy success in the real world	85.37%	56.36%	15.38%
First-try real-world success	21.95%	43.64%	5.77%
Real-world throughput, successes/minute	1.49	0.63	0.31
Real-only BC on 100 demonstrations	2.44%	1.82%	1.92%

These numbers show both the strength and the remaining gap. The distilled policy can retry and recover, but first-try performance is much lower than eventual success, and Drawer transfer remains weak despite 84% image-policy success in simulation.

Reward And Curriculum Claim Audit

The official project page uses the shorthand No demos, no reward shaping, just RL. The paper supports a narrower claim: no task-specific reward tuning for the simulation behavior-generation experiments.

The actual fixed reward is:

$$r_t=r_{\mathrm{success}}+r_{\mathrm{dist}}+r_{\mathrm{reach}}+r_{\mathrm{smooth}}+r_{\mathrm{term}}.$$

It includes dense distance-to-goal and end-effector-to-object terms, action smoothness penalties, a sparse success reward, and unsafe-state penalties. The weights are shared across tasks, but this is still reward shaping in the ordinary technical sense. The reliable wording is therefore fixed task-agnostic shaping, not no reward shaping.

The paper also says that an action-space-reduction curriculum is used during sim-to-real finetuning. The authors explicitly separate that transfer curriculum from the simulation-only claim that diverse resets can learn high-performing state policies without an ordered behavior curriculum.

Why It Matters For Dynamic Curriculum JEPA

OmniReset adds a second curriculum control surface to Dynamic Curriculum Learning For JEPA. The current idea mostly asks which observed windows deserve compute. OmniReset asks which state neighborhoods and prediction starts must be exposed so long-horizon competence can emerge.

For a JEPA-style latent-state model, the adaptation path is:

state or trajectory coverage proposal
  -> candidate context/target pairs at several distances from known outcomes
  -> latent prediction over locally learnable transitions
  -> competence/coverage measurement by bucket
  -> expand toward longer horizons and harder state neighborhoods

This is directly usable only when valid states can be generated or reset. In offline time series, a sampler can vary observed context/target endpoints and horizon, but it cannot invent reachable system states without a trusted simulator or generator. The JEPA experiment should therefore distinguish:

• resampling real trajectory windows;
• resetting a known simulator;
• synthesizing candidate states with a learned model;
• validating that generated states remain reachable and preserve action/control-input history.

OmniReset also argues against a curriculum based on difficulty alone. Broad coverage and large batches matter because the learner must retain progress across several interaction modes while value propagates backward. A JEPA analogue should monitor regime coverage, normal-state retention, rare-state accessibility, and stale-score drift rather than only maximize latent surprise.

Relation To Curriculum Mining

Curriculum Mining is a same-day board-game project that explicitly cites OmniReset as inspiration. It turns broad resets into random legal mid-game positions and combines them with sparse-reward self-play PPO plus a frozen opponent pool. The board-game project’s no reward shaping claim applies to that project, not to OmniReset.

Together the two sources suggest a method class:

generate a broad set of valid intermediate starts
  -> let current learnability induce an emergent curriculum
  -> preserve coverage and stability while the competence frontier expands

The evidence remains domain-specific: ICLR robotics plus an unreviewed board-game project, not JEPA or multivariate time-series pretraining.

Limitations And Gotchas

• OmniReset is a policy-training and data-generation framework, not a learned action-conditioned world model.
• The simulation actor uses compact privileged object/robot state; RGB control is learned later through distillation.
• The method needs simulator access, target-object identity, goal configurations, workspace bounds, and a grasp sampler.
• Grasp quality can fail on complex non-convex objects; bimanual and dexterous-hand scaling remains open.
• The generic reward still contains dense shaping terms, despite simplified project-page wording.
• The sim-to-real stage uses action-space curriculum, system identification, controller matching, visual randomization, and dynamics randomization.
• Real-world Drawer success is 15.38%, and first-try success is low on Peg and Drawer.
• The four-leg task uses separately trained policies plus a scripted high-level controller.
• Released code/checkpoints improve reproducibility prospects, but they were not executed in this ingest.
• Direct transfer to offline multivariate time series or JEPA is a hypothesis.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Data diversity, curriculum, and long tail	partially closes outside time series	ICLR 2026 evidence that broad, structured reset distributions plus large-batch PPO can expose intermediate states and induce backward competence expansion without an explicit ordered simulation curriculum.	Needs JEPA/multivariate time-series experiments, support-preserving temporal-window policies, and validity checks for generated states.
Action-conditioned trajectories	adjacent	Trains policies over explicit robot state histories and control inputs in a simulator, then transfers distilled RGB policies to hardware.	It does not learn a general dynamics model or evaluate candidate control sequences through learned counterfactual rollout.
Data generation	partially closes outside time series	Turns state-policy experts into 80,000 simulated trajectories for visuomotor distillation and publishes code/checkpoints.	Needs corpus metadata, contamination/duplication accounting, data license details, and evidence that the recipe transfers across embodiments.
Latent-state prediction	insufficient evidence	The curriculum mechanism suggests how to select local versus long-horizon state-prediction problems.	No JEPA, latent dynamics objective, or representation probe is tested.
Benchmark hygiene	warning	Separates simulation state-policy success, image-policy simulation success, real-world eventual success, first-try success, and throughput.	Needs independent reproduction and clearer matched-resource accounting across demonstrations, simulator steps, reset generation, and distillation.

Links Into The Wiki

• OmniReset
• Curriculum Mining
• Dynamic Curriculum Learning For JEPA
• Robotics Time-Series Modeling
• Action-Conditioned Time-Series Datasets
• World Models
• Foundation Time-Series Model Research Agenda

Open Questions

• Can reset-state populations be scored jointly by learnability, novelty, coverage, and downstream utility instead of sampled from fixed mixtures?
• Which parts of OmniReset’s gain come from reset diversity, large batch size, generic reward shaping, gSDE, privileged critic inputs, or simulator scale?
• Can the method generate valid resets for deformable objects, dexterous hands, bimanual manipulation, and partially observed tasks?
• Can a JEPA curriculum vary context/target starts and horizons without deleting normal-state support or overselecting high-surprise corruption?
• When a learned world model proposes restart states, how should reachability and model exploitation be audited?
• Can released checkpoints and code reproduce the reported hard-task curves and real-world transfer under the documented hardware budgets?