# Curriculum Mining ## Provenance - Source type: author technical blog, code/checkpoint release, and X thread; not a paper. - Author: Matteo Peluso (`@MozarellaPesto`, GitHub `the-puzzler`). - Blog: - Code: - X root supplied by Alex: - Canonical X root: - X timestamp: 2026-07-26T18:25:33Z. - Code revision: `853743229b8ef842678327b26ad5a5963049306d`, committed 2026-07-26T18:37:20Z. - Local pinned code archive: `source_code_853743229b8ef842678327b26ad5a5963049306d.tar.gz`. - Snapshot date: 2026-07-26. - Local artifacts: blog redirect and article HTML/Markdown, pinned code README and source archive, X API root/thread/media snapshots, normalized thread Markdown, and supporting visual assets. No repository license file was visible at the pinned code revision. Treat source-code reuse terms as unspecified until the author adds a license. ## Source Status This is a current, inspectable weekend project with public code, released checkpoints, result tables, and verification scripts. It is not peer reviewed, has no independent replication verified here, and characterizes training variance with only one training run per game. The source is credible as evidence of what the author implemented and reported, but not as a settled board-game-training result. The code and checkpoints were not executed during this ingest, so reproducibility remains unassessed. ## Exact Social Claim The X root says: > I trained policies that beat Pgx’s published Gumbel AlphaZero policy networks on Go 9×9, Othello and Hex! > > No MCTS. No teacher. No reward shaping. > > Only self-play PPO with sparse terminal rewards, using comparable or fewer environment interactions. A same-author follow-up narrows the comparison: the learned policies beat the **raw published policy networks**, not AlphaZero with MCTS at inference. ## Method The project holds the policy architecture fixed to a Flax port of Pgx's published `AZNet v0` and changes the training objective from Gumbel AlphaZero search-improved targets to self-play PPO with sparse terminal $\pm 1$ rewards. The release says the port has 1,793,995 trainable parameters and reproduces the published baseline outputs bit-exactly; this ingest did not rerun that check. Two mechanisms are central: 1. **Random legal mid-game resets.** After termination, the environment starts from a position generated by up to 40–50 random legal plies from the opening. This exposes PPO to short, initially solvable credit-assignment problems. The claimed emergent order is near-terminal competence first, followed by backward value propagation toward earlier game states. 2. **Opponent checkpoint pool.** Half of the games use an opponent sampled from a 12-policy frozen ring, updated every 100 iterations, and the loss is applied only to learner moves. The author reports that this prevents the peak-and-drift instability seen in naive self-play, but the release does not provide a controlled ablation isolating the pool's contribution. The author calls the reset mechanism **curriculum mining**: the reset distribution proposes many candidate intermediate problems, while the current learner determines which ones are learnable enough to generate useful updates. The curriculum is not a hand-authored ordering, but it is still induced by a deliberately chosen state-generation process. ## Reported Results The pinned README reports greedy raw-policy head-to-heads over 10,240 games per comparison: | Game | Score versus published raw baseline | W / D / L | |---|---:|---:| | Othello 8×8 | $0.671 \pm 0.005$ | 66.0 / 2.3 / 31.8 | | Othello, compute-matched checkpoint | $0.579 \pm 0.005$ | 56.8 / 2.1 / 41.0 | | Hex 11×11 | $0.835 \pm 0.004$ | 83.5 / – / 16.5 | | Go 9×9 | $0.817 \pm 0.004$ | 81.7 / – / 18.3 | The source says each evaluation uses five seeds, 1,024 random openings, and both seat assignments. Training uses one run (`seed 0`) per game, so the table does not characterize training-run variance. ## Compute Accounting Audit The release counts every simulator transition used by MCTS as an environment interaction: ```text Curriculum Mining final policy: 8,000 iterations × 4,096 environments × 32 steps = 1,048,576,000 interactions Pgx Othello / Hex published baseline: 100 iterations × 1,024 environments × 256 frames × (32 MCTS simulations + 1 played move) = 865,075,200 interactions Pgx Go 9×9 published baseline: 200 iterations × 1,024 environments × 256 frames × 33 = 1,730,150,400 interactions ``` Under this accounting, the final PPO run uses about 1.21 times the Othello/Hex interaction count and 0.61 times the Go count. The lower-cost Othello iteration-3000 checkpoint uses 393M interactions and still scores 0.579. Thus `comparable or fewer` is game- and checkpoint-dependent rather than one universal reduction. This is **environment-interaction accounting**, not a matched total-training-compute result. It does not equate neural-network forward/backward FLOPs, accelerator utilization, wall-clock time, memory traffic, hyperparameter-search cost, or the value of an MCTS simulator step versus a PPO rollout/update. The baseline networks are Pgx reference models from the NeurIPS 2023 Pgx paper, whose authors explicitly did not present them as state of the art. ## Link To OmniReset The thread says the reset idea was directly inspired by [OmniReset](https://arxiv.org/abs/2603.15789), an ICLR 2026 robotics method. The shared mechanism is **broad initial-state coverage** that makes long-horizon learning look like a sequence of shorter problems. The sources must remain separate: - Curriculum Mining samples random legal board states, uses sparse terminal rewards, and evaluates raw policy networks in finite perfect-information games. - OmniReset constructs manipulation-centric simulator reset distributions and uses large-batch PPO with a fixed, task-agnostic reward containing dense reach/distance/smoothness terms plus sparse success and safety terms. - Curriculum Mining also adds an opponent checkpoint pool, which is not the OmniReset mechanism. ## Transfer To Dynamic Curriculum JEPA The durable transfer is not `use PPO for JEPA`. It is to expand curriculum design from **scoring observed examples** to **controlling where temporal prediction problems begin**. For JEPA or latent-state time-series pretraining, an analogue could be: ```text trajectory or event stream -> propose context/target start points and horizon lengths -> evaluate current learnability, novelty, support coverage, and gradient value -> sample a mixture of solvable local transitions and harder frontier transitions -> refresh the frontier as the representation improves ``` This creates two complementary curriculum axes: 1. **Example-value curriculum:** reweight existing windows by latent surprise, gradient alignment, downstream probe value, or another training-effect signal. 2. **Reset/start-state curriculum:** choose context endpoints, target start points, horizon lengths, or simulator restart states so the model sees intermediate predictive problems that bridge toward longer-horizon state prediction. In offline time series, the method cannot reset the real system; it can only resample valid observed windows or use a trusted simulator/generative model. Any synthetic reset must preserve temporal validity, causal order, exogenous variables, action/control-input history, and rare-regime support. In an action-conditioned simulator, stronger population-based extensions could maintain candidate restart states and optimize a joint criterion over learnability, novelty, state-space coverage, and downstream utility. The thread's POET/quality-diversity/dominated-novelty-search suggestion is a useful hypothesis for this branch, not current evidence. ## Limitations - Not peer reviewed and not independently replicated in this ingest. - One training run per game; evaluation variation does not substitute for training-seed variation. - Published Pgx raw networks are reference baselines, not current state of the art. - Raw-policy versus raw-policy evaluation does not test AlphaZero's usual MCTS-assisted playing regime. - The compute comparison counts simulator interactions but does not match total FLOPs or wall-clock cost. - The checkpoint-pool contribution is described as essential but lacks a public controlled ablation in the captured blog/README. - Random legal-state generation is especially convenient in finite board games; arbitrary event streams, physical trajectories, and real systems may not admit valid reset states. - Informal GRPO/generalization claims in the X replies are not reported experiments. - No code license file was visible at the pinned revision. - Engagement metrics from X are not technical evidence.