Alex Open Research Wiki

Param Decomp

3 min read

Param Decomp

Summary

Param Decomp is Goodfire’s public code repository for parameter decomposition on neural networks. It contains the implementation lineage used by Goodfire’s SPD and VPD work, including the VPD code release for Interpreting Language Model Parameters and the compact nano_param_decomp/ implementation noted in the README.

Official Artifacts

What It Provides

The README describes the repository as training tools for parameter decomposition on neural networks. It exposes two Python distributions:

  • param-decomp, the core library imported as param_decomp;
  • param-decomp-lab, in-repo experiments, app, postprocessing, and CLI tooling imported as param_decomp_lab.

It also provides installed pd-* commands for experiments:

  • pd-tms for toy-model-of-superposition experiments;
  • pd-resid-mlp for residual MLP experiments;
  • pd-lm for language-model experiments.

The source snapshot records the repository as public, MIT-licensed, and owned by the goodfire-ai GitHub organization. The root pyproject.toml targets Python 3.13 and uses a uv workspace for the lab package.

Method Lineage

flowchart LR
  APD[Attribution-based Parameter Decomposition] --> SPD[Stochastic Parameter Decomposition]
  SPD --> VPD[adVersarial Parameter Decomposition]
  VPD --> Repo[goodfire-ai/param-decomp]
  Repo --> Edit[Scalar component-basis model edits]
  • SPD makes parameter decomposition more scalable and robust than APD by learning which components can be removed on each datapoint rather than relying on gradient-attribution proxies.
  • VPD extends the line to a 67M-parameter language model, adds adversarial ablation pressure for mechanistic faithfulness, decomposes attention-layer computations, and supports attribution graphs and hand-written edits.
  • The German-removal demo uses VPD-derived subcomponents as a component basis for post-hoc scalar fine-tuning.

Why It Matters For The Wiki

Param Decomp is an implementation handle for weight-space interpretability. It is relevant to the wiki when a source asks whether adaptation should happen through:

  • adding adapter capacity such as LoRA;
  • sparse or dense full-parameter updates;
  • fast editable context;
  • synthetic data that indirectly steers weights; or
  • explicit decomposed parameter components.

The key difference is that Param Decomp tries to learn a component basis with causal semantics before editing. If that basis is trustworthy, the edit can be lower-dimensional and more inspectable than a fresh adapter. If the basis is incomplete or mislabeled, the edit can still have hidden collateral effects, as the German-component edit’s Italian damage shows.

Relation To Foundation TSFM Agenda

This is not a time-series entity. It is adjacent tooling for the Foundation Time-Series Model Research Agenda because it suggests a way to expose and edit learned mechanisms in model weights. A TSFM analogue would need decomposed components that correspond to numeric channels, regimes, event-stream motifs, latent-state transitions, or intervention responses, followed by retention tests on neighboring regimes and rare events.

Gotchas

  • Param Decomp evidence is currently strongest on toy models and a small 67M language model, not frontier-scale models.
  • Decomposition cost is a first-class part of any fair comparison with LoRA, SFT, RL, or ES.
  • Autointerp labels are not complete causal specifications. The German-labeled component also affected Italian.
  • Repository availability improves reproducibility, but the VPD scaling and efficiency claims still need independent measurement.

Graph View

Backlinks