Appendix B — Reproducibility

This appendix states what is reproducible, by what mechanism, and what is not claimed.

B.1 Randomness

All simulation randomness descends from a single root token through a content-addressed seed tree: each dataset, chain, extraction and bootstrap carries a seed derived by hashing its identity path, so seeds are collision-checked, order-independent, and recoverable from names. The generator is L’Ecuyer-CMRG with inversion sampling for normals, pinned at runtime by a guard that refuses to run under any other configuration. Bootstrap and operating- characteristic seeds are recorded in their outputs. Named seeds guarantee that reruns draw the same randomness; they do not by themselves guarantee bitwise identity of MCMC output across platforms or library builds, and no such claim is made.

B.2 Content identity

Frozen artifacts are addressed by SHA-256: the response store’s manifest hash names the store, every fit bundle’s hash travels in its loss rows, the preregistration’s hash travels in every confirmatory row, and the analysis outputs are validated against a schema contract before writing and after reading. The book’s own build recomputed a sample of losses from stored vectors (Section A.4); its exemplar figures carry the same check.

B.3 Environment

Fitting ran under a frozen package set (the project’s DPprior, DPMirt and IRTsimrel packages with NIMBLE pinned); the analysis and this book run under R 4.6.0 with package versions locked by the repository’s lockfile. The compiler toolchain of the fitting hosts was not frozen, which is one reason bit-level reproduction is not claimed.

B.4 Rebuilding this book

The book’s data layer is a frozen copy of the production analysis row-sets; nothing in the book refits models. From the repository root, the stale-safe build order is:

  1. run 03-derive.R and 03b-exemplars.R (the latter requires read access to the V3 stores);
  2. run 04-facts.R, then 05-tables.R;
  3. run every fig-*.R script and copy all master PNG, PDF and caption sidecars into the corresponding book/figures/ directories;
  4. run 06-semantic-qa.R, which checks export completeness, master/copy hashes and reader-visible figure semantics;
  5. remove book/_freeze and book/.quarto, then run quarto render from book/.

Each R script is idempotent. The explicit copy check is necessary because co-generation alone cannot prevent a stale copied image or caption, and Quarto’s automatic freeze does not track every upstream RDS or copied-sidecar dependency. Analytical findings in prose and tables are read from the facts object or derived rows where available. Protocol constants and historical or operational facts may remain literal or quoted, but their registration, configuration, registry, manifest, or build-log source must be explicit.

Table B.1: Provenance classes used by the book build. Source: tables/T-provenance.rds.
Provenance class Examples in the book Authoritative source Build treatment
Frozen analytical rows estimates, intervals, decisions, diagnostics, adequacy, evidence labels data/tidy (11 frozen row-set files + 1 JSON manifest, 15.2 MB on disk) derived by 03/04/05 scripts; display values are not hand-typed
Generated design facts cell counts, target ladder, calibration range, thresholds imported from the design data/design and the generated F$design component read or summarized by 04-facts; protocol constants retain explicit source
Quoted operational facts runtime, cost, archive size, hosts and workers production archive manifest and build logs 058/060 quoted in F$compute with source; not claimed to be recomputed from row-sets
Protocol / historical record estimand definitions, registered cutoffs, addendum timing and scope preregistration, addenda registry, and frozen configuration literal wording/numbers allowed only when the source is named
This is a provenance-class audit, not a claim that every displayed string is generated. A future F$provenance_audit component will replace the fallback map automatically.

B.5 What is reproducible, ranked

In descending strength: (a) analytical findings, tables, and generated figure data from the frozen row-sets, subject to the build’s semantic and copy-equality checks; (b) the analysis stage from the fit archive, as its replay gate demonstrated on a sample; (c) the fitting wave from the frozen store and seeds, up to MCMC platform variation; (d) the store itself from the design freeze under the pinned generator. Historical and operational facts belong to a different class: they are traceable to registries, manifests, and logs, but are not recomputed from the analytical row-sets. The weakest computational link is (c), the standard limitation for MCMC at scale.