| Quantity | Value |
|---|---|
| Fits attempted / completed | 7,200 / 7,200 |
| Wall clock (fitting wave) | 147.7 host-hours (plan 70.5) |
| Compute cost (GCP portion) | about $291 (plan $162) |
| Fit archive | 174.0 GB |
| Hosts | MacBook Pro 2,928 fits (12 workers); GCP 4,272 fits (44 workers) |
| Postrun analysis | 2 h 02 m (7,200 fits) |
| Operating characteristic | 2 h 42 m |
| Aggregation | 10 m 52 s |
| Quoted operational facts, not recomputed analytical outcomes. Sources carried into book-facts.rds from the production archive manifest and build logs 058/060; plan values come from the production authorization record. | |
Appendix A — Computation and execution
This appendix records how the study was computed: the engine, the budgets, the production wave as it actually ran, and the verification that ties analysis rows to fitted draws. It condenses the build logs; the full logs live with the codebase.
A.1 Engine and fitting
Models are fitted by MCMC in NIMBLE (Valpine et al., 2017), with the DPM machinery from the project’s DPprior and DPMirt packages (Lee, 2026b, 2026a) and the data factory from IRTsimrel (Lee, 2026c). Each fit runs 4 chains from over-dispersed starts under per-fit seeds derived from a content-addressed seed tree (Appendix B). The DP arms sample the truncated stick-breaking representation; truncation levels are recorded per fit and their adequacy monitored by an occupancy guard, which flagged 5 of 4,800 DP fits (none at a level affecting analysis; Section 14.2). Compilation is cached per model structure, so a wave amortizes compilation across fits.
A.2 Budgets
Budgets follow the raise-only rule: they may be increased before production on pilot evidence, never decreased, and never changed after the wave starts. The realized schedule (iterations, burn-in, thinning per chain): Rasch Gaussian 8,000 / 4,000 / 2; Rasch DP 12,000 / 4,000 / 2; 2PL Gaussian 40,000 / 20,000 / 10; 2PL DP 36,000 / 12,000 / 6. The 2PL figures are the P-2 raise described in Section 6.2, sized from a diagnostic that showed discrimination-block effective sample size growing proportionally with iterations; thinning was raised in proportion, so retained draws per chain are 2,000 (Gaussian) and 4,000 (DP) in every family.
A.3 The production wave
The wave ran on two hosts (a MacBook Pro and a GCP instance) under seven authorization packets, an operational bookkeeping that never enters analysis. All 7,200 fits completed, none missing, none duplicated, none rerun. Wall clock exceeded plan by a factor of 2.1 and cost by 1.8, almost entirely from the raised 2PL budgets; the archive holds 174 GB of fit bundles whose hashes every analysis row carries.
Execution was not incident-free, and the incident record is part of the study’s evidence about itself. The wave driver’s early runs exposed a series of defects at the driver-builder boundary (path quoting, an environment variable dropped under --vanilla, a job-index mismatch), and one design defect with real cost: the driver initially dispatched fits serially, idling the worker pool until it was caught by throughput monitoring. Each was fixed before or during the wave under the raise-only and identity-binding rules, and two temporarily missing fit bundles were recovered from their content hashes. The general lesson the project draws, that checking outputs is not checking throughput, and that every gate must bind to content identity rather than to labels, is written into the verification design of Appendix D.
A.4 Analysis-stage verification
The analysis stage is resumable per fit, with five tracked sub-stages per fit (assembly, diagnostics, extraction, losses, replay). Before aggregation, a replay gate re-derived a random sample of stored loss values from stored posterior summaries and the frozen data: 36 of 36 replays matched to numerical precision. Reconciliation then verified five complete stages for every fit (0 missing, 0 unexpected), and the aggregation wrote the eleven contract-validated row-sets this book reads. The exemplar extractions used for Figure 8.4 and Figure 8.5 were verified the same way at book-build time: each displayed estimate set’s MSEL and KS were recomputed from the plotted vectors and matched the frozen loss rows to within \(10^{-8}\).
A.5 Analysis-stage cost
The full postrun (7,200 fits through the five sub-stages) took 2 hours 2 minutes on 14 workers; per-fit medians ran from 6.0 seconds at N = 50 to 32.3 at N = 500. Aggregation took 11 minutes; the realized operating characteristic required 2 hours 42 minutes of simulation. These figures quantify the additional runtime required for verification, which was small relative to the fitting wave it audited.
Lee, J. (2026a). DPMirt: Bayesian semiparametric item response theory models using Dirichlet process mixture priors. https://joonho112.github.io/DPMirt/
Lee, J. (2026b). DPprior: Principled prior elicitation for Dirichlet process mixture models. https://joonho112.github.io/DPprior/
Lee, J. (2026c). IRTsimrel: Reliability-targeted simulation for item response data. https://joonho112.github.io/IRTsimrel/
Valpine, P. de, Turek, D., Paciorek, C., Anderson-Bergman, C., Temple Lang, D., & Bodik, R. (2017). Programming with models: Writing statistical algorithms for general model structures with NIMBLE. Journal of Computational and Graphical Statistics, 26(2), 403–413. https://doi.org/10.1080/10618600.2016.1172487