Skip to contents

Preset simulation designs — defensible starting points for common multisite scenarios

Source: R/presets.R
presets.Rd

Presets package a defensible starting design — site count, latent-effect shape, heterogeneity, sample-size or precision targets — for a common multisite-trial or meta-analysis scenario. Each preset returns a locked multisitedgp_design object whose parameter values are anchored to a published reference, so a reviewer can defend the choice. Pass any named multisitedgp_design() argument through ... to override the locked defaults.

Usage

preset_education_small(...)

preset_education_modest(...)

preset_education_substantial(...)

preset_jebs_paper(...)

preset_jebs_strict(...)

preset_walters_2024(...)

preset_twin_towers(...)

preset_meta_modest(...)

preset_small_area_estimation(...)

Arguments

...

Named overrides passed to multisitedgp_design. Override field names must match multisitedgp_design() arguments; unknown names trigger an informative error.

Value

A multisitedgp_design object with paradigm = "site_size" or paradigm = "direct" depending on the preset. Pass to sim_multisite or sim_meta respectively.

Details

Decision table — pick by trial type.

Education multisite trial

preset_education_small (\(\sigma_\tau = 0.05\)) / preset_education_modest (\(\sigma_\tau = 0.20\)) / preset_education_substantial (\(\sigma_\tau = 0.30\)). Calibrated to Weiss et al. (2017) cross-site benchmarks from 16 multisite RCTs.

JEBS reference design (Mixture, engine A1)

preset_jebs_paper (UX-anchored, \(\sigma_\tau = 0.20\)) for general use; preset_jebs_strict (\(\sigma_\tau = 0.15\), J = 100) for paper-grid bit-identical reproduction. Both use A1 legacy engine — the JEBS Mixture fixture.

Empirical-Bayes school value-added

preset_walters_2024 — J = 46 schools, Boston charter value-added, calibrated to the Walters (2024) Handbook chapter reanalysis of the BPS dataset.

Deconvolution / shape-recovery benchmark

preset_twin_towers — bimodal mixture, J = 1000, equal-sample-size sites; the canonical deconvolution-method test bed (Efron-style benchmark).

Meta-analysis (direct precision)

preset_meta_modest — J = 50, \(I = 0.30\), \(R = 1.5\), modest-informativeness meta-analysis design.

Small-area estimation (direct precision)

preset_small_area_estimation — J = 30, \(I = 0.20\), \(R = 3.0\), Fay-Herriot small-area design.

Override pattern. Every preset accepts ... for named overrides; the \code{\link{multisitedgp_design}} argument set is the valid name space. To use a preset's heterogeneity scale but a different site count: preset_education_modest(J = 100L). To use a preset's calibration but switch on rank-correlated dependence: preset_education_modest(dependence = "rank", rank_corr = 0.3).

Citing presets in published work. Each preset locks parameter values from a published reference; cite that reference, not the preset name. The relevant citations are listed in multisiteDGP package help and in citation("multisiteDGP").

For a workflow walkthrough see the Choosing a preset vignette.

Functions

  • preset_education_small(): Education multisite trial with small effect-size heterogeneity (\(\sigma_\tau = 0.05\)). Calibrated to Weiss et al. (2017) education programs at the small end of the cross-site benchmark range. J = 50, sample sizes ~ 40 per site.

  • preset_education_modest(): Education multisite trial with moderate effect-size heterogeneity (\(\sigma_\tau = 0.20\)). The package's default applied starting point; J = 50, sample sizes ~ 50 per site, calibrated to the Weiss et al. (2017) middle of the cross-site range and used as the JEBS UX anchor.

  • preset_education_substantial(): Education multisite trial with substantial effect-size heterogeneity (\(\sigma_\tau = 0.30\)). Larger trial (J = 100, sample sizes ~ 80) calibrated to the upper end of the Weiss et al. (2017) cross-site benchmarks.

  • preset_jebs_paper(): JEBS Mixture reference design with the UX-anchored heterogeneity (\(\sigma_\tau = 0.20\)). Uses the legacy A1 site-size engine for bit-identical reproduction of the JEBS paper's general-use Mixture fixture (J = 50, mixture parameters \(\delta = 5\), \(\epsilon = 0.3\), \(\Upsilon = 2\)).

  • preset_jebs_strict(): JEBS strict paper-grid Mixture (J = 100, \(\sigma_\tau = 0.15\)). Use this for bit-identical reproduction of the JEBS paper's strict simulation grid (the T1a invariant). Engine A1 legacy.

  • preset_walters_2024(): Boston charter / BPS value-added empirical-Bayes design from Walters (2024) Handbook of Labor Economics chapter. J = 46 schools, \(\sigma_\tau \approx 0.197\), mean per-school sample 240, with \(R^2 = 0.40\) from covariate adjustment. Use when calibrating to school-effect or teacher-effect Empirical Bayes literature.

  • preset_twin_towers(): Twin-towers bimodal mixture benchmark for deconvolution and shape-recovery method evaluation. J = 1000 sites with equal sample sizes; mixture parameters \(\delta = 4\), \(\epsilon = 0.5\), \(\Upsilon = 1\). Standard test bed for nonparametric Empirical Bayes methods.

  • preset_meta_modest(): Direct-precision (Paradigm B) meta-analysis design with moderate informativeness (\(I = 0.30\), \(R = 1.5\)). J = 50 studies. Use with sim_meta for meta-analysis simulations where study-level standard errors are specified directly.

  • preset_small_area_estimation(): Direct-precision Fay-Herriot small-area estimation design (J = 30, \(I = 0.20\), \(R = 3.0\)). Wider precision spread than the meta-analysis preset; reflects the heterogeneous-precision character of small-area survey data. Use with sim_meta.

References

Lee, J., Che, J., Rabe-Hesketh, S., Feller, A., & Miratrix, L. (2025). Improving the estimation of site-specific effects and their distribution in multisite trials. Journal of Educational and Behavioral Statistics, 50(5), 731–764. doi:10.3102/10769986241254286 .

Walters, C. (2024). Empirical Bayes methods in labor economics. In Handbook of Labor Economics (Vol. 5, pp. 183–260). Elsevier. doi:10.1016/bs.heslab.2024.11.001 .

Weiss, M. J., Bloom, H. S., Verbitsky-Savitz, N., Gupta, H., Vigil, A. E., & Cullinan, D. N. (2017). How much do the effects of education and training programs vary across sites? Evidence from past multisite randomized trials. Journal of Research on Educational Effectiveness, 10(4), 843–876. doi:10.1080/19345747.2017.1300719 .

See also

multisitedgp_design for the constructor that backs every preset; sim_multisite and sim_meta for the wrappers that consume preset designs; design_grid for sweeping multiple presets in a grid; the A2 Choosing a preset vignette.

Examples

# The default applied starting point.
design <- preset_education_modest()
sim_multisite(design, seed = 1L)
#> # A multisitedgp_data: 50 sites, paradigm = "site_size"
#> # Realized vs intended:
#> #   I: realized=0.303 (no target)
#> #   R: realized=10.167 (no target)
#> #   sigma_tau: target=0.200, realized=0.166, FAIL
#> #   rho_S: target=0.000, realized=0.254, PASS
#> #   rho_S_marg: realized=0.254 (no target)
#> #   Feasibility: WARN (n_eff=15.693)
#> # A tibble: 50 × 7
#>   site_index    z_j   tau_j tau_j_hat  se_j  se2_j   n_j
#>        <int>  <dbl>   <dbl>     <dbl> <dbl>  <dbl> <int>
#> 1          1 -0.626 -0.125    -0.329  0.329 0.108     37
#> 2          2  0.184  0.0367    0.0481 0.270 0.0727    55
#> 3          3 -0.836 -0.167    -0.399  0.254 0.0645    62
#> 4          4  1.60   0.319     0.410  0.577 0.333     12
#> 5          5  0.330  0.0659   -0.146  0.324 0.105     38
#> 6          6 -0.820 -0.164     0.265  0.243 0.0588    68
#> # ℹ 44 more rows
#> # Use summary(df) for the full diagnostic report.

# Override the site count while keeping the calibrated heterogeneity.
large <- preset_education_modest(J = 200L)

# Switch on rank-correlated precision dependence.
dep <- preset_education_modest(dependence = "rank", rank_corr = 0.3)