IRTsimrel

IRTsimrel calibrates unidimensional IRT simulations to a requested marginal reliability. Version 0.3.0 supports Rasch, 2PL, and D = 1 3PL data-generating models, preserves the realized item design, and reports the estimand and root topology used to obtain the global discrimination scale.

The package is designed for inverse simulation questions: instead of generating item parameters first and accepting whatever reliability follows, specify the reliability condition as part of the design.

Installation

Install the development version from GitHub:

# install.packages("remotes")
remotes::install_github("joonho112/IRTsimrel")

Quick start: a reliability-targeted 3PL form

The following example calibrates a 20-item 3PL form to average-information reliability 0.75. The lower asymptote is fixed at 0.20 for every item.

library(IRTsimrel)

fit_3pl <- eqc_calibrate(
  target_rho = 0.75,
  n_items = 20,
  model = "3pl",
  item_source = "parametric",
  item_params = list(
    guessing_params = list(distribution = "fixed", value = 0.20)
  ),
  reliability_metric = "info",
  M = 2500,
  seed = 42
)

c(c_star = fit_3pl$c_star, achieved_rho = fit_3pl$achieved_rho)
#>       c_star achieved_rho
#>    1.1735121    0.7500001
fit_3pl$item_scope
#> [1] "fixed_form"

Generate responses from the stored calibrated form:

sim <- simulate_response_data(fit_3pl, n_persons = 250, seed = 123)
dim(sim$response_matrix)
#> [1] 250  20
range(sim$response_matrix)
#> [1] 0 1

achieved_rho above is the fitted value on EQC’s fixed empirical quadrature. It is a root-fit diagnostic, not a guarantee of exact reliability in every new person sample. Use independent holdout draws when population generalization error matters.

The 3PL parameter contract

IRTsimrel uses

P(Y_{pi}=1\mid\theta_p) = g_i + (1-g_i)\operatorname{logit}^{-1} \{c\lambda_{i,0}(\theta_p-\beta_i)\}, \qquad D=1.

The names deliberately avoid a common ambiguity:

• guessing is the item lower asymptote g_i;
• c is the global discrimination scale and c_star is its calibrated value;
• calibration changes c\lambda_{i,0}, but does not change guessing or item difficulty.

sim_item_params() can generate fixed, beta-distributed, or uniformly distributed guessing values. A custom 3PL form supplies them through custom_params$guessing.

items <- sim_item_params(
  n_items = 8,
  model = "3pl",
  source = "parametric",
  guessing_params = list(distribution = "uniform", min = 0.10, max = 0.25),
  seed = 7
)

range(items$data$guessing)
#> [1] 0.1123854 0.2356327

Two calibration contracts

Algorithm	Public function	Reliability metric	Item scope
Empirical Quadrature Calibration	eqc_calibrate()	info / tilde	One realized fixed form
Stochastic Approximation Calibration	sac_calibrate()	info / tilde or msem / bar	Fixed form (resample_items = FALSE) or item superpopulation (TRUE)

Both algorithms scan the requested range on log(c) and select an explicit root branch. This matters because 3PL guessing and limited latent–item coverage can produce non-monotone reliability curves and multiple roots.

SAC adds a preflight branch guard, Robbins–Monro updates, and an independent final evaluation. It is sensitive to the step schedule and finite iteration budget, so inspect summary(), convergence flags, branch diagnostics, and the achieved distribution rather than reporting c_star alone.

# Item-superpopulation MSEM target: a new form is drawn across iterations.
sac_3pl <- sac_calibrate(
  target_rho = 0.70,
  n_items = 20,
  model = "3pl",
  item_params = list(
    guessing_params = list(distribution = "fixed", value = 0.20)
  ),
  reliability_metric = "msem",
  resample_items = TRUE,
  seed = 42
)

Run EQC and SAC on the same fixed form and metric before interpreting compare_eqc_sac() as a cross-algorithm check. The comparison helper records why results with different estimands or item designs are not directly comparable.

Reliability and validation boundaries

• Average-information (info) and MSEM-based (msem) reliability are distinct estimands. For the same information distribution, Jensen’s inequality gives rho_info >= rho_msem; fitted-score reliability is a separate quantity.
• Population MSEM reliability is non-integrable for the built-in Student-t heavy_tail distribution. SAC rejects that combination. Choose info, or explicitly define a finite truncated/empirical estimand.
• compute_reliability_tam() supports its documented Rasch/2PL paths and returns TAM WLE and EAP diagnostics. The external 3PL validation used in the 0.3.0 evidence was EAP-only: TAM 4.3-25 did not support tam.wle() for the fitted tam.mml.3pl object. IRTsimrel therefore makes no 3PL WLE claim.
• The current package is unidimensional. Multidimensional calibration is not part of version 0.3.0.

Large reliability reductions automatically use bounded row chunks for transient item-information matrices. This changes peak-memory use, not the returned reliability value or the matrix-returning kernel contract.

Backward compatibility

• spc_calibrate() remains a deprecated alias for sac_calibrate(), and compare_eqc_spc() remains a deprecated comparison alias.
• The historical argument prefix of sac_calibrate() is unchanged; new root, preflight, and evaluation controls are appended.
• Older Rasch/2PL result objects without a guessing field are interpreted as zero-guessing designs without mutating the object. Legacy spc_result objects that do not contain enough item-design information must be recalibrated with sac_calibrate().

Documentation

Start with the Quick Start or the Applied Guide. The methodological articles describe reliability theory, EQC, SAC, and the validation framework. See the API reference for function-level documentation.

Citation

Lee, J.-H. (2026). Reliability-Targeted Simulation of Item Response Data:
Solving the Inverse Design Problem. arXiv:2512.16012v2.
https://doi.org/10.48550/arXiv.2512.16012

License

MIT