Compute Marginal Reliability from Simulated Test Information

Low-level utilities for mapping a discrimination scale \(c\) and simulated person/item parameters to marginal reliability.

These functions implement the same reliability definitions used inside both eqc_calibrate() and sac_calibrate():

compute_rho_bar(): MSEM-based marginal reliability \(\bar{w}(c) = \sigma_\theta^2 / (\sigma_\theta^2 + E[1/\mathcal{J}(\theta;c)])\)
compute_rho_tilde(): Average-information reliability \(\tilde{\rho}(c) = \sigma_\theta^2 \bar{\mathcal{J}}(c) / (\sigma_\theta^2 \bar{\mathcal{J}}(c) + 1)\)

Usage

compute_rho_bar(
  c,
  theta_vec,
  beta_vec,
  lambda_base,
  theta_var = NULL,
  guessing = NULL,
  weights = NULL,
  return_diagnostics = FALSE
)

compute_rho_tilde(
  c,
  theta_vec,
  beta_vec,
  lambda_base,
  theta_var = NULL,
  guessing = NULL,
  weights = NULL,
  return_diagnostics = FALSE
)

Arguments

c: Numeric scalar. Global discrimination scaling factor.
theta_vec: Numeric vector of abilities \(\theta_m\).
beta_vec: Numeric vector of item difficulties \(\beta_i\).
lambda_base: Numeric vector of baseline discriminations \(\lambda_{i,0}\) (before scaling by c).
theta_var: Optional numeric. Pre-calculated variance of theta. If NULL, computed from theta_vec.
guessing: Optional numeric scalar or item-length vector of 3PL lower-asymptote parameters in [0, 1). A scalar is recycled. NULL is equivalent to zero guessing and gives the Rasch/2PL special case.
weights: Optional non-negative integration weights, one per value in theta_vec. Weights are normalized internally.
return_diagnostics: Logical. If FALSE (the default), return the legacy scalar result. If TRUE, return the reliability together with numerical diagnostics and estimand metadata.

Value

By default, a numeric scalar reliability value. With return_diagnostics = TRUE, a named list containing the value, logit-scale value, numerical summaries, and estimand metadata.

Details

The computation proceeds as:

Form scaled discriminations \(\lambda_i(c) = c \cdot \lambda_{i,0}\).
With lower asymptote \(g_i\), compute \(p_{mi}=g_i+(1-g_i)\operatorname{logit}^{-1} \{\lambda_i(c)(\theta_m-\beta_i)\}\) under the fixed D = 1 convention. Setting every \(g_i=0\) gives Rasch/2PL exactly.
Compute 3PL item information \(\mathcal{J}_{mi}=\lambda_i(c)^2 (p_{mi}-g_i)^2(1-p_{mi})/\{(1-g_i)^2p_{mi}\}\). For \(g_i=0\), this reduces to \(\lambda_i(c)^2p_{mi}(1-p_{mi})\).
Test information at each \(\theta_m\): \(\mathcal{J}_m = \sum_i \mathcal{J}_{mi}\).
Reliability:
- compute_rho_bar(): harmonic-mean-based MSEM \(\text{MSEM} = E[1/\mathcal{J}_m]\), \(\bar{w}(c) = \sigma_\theta^2 / (\sigma_\theta^2 + \text{MSEM})\).
- compute_rho_tilde(): arithmetic-mean-based information \(\bar{\mathcal{J}} = E[\mathcal{J}_m]\), \(\tilde{\rho}(c) = \sigma_\theta^2 \bar{\mathcal{J}} / (\sigma_\theta^2 \bar{\mathcal{J}} + 1)\).

Information and its expectations are evaluated in the log domain. No artificial information floor is applied. Consequently, zero information at a positively weighted node correctly implies infinite MSEM and zero MSEM-based reliability.

Examples

# Simple toy example
set.seed(1)
theta <- rnorm(1000)
beta  <- rnorm(20)
lambda0 <- rep(1, 20)

compute_rho_bar(1, theta, beta, lambda0)
#> [1] 0.7795393
compute_rho_tilde(1, theta, beta, lambda0)
#> [1] 0.7870079

# With pre-calculated theta variance (recommended for SAC)
theta_var_fixed <- var(rnorm(10000))  # Pre-calculate from large sample
compute_rho_bar(1, theta, beta, lambda0, theta_var = theta_var_fixed)
#> [1] 0.7684109

# 3PL lower asymptotes use the same public kernel.
g <- rep(0.20, length(beta))
compute_rho_tilde(1, theta, beta, lambda0, guessing = g)
#> [1] 0.7055277