11  Rasch versus 2PL

Most simulation evidence on nonparametric ability priors comes from Rasch-type models, and a recurring question in review is whether conclusions drawn there carry to models with estimated discriminations. The design makes the two families co-primary: every condition exists in a Rasch and a 2PL version at matched values of the achieved information-based coefficient, with generation and fitting matched in family. The comparison therefore has the same replication structure as everything else in the study. It comes with two disclosed asymmetries, treated at the end of the chapter, that are consequences of matching on that joint length-plus-scale ladder rather than on item count, item parameters, or compute.

11.1 The headline: conclusions transfer

The core KS and MSEL patterns of the preceding chapters appear within each family separately. The winner maps of Chapter 7 have the same structure row by row: GR wins distribution recovery and PM wins individual accuracy in both families, the flexible prior’s territory concentrates at high reliability and large N in both, and the normal columns are quiet in both. The mean primary-contrast advantage over non-normal cells is similar in the two families (mean log ratios -0.179 under Rasch and -0.270 under the 2PL, an exploratory marginal). What differs between the families is not whether the flexible prior works but where it works best, and that difference is itself a registered result.

11.2 The registered dissociation

H3 states a crossed pattern: the flexible prior’s advantage should be larger for bimodal populations under Rasch and larger for skewed populations under the 2PL, expressed as a difference in differences. The estimate is -0.127 (SE 0.023, 95% CI [-0.175, -0.078], two-sided Holm-adjusted p \(= 0.000\)). Figure 11.1 displays it: the two family profiles cross, with the Rasch mean advantage deeper on the bimodal side and the 2PL mean advantage deeper on the skewed side. Its ratio ticks are positioned on a log2 axis for display; H3’s reported coefficient is on the preregistered natural-log response, so the visual height and coefficient should not be compared without converting bases.

Figure 11.1: Each model family earns the flexible prior’s advantage on a different population shape. Cell-level KS loss ratios of DP (focused) + GR against Gaussian + GR in the non-normal columns, jittered points over the 20 cells per model-by-shape group, with large markers at group means. Color, marker shape, and line type all identify model family. The plotted ordinate is log2 of the ratio so that ratio ticks are evenly spaced; the preregistered H3 model instead uses the natural-log response. Under Rasch the mean advantage is larger for bimodal populations than skewed ones; under the 2PL the order reverses. The preregistered interaction contrast H3 estimates the difference in differences at -0.127 natural-log units (95% CI -0.175 to -0.078, Holm-adjusted p = 1e-04). The crossing displays direction, but its vertical height must not be read as that natural-log coefficient without base conversion. A mechanism consistent with this pattern is discussed in the text; the design establishes the crossed pattern, not the proposed mechanism.

A mechanism consistent with the pattern, though the design cannot isolate it, runs through what each family’s likelihood preserves. Under the Rasch model the sum score is sufficient for ability, so the shape of the score distribution is a direct, if noisy, image of the shape of the ability distribution; a bimodal population leaves a bimodal trace that a long enough form transmits to the prior. The 2PL’s estimated discriminations reweight items, which blurs that direct image somewhat for bimodality but helps against skew: heterogeneous slopes let the item response functions carry an asymmetric mapping between score and ability that the Rasch family must leave to the prior alone, and the flexible prior then works with a likelihood that has already absorbed part of the distortion. We flag this reading as interpretation; what the data establish is the crossed pattern itself, at the size above.

The dissociation surfaces elsewhere in the results. In the crossed-lever paired-geometric sensitivity of Section 9.4, 16 of the 20 exceptions to Gaussian + GR are 2PL cells, and the skew column flips only under the 2PL; in Figure 8.3, the low-reliability skew lines fall further under the 2PL than under Rasch. Where the family gives the prior more to work with, the prior matters more.

11.3 The operational asymmetries

Reliability parity between families is a design achievement with two operational asymmetries, and both belong in any comparison a reader makes across families.

Figure 11.2: Matched information uses different calibrated forms and computational budgets. Panel (a): the nested-length plus cell-specific-discrimination ladders hit the same achieved design coefficient (within 0.003 of target everywhere) with about 8 to 68 items under Rasch and 7 to 62 under the 2PL. Family names at the line ends make color nonessential. Panel (b): the 2PL uses 5.0 times the Gaussian-arm iterations and 3.0 times the DP-arm iterations of the Rasch budgets. Direct labels identify each bar; retained draws per chain are 2,000 Gaussian and 4,000 DP. Panel (c): realized median bulk effective sample size for the person block, by arm and family, again directly labeled. The Gaussian arms retain half as many draws as the DP arms; higher 2PL ESS reflects the larger budgets and is not evidence that 2PL mixes intrinsically better. Diagnostics rows yield 0 hard failures and 0 exclusions among 7,200 fits under REQ-07d. The panel displays item counts, not the accompanying c_star variation; the latter is reported in the reliability and design chapters.

First, the calibrated forms differ at matched values of the operational design coefficient. Their nested-length component runs from about 8 to 68 items under Rasch and 7 to 62 under the 2PL, with achieved analytic reliabilities within 0.000 of target everywhere. Each model-by-form-by-tier-by-shape cell also carries the calibrated global discrimination multiplier described in Section 3.3. The family comparison is therefore a comparison at matched values of the joint length-plus-scale ladder, not at matched item counts or identical item parameters. Second, compute differs: after a pilot exposed slow mixing in the discrimination block, the 2PL budgets were raised to five times (Gaussian arm) and three times (DP arms) the Rasch iteration counts, with thinning raised in proportion so retained draws are unchanged. The raise overshot the deficit it corrected, so realized effective sample sizes are somewhat higher under the 2PL. Neither asymmetry touches the loss comparisons, which are within-family by construction; both matter for anyone budgeting a study.

11.4 What this chapter establishes

Within this dichotomous, unidimensional grid and its one item-bank template per family, the study’s conclusions are family-robust in structure and family-specific in geography. The Rasch results place the flexible-prior advantage mainly in bimodal populations at the top of the reliability ladder; the 2PL results show broader gains under skew, an earlier crossover in the crossed-lever competition, and meaningfully greater computational demand. These patterns should be tested before being transferred to other banks or response models. The dissociation is registered, two-sided, and survives every preregistered sensitivity (Section 14.5).