Congeneric reliability

For broader coverage of this topic, see Reliability (statistics).

In statistical models applied to psychometrics, congeneric reliability ${\displaystyle \rho _{C}}$ ("rho C")^[1] a single-administration test score reliability (i.e., the reliability of persons over items holding occasion fixed) coefficient, commonly referred to as composite reliability, construct reliability, and coefficient omega. ${\displaystyle \rho _{C}}$ is a structural equation model (SEM)-based reliability coefficients and is obtained from a unidimensional model. ${\displaystyle \rho _{C}}$ is the second most commonly used reliability factor after tau-equivalent reliability(${\displaystyle \rho _{T}}$; also known as Cronbach's alpha), and is often recommended as its alternative.

History and names

A quantity similar (but not mathematically equivalent) to congeneric reliability first appears in the appendix to McDonald's 1970 paper on factor analysis, labeled ${\displaystyle \theta }$.^[2] In McDonald's work, the new quantity is primarily a mathematical convenience: a well-behaved intermediate that separates two values.^[3][4] Seemingly unaware of McDonald's work, Jöreskog first analyzed a quantity equivalent to congeneric reliability in a paper the following year.^[4][5] Jöreskog defined congeneric reliability (now labeled ρ) with coordinate-free notation,^[5] and three years later, Werts gave the modern, coordinatized formula for the same.^[6] Both of the latter two papers named the new quantity simply "reliability".^[5][6] The modern name originates with Jöreskog's name for the model whence he derived ${\displaystyle \rho _{C}}$: a "congeneric model".^[1][7][8]

Applied statisticians have subsequently coined many names for ${\displaystyle {\rho }_{C}}$. "Composite reliability" emphasizes that ${\displaystyle {\rho }_{C}}$ measures the statistical reliability of composite scores.^[1][9] As psychology calls "constructs" any latent characteristics only measurable through composite scores,^[10] ${\displaystyle {\rho }_{C}}$ has also been called "construct reliability".^[11] Following McDonald's more recent expository work on testing theory, some SEM-based reliability coefficients, including congeneric reliability, are referred to as "reliability coefficient ${\displaystyle \omega }$", often without a definition.^[1][12][13]

Formula and calculation

Congeneric measurement model

Congeneric reliability applies to datasets of vectors: each row X in the dataset is a list X_i of numerical scores corresponding to one individual. The congeneric model supposes that there is a single underlying property ("factor") of the individual F, such that each numerical score X_i is a noisy measurement of F. Moreover, that the relationship between X and F is approximately linear: there exist (non-random) vectors λ and μ such that $${\displaystyle X_{i}=\lambda _{i}F+\mu _{i}+E_{i}{\text{,}}}$$ where E_i is a statistically independent noise term.^[5]

In this context, λ_i is often referred to as the factor loading on item i.

Because λ and μ are free parameters, the model exhibits affine invariance, and F may be normalized to mean 0 and variance 1 without loss of generality. The fraction of variance explained in item X_i by F is then simply $${\displaystyle \rho _{i}={\frac {\lambda _{i}^{2}}{\lambda _{i}^{2}+\mathbb {V} [E_{i}]}}{\text{.}}}$$ More generally, given any covector w, the proportion of variance in wX explained by F is $${\displaystyle \rho ={\frac {(w\lambda )^{2}}{(w\lambda )^{2}+\mathbb {E} [(wE)^{2}]}}{\text{,}}}$$ which is maximized when w ∝ 𝔼[EE^*]^-1λ.^[5]

ρ_C is this proportion of explained variance in the case where w ∝ [1 1 ... 1] (all components of X equally important): $${\displaystyle \rho _{C}={\frac {\left(\sum _{i=1}^{k}\lambda _{i}\right)^{2}}{\left(\sum _{i=1}^{k}\lambda _{i}\right)^{2}+\sum _{i=1}^{k}\sigma _{E_{i}}^{2}}}}$$

Example

Fitted/implied covariance matrix

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$	${\displaystyle X_{4}}$
${\displaystyle X_{1}}$	${\displaystyle 10.00}$
${\displaystyle X_{2}}$	${\displaystyle 4.42}$	${\displaystyle 11.00}$
${\displaystyle X_{3}}$	${\displaystyle 4.98}$	${\displaystyle 5.71}$	${\displaystyle 12.00}$
${\displaystyle X_{4}}$	${\displaystyle 6.98}$	${\displaystyle 7.99}$	${\displaystyle 9.01}$	${\displaystyle 13.00}$
${\displaystyle \Sigma }$	${\displaystyle 124.23=\Sigma _{diagonal}+2\times \Sigma _{subdiagonal}}$

These are the estimates of the factor loadings and errors:

Factor loadings and errors

	${\displaystyle {\hat {\lambda }}_{i}}$	${\displaystyle {\hat {\sigma }}_{e_{i}}^{2}}$
${\displaystyle X_{1}}$	${\displaystyle 1.96}$	${\displaystyle 6.13}$
${\displaystyle X_{2}}$	${\displaystyle 2.25}$	${\displaystyle 5.92}$
${\displaystyle X_{3}}$	${\displaystyle 2.53}$	${\displaystyle 5.56}$
${\displaystyle X_{4}}$	${\displaystyle 3.55}$	${\displaystyle .37}$
${\displaystyle \Sigma }$	${\displaystyle 10.30}$	${\displaystyle 18.01}$
${\displaystyle \Sigma ^{2}}$	${\displaystyle 106.22}$

${\displaystyle {\hat {\rho }}_{C}={\frac {\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}}{{\hat {\sigma }}_{X}^{2}}}={\frac {106.22}{124.23}}=.8550}$

${\displaystyle {\hat {\rho }}_{C}={\frac {\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}}{\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}+\sum _{i=1}^{k}{\hat {\sigma }}_{e_{i}}^{2}}}={\frac {106.22}{106.22+18.01}}=.8550}$

Compare this value with the value of applying tau-equivalent reliability to the same data.

Related coefficients

Tau-equivalent reliability (${\displaystyle \rho _{T}}$), which has traditionally been called "Cronbach's ${\displaystyle \alpha }$", assumes that all factor loadings are equal (i.e. ${\displaystyle \lambda _{1}=\lambda _{2}=...=\lambda _{k}}$). In reality, this is rarely the case and, thus, it systematically underestimates the reliability. In contrast, congeneric reliability (${\displaystyle \rho _{C}}$) explicitly acknowledges the existence of different factor loadings. According to Bagozzi & Yi (1988), ${\displaystyle \rho _{C}}$ should have a value of at least around 0.6.^[14] Often, higher values are desirable. However, such values should not be misunderstood as strict cutoff boundaries between "good" and "bad".^[15] Moreover, ${\displaystyle \rho _{C}}$ values close to 1 might indicate that items are too similar. Another property of a "good" measurement model besides reliability is construct validity.

A related coefficient is average variance extracted.