The absolutely fantastic statistical magicians Ashley Watts had a new pre-print, and Eiko Fried wrote a recent wonderful blog post to show her evidence that comorbidity within our assumed models of psychopathology may not hold up well (e.g., Internalization/Externalization models like HiTop).
https://eiko-fried.com/blog/psychopathology-structure-changes-with-severity
As someone working in validity theory, I see a direct parallel that changes how we should think about performance validity, symptom validity, and the very structure of what we’re measuring. Fried’s blog (discussing Watts et al.’s preprint) demonstrates the correlation between psychopathology domains like internalizing and externalizing is largely an artifact of mixing clinical and healthy populations. When you have a heavily zero-inflated sample where most people have no symptoms, you artificially inflate correlations between domains that may be entirely independent within each subgroup. One of the things he points to is the change in SD as a function of severity, which suggests to me a very close parallel to SVT validity patterns. Over-reporting, after all, more often produce many of the same patterns – stable effect sizes and classification patterns and difficulty isolating consistent theoretical elements (E.g., SVT-PVT correlation, as well as domain level distinctions). Moreover, I am assuming and proposing that the way people think and engage in tests reflect a pattern of unlikely high comorbidity compared to their baseline pathology they possess.
I took the basic premise of his simulation and extended it here to symptom validity. Given our poor understanding of what PVTs are, I wanted to stay focused on at least the same dimensional, individualistically measured traits that represent symptom tests. Its increasingly clear that they are distinct and can be at times, entirely separate constructs so its better to treat them that way (. Fried’s blog demonstrates that the correlation between psychopathology domains (eg internalizing and externalizing) is largely an artifact of mixing clinical and healthy populations. When most people have zero symptoms, you artificially inflate correlations between domains that may be entirely independent within each subgroup.
The simulation extends this logic to validity testing. Most SVTs approach classification from “fabrication vs. honest” models where we typically apply in tests like T-test, unidimensional AUC, or ANOVA to evaluate SVTS (a necessary clinical reality for any sort of determination-based evaluative decision). Instead, this simulator implements the more realistic case of partial malingering. Just like with non-content responding (e.g., random, inattentive, as measured by VRIN/TRIN on the MMPI), mixed responding of symptom validity is also the most common. Individuals are more likely to exaggerate or misrepresent than outright fabricate, and this can occur both intentionally and not (misremembering, emotional salience, awareness, etc.). As a result, many individuals who are misrepresenting symptom presentation are also genuinely symptomatic in some ways that may, in truth, be impairing and qualifying of services or whatever the evaluation is for. This is what dominates real compensation-seeking samples. To me, this sounds like the same issue of nomorbidity/comorbidity. In effect, nomorbidity is the honesty (but pathological) control who represents the null state of how mental health is structured in their experience with respect to frequency, intensity, duration, and scope of symptom as ipsative normal. Comorbidity becomes the distorted response pattern, exaggerating across symptoms measures and items and often theorized as symptom domain elements of somatic, cognitive and psychologically based symptoms (see AACN/NAN consensus status on symptom validity).
This explains why we struggle to find clean, replicable factor structures in validity samples. The exaggeration strategy (see Rogers & Bender, 2018) doesn’t respect the theoretical boundaries between somatic, cognitive, and psychological symptoms the way genuine pathology does. It splashes across everything, artificially inflating the apparent coherence of psychopathology and making it look like there’s a strong general factor when most of that structure is actually just the signal of people saying “yes” to too many things. This is why we don’t always see it (e.g., Gervais et al., 2007). It’s often been invalidation studies, which makes sense because we are pulling a situationally bound, context specific effect that emerged in that case. These referral and clinical context patterns are showing influence on these basic assumptions with SVT-PVT relationships.
Until I figure out how to plug this into an embedded window like he did, the simulation models showed symptom scores as an additive decomposition of genuine pathology and exaggeration: O = G + I, where O is the observed score, G is the bona fide symptom level (correlated across domains at r_G ≈ .30), and I is the invalid exaggeration component added only by non-credible responders according to base rate. The invalid component itself contains both a coherent exaggeration strategy (domains correlated at r_I) and a magnification term coupling it to genuine pathology (c). The pooled observed variance decomposes as Var(O) = Var(G) + Var(I) + 2·Cov(G,I), meaning the observed correlation between domains becomes weighted : r_O ≈ [Var(G)·r_G + Var(I)·r_I + 2·Cov(G,I)] / Var(O). When BR = 20% and exaggeration adds ~1.5–1.8 SDs, this inflates observed inter-domain correlations from r = .30 to r ≈ .70–.75, and the first eigenvalue (general factor strength) jumps from ~53% to ~75% of variance—meaning most of the apparent “general factor” in validity samples is artifact, not psychopathology. The inflated eigenvalue is the mathematical signature of a contextually activated exaggeration strategy. This is just what Eiko’s post was talking about, except in pathology.
And because partial malingerers still have real symptoms underneath, you can’t just remove them from your sample without also removing genuine pathology. Doing so changes the underlying comoribidty pattern observed across endorsements, and echoes the nomorbidity/comorbidity issue of internalizing/externalizing dimensions and the whole underlying ‘what causes problems and when do they occur’ debate.
I called the general factor scale of the MMPI the Scale of Scales validity indicator the SOS for a reason (Ingram et al., 2024). I found no matter what content we included via the RC scales, the outcome was the same. Gaines et al (2013) and Aita et al (2025) found the same thing on the PAI, and the literature is quickly growing across measures of the same pattern generally. We really need to stop and rethink what we assume is true and test it. Tristan and I recently wrote an invited commentary (in review) about some of these points, focusing on dispelling domain level distinction myths and critiquing the truly nascent research in this field. This is a flashback to one of my first research projects on validity, I called it “Validity is invalidity” and I looked at the MMPI-2 scales in a large (9k), nationally sampled VA sample. I did it poorly, but I still thinking the same thing I am now- How we measure validity isn’t fitted to what we say we are measuring. This seems like a good measurement step to resolving that problem. We have a few papers coming out looking at this exact problem in various self-report instruments.
-Rushed post, I need to double check my maths.
Dr. Paul Ingram, Ph.D 