PLOS Digital Health · 12 Nov 2026
Text Psychometrics: Assessing psychological constructs in text using natural language processing
Daniel M. Lowh-index 7, Patrick Mair, Matthew K. Nock, Satrajit S. Ghoshh-index 50
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Large language models (LLMs) have revolutionized natural language processing (NLP). Yet when used to assess psychological constructs in text, they are generally not evaluated for the types of validity, reliability, and standardization typically expected from traditional questionnaires with rating scales. This study bridges that gap by proposing a framework for evaluating the psychometric properties of NLP-based psychological assessments (including LLMs-based ones), an area of research we refer to as Text Psychometrics. We first review different NLP methods, compare their ability to address key challenges in psychological research such as explainability, and outline ways of evaluating them on many desirable psychometric properties. Our goal is to describe most psychometric properties that may be of interest, especially for psychology and NLP researchers who are less familiar with psychometrics or psychometricians who are less familiar with NLP. We then illustrate the tradeoffs of different NLP methods through two empirical studies. Study 1 classifies thousands of crisis counseling conversations and Reddit posts into different types of mental health issues and introduces a novel method to evaluate text models for content validity—the extent to which a test captures the full range of expressions of a single construct. Study 2 examines prospective criterion validity by estimating how 49 known suicide risk factors predict imminent risk in crisis counseling conversations. We find that mentioning lethal means for suicide, the police, depressed mood, and anxiety in text as well as the texter writing more than the counselor characterize the conversations that end up needing an emergency service intervention for imminent suicide risk, which showcases the interpretability of the method that was used. In sum, NLP studies in psychology often rely on only a few validation metrics; here, we demonstrate the need for broader psychometric evaluation and, as a step towards Text Psychometrics, provide practical examples and future directions that can help researchers evaluate these models.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Satrajit S. Ghosh has the highest h-index among the authors (50).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (72)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 52 more references
A free account opens the rest of the citing papers and references on every paper’s page.