Journal Article
A contemporary research agenda in behavioral and neuroeconomics aims to identify individual differences and (neuro-)psychological correlates of rationality. This research has been widely received in important interdisciplinary and field outlets. However, the psychometric reliability of such measurements of rationality has been presumed without enough methodological scrutiny.
Drawing from multiple original and published datasets (in total over 1600 participants), the authors unequivocally show that contemporary measurements of rationality have moderate to poor reliability according to common standards. Further analyses of the variance components as well as a allowing participants to revise previous choices suggest that this is driven by low between-subject variance rather than high measurement error.
As has been argued previously for other behavioral measurements, this poses a challenge to the predominant correlational research designs and the search for sociodemographic or neural predictors. While the authors' results draw a sobering picture of the prospects of contemporary measurements of rationality, they are not necessarily surprising from a theoretical perspective, which they outline in their discussion.
