HPS Research Seminar - Construct Validity in Automated Counter-Terrorism Analysis

Governments and social scientists are increasingly developing machine learning methods to automate the process of identifying terrorists in real-time and predicting future attacks. However, current operationalizations of ‘terrorist’ in artificial intelligence are difficult to justify given four issues: insufficient construct legitimacy, insufficient criterion validity, insufficient construct validity, and considerable distribution shifts. I argue for the more general conclusion that any socially constructed entity that is value-laden appearing in machine learning models requires sufficiently stable construct validity over time, and otherwise renders moot diachronic analyses of a given phenomenon.