Can Statistical Methods Reliably Detect Fraudulent Data? Examining The Utility Of P-Value Analyses, Extreme Effect Sizes, GRIM, And GRIMMER

Data fraud occurs when one creates fake data (i.e., fabrication) or alters real data (i.e., falsification), often to support a desired research hypothesis. It is detrimental to science and occurs frequently, making it a pressing concern. Fortunately, there exist several statistical tools to detect it. Extant research, however, is largely inconsistent regarding which tools work well, and no research examines how well they differentiate fraudulent articles (containing fake data) from legitimate controls. The present thesis investigated how well four popular methods to detect data fraud differentiated retracted psychology articles from legitimate controls. I included the method of extreme effect sizes, p-value analysis, GRIM, and GRIMMER. Extreme effect sizes performed quite well: standardized effect sizes for retracted articles were noticeably larger than controls. The other methods performed at chance levels or worse. I contend that the method of extreme effect sizes could provide valuable information during investigations of potentially fraudulent studies.