Exploration Versus Exploitation Decisions in the Human Brain: A Systematic Review of Functional Neuroimaging and Neuropsychological Studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, the heterogeneity of experimental protocols and paradigms used to study these choice behaviours, as well as the predominance of reinforcement learning studies to study the neurocomputational basis of choice behaviours. We conducted a systematic review of functional neuroimaging (fMRI) studies of exploration- versus exploitation- based decision-making in healthy adult humans during reinforcement learning, information search, and foraging. Eleven fMRI studies met inclusion criterion. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. Building from these findings in typical adult development, we next surveyed exploration-exploitation behaviours in neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative disease. Findings revealed differing exploration and exploitation decision-making biases across populations. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploitation and exploration in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.