Representational similarity analysis – connecting the branches of systems neuroscience

TLDR

Systems neuroscience faces the challenge of quantitatively linking brain‑activity measurement, behavioral measurement, and computational modeling, complicated by the need to map model units to brain channels and to reconcile activity patterns across modalities, subjects, and species. The authors propose representational similarity analysis (RSA), a framework that abstracts activity patterns into representational dissimilarity matrices (RDMs) to quantitatively relate neural data, computational models, and behavior. RSA is demonstrated by comparing fMRI‑derived RDMs of visual objects in early visual cortex and fusiform face area with computational models of varying complexity, using multidimensional scaling, randomization, and bootstrap techniques to relate and test the RDMs. The authors argue that RSA offers broad potential for experimental design and enables integrated quantitative analysis across all three branches of systems neuroscience, advancing a more unified field.

Abstract

A FUNDAMENTAL CHALLENGE FOR SYSTEMS NEUROSCIENCE IS TO QUANTITATIVELY RELATE ITS THREE MAJOR BRANCHES OF RESEARCH: brain-activity measurement, behavioral measurement, and computational modeling. Using measured brain-activity patterns to evaluate computational network models is complicated by the need to define the correspondency between the units of the model and the channels of the brain-activity data, e.g., single-cell recordings or voxels from functional magnetic resonance imaging (fMRI). Similar correspondency problems complicate relating activity patterns between different modalities of brain-activity measurement (e.g., fMRI and invasive or scalp electrophysiology), and between subjects and species. In order to bridge these divides, we suggest abstracting from the activity patterns themselves and computing representational dissimilarity matrices (RDMs), which characterize the information carried by a given representation in a brain or model. Building on a rich psychological and mathematical literature on similarity analysis, we propose a new experimental and data-analytical framework called representational similarity analysis (RSA), in which multi-channel measures of neural activity are quantitatively related to each other and to computational theory and behavior by comparing RDMs. We demonstrate RSA by relating representations of visual objects as measured with fMRI in early visual cortex and the fusiform face area to computational models spanning a wide range of complexities. The RDMs are simultaneously related via second-level application of multidimensional scaling and tested using randomization and bootstrap techniques. We discuss the broad potential of RSA, including novel approaches to experimental design, and argue that these ideas, which have deep roots in psychology and neuroscience, will allow the integrated quantitative analysis of data from all three branches, thus contributing to a more unified systems neuroscience.