Representational Similarity Analysis (RSA) is a popular method for analyzing neuroimaging and behavioral data. Here we evaluate the accuracy and reliability of RSA in the context of model selection, and compare it to that of regression. Although RSA offers flexibility in handling high-dimensional, cross-modal, and cross-species data, its reliance on a transformation of raw data into similarity structures may result in the loss of critical stimulus-response information. Across extensive simulation studies and empirical analyses, we show that RSA leads to lower model selection accuracy, regardless of sample size, noise level, feature dimensionality, or multicollinearity, relative to regression. While principal component analysis and feature reweighting mitigate RSA's deficits driven by multicollinearity, regression remains superior in accurately distinguishing between models. Empirical data and a follow-up fMRI simulation further support these conclusions. Our findings suggest that researchers should carefully consider which approach to use: RSA is less effective than linear regression for model selection and fitting when direct stimulus-response mappings are available.
@article{arxiv.2511.00395,
title = {Is Representational Similarity Analysis Reliable? A Comparison with Regression},
author = {Chuanji Gao and Gang Chen and Svetlana V. Shinkareva and Rutvik H. Desai},
journal= {arXiv preprint arXiv:2511.00395},
year = {2025}
}