Multivariate Analysis: The Need for Data, and other Problems

TLDR

Multivariate analyses aid but do not replace critical thinking; meaningful results require careful attention to sample size, variable type, and distribution, yet the ease of accessing software and editorial pressure to include multivariate tests and significance levels has fostered widespread misuse. Researchers should perform initial graphical screening of data to ensure suitability before applying complex multivariate analyses. Abuses of univariate methods are unlikely to produce the severely misleading results seen with multivariate misuse.

Abstract

Multivariate analyses are an aid to, not a substitute for critical thinking in the area of data analysis. Meaningful results can only be produced by these methods if careful consideration is given to questions of sample size, variable type, variable distribution etc., and accusations of subjectivity in interpretation can only be overcome by replication. The computer revolution has produced many problems for statisticans, not least of which is the ease with which experiments may access packages of programs for multivariate analysis, and so bypass a "difficult" (by which is meant one who will not do simply as he is told) statistician. Of course there are many abusers of univariate statistical methods. Here, however, the abuses are not likely to lead to such seriously misleading results as in the multivariate case. Perhaps a major cause of the continuing misuse of statistical methods is the insistence of many journal editors in psychology and related areas, on articles being laced with multivariate analyses, and on encouraging the pedantic use of signifance levels, i.e. the inevitable p less than minus, as if such inclusions lent an air of respectability to their journal which it might not otherwise have had. Research workers in these fields would be better encouraged to devote more time to an initial screening of their data using simple graphical techniques, to ensure that their data are at least approximately suitable for more complicated multivariate analyses.