Statistical Properties of the Maximum Likelihood Method of Phylogenetic Estimation and Comparison With Distance Matrix Methods

Abstract

A proof is presented that the maximum likelihood (ML) method of phylogenetic estimation from DNA sequences (Felsenstein, 1981, J.'Mol. Evol. 17:368-376) is statistically consistent despite the irregularity of the parameter space of the estimation problem. The distance matrix method using the least squares (LS) criterion is also consistent, but disconnection of two steps in the method, i.e., estimation of sequence divergence and construction of the tree topology, appears to lead to both theoretical contradictions and practical problems. Comparison of the ML and LS methods shows that the ML method is much more, indeed extremely, tolerant to violation of its assumptions and also has smaller sampling errors caused by limited data. This conclusion should be general, independent of particular models, tree topologies, and number of species in the data set. The problem of evaluating the reliability of the estimated tree topology was examined. The test of positivity of the interior branch length in the estimated tree is not a test of the significance of the ML tree but may be taken as a test of the significance of the LS tree.