Missing Data in Phylogenetic Analysis: Reconciling Results from Simulations and Empirical Data

Abstract

This paper will attempt to resolve some controversies about the effects of missing data on phylogenetic analysis. Whether missing data are generally problematic is a critical issue in modern phylogenetics, especially as wildly different amounts of molecular data become available for different taxa, ranging from entire genomes, to single genes, to none (e.g., fossils). Our perception of the impact of missing data (or lack thereof) may strongly influence which taxa and characters we include in a phylogenetic analysis (Wiens 2006) and may lead to a diversity of serious errors. For example, if we think that missing data are problematic when they are not, then we may exclude taxa and characters that would otherwise benefit our analyses, given the abundant evidence that increasing numbers of both taxa and characters can potentially improve the accuracy of phylogenetic analyses (e.g., Huelsenbeck 1995; Rannala et al. 1998; Poe 2003), where accuracy is generally defined as the similarity between the estimated tree and the correct, known phylogeny. In contrast, if missing data cells are themselves intrinsically problematic (e.g., Huelsenbeck 1991), including taxa or characters with many missing data cells may lead to inaccurate phylogenetic estimates. Several studies have explored how missing data may impact phylogenetic analyses, using both empirical and simulated data. Many simulation and empirical studies now suggest that it is often possible to include taxa that have large amounts of missing data without ill effects (e.g., Wiens 2003b; Driskell et al. 2004; Philippe et al. 2004; Wiens et al. 2005; Wiens and Moen 2008; Lynch and Wagner 2010; Thomson and Shaffer 2010; Wiens, Kuczynski, Townsend, et al. 2010). However, a recent simulation study (Lemmon et al. 2009) suggested instead that missing (“ambiguous”) data are generally problematic for phylogenetic analysis and implied that these previous simulation and empirical studies are therefore incorrect. They justified their study based on the grounds that previous studies were supposedly in conflict about the impacts of missing data (p. 131). In this paper, we will show that the paper by Lemmon et al. (2009; LEA hereafter) is problematic for several reasons. First, despite their statement that previous studies are in conflict, most simulation and empirical results on missing data can be easily explained within an existing theoretical framework (Wiens 2003b). Furthermore, many contradictory studies suggesting that missing data are not generally problematic for Bayesian and likelihood analyses (given some assumptions) were not addressed by LEA. Second, the sweeping negative conclusions of LEA are not necessarily supported by their results. LEA find missing data to be problematic primarily when using sets of invariant or saturated characters and/or when obvious rate heterogeneity is ignored. Their results do not support the idea that missing data generally lead to incorrect inferences about topology when informative data are analyzed with appropriate methods. We conduct new simulations under more realistic conditions, and these results show no evidence that missing data generally lead to inaccurate Bayesian estimates of phylogeny. In fact, we show that the practice of excluding characters simply because they contain missing data cells may itself reduce accuracy. We reanalyze the “manipulated” empirical example from LEA and find that, without these artificial “manipulations” of the data, their conclusions are not supported. We also analyze eight empirical data sets, each containing many taxa with extensive missing data. We show that these incomplete taxa are consistently placed into the expected higher taxa, often with very strong support. Overall, our results confirm previous simulation and empirical studies showing that taxa with extensive missing data can be accurately placed in phylogenetic analyses and that adding characters with missing data can be beneficial (at least under some conditions). We conclude by pointing out important areas for future research on the topic of missing data and phylogenetic analysis. There are two main ways that missing data might be added to a phylogenetic analysis, either through the addition of incomplete characters or incomplete taxa. For example, imagine having data from two genes for a given genus of organisms, in which the first gene has been sequenced for all 10 species and the second gene has been sequenced for only 5 species. Given this situation, one might (a) analyze only the first gene for all 10 species and then decide whether or not to (b) add the second gene (adding incomplete characters that are missing data for 5 of the species). Or, one might (c) analyze only the 5 species having data for both genes, and then decide whether or not to (d) add the 5 species lacking data for the second gene (adding incomplete taxa that are lacking data for the first gene). Note that (b) and (d) are effectively identical. Most of the literature on missing data has focused on whether to include taxa lacking data for some characters (c versus d). LEA did not actually address this question, they that their results studies that did (e.g., we previous studies on incomplete taxa studies and address incomplete characters in our simulations and under for in conflict, we that most of the empirical and simulation studies on missing data are when in of the that incomplete taxa can potentially be accurately placed if informative characters are (Wiens 2003b). the impacts of extensive missing data in these studies a negative to based on the of characters in the analysis. issue of missing data first in with analyses of data for taxa (e.g., et al. et al. studies have incomplete taxa to be problematic in some (e.g., many and 1995; not (e.g., et al. However, these studies characters to a often simulations of Huelsenbeck only characters and incomplete taxa missing were often Wiens and that including incomplete taxa accuracy in analyses of known their two data sets and each informative and that their analyses were to missing data with characters genes for with no on topology when one or two of the genes were with missing data that some when some taxa missing data. simulations of Wiens also that incomplete taxa (e.g., missing can be accurately placed given characters in and analyses, the of on the phylogenetic of missing data and (e.g., is more with and when missing data are taxa, and when are and/or characters et al. a of simulations based on their data for characters for and that the impact of incomplete taxa on the from negative to none to simulations in which all taxa were Philippe et al. characters from and that incomplete taxa were placed with strong support in their empirical likelihood analyses the most incomplete taxa and missing data, and the likelihood with their taxa are and They also accuracy in their simulations based on data (e.g., accuracy for all when of the data were and accuracy when were Driskell et al. data sets with very large numbers of characters for and for and extensive missing data and and that incomplete taxa were placed in expected from previous with strong support based on Wiens et al. characters for and that incomplete taxa were placed in the expected with very strong support by and Bayesian analyses (e.g., 10 species with missing data each were all placed in the expected with of these each supported with a Bayesian of recent empirical studies have also that incomplete taxa are placed in the expected with strong and most of these studies characters each (e.g., Lynch and Wagner 2010; Thomson and Shaffer 2010; Wiens, Kuczynski, Townsend, et al. 2010). Wiens simulations to show that adding incomplete taxa missing be as as taxa in likelihood and Bayesian analyses from when the in these analyses were rate heterogeneity and were simulated not in the given a of simulations of data by and accuracy with extensive missing data for and analyses, only Wiens and Moen their simulations to show that incomplete taxa be accurately placed in Bayesian analyses given characters (e.g., when rate heterogeneity and were simulated not in the Bayesian In all of these simulation and empirical studies to into this with incomplete taxa potentially problematic when the of characters is and generally when the is This framework to to all phylogenetic not simply likelihood and Bayesian analysis. results of many of these studies the conclusions of LEA were not by including the that addressed the impact of missing data on likelihood and Bayesian analyses (e.g., the results of Philippe et al. 2004; Wiens 2005; Wiens et al. were not and the two studies were not For example, LEA conclude that both and Bayesian rate can with data to estimates of (p. and that problematic rate is not (p. the simulation studies by Wiens and Wiens and Moen of with incomplete taxa by Bayesian and likelihood when rate simulated ignored. LEA analyzed a very of simulated and results that were with of simulation studies of the they sweeping conclusions from their results (e.g., that their results have for analyses that on estimates of topology or They also that their results of previous is therefore important to they did and more LEA the with the of and (given that an tree is For each of conditions, they simulated two data sets and one with data for all taxa and and lacking data for all characters for two taxa. genes were simulated under either the or different of They then Bayesian for the single for and for and For they the with which this They that the data would the results as because data were only in two of the taxa for under the that this is not necessarily for likelihood or Bayesian analysis, They that Bayesian for the data from the based on for likelihood a only when were to They to these as In some these to be as when for the tree that the is not they some where very for the both data sets were effectively They suggested that these were to the Bayesian (e.g., et al. the for Bayesian analysis to strongly one tree when is with which to of missing However, they none of these the characters were effectively invariant or by LEA as as to be effectively and rate heterogeneity between genes simulated and then by to by only we find is in their for one of with very in both genes and data missing in taxa. their results do not support their sweeping about the negative impact of missing data, especially for to be by most empirical This many previous simulation and empirical studies their conclusions about the negative impact of missing data on Bayesian and likelihood analyses In some they show that data from for (e.g., their when rate of is However, that these are that has no influence on topology for the analysis be the as for this is for it is not necessarily for Bayesian or likelihood For example, if has no influence on Bayesian estimates of topology are based on then how can it influence the that incomplete characters in have no impact be Furthermore, that data from for is not evidence that data estimates of accuracy in this the that the is by the under a given of conditions). would the between accuracy that the is by Bayesian analysis of and and the for these studies the between Bayesian and accuracy et al. et al. Huelsenbeck and Rannala LEA did not the between accuracy and when missing data are and for these conditions, their about by missing data are not actually based on In the results of LEA suggest that missing data are primarily problematic when characters and/or when to data sets, that may not be by most This is not to that we think that data sets with missing data with support that the simulation results of LEA can have a very different from their sweeping negative if one simply which of their results are to actually critical issue is the addition of sets of characters with data for only two which are expected to have impact on the analysis which would not be by empirical is if their results are to adding only two species or if they also to numbers of species. We have therefore new simulations to address the of the results of LEA to numbers of taxa. We addressed how adding data from a gene with incomplete to the one with the accuracy of Bayesian generally Wiens and Moen We simulated that were either or we simulated data with the with the We generally characters also simulated all characters for all taxa. For we simulated the data, and then a of taxa in each to have all their characters with missing data In one of eight taxa were incomplete in of the of LEA to numbers of In of taxa were only taxa in with data of the of LEA to more were simulated under a of of in each data from very of in a given a given of to and analyses on the tree that the most accuracy for these of tree or We simulated both in each data and many not possible of We analyzed for each of We analyzed and then analyzed data sets and sets were analyzed using and a with a for of and to analyses were a different for in each data were for and excluding the first as tree for these (Wiens and Moen We accuracy for each as the of in the Bayesian tree that are with the and accuracy a given of is based on the for This of accuracy is in many previous simulation studies (e.g., Wiens 2003b; Wiens and Moen are they also the similarity between the and estimated We did not support for a will not be is and LEA did not the between accuracy and of simulations showing the impact of adding sets of characters with missing data on the accuracy of Bayesian phylogenetic analysis. each the is accuracy for the is for analysis of sets and on the are for simulations in which has eight taxa and the results in which has only two taxa. We find that a of adding the data of missing data of taxa either or has no on to the data the are under some conditions, the can be (e.g., in accuracy may when the of in the two data sets are or when they are very as the added data has only two taxa in the simulations of accuracy may be higher or and is consistently within results suggest that adding sets of characters with only two species has influence on the accuracy of analyses with numbers of taxa, and that the and results of LEA do not to more realistic to the conclusions of we find no evidence that adding sets of characters with extensive missing data to estimates of Bayesian or support only with are our results suggest that under some conditions, to add characters with missing data can lead to phylogenetic accuracy. about excluding characters simply because they have missing data can lead to phylogenetic accuracy. This is a critical that LEA do not Our simulation are not to of LEA. For example, we that will not to analyze data sets that lack phylogenetic to that are or we do not these conditions, we do that are both and We also that most will data sets different most our results suggest that the Bayesian estimates by LEA do not necessarily under more realistic (e.g., more taxa, data, and of one example, LEA suggest that Bayesian may be strongly by whether the taxa with data are or taxa, this when taxa are For example, given a tree with missing data in species and data are in both taxa and taxa simulations are also very and very from Many that have been were not (e.g., more in within in to the results more with of LEA instead Wiens 2005; Wiens and Moen the most important is the of the In to address how missing data influence Bayesian and likelihood analyses under realistic conditions, we also analyses of eight empirical data we do that, we address the empirical example by LEA. LEA analyzed an empirical data many that these data very different from analyzed by empirical They analyzed data from a single gene from eight species of for which entire were et al. a very of We not find an for these characters and taxa were They then added a second data of missing data for of the eight species and empirical for the data of from the gene for the to be either all invariant or all between the two species. they that adding this second of characters Bayesian and likelihood estimates of and this analysis many about not data (e.g., instead of from the only and invariant are their results an of these We addressed this using very empirical data, and our results a to of LEA We the data, instead of adding only invariant or sets of characters from the we added data from gene from et al. to the species to which LEA added data. adding gene is more to empirical actually of that characters can strongly estimates of support and (p. we find that Bayesian and likelihood estimates of and are adding with data missing in species for in their it that the results of LEA of adding invariant and saturated characters to data and therefore may have to most empirical of data in show that adding sets of characters with extensive missing data may have impacts on and with of We the data for the species as LEA using instead of adding invariant and sets of characters from we added data from gene are added for the of and species by all species have missing data cells for Bayesian analyses and with the and analyses 2006) with the with and analyses were by to or are et al. for for and from their example “manipulated” empirical data, LEA do not show empirical studies in which missing data to be problematic for Bayesian or likelihood methods. Note that on LEA of has an example of gene in empirical data from were from the analysis, gene tree et al. However, the only to missing data in that paper were the to of missing data in taxa to in and and the that these influence the phylogenetic analysis (Lemmon et al. the to (p. and also (p. to have a on the of a and/or Bayesian analysis (Lemmon et This to exclude with missing data for more one from our the et al. paper not contain the empirical results that LEA that it only to LEA. of missing data is that empirical may LEA that the negative impacts of missing data on phylogenetic analysis are to that and (p. this were we would to negative impacts in empirical analyses that include extensive missing data. We have empirical studies that evidence that impacts can be or (e.g., Philippe et al. 2004; Wiens et al. for likelihood and Bayesian we analyses of eight empirical data sets that show the is in most empirical data However, one can about how will with data given the results of is not empirical can be from the simulations of LEA. they that extensive missing data may (p. estimated in likelihood and Bayesian this were the then we that incomplete taxa will be placed in that to be incorrect based on previous and research accuracy based on In contrast, if the of Wiens is correct, and if characters have been then we that incomplete taxa will be placed in the expected higher taxa (e.g., and with strong support. In be strong support for the of these species within these higher taxa characters were Wiens et al. we for a negative between support and of taxa. We eight empirical data sets all Bayesian or likelihood analyses of eight have least one species with missing data, and data sets have least one species with missing data. Given that all eight studies are from our we that taxa were not simply because of or because of how the results is for eight studies include genes, and of eight include both and is rate heterogeneity genes and data on the eight data sets in analyses of incomplete taxa and support Bayesian on the eight data sets in analyses of incomplete taxa and support Bayesian For each data we the of missing data cells in each species. Most missing data from the of data from one or more genes (or of in analyses, a also from in of support were based on in the study Bayesian we likelihood for et al. which For the and data sets, we as the as were for for and using (e.g., results. are in the However, given that effects of missing data may on how a these we that likelihood analyses 2006) and Bayesian analyses and We first whether incomplete taxa are placed in the expected based on previous and whether they are placed in these with strong support. incomplete taxa are generally then they not be consistently placed in the by previous or if they the support for these be For each data we a of from previous of as the of most of these species based on data. However, for with only one species we for For we given that the for these is (e.g., et al. et al. 2006) is not (e.g., et al. Wiens et al. We then the support for each or Bayesian and the species with the of missing data in that the that be most to accurately We the that these higher taxa may be with a of within the this of not our results or Wiens et al. we then the of support for the of each species based on either the support for the with their species species that are to a single or the of the support for the with their and the support for the excluding the species from that species that are to species). Given the of and support for each we then the between these using analysis, in Note that a negative between support and is not with the by Wiens are data to accurately a on the then the support for be However, the simulations of Wiens suggest that, given incomplete taxa will be accurately placed in the with This be with support we that this analysis not necessarily address whether support are by missing data, they are strongly to be consistently or negative LEA do not address either because they did not how accuracy and support are eight data sets include species and higher taxa, and all of these species are placed in the expected higher taxa, despite many having extensive missing data and Furthermore, the of most of these is strongly supported likelihood most In the in which are not in these data sets, are missing data that are In and both to be et al. previous phylogenetic studies with missing data suggest that this and and of to conflict between and genes, not missing data Kuczynski, et al. 2010). In if missing data are generally problematic as LEA not to be evidence for it in these eight data sets the previous in these have been in a that is with the effects of missing data on likelihood and Bayesian analyses of of support for higher taxa within the eight data sets, showing that all higher taxa are strongly supported as despite many of containing one or more taxa with extensive missing data Bayesian of support for higher taxa within the eight data sets, showing that all higher taxa are strongly supported as despite many of containing one or more taxa with extensive missing data Bayesian two of the eight studies show negative between support and results suggest that missing data have negative impact on of and is strong support for the phylogenetic of taxa with missing data within these expected higher taxa. the two data sets with between support and have the numbers of taxa only numbers of characters we that when informative characters have been in a we only support for in the between the of a missing and the support for in phylogenetic analyses using likelihood and Bayesian analysis. In addition to these eight studies and (e.g., Driskell et al. 2004; Philippe et al. 2004; Wiens et al. recent studies have also (e.g., Lynch and Wagner 2010; Thomson and Shaffer 2010; Wiens, Kuczynski, Townsend, et al. 2010; et al. For example, Lynch and Wagner with a Bayesian analysis of molecular with some taxa incomplete and each having an of missing data. their is generally strongly supported and with previous and (e.g., of with are strongly supported as with Wiens, Kuczynski, Townsend, et al. that addition of molecular characters to a data of characters for did not the of most taxa in a Bayesian analysis the having missing data in this and no in Bayesian for taxa. Furthermore, the of taxa with previous (e.g., placed in both and addition of molecular data. There are now many studies showing support for the idea that incomplete taxa can be accurately placed in analyses, and sweeping about the negative impacts of missing data are not many of the impact of missing data on phylogenetic analysis are in of In addition to the effects of incomplete taxa, given a of characters for a of taxa, is it to add a second of characters that are incomplete they include data for only some of the In when do the of adding more characters the of increasing missing data in the it might that the simulations of LEA addressed this However, their results may be of to empirical studies because only two species were Our simulations suggest that adding a of characters with data for of the species is generally either beneficial or for Bayesian analysis. However, these simulations were not and analyses are (e.g., different numbers of and different of simulation and empirical studies have also results suggesting that incomplete characters can be with some Wiens that adding sets of incomplete characters can potentially accuracy for that accuracy more by the of added data taxa and more characters with missing also of when a of incomplete characters is Wiens et al. that adding a of characters available for only some taxa with missing to improve results to from only characters for a of taxa. some taxa are in the analysis of characters on previous not in the analysis. simulations of et al. that accuracy of likelihood analyses higher when with missing are Wiens simulations to address whether adding molecular data phylogenetic accuracy for taxa, in a analysis of molecular and data, with and Bayesian analysis the molecular data are missing in the simulations that under many conditions, adding molecular data accuracy for taxa. of empirical studies (Wiens 2009) that adding molecular data can improve resolve in for the of taxa, least in some analyses et al. analysis of Kuczynski, Townsend, et al. that molecular data can the of some taxa, in addition to increasing would also be to the effects of missing data on of LEA that their results on are to this they that their results may be an of not including rate heterogeneity in the likelihood (p. and this is supported by our analyses also Furthermore, their study no of We have several analyses using that contain extensive missing data (e.g., Wiens, et al. et al. Wiens et al. using both likelihood and Bayesian et al. we have no evidence to suggest that these estimates are generally by missing data. these estimates are generally to for the based on data sets with missing data cells (e.g., et al. Wiens et al. for Wiens et al. for this is an in of Many areas to be For example, it is how gene may with missing data to impact phylogenetic accuracy. simulation studies have that different genes the and have been based on analysis of genes or impact of missing data on that species without (e.g., et al. also impact of missing data on support would also benefit from For simulations are to address whether the of support (e.g., likelihood Bayesian for taxa with extensive missing data. LEA (p. that the results of their study for all analyses that on estimates of topology or including rate analysis, phylogenetic and However, of their results that their evidence for the negative impacts of missing data on that would not be by most empirical (e.g., adding data sets of invariant or to data sets different are their sweeping are not supported by their results. are also by many previous simulation and empirical and also by new results from simulations that numbers of taxa and data from of their example and from eight empirical data sets analyzed In to the idea of studies by we that most results on missing data can be explained in a theoretical framework (Wiens and that most studies suggest that it generally be possible to accurately incomplete taxa in if informative characters are We think that is a for of the impact of missing data on phylogenetics, and we out in of focused However, future studies to their new results with from previous studies in to in this This supported by the to For on the we For on the we the and and