Approximating discrete probability distributions with dependence trees
IEEE Transactions on Information Theory · 1968 · 2.6K citations · 9 references
EngineeringDiscrete ProbabilityInformation TheoryDecision TreeUncertainty QuantificationUnknown DistributionRandomized AlgorithmProbabilistic AnalysisStatistical InferenceProbability TheoryComputer ScienceBayesian MethodsCombinatorial OptimizationTree DependenceStatisticsOrder Dependence RelationshipDependence Trees
A method is presented to approximate optimally an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</tex> -dimensional discrete probability distribution by a product of second-order distributions, or the distribution of the first-order tree dependence. The problem is to find an optimum set of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n - 1</tex> first order dependence relationship among the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</tex> variables. It is shown that the procedure derived in this paper yields an approximation of a minimum difference in information. It is further shown that when this procedure is applied to empirical observations from an unknown distribution of tree dependence, the procedure is the maximum-likelihood estimate of the distribution.
9
On Information and Sufficiency
S. Kullback, R. A. Leibler · The Annals of Mathematical Statistics · 1951
19.5K citations
On the shortest spanning subtree of a graph and the traveling salesman problem
Joseph B. Kruskal · Proceedings of the American Mathematical Society · 1956
5K citations
On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem
Joseph B. Kruskal · Proceedings of the American Mathematical Society · 1956
1.1K citations
Approximating probability distributions to reduce storage requirements
Philip Lewis · Information and Control · 1959
164 citations
A note on approximations to discrete probability distributions
David T. Brown · Information and Control · 1959
94 citations