Distance between sampling with and without replacement

Abstract

Summary Two random samples of size n are taken from a set containing N objects of H types, first with and then without replacement. Let d be the absolute (L 1 ‐)distance and I the K ullback ‐L eibler information distance between the distributions of the sample compositions without and with replacement. Sample composition is meant with respect to types; it does not matter whether order of sampling is included or not. A bound on I and d is derived, that depends only on n, N, H. The bound on I is not higher than 2 I. For fixed H we have d 0, I 0 as N if and only if n/N 0. Let W r be the epoch at which for the r‐th time an object of type I appears. Bounds on the distances between the joint distributions of W 1 ., W r without and with replacement are given.