2012 · 10 citations · 7 references
ComputingComputer ScienceComputer SystemsData ManagementDatabasesStorage SystemsBlock StorageData DeduplicationData RedundancyData Storage SystemsStorage ManagementBackup DeduplicationBlock-level DeduplicationData Redundancy ReductionDeduplication Performance EvaluationFile-level DeduplicationStorageStorage DeduplicationStorage-aware Deduplication
While the volume of emails are ever-increasing, email servers tend to have large redundancies demanding huge storage unnecessarily. MS exchange single instance (SIS) attempts storage-efficiency by removing redundant copies of an entire email-level when it is destined for multiple recipients' mailboxes within a server. However, redundant attachments or parts of emails due to threads are left storage-inefficient by SIS. On the other hand, existing deduplication systems are mainly for backup systems, and execute deduplication in block-level that makes it difficult to be used in in-line systems. We propose a hybrid scheme that adaptively performs deduplication at the granularity of either the file-level or chunk-level. We have designed and implemented a hybrid email deduplicaiton system, named HEDS, and evaluated it with real email datasets.We show that it achieves a high data reduction rate while keeping the CPU and memory overhead small;thus, it is feasible to be used as an in-line system on email servers.
7
Space/time trade-offs in hash coding with allowable errors
Burton H. Bloom · Communications of the ACM · 1970 · 7.5K citations · Full text
A low-bandwidth network file system
Athicha Muthitacharoen, Benjie Chen, David Mazières · 2001 · 786 citations
Venti: A New Approach to Archival Storage
Sean Quinlan, Sean Dorward · 2002 · 728 citations
A study of practical deduplication
Dutch T. Meyer, William J. Bolosky · ACM Transactions on Storage · 2012 · 472 citations