DBLK: Deduplication for primary block storage

Abstract

The deduplication block-device (DBLK) is a deduplication and compression system with a block device interface. It is used as a primary storage and block-wise deduplication is done inline. Since deduplication for primary storage requires low latency and detecting block-wise deduplication creates a large amount of metadata, it is necessary to efficiently use the memory of the system. We solved this problem by developing a multilayer Bloom filter (MBF) to reduce the size of the data structure in the memory for indexing duplicate data.