Building a database on S3

TLDR

Amazon S3 is widely promoted for its infinite scalability, high availability, and low cost, and is primarily used to store rarely updated media files. The study aims to evaluate the suitability of S3 for general‑purpose database workloads with small objects and frequent updates, examining its cost, performance, and consistency. The authors present read, write, and commit protocols for S3 and analyze its cost, performance, and consistency characteristics.

Abstract

There has been a great deal of hype about Amazon's simple storage service (S3). S3 provides infinite scalability and high availability at low cost. Currently, S3 is used mostly to store multi-media documents (videos, photos, audio) which are shared by a community of people and rarely updated. The purpose of this paper is to demonstrate the opportunities and limitations of using S3 as a storage system for general-purpose database applications which involve small objects and frequent updates. Read, write, and commit protocols are presented. Furthermore, the cost ($), performance, and consistency properties of such a storage system are studied.