CloudJump: Optimizing Cloud Databases for Cloud Storages

Download PDF

Abstract

There has been an increasing interest in building cloud-native databases, which decouple computation and storage and allow them to scale independently. This enables high availability, elasticity, and on-demand pricing for cloud applications. A cloud-native database often adopts a cloud storage in its storage engine; it leverages another layer of virtualization and provides a high-performance storage service without exposing complex details such as fault tolerance and load balancing. Such a design choice helps reduce the maintenance cost and expedite development cycles for the database kernels. However, given the drastic differences between a cloud storage and a local storage, when building a cloud-native database through migrating an on-premise database kernel to the cloud using a cloud storage, many challenges exist in using the cloud storage “efficiently”. In this paper, we analyze the challenges and opportunities of both B-tree and LSM-tree based storage engines when they are deployed on a cloud storage. We then propose an optimization framework that guides database developers to transform on-premise databases into their cloud-native counterparts. We use a B-tree based cloud-native database, PolarDB, as a demonstration vehicle where we have implemented a suite of optimizations using the proposed framework, and extend such efforts to RocksDB which uses the popular LSM-tree based storage engine.We perform an extensive evaluation and demonstrate that the adoption of our proposed optimization framework leads to significant performance improvement towards building a cloud-native database.