Robust and Budget-Constrained Encoding Configurations for In-Memory Database Systems

Download PDF

Abstract

Data encoding has been applied to database systems for decades as it mitigates bandwidth bottlenecks and reduces storage requirements and thus costs. But even in the presence of these advantages, most in-memory systems only defensively employ compression as the negative impact on runtime performance can be severe. Nevertheless, real-world systems with large parts being infrequently accessed and cost efficiency constraints in cloud installations necessitate solutions that automatically and efficiently select encoding techniques, including heavy ones. In this paper, we study approaches to determine memory budget-constrained encoding configurations using greedy heuristics and linear programming. We show that optimal configurations can reduce the main memory footprint by 60% for TPC-H without a loss in runtime performance over state-of-the-art dictionary encoding. To yield robust selections, we extend the linear programming-based approach to incorporate query runtime constraints and mitigate unexpected performance regressions.