Database Technology for the Masses: Sub-Operators as First-Class Entities

Download PDF

Abstract

Relational databases have developed a wealth of technology over decades that has been successfully tried and tested in many settings and use cases. Yet, often most of it is left aside in the pursuit of performance (e.g., NoSQL) or new functionality (e.g., graph data, machine learning). In this paper, we argue that a wide range of techniques readily available in databases are crucial to tackling the challenges the IT industry faces in managing hardware trends, growing workloads, and the overall complexity of a quickly changing application and platform landscape. However, to be truly useful, these techniques must be freed from the legacy component of database engines: relational operators. Therefore, we argue that in order to make databases a more flexible platform and extend their functionality to new data types and operations, it requires to expose a lower level of abstraction: instead of working with SQL, database engines should compile, optimize, and run a collection of sub-operators for manipulating and managing data, offering them as an external interface. In this paper, we discuss the advantages of doing so, provide a first list of such sub-operators, and show how they can be used in practice.