VLDB Endowment Logo

go back

Volume 14, No. 12

Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile

Authors:
Can Gencer (Hazelcast Inc.), Marko Topolnik (Hazelcast Inc.), Viliam Ďurina (Hazelcast Inc.), Emin Demirci (Layer Co), Basri Kahveci (-), Ali Gürbüz (Hazelcast Inc.), Ondřej Lukas (Hazelcast Inc.), Jozsef Bartok (Hazelcast Inc.), Grzegorz Gierlach (Hazelcast Inc), František Hartman (Hazelcast Inc.), Ufuk Yilmaz (Hazelcast Inc.), Mehmet Doğan (Layer Co), Mohamed Mandouh (Mansoura University), Marios Fragkoulis (TU Delft), Asterios Katsifodimos (TU Delft)
Download PDF

Abstract

Jet is an open-source, high-performance, distributed stream proces-sor built at Hazelcast during the last five years. Jet was engineeredwith millisecond latency on the 99.99th percentile as its primarydesign goal. Originally Jet’s purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast’s In-memory Data Grid (IMDG): a set of in-memory, partitioned and replicated data structures. With time, Jet evolvedinto a full-fledged, scale-out stream processor that can handle out-of-order streams and provide exactly-once processing guarantees.Jet’s end-to-end latency lies in the order of milliseconds, and itsthroughput in the order of millions of events per CPU-core. Thispaper presents the main design decisions we made in order to max-imize the performance per CPU-core, alongside lessons learned,and an empirical performance evaluation

PVLDB is part of the VLDB Endowment Inc.

Privacy Policy