Abstract

Vectors have quickly become the de facto standard in modern search-oriented applications due to their ability to represent complex data in a structured and effective manner. While existing work has focused mainly on scalable retrieval over vector databases, this paper is the first to tackle unique challenges and opportunities in a streaming environment, where continuous and scalable vector processing is required. Streaming settings present distinctive constraints, such as low-latency processing requirements and dynamic data flows, which necessitate new trade-offs and the application of both novel and established techniques. To this end, we are developing VectraFlow, a stream-oriented data flow engine to support scalable monitoring applications involving vector data. We argue that VectraFlow can be effectively used to support a large suite of online applications such as continuous prompts, copyright infringement detection, and video-based surveillance where vector streams need to be processed continuously and with low latency. VectraFlow’s design emphasizes efficient handling of data streams, ensuring that latency is minimized while maintaining accuracy and scalability. We detail VectraFlow’s architecture, focusing on vectorbased streaming filtering, top-k, and join operations implemented through efficient clustering and novel indexing structures. We also investigate the use of alternative data representation techniques, such as quantization, which demonstrate substantial performance improvements over current methods while maintaining high-quality results. We conclude with ongoing and future research directions in this area.