BABOONS: Black-Box Optimization of Data Summaries in Natural Language

Download PDF

Abstract

BABOONS (BlAck BOx OptimizatioN of data Summaries) is a system that automatically optimizes text data summaries for an arbitrary, user-defined utility function. Data summaries use relational data to compare user-defined items to others in terms of aggregate values for data subsets. For instance, BABOONS supports text evaluation by user-provided models for text analysis. BABOONS uses reinforcement learning to explore the space of possible descriptions. In each iteration, BABOONS generates summaries and evaluates their utility. To reduce data processing overheads during summary generation, BABOONS uses a proactive processing strategy that dynamically merges current with likely future queries for efficient processing. Also, BABOONS supports scenario-specific sampling and batch processing strategies. These mechanisms allow to scale processing to large data and item sets. The experiments show that BABOONS scales significantly better than baselines. Also, they show that summaries generated by BABOONS receive higher average grades from users in a large survey.