Demonstration of VCR: A Tabular Data Slicing Approach to Understanding Object Detection Model Performance

Download PDF

Abstract

In this demonstration, we present VCR, an automated slice discovery method (SDM) for object detection models that helps practitioners identify and explain specific scenarios in which their models exhibit systematic errors. VCR leverages the capabilities of vision foundation models to generate segment-level visual concepts that serve as interpretable explanation primitives. By integrating these visual concepts with additional image metadata in a tabular format, VCR uses a scalable frequent itemset mining-based technique to identify common patterns associated with model performance. We will demonstrate VCR’s capabilities through three usage scenarios. First, users can explore the automatically extracted visual concepts and their associated labels. Second, users can run slice finding on a large object detection dataset and visually inspect the results to discover systematic errors. Finally, users can iteratively refine their slicing results by providing feedback on the granularity of visual concepts and the quality of the generated labels. These scenarios will illustrate how VCR can aid practitioners in discovering non-trivial gaps in their models’ performance, providing actionable insights for model improvement.