Heuristic subset clustering for consideration set analysis

The term consideration set is used in marketing to refer to the set of items a customer thought about purchasing before making a choice. While consideration sets are not directly observable, finding common ones is useful for market segmentation and choice prediction. We approach the problem of inducing common consideration sets as a clustering problem. Our algorithm combines ideas from binary clustering and itemset mining, and differs from other clustering methods by reflecting the inherent structure of subset clusters. Further, we introduce two speed-up methods to make the algorithm more efficient and scalable for large datasets. Experiments on both real and simulated datasets show that our algorithm clusters effectively and efficiently even for sparse datasets. A novel evaluation method is also developed to compare clusters found by our algorithm with known ones. Based on the clusters found by our algorithm, different classification models are built for each particular consideration set. The advantages of the two-stage model are it builds specific model for different clusters, and it helps us to capture the characteristics of each group of the data by analyzing each model.