A probabilistic approach to diversified query recommendation

Abstract

The effectiveness of keyword-based search engines depends largely on the ability of a user to formulate proper queries that are both expressive and selective. However, web search queries issued by casual users are often short and with limited expressiveness. Query recommendation is a popular technique employed by search engines to help users refine their queries. Traditional similarity-based methods, however, often result in redundant and monotonic recommendations. We identify five basic requirements of a query recommendation system, namely relevancy, redundancy-free, diversity, ranking and efficiency. In particular, we focus on the requirements of redundancy-free and diversified recommendations. We propose the DQR framework, which mines a search log to achieve two goals: (1) It clusters search log queries to extract query concepts, based on which recommended queries are selected. Through query construction from the query concepts, we are able to avoid recommendation redundancy. (2) It employs a probabilistic model and a greedy heuristic algorithm to achieve recommendation diversification. Through a comprehensive user study we compare DQR against five other recommendation methods on real search log datasets. Our experiment shows that DQR outperforms the other methods in terms of relevancy, diversity, and ranking performance of the recommendations. At the same time, DQR also achieves high efficiency performance.