Various approaches are developed for evaluation of query-oriented text summarization. However, for text forums this procedure is not well-defined, and standard approaches are not suitable. Evaluation of query-oriented text summarization greatly depends on the query type. We compare two typical scenarios of search of professionally significant information on Internet forums. Our subject of interest is the similarities and differences between relevance-oriented queries and usefulness-oriented queries. To compare these query types we have collected dataset, extracted textual, structural features and social graph features, constructed different ranking models, used suitable quality measure (NDCG), and applied feature selection techniques to investigate causes of differences. We have found out that these query types are very different by their nature, have weak correlation. Distinct model types and features should be used in order to create an efficient information retrieval system for each query type.
展开▼
