Book chapter
Human Computation for Information Retrieval
Handbook of Human Computation, pp.205-214
Springer New York
11/20/2013
DOI: 10.1007/978-1-4614-8806-4_18
Abstract
Human computation techniques, such as crowdsourcing and games, have demonstrated their ability to accomplish portions of information retrieval (IR) tasks that machine-based techniques find challenging. Query refinement is one such IR task that may benefit from human involvement. We conduct an experiment that evaluates the contributions of participants from Amazon Mechanical Turk (N = 40). Each of our crowd participants is randomly assigned to use one of two query interfaces: a traditional web-based interface or a game-based interface. We ask each participant to manually construct queries to respond to a set of OHSUMED information needs and we calculate their resulting recall and precision. Those using a web interface are provided feedback on their initial queries and asked to use this information to reformulate their original queries. Game interface users are provided with instant scoring and asked to refine their queries based on their scores. In our experiment, crowdsourcing-based methods in general provide a significant improvement over machine algorithmic methods, and among crowdsourcing methods, games provide a better mean average precision (MAP) for query reformulations as compared to a non-game interface.
Details
- Title: Subtitle
- Human Computation for Information Retrieval
- Creators
- Christopher G Harris - SUNY Oswego, Oswego, USAPadmini Srinivasan - Iowa City, USA
- Resource Type
- Book chapter
- Publication Details
- Handbook of Human Computation, pp.205-214
- DOI
- 10.1007/978-1-4614-8806-4_18
- Publisher
- Springer New York; New York, NY
- Language
- English
- Date published
- 11/20/2013
- Academic Unit
- Nursing; Computer Science; Business Analytics
- Record Identifier
- 9984003187102771
Metrics
21 Record Views