Book chapter
Comparing Crowd-Based, Game-Based, and Machine-Based Approaches in Initial Query and Query Refinement Tasks
Advances in Information Retrieval, pp.495-506
Lecture Notes in Computer Science, Springer Berlin Heidelberg
2013
DOI: 10.1007/978-3-642-36973-5_42
Abstract
Human computation techniques have demonstrated their ability to accomplish portions of tasks that machine-based techniques find difficult. Query refinement is a task that may benefit from human involvement. We conduct an experiment that evaluates the contributions of two user types: student participants and crowdworkers hired from an online labor market. Human participants are assigned to use one of two query interfaces: a traditional web-based interface or a game-based interface. We ask each group to manually construct queries to respond to TREC information needs and calculate their resulting recall and precision. Traditional web interface users are provided feedback on their initial queries and asked to use this information to reformulate their original queries. Game interface users are provided with instant scoring and ask to refine their queries based on their scores. We measure the resulting feedback-based improvement on each group and compare the results from human computation techniques to machine-based algorithms.
Details
- Title: Subtitle
- Comparing Crowd-Based, Game-Based, and Machine-Based Approaches in Initial Query and Query Refinement Tasks
- Creators
- Christopher G Harris - Informatics Program, The University of Iowa, Iowa City, USAPadmini Srinivasan - Computer Science Department, The University of Iowa, Iowa City, USA
- Resource Type
- Book chapter
- Publication Details
- Advances in Information Retrieval, pp.495-506
- Publisher
- Springer Berlin Heidelberg; Berlin, Heidelberg
- Series
- Lecture Notes in Computer Science
- DOI
- 10.1007/978-3-642-36973-5_42
- eISSN
- 1611-3349
- ISSN
- 0302-9743
- Language
- English
- Date published
- 2013
- Academic Unit
- Nursing; Computer Science; Business Analytics
- Record Identifier
- 9984003177002771
Metrics
25 Record Views