Using Crowdsourcing to Code Open-Ended Responses

A Mixed Methods Approach

Datos Bibliográficos

ID12470636
AutoresMiriam R Jacobson (ICF, Los Angeles, CA, USA), Cristina E Whyte (Claremont Graduate University), Tarek Azzam (0000-0003-3864-0217, Claremont Graduate University, autor de correspondencia)
Año2017
Volumen39
Número3
Páginas413-429
Fecha de publicación2017-08-31
Peer ReviewedSí
Open AccessSí
TipoARTICLE
RevistaAmerican Journal of Evaluation (JOURNAL)
Identificadores de la revistaISSN: 1098-2140 • E-ISSN: 1557-0878
EditorialSAGE Publishing (PUBLISHER • US)
DOI10.1177/1098214017717014
OpenAlexW2751392745
IdiomaEN
Citas recibidas1
Referencias citadas18

Evaluators can work with brief units of text-based data, such as open-ended survey responses, text messages, and social media postings. Online crowdsourcing is a promising method for quantifying large amounts of text-based data by engaging hundreds of people to categorize the data. To further develop and test this method, individuals were recruited through online crowdsourcing to code open-ended survey responses, using a predetermined list of thematic codes that were derived from the responses. The study compared the coding results obtained from online crowdsourcing with coding results obtained from researcher coders. Additionally, the study examined feedback from the crowdsourced coders about their experiences with the task. The results suggested that online crowdsourcing can produce comparable results to researcher coding, but that the comparability of the results may differ across codes. This method may increase the efficiency of quantifying text-based data and provide evaluators with valuable feedback on their coding schemes

Categorization · Coding (social sciences · Comparability · Crowdsourcing · Data science · Information retrieval · Statistics · World Wide Web · Computer Science · Mathematics · Mobile Crowdsensing and Crowdsourcing · Social Media and Politics · Survey Methodology and Nonresponse · Artificial Intelligence